KR20210058622A - Automatic medical drawing generator, the method for generating automatic medical drawing thereof and machine learning based automatic medical drawing generator - Google Patents

Abstract

The present invention relates to an automatic medical image generating system, an automatic medical image generating method using the same and a machine learning based automatic medical image generating system. The automatic medical image generating system comprises: an information extraction unit (100) configured to extract information of at least one of character data (130) and a medical video (110) which are made during diagnosis; a shape generating unit (200) for generating a basic model (210) and a disease model (220); and a display unit (600) for visualizing the basic model (210) and the disease model (220). Therefore, the automatic medical image generating system delivers accurate information in detail.

Description

Automatic medical picture generation system, automatic medical picture generation method using the same, and automatic medical picture generation system based on machine learning {AUTOMATIC MEDICAL DRAWING GENERATOR, THE METHOD FOR GENERATING AUTOMATIC MEDICAL DRAWING THEREOF AND MACHINE LEARNING BASED AUTOMATIC MEDICAL DRAWING GENERATOR}

The present invention relates to an automatic medical picture generation system and a medical picture generation using the same. The system clarifies what the patient's condition is like in a picture. Therefore, compared to the medical records previously checked only with text, the medical staff can check them in more detail and more accurately, and the patient can understand the disease more easily and deeply.

Medical pictures represent the patient's condition in place of text. Medical pictures are simpler than text medical records, but can convey the location of symptoms, shape and size of tumors, etc. more accurately than text records. In addition, by accurately conveying the size and shape of the disease to general patients who are not familiar with medical terms, it motivates treatment and enables them to better cooperate in treatment.

Conventional medical drawings were created by humans. Prior to the use of electronic charts, after surgery, surgeons drew a simple picture of the operation and recorded the procedure, or the location of symptoms was marked on a simple drawing of the whole body. Medical pictures were also a means for doctors to leave notes for medical activities, but they were also a means for explaining to patients. In modern times, this medical picture has developed into a form in which medical staff can directly mark and color them with the background of a basic human body picture drawn on an electronic chart system. However, drawing on the monitor with a mouse rather than paper and pen requires a lot of time, and the medical staff consulted the patient and had to draw it in a limited amount of time. Therefore, it was difficult to display detailed information for medical pictures, and there was a high possibility that inaccurate medical pictures would be made.

In addition, medical pictures are likely to be drawn in various shapes depending on the medical staff or body part for the same medical information. Therefore, if you can set the principles for how to express the type of disease or body part, it will help to increase the accuracy and reusability of the information that the healthcare provider wants to convey when sending pictures to other medical institutions or explaining symptoms to patients. Can be.

In addition, various parameters that inform the patient's condition, such as blood tests and hormone levels, will be of great help in understanding the patient if the values can be accurately expressed through attribute values such as color, brightness, and transparency. I can.

In addition, whenever a patient moves to a hospital for treatment, an element of an added medical picture can be automatically added. According to this added picture, it is possible to examine the trend of the treatment process, such as worsening and alleviating patient symptoms, and these results can be shared with each hospital.

Meanwhile, in order to establish medical big data, the governments of each country are in a hurry to improve various technologies and laws. By establishing a big data platform, it is trying to collect data through the linkage between hospitals and health insurance, or between hospitals. In particular, large hospitals are making efforts to standardize the data created in hospitals. Therefore, if standardized medical picture data can be exchanged between medical institutions, it will be of great help in constructing medical big data.

Korean Patent Application Publication No. 2018-0098869

The present invention was devised to solve the above problems. It is an object of the present invention to convert medical information of drawings, which have been previously expressed in text or inaccurately, into quantified and detailed images or drawings. Through this, it is possible to provide an automatic medical picture generation system that avoids the disadvantages of existing data and delivers detailed and accurate information.

The problems to be solved by the present invention are not limited to the technical problems mentioned above, and other technical problems that are not mentioned will be clearly understood by those with expertise in the technical field to which the present invention belongs.

The automatic medical picture generation system according to the present invention,

An information extracting unit 100 for extracting information from at least one of the text data 130 and the medical image 110 created during treatment;

A shape generating unit 200 for generating a basic model 210 and a disease model 220 by shaping a medical picture to represent medical information generated from the information extracted by the information extracting unit 100;

A display unit 600 that visualizes the basic model 210 and the disease model 220 expressed by the shape generation unit 200, but selectively presents differently depending on the viewer; includes,

The basic model 210,

It may be a model representing the whole body or a part of the body, and it may be a model such as organs, blood vessels, brain, etc.

The disease model 220,

It may be a model having a shape such as a tumor or a lump, or a model having a range such as an inflammation, burn, or metastasized part of the cancer.

It can be an organ, or a model that has a shape similar to an organ,

It is prepared to express any one or more of the location, size, shape, color, transparency, and texture of the disease according to the characteristics of medical information,

A position calculation unit 300 for calculating the position of the disease model 220 in the basic model 210 by measuring the distance between the organ and the disease position in the medical image 110; And

The disease model 220 by the size calculation unit 400 for calculating the size of the basic model 210 by using one or more of the length, size, and volume of the lesion on which the disease occurred in the medical image 110 Characterized in that it expresses.

In addition, the automatic medical picture generation method according to the present invention,

An information extraction step (S100) of extracting information from at least one of the text data 130 and the medical image 110 created during treatment;

A shape generation step (S200) of creating a disease model 220 by shaping it into a medical picture to represent medical information made from the information extracted in the information extraction step;

A display step (S600) of visualizing the basic model 210 and the disease model 220 expressed by the shape generation step, but selectively presenting differently according to the viewer (S600); including,

The disease model 220,

Depending on the characteristics of medical information, the disease is prepared to be expressed in a different way at least one of the location, size, shape, color, transparency, and texture.

The shape generation step (S200),

A position calculation step (S300) of calculating the position of the disease model 220 in the basic model 210 by measuring the distance between the organ and the disease position in the medical image 110; And

A size calculation step (S400) of calculating the size in the basic model 210 by measuring the ratio between the size of the organ in the medical image 110 and the size of the diseased lesion (S400); by the disease model 220 It is characterized by expressing.

By means of solving the above problems, the present invention can automatically generate a medical picture, thereby saving time for the medical staff to draw the picture.

In addition, the present invention can save time for medical staff, including doctors, to grasp the patient's condition.

In addition, the present invention enables the creation of detailed and accurate medical pictures.

In addition, the present invention enables the patient to have more understanding of the disease and to actively cooperate with the treatment.

In addition, the present invention can input structured data in a short time, thereby improving work efficiency of medical staff and helping to increase data accuracy.

In addition, the present invention can accurately represent data in numerical form than before.

In addition, the present invention facilitates information sharing among other medical institutions through the sharing of quantified and standardized medical pictures.

1 is a block diagram showing the configuration of an automatic medical picture generation system 1 according to the present invention.
FIG. 2 is an embodiment showing the generation of a disease model 220 in a two-dimensional basic model 210 by receiving text data 130 from an electronic medical record (EMR, 2).
3 is an embodiment showing that the liver disease model 220 can be inserted into the whole body basic model 210 by the shape generating unit 200.
FIG. 4 is an embodiment showing a basic model 210 by generating various disease models 220 in two dimensions and three dimensions.
FIG. 5 shows that, when medical information according to the time and situation of the same disease is mentioned differently over several times during treatment, the reliability application unit 500 selects a weight for the importance or reliability of the disease model 220 It is an example
6 is a medical image 110 after correcting the position of the disease model in the basic model 210 by measuring the distance between a certain part of the body and the location of the lesion in the medical image 110 by the position calculation unit 300. ) Is converted into a medical picture.
7 is an embodiment in which different medical pictures are presented to a doctor and a patient in the display unit 600 of the same extracted medical information for a spine fracture.
8 is an embodiment showing the location and size of a disease by applying an automatic medical picture generation system 1 of the present invention and an automatic medical picture generation method using the same through machine learning.
9 is an embodiment in which a part of the medical image 110 is used as the disease model 220.
10 illustrates an example in which the display unit 600 provides a medical picture by interlocking an electronic medical system screen to a doctor's terminal.
11 shows a round pattern (a), a thin diagonal pattern (b), a thick diagonal pattern (c) and a dotted line pattern (d), a heart pattern (e), and a pattern using letters (f) during the generation of the disease model 220. ), a logo (g), and a pattern (h) using a logo.
12 is a photograph showing an embodiment in which an image directly extracted from a medical image, an anatomical pathology finding, a photograph of a skin disease, etc. can be used as a texture of the disease model 220.
13 is a block diagram showing the configuration of an automatic medical picture generation method according to the present invention.
14 is an embodiment showing that medical information treated at each medical institution is combined into one.
15 is an embodiment showing medical information stored in a disease model through a GUI interface.

The terms used in the present specification will be briefly described, and the present invention will be described in detail.

As for terms used in the present invention, general terms that are currently widely used as possible are selected while considering functions in the present invention, but this may vary according to the intention or precedent of a technician engaged in the relevant field, the emergence of new technologies, and the like. Therefore, the terms used in the present invention should be defined based on the meaning of the term and the overall contents of the present invention, not a simple name of the term.

When a part of the specification is said to “include” a certain component, it means that other components may be further included rather than excluding other components unless otherwise stated.

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings so that those of ordinary skill in the art can easily use the embodiments of the present invention. The present invention may be implemented in various different forms and is not limited to the embodiments described herein.

Specific matters including the problems to be solved, means for solving the problems, and effects of the invention are included in the following examples and drawings. Advantages and features of the present invention, and a method of achieving them will become apparent with reference to the embodiments described below in detail together with the accompanying drawings.

Hereinafter, the present invention will be described in more detail with reference to the accompanying drawings.

As shown in Fig. 1, the present inventors automatic medical picture generation system includes an information extraction unit 100, a shape generation unit 200, a position calculation unit 300, a size calculation unit 400, a reliability application unit 500, It is composed of a display unit 600.

First, the information extracting unit 100 transmits/receives and extracts information from at least one of the text data 130 and the medical image 110 created during treatment. As shown in FIG. 1, the information extracting unit 100 receives data from an electronic medical record (EMR) 2 and a medical institution server 3.

The medical image 110 may be an X-ray, MRI, CT, ultrasound and nuclear medicine image, a positron emission tomography (PET) image, which is a nuclear medicine test including a picture taken in a medical office.

In addition, the medical information source to be extracted may be a clinical document delivered to each medical staff via USB or e-mail, or electronic chart data stored in a hospital. These data may be structured and classified into fields or columns, but may be stored in a simple document form or may be stored as structured data in a database.

On the other hand, when a patient visits the outpatient clinic several times or is hospitalized, medical information is also written several times. The information extracting unit 100 also serves to extract data necessary to create a medical picture from the data written several times. In addition, the information extracting unit 100 can also be used for data received from other medical institutions, data stored in servers outside the hospital, data stored in insurance companies, data stored in personal smartphones or personal medical devices, data transmitted from various medical devices, etc. Data can be extracted.

Here, the electronic medical record (EMR, 2) computerizes and stores at least one medical information of the patient's personal information, medical history, medical records, and admission/discharge records, and responds to a request from a medical institution to which the patient has agreed to provide the information. Accordingly, it may refer to a system that provides patient medical information in the form of an electronic chart to a server of a corresponding medical institution.

As shown in FIG. 2, the text data 130 includes medical terms used in various health records, ie, progress records, surgical records, and discharge summary sheets, in addition to the electronic medical records (EMR, 2). can do. In addition, it is data extracted from medical data displayed in a number of letters or numbers, such as disease name or diagnosis name, symptom, blood test result, reading paper, operation name, nursing record, nursing treatment, consent form, EMG, ECG, etc. It refers to textual data included in the category of medical data.

The text data 130 is not limited to a diagnosis name, and may represent an anatomical part, a procedure name, and a measured blood pressure value. Or, data created by the activities of not only medical personnel, but also rehabilitation therapists, medical assistants, and patients, or various text data indicating the patient's condition such as "critical", "light", "large", and "small" may be included. have. For example, the text data 130 may be letters (alphabets) of countries around the world such as'fat liver','ankle pain', and'heart failure', or'K76.0' which is a number or a combination of letters and numbers. It may be standardized data such as', '61515','N05', and'M51.0' or a medical term code. The standardized medical terminology code refers to data presented by SNOMED-CT, ICD-9, ICD-10, ICD-11, LOINC, CPT, ATC, RxNorm, ICNP, NMDS, etc. for medical concepts. In addition, the test result, such as a hemoglobin level of 10.3 gram/deciliter, may be data expressed as a number.

In addition, the character data 130 may be a free-speech character string whose data structure is not specified separately or is not standardized. In this case, only necessary values can be extracted and used from the character data 130. The information extracting unit 100 is configured to collect information from data scattered in several hospitals or distributed in external servers.

As an example, the information extracting unit 100 selects a menu related to an event, that is, a medical history inquiry, a prescription (prescription) by a user on the EMR treatment screen of a patient, or hospitalization, surgery, examination, etc. When an order is entered or treatment is initiated by selecting a patient from a list of multiple patients, the system may recognize that the medical picture is requested.

The information extracting unit 100 may search the patient's medical record in the electronic medical record (EMR, 2), and retrieve all diseases (diseases) previously occurring in the patient based on the requested time point.

In this case, the information extracting unit 100 may search not only the medical institution currently visited by the patient, but also all medical records (including examination records) of other medical institutions (including examination institutions) visited in the past.

As another example, the information extracting unit 100 excludes diseases of low severity such as cold, abdominal pain, and headache from the patient's medical record, and treats diseases of high severity such as thyroid disease, heart disease, and lung disease. You can select and identify.

In addition, the information extracting unit 100 may select and identify a specific disease (eg,'cerebral hemorrhage') from the patient's medical record, and diseases that are related to each other (eg,'diabetes' and'diabetic kidney Diseases', etc.) can be selected and identified, diseases that have occurred in a specific body part (eg,'lungs','brain') designated by the user as an area of interest, or'after the time of delivery to the emergency room and administration of drugs' As such, it is also possible to select and identify diseases that occurred at a specific time point.

In addition, when identifying medical data from the patient's medical record, the information extracting unit 100 includes the time of occurrence of the disease, the time of cure, the site of occurrence, size and shape, medications and surgery, hospitalization, various test results, nursing records, etc. Data on the course of treatment can also be identified.

For example, referring to FIG. 10, the information extracting unit 100 may search for a patient's medical record from the electronic medical record (EMR, 2) at an arbitrary point in time to identify past diseases that have occurred in this patient. have.

In addition, the information extraction unit 100 retrieves the patient's medical record from the electronic medical record (EMR, 2) when patient information is registered, when a patient's outpatient treatment is requested, or at a specified time every day. It is also possible to identify past diseases that have occurred in patients.

In addition, when the'prescription' menu is selected on the electronic medical record (EMR, 2) screen output from the terminal of the doctor treating the patient, the information extracting unit 100 You can also search medical records and identify past illnesses that have occurred in this patient.

Meanwhile, the medical image 110 may be an X-ray, MRI, CT, ultrasound, nuclear medicine, or PET (positron emission tomography), or a general camera photograph of the skin or face in order to check the patient's condition in a dermatology or plastic surgery clinic ( camera picture) or an image of a slide made for pathological biopsy.

The text data 130 and the medical image 110 become a basic source of a medical picture. However, the text data 130 alone has limitations in expressing the exact location of the symptoms and the range of the lesion, and the user may need to know which medical image is the source of the corresponding medical picture. Therefore, the text data 130 and the medical image 110 used as sources are stored together with the medical picture created by the user, and can be viewed if the user desires.

Next, the shape generation unit 200 converts the type and state of the medical data extracted from the information extraction unit 100 into a suitable symbol and a picture symbolizing a disease, and the basic model 210 and the disease model 220 Is created. The basic model 210 and the disease model 220 may be 2D or 3D data and may form an organ or part of an organ.

The basic model 210 is a model showing part or all of the body. As shown in Figs. 2 and 3, the basic model 210 may be the whole body of the human body, or only a partial system of the human body such as the digestive system or the respiratory system, or only a certain area of the body such as the left leg or the head. It may be expressing.

The disease model 220 can be created by taking a boundary of a shape where a disease has occurred, such as'lung cancer', and the appearance design, size, location, shape, and color according to the characteristics of medical information and the patient's condition. , Transparency, brightness, and texture may be a model in which one or more of these are different.

In addition, the disease model 220 may be a model representing the type and state of a disease, or medical information. As shown in FIG. 4, not only the organs that can confirm the disease, but also the procedure or surgery site, the nursing observation site, the nursing intervention site, the rehabilitation treatment area, the biopsy area, the imaging test, the organs and the patient's symptoms This indicates the location of the device, the location of interest to the medical staff, and so on.

In addition, the disease model 220 can be extracted from the shape generating unit 200 and can be directly added or drawn by a user to be imaged. The disease model 220 is a place where a patient or a medical staff directly draws the corresponding information, so that a spot with a spot, an itchy spot, a place where the blood pressure was measured, an organ that has undergone surgery, a body part that has received various nursing care, a body part that has received rehabilitation treatment, and an injection. You can express the right spot, etc. For example, when a patient has fatty liver, the medical staff can directly draw the shape of the fatty liver against the background of the basic model 210 without recording it in the electronic medical record (EMR, 2). Alternatively, a medical staff or a patient may select any one of the disease models 220 stored in advance to express the disease state.

The disease model 220 is displayed as a two-dimensional or three-dimensional image, and is provided to adjust one or more of a position, size, shape, color, transparency, and texture.

The disease model 220 may be implemented as an image representing the whole, part, and cross-section of the human body (eg, an image of a human body cut through CT or MRI). In addition, as shown in FIG. 9, the disease model 220 may be configured by editing one or more medical images taken of a patient.

When generating the disease model 220 in the shape generating unit 200, the name of the disease, the severity of the disease, the frequency, the degree of acute/chronic and malignancies, various clinical tests, blood test values, whether or not the treatment is terminated, According to medical information such as the frequency of the disease, the image may be generated by varying one or more of the location, shape, size, color, brightness or transparency, and pattern. In one embodiment, when a patient's tibialis anterior is paralyzed, the color of the disease model 220 is selected or combined with one or more colors of red, green, and blue to reduce muscle strength (muscle strength) can be expressed.

For example, if the muscle strength of the relevant muscle is 20% of normal and the color expressing the muscle strength is red, the maximum value of the red channel is 255 (maximum value of the red channel) X 0.2 ( Muscle strength) = 51 (the degree of red color appearing on the display) can be expressed. On the other hand, when expressing the function of the kidney, the value of eGFR (estimated glomerular filtration rate), which is one of the blood tests, can be expressed by linking the G value representing the green value. When eGFR is 100, 255 (maximum value of green channel) X 1 (eGFR) = 255 (the degree of green color appearing on the display), when eGFR is 50, 255 (maximum value of green channel) X 0.5 (eGFR) = 255/2 (The degree of green color appearing on the display), when eGFR is 0, the G value can be displayed as 0. That is, the patient's condition can be indicated by changing one or more attributes of an image such as color, brightness, transparency, and pattern by a function that takes the result of a blood test as a factor. Color can be determined not only by blood tests, but by any function that takes medical data as a factor. For example, if a tumor in the lungs is malignant, it may be marked in red, if it is benign, it may be marked in green, a disease that has been treated may be marked in gray, and a disease under treatment may be marked in red.

In addition, the disease model 220 may be completed by adding a texture in addition to general image features such as color, brightness, and transparency. That is, the disease model 220 uses the texture to express the type of disease, acute/chronic degree, severity, malignancy, frequency, and whether or not treatment has ended. For example, using the patterns shown in Fig. 11, stenosis can be expressed as a round pattern (a), for squamous cell carcinoma, it can be expressed as a thin diagonal pattern (b), and hemangioma ) Can be expressed as a thick diagonal pattern (c), and muscle paralysis can be expressed as a dotted diagonal pattern (d). The patterns presented here are just a few examples, and the texture is not limited thereto and may be expressed in various ways. In addition, a heart shape (e) can be used to express a human-drawn figure or heart disease, and a letter can be added to a part of the pattern (f), and ischemic heart disease can be expressed. Specially designed disease-expressing logos (g), icons, alphabets, numbers, or a combination of these (h) can create textures for a wide variety of disease models.

In addition, an image extracted from the medical image 110, a photograph or image showing an anatomical pathology finding, a skin disease photograph, or the like, is provided to be directly converted into the texture. That is, all medical images may be applied to the disease model 220 as a model or texture. For example, the microscopic pathological tissue findings in Fig. 12(a) can be used as the texture, and as shown in Fig. 9, only the outline of the cerebral hemorrhage seen in the cerebral hemorrhage can be selected along the hematoma shape and used as the outline of the disease model. . Or it may be a compilation of typical pathological findings from a pathology textbook, or a compilation of a photograph of the patient. 12(b) is an image taken directly of a patient's skin lesion, and the image may be used as the texture on a corresponding region. 12(c) is a part of an image taken by MRI. In addition, FIG. 9 is a diagram showing a case in which a part of a CT image is taken to create a texture in a disease model of a patient with cerebral hemorrhage. By doing so, more various clinical data can be imaged.

The basic model 210 and the disease model 220 may be formed in two or three dimensions, and the disease model 220 is expressed in one or more of the basic model 210 to express the condition of patients with various diseases. I can.

In addition, the medical picture generation system is capable of machine learning. This machine learning system can predict medical pictures when one or more of text data or medical images are input. In addition, it may be trained to predict one or more of the location, shape, size, color, pattern, transparency, and pattern of the disease model of the disease model 220 in the medical picture.

As shown in FIG. 3, the shape generation unit 200 may generate a disease model 220 representing a liver with the basic model 210. In addition, as shown in FIG. 4, in order to express various medical information, the shape generating unit 200 may generate a medical picture by combining the various disease models 220 at a specific position of the basic model 210. .

Meanwhile, the function of the shape generating unit 200 may be performed by machine learning, and the disease model 220 may be trained to be made according to the shape of an actual lesion. As shown in FIG. 8, when the patient's tumor has a round shape, a round-shaped disease model in a round shape can be generated (A), and in a case in a star shape, a star-shaped disease model can be generated (B).

Next, the location calculation unit 300 calculates the location of the disease model 220 by measuring the distance between the location of the organ and the disease lesion in the medical image 110 and calculating the relative distance in the medical picture. . The location of the basic model 210 can also be confirmed by indicating a left sign value in the medical image 110.

As shown in FIG. 6, the location calculation unit 300 may measure a distance between a part of an organ in an X-ray image of a patient and a disease location. In the basic model 210, the position of the disease model 220 may be calculated by converting it to a relative distance. In the medical image 110 of FIG. 6 of the location calculation unit 300, the apex of the heart is set as a reference point (0, 0), and after the coordinates of the disease model 220 are extracted, the coordinates of the disease location ( x, y) is extracted, and the horizontal value (x1) of the distance between the edge of the right lung and the distance and the vertical value (y1) of the distance from the left heart attachment are calculated. Next, after measuring the size of the medical image 110 and the relative image size of the basic model 210, the same reference point is also set in the basic model 210, and a relative horizontal value (x2) and a relative vertical value (y1) are measured. ) Is calculated and the location value of the disease model 220 is generated as (x', y'). At this time, the reference point may be variously selected depending on the structure, such as a lung apex or a costophrenic angle. The relative length using the distance measured in the image can be calculated by various functions using the distance. For example, when obtaining the distance between two or more structures, the distance may differ depending on the case of the measurement. In this case, it is also possible to select the location of the disease model by adopting the average of several measurements.

Next, the size calculation unit 400 uses one or more of the length, size, and volume of the lesion where the disease occurs in the medical image 110 to determine the relative size of the disease model 220 in the basic model 210. Calculate the size. For example, the size calculation unit 400 may calculate the relative size of the disease model in the basic model 210 by measuring a ratio between the size of an organ and the size of a diseased lesion in the medical image 110. In addition, at this time, there may be two or more organs selected to compare the size of the lesion, and when the ratios measured in two or more selected organs are different, the size of the disease model 220 may be determined by adopting an average of the ratios. Meanwhile, the position calculation unit 300 and the size calculation unit 400 can be performed by a machine learning system, similar to the shape generation unit 200.

After learning by performing labeling of the lesion area of the disease image in the medical image, the medical picture can be automatically generated by machine learning. Through this, at least one of the location, size, and color of the disease model is learned so that the size and location of the disease model can be predicted when image data are input.

Next, the reliability application unit 500 sets the reliability of the disease model. Reliability is one of the attribute values of the disease model shown in the medical picture, for example, the accuracy of the disease model as a value from 0 to 1. The reasons for setting the reliability are as follows.

The disease model 220 drawn on the basic model 210 may be medical data such as text, images, or images input to the system, and may be automatically extracted from medical pictures drawn by the user. However, since the picture is extracted by machine learning, there is a possibility that the picture is different from the actual picture, and since there is a possibility that the information on the location or size of the disease model 220 may not be accurate, it is necessary to set the reliability.

In addition, the source (SOURCE) that is the basis of the disease model 220 extracted by machine learning is also stored. At this time, the stored source may be a medical code, a disease name, an X-RAY picture, a CT image, a picture drawn by the user through EMR, etc., and is not limited thereto, but all the basis for drawing a disease model by a machine. It could be medical information.

The stored reliability information or source can be displayed when a user clicks a picture using a GUI interface or the like.

In addition, reliability can be applied to the disease model 220 through this process. When a disease occurs repeatedly in the same region in the same patient, a similar disease model 220 has been formed several times, so that reliability can be applied.

For example, if a patient receives treatment at several hospitals due to the same discomfort and data is accumulated, multiple medical models will be created based on the data at each hospital. In this case, the disease model 220 input by each hospital may have a difference in size or shape, and the disease models 220 may be overlapped to apply a weight. As another example, when a patient visits the hospital for appendicitis, the location and size of abdominal pain may change over time even during hospitalization. At this time, the entire abdomen hurts, only the lower abdomen hurts, and later, the process of localizing to the area where the appendix is located may be seen. As described above, when a patient needs various disease models 220 for the same injured disease, it is necessary to display changes according to time points. In this case, the symptoms may be expressed by multiple disease models 220, or the disease model 220 may be put into one, but a weight of higher reliability may be placed in the region that overlaps the most. In addition, for disease information that is determined to be more important than the past state, the reliability application unit 500 places the disease model 220 on the basic model 210 according to the occurrence and treatment process of the disease. It can be expressed by changing it.

The reliability application unit 500 sets the reliability of the disease model 220 according to the clinical importance of each area when an overlapped area is formed on the disease model 220. It is desirable to be able to set the reliability so that one or more of the attribute values of the disease model, such as brightness, color, transparency, and texture, can be expressed according to the importance or reliability. For example, the importance of the lightest area is expressed as 30, and the importance of the darkest area is expressed as 70, so that information on the importance of each area is visually provided to the medical staff.

The reliability application unit 500 may apply the reliability according to the time point of occurrence of the disease. For example, the more recent findings may be of greater importance. Or, conversely, the first symptom, the more important the symptom. In this case, it is preferable to generate the brightness or saturation of the disease model 220 differently in stages according to the importance. For example, as shown in FIG. 5, when the location and size of symptoms change in the order of (a) to (c) over time (when the occurrence of the disease is generated), the location of the lesions overlap. By setting the brightness or saturation of the disease model 220 to be darker in stages according to the degree of overlap, the medical personnel can confirm the position considered to be more clinically important.

In addition, the reliability application unit 500 may indicate a change in the time point at which the disease progresses on the basic model 210 through a color or a pattern, as shown in FIG. 5.

Hereinafter, a medical picture system according to the present invention will be described in more detail through an embodiment.

14 is an embodiment showing that medical information treated at each medical institution can be combined into a single medical picture. For example, let's say that one patient had a stroke and was treated at Hospital A. Through this treatment, the stroke disease model 221 created in Hospital A is stored in the medical institution server 3-1 of Hospital A. This can be checked on the patient's personal terminal and can be shared with medical staff at other hospitals.

Over time, this patient develops nephritis and is treated at Hospital B. At this time, you can receive a record of being treated at Hospital A at Hospital B. In the existing hospital A, the patient's stroke was partially treated, but if the aftereffects remain, the brain part of the stroke disease model 221 may be displayed in yellow, and nephritis is a state that continues to be treated, so the nephritis disease model (222). ) Can be displayed in red light. The nephritis disease model 222 is stored in the medical institution server 3-2 of hospital B.

The nephritis disease model 222 extracted from hospital B may represent a process that changes according to the trend of symptoms. At this time, it is also possible to adjust the color characteristics by linking the blood test results. For example, the color of the disease model is determined by determining the rgb transparency value by adding an algorithm that determines the color to the value converted by a function that takes the level of creatinine in the blood, which indicates kidney function, as a factor.

Again, this patient has hepatitis and visits C hospital, and C hospital also receives patient's medical records from previous hospitals A and B. The patient receives treatment at C hospital, and in this process, a hepatitis disease model 223 is created, and the hepatitis disease model 223 is stored in the medical institution server 3-3 of C hospital or a personal terminal.

Thereafter, the patient is hospitalized at D hospital for treatment of a fracture of the left leg and then discharged. Hospital D confirms that the nephritis of the patient who was treated at Hospital B has been cured and updates the information. As a result, the nephritis disease model 222, which was marked in red for "treating" in the medical picture of the full-body basic model 210 of the terminal, disappears. Meanwhile, the left leg fracture disease model 224 that needs continuous observation may be displayed in green or the like.

As in the above example, the disease model and additional information related thereto, namely, the name of the disease such as stroke and hepatitis, the course of treatment, the drug being treated, the blood test related to the disease, the results of various function tests, etc., are stored and can be observed at once. Medical models may be collected and displayed in one or more medical institutions or servers existing in a network. By doing this, each hospital that has shared this can update the details of the treatment that occurred at the hospital or the trend of previous symptoms, so that the treatment or worsening of the disease can be examined.

In addition, this method of collecting data from multiple hospitals can also be applied to the stage in which medical information is extracted before the medical picture is created. It is a method of generating pictures by extracting the information necessary to create medical pictures from one or more hospitals. In addition, such medical information may exist in the network or may be extracted from one or more servers to generate a picture. Alternatively, such information may be extracted from one or more personal devices in which data owned by the patient is stored, and a medical picture may be created.

Next, the display unit 600 visualizes the basic model 210 and the disease model 220 expressed through the reliability application unit 500, but may selectively present the model according to the viewer. This has the advantage of being able to differentiate the expertise of the transmitted information according to the degree of understanding of medical information of the person viewing the basic model 210 and the disease model 220. For example, a specific and direct image may be presented to a professional medical practitioner, and a patient may be presented with a simple yet relatively easy-to-understand disease and symptoms. For example, as shown in FIG. 7, a spine fracture model in the display unit 600 may be presented differently to a doctor and a patient, respectively. That is, a bone that provides more anatomical information to the doctor was selected as the basic model 210, and the location of the fracture was also specifically expressed in the vertebral segment, whereas for the patient, the human back view was selected as the basic model 210. It was selected and the location of the fracture was also briefly expressed against the background of the skin.

The display unit 600 is

In addition, the viewpoint of the medical picture implemented by the display unit 600 may be switched through manipulation of the GUI interface according to the occurrence time of the disease. For example, depending on the operation of the GUI, you can check the disease state 1 or 5 years ago, and you can find the time point of occurrence of a specific disease by operation of the GUI. Here, the GUI control provides the user with a means to select the viewpoint of the medical picture.

The display unit 600 may be output in the form of at least one of GUI elements such as a window window, an icon, and a scroll bar, and may include a human body image and a GUI menu (for example, a “scroll bar”). In addition, the display unit 600 may further include a clock, a calendar, a date, a graph, coordinates, a progress bar, a viewpoint display column, and the like to indicate the time point indicated by the GUI control or the flow of time.

Here, the GUI control is a part for recognizing a user's operation on the patient status inquiry screen, and is configured to be operated by inputting one or more combinations of, for example, a mouse, gravity, vibration, touch sensor, microphone, and camera. I can. Additionally, the display unit 600 may inform the user by expressing the passage of time or time point indicated by the GUI control in the form of a graph on the screen using length, angle, volume, number, or text.

For example, the GUI control may be implemented in the form of a scroll bar that switches the viewpoint of a medical picture, and in addition, various buttons (play button, back button, fast forward button, setting button, radio button), check It may be implemented as a box, a drop-down menu, a slider, a progress bar, a screen area that recognizes at least one of a touch, a click, and a cursor movement.

In addition, when the GUI interface is executed at the disease occurrence location in the basic model 210, the display unit 600 displays a speech balloon in association with the disease occurrence location, and the disease name, occurrence time, severity, and It is also possible to visualize information about the current state in text form. Here, the GUI event is an input event generated by a user clicking a mouse on the patient status inquiry screen, mouse over, mouse wheel, mouse leaving, GUI focusing, keyboard input, finger touch, sound recognition, human body motion recognition, etc. I can.

For example, as shown in FIG. 15, the display unit 600 outputs a speech balloon in association with an area where a user touches or a mouse cursor is located in the human body image, and a disease occurring in the corresponding area through the speech balloon You can visualize detailed information about the text in the form of text.

For example, when the GUI interface such as mouse focusing, clicking, or touching occurs on the human body image, the display unit 600 displays the EMR (2) treatment screen driven by the terminal of the doctor requesting the display unit 600. You can switch. As such, the display unit 600 may provide a function of switching the EMR (2) treatment screen through the displayed screen without providing a separate context menu.

As shown in Fig. 1, the medical picture is printed on a part of the examination screen of the electronic medical record (EMR, 2) in the terminal of the doctor who treats the patient via the medical institution server 3 or the electronic medical record (EMR) , 2) It can be output by replacing the treatment screen.

Hereinafter, as shown in FIG. 13, a method of generating an automatic medical picture using the automatic medical picture generating system according to the present invention will be described.

First, the first step (S100) is an information extraction step, and extracts information from one or more of the text data 130 created during treatment and the medical image 110. In the information extraction step, as shown in FIG. 13, data is received from the electronic medical record (EMR, 2) and the medical institution server (3).

The medical image 110 is an image extracted from positron emission tomography (PET), which is a nuclear medicine test including an X-ray, MRI, CT, ultrasound and nuclear medicine image, and a picture taken in a medical office. In addition, the text data 130 refers to text data representing a patient's condition.

Next, the second step (S200) is a shape generation step, in which the information extracted in the information extraction step is shaped into symbols or images to express the type and state of the disease, and the basic model 210 and the disease model 220 Is created. The basic model 210 and the disease model 220 are generated as a two-dimensional or three-dimensional image. The basic model 210 may be a whole body, a part of a body, an organ, a blood vessel, etc., and the disease model 220 is provided to be selected by varying one or more of a location, size, shape, color, transparency, and texture. do.

Next, the third step (S300) is a position calculation step, by measuring the distance between the organ and the disease position in the medical image 110 to calculate the position of the disease model 220 in the basic model 210. The position calculation in the basic model 210 may be expressed or calculated as a two-dimensional or three-dimensional left sign value in the medical image 110.

Next, the fourth step (S400) is a size calculation step, in which the relative size in the basic model 210 is calculated using at least one of the length, size, and volume of the diseased lesion in the medical image 110. .

Next, the fifth step (S500) is a reliability application step. When the same disease is mentioned in the medical record several times during treatment and the disease model 220 is formed several times, the weight according to the reliability or frequency of occurrence of each disease model Apply. In the reliability application step, a weight is set according to the reliability of the disease model 220 and the number of times the models are overlapped. In the reliability application step, the brightness or saturation of the disease model 220 is differently generated in stages according to the time when the disease occurs.

In addition, in the reliability application step, the color of the disease occurrence location in the basic model 210 is changed, or the user adjusts one or more of the size, shape, brightness, transparency, and texture of the disease model to prevent disease occurrence and treatment. You can visualize the changes in the organs that follow.

In addition, in the step of applying the reliability, a sequence number (number, number) is assigned to a plurality of diseases that have occurred in the patient from the past to the present time according to the occurrence time of the disease, and the assigned sequence number is the occurrence of the disease in the basic model 210. By sequentially visualizing the location according to the time of occurrence, it is possible to guide the occurrence, location and order of occurrence of each disease.

Next, the sixth step (S600) is a display step, which visualizes the basic model 210 and the disease model 220 expressed through the shape generation step and the reliability application step, but may be selectively presented differently depending on the viewer. I can. The point of view of the medical picture implemented in the display step is switched according to the point of occurrence of the disease by manipulation such as GUI control. The size or direction of the disease model 220 implemented in the display step may be changed by manipulation of GUI control or the like according to the occurrence time of the disease.

In addition, the automatic medical picture generation system 1 of the present invention and an automatic medical picture generation method using the same can be implemented by applying to machine learning (hereinafter referred to as machine learning). This includes algorithms that can be classified by conventional machine learning, such as a support vector machine, a convolutional neural network, a generative adversarial neural network, and a generative model. The machine learning system can perform learning and prediction using at least one of the text data 130 and the medical image 110 as input data, and configure a system for predicting a medical picture. The machine learning system may learn a disease model and predict one or more of shape, brightness, transparency, location and texture, and size.

For example, as shown in FIG. 8, one or more of the basic model 210 and the disease model 220 may be generated through machine learning.

In addition, the machine learning-based automatic medical picture generation system 1 according to the present invention has the same configuration as the automatic medical picture generation system 1. When data is input to the information extraction unit 100 that extracts information from one or more of the text data 130 and medical image 110 created during treatment, the disease model 220 is predicted in the basic model 210 So that it can be expressed.

The disease model 220 is characterized in predicting one or more of a disease location, size, shape, color, transparency, texture, and size according to the characteristics of medical information, so that machine learning can perform one or more roles. Can be made.

By means of solving the above problems, the present invention can automatically generate a medical picture, thereby saving time for a doctor to draw a picture.

In addition, according to the present invention, it is possible to save time required for medical staff, including doctors, to determine a patient's condition than a text medical record.

In addition, the present invention enables the creation of detailed and accurate medical pictures.

In addition, the present invention enables the patient to have more understanding of the disease and can actively cooperate in treatment.

In addition, the present invention can input structured data in a short time, thereby improving work efficiency of medical staff and helping to increase data accuracy.

In addition, the present invention can be integrated into a medical picture by quantifying data in numerical form.

In addition, according to the present invention, the user checks and corrects the automatically generated medical image to increase the reliability of the medical picture.

In addition, the present invention increases the reliability of medical pictures representing patient symptoms by expressing weights in the case of symptoms treated several times.

In addition, the present invention increases the reliability of patient symptoms by allowing such disease images to be corrected and supplemented in various hospitals.

In addition, the present invention facilitates communication between hospitals about a patient's disease by allowing hospitals or medical staff to share such disease images.

As described above, it will be appreciated that the above-described technical configuration of the present invention can be implemented in other specific forms without changing the technical spirit or essential features of the present invention by those skilled in the art.

It should be understood that the embodiments described above are illustrative and non-limiting in all respects, and the scope of the present invention is indicated by the claims to be described later rather than the above-described description, and the meaning and scope of the claims and the All changes or modifications derived from the equivalent concept should be construed as being included in the scope of the present invention.

1. Automatic medical picture creation system
100. Information Extraction Department
110. Medical Imaging
130. Character data
200. Shape generator
210. Basic Model
220. Disease Model
300. Position calculator
400. Size calculation section
500. Reliability Application Department
600. Display
2. Electronic Medical Record (EMR)
3. Medical institution server
S100. Information extraction step of extracting information from text data 130 and medical image 110 created during treatment
S200. A shape generation step of creating a basic model 210 and a disease model 220 by shaping the information extracted in the information extraction step into symbols and images to express the type and condition of the disease.
S300. Position calculation step of calculating the position of the disease model 220 in the basic model 210 by measuring the distance between the organ and the disease in the medical image 110
S400. A size calculation step of calculating the relative size in the basic model 210 by measuring at least one of the length, size, and volume of the lesion where the disease has occurred in the medical image 110
S500. When the disease model 220 is formed multiple times because the same disease is mentioned several times during treatment, the disease model 220 is selected by selecting the location of the disease model 220 in the most overlapped area, and automatically extracted by the machine. Reliability application step in which the user checks and corrects the reliability of) and imposes reliability
S600. Display step of visualizing the basic model 210 and disease model 220 expressed by the reliability application step, but selectively presenting differently depending on the viewer

Claims (19)

An information extracting unit 100 for extracting information from at least one of the text data 130 and the medical image 110 created during treatment;
A shape generating unit 200 for generating a basic model 210 and a disease model 220 by shaping a medical picture to represent medical information generated from the information extracted by the information extracting unit 100;
A display unit 600 that visualizes the basic model 210 and the disease model 220 expressed by the shape generation unit 200, but selectively presents differently depending on the viewer; includes,
The basic model 210,
It may be a model representing the whole body or a part of the body, and it may be a model such as organs, blood vessels, brain, etc.
The disease model 220,
It may be a model having a shape such as a tumor or a lump, or a model having a range such as an inflammation, burn, or metastasized part of the cancer.
It can be an organ, or a model that has a shape similar to an organ,
It is prepared to express any one or more of the location, size, shape, color, transparency, and texture of the disease according to the characteristics of medical information,
A position calculation unit 300 for calculating the position of the disease model 220 in the basic model 210 by measuring the distance between the organ and the disease position in the medical image 110; And
The disease model 220 by the size calculation unit 400 for calculating the size of the basic model 210 by using one or more of the length, size, and volume of the lesion on which the disease occurred in the medical image 110 Automatic medical picture generation system, characterized in that to express.
The method of claim 1,
Automatic medical picture generation system, characterized in that it further comprises a; reliability application unit (500) for applying a weight when the disease model 220 is formed multiple times because the same disease is mentioned several times during treatment.
The method of claim 2,
The reliability application unit 500,
Automatic medical picture generation system, characterized in that a weight for the importance or reliability of a disease is set for the disease model 220.
The method of claim 1,
The position calculation in the basic model 210 in the position calculation unit 300,
An automatic medical picture generation system that calculates the location of the disease model 220 by measuring the ratio of the distance between the location of the organ and the disease in the medical image 110.
The method of claim 1,
The disease model 220 is
A medical picture creation system in which one or more of the appearance design, size, location, shape, color, transparency, brightness, and texture change depending on the characteristics of medical information and the patient's condition.
The method of claim 1,
The texture of the disease model 220 is
An automatic medical picture generation system, characterized in that a part of the actual patient's medical image 110 is used as it is or edited.
The method of claim 1,
The position calculation in the basic model 210 in the position calculation unit 300,
Automatic medical picture generation system, characterized in that represented by the left sign value in the medical image (110).
The method of claim 1,
In the size calculation unit 400, the disease model 220,
In the medical image 110, a size calculation unit 400 for calculating a relative size in the basic model 210 by measuring the ratio of at least one of the length, size, and volume of a lesion with a certain organ and a disease. Automatic medical picture creation system.
The method of claim 2,
The reliability application unit 500,
An automatic medical picture generation system, characterized in that the brightness or saturation of the disease model 220 is differently generated in stages according to the occurrence time of the disease.
The method of claim 1,
By operating the GUI control according to the time of occurrence of the disease,
Automatic medical picture generation system, characterized in that the size or direction of the disease model 220 implemented in the display unit 600 is changed.
An information extraction step of extracting information from at least one of the text data 130 and the medical image 110 created during treatment;
A shape creation step of creating a basic model 210 and a disease model 220 by shaping a medical picture to represent medical information created from the information extracted in the information extraction step;
A display step of visualizing the basic model 210 and the disease model 220 expressed by the shape generation step, but selectively presenting differently depending on the viewer; including,
The disease model 220,
Depending on the characteristics of medical information, the disease is prepared to be expressed in a different way at least one of the location, size, shape, color, transparency, and texture.
The shape generation step,
A position calculation step of calculating the position of the disease model 220 in the basic model 210 by measuring the distance between the organ and the disease position in the medical image 110; And
Representing the disease model 220 by a size calculation step of calculating the size in the basic model 210 by measuring the ratio between the size of the organ in the medical image 110 and the size of the lesion where the disease has occurred. Automatic medical picture creation method characterized by.
The method of claim 11,
The method of generating an automatic medical picture, further comprising: a reliability application step of applying a weight when the disease model 220 is formed multiple times by the model combination step because the same disease is mentioned multiple times during treatment.
The method of claim 12,
The reliability application step,
Automatic medical picture generation method, characterized in that a weight for the importance or reliability of a disease is set for the disease model 220.
The method of claim 11,
In the position calculation step, the position calculation in the basic model 210,
An automatic medical picture generation system that calculates the location of the disease model 220 by measuring the ratio of the distance between the location of the organ and the disease in the medical image 110.
The method of claim 11,
The texture of the disease model 220 is
An automatic medical picture generation system, characterized in that a part of the actual patient's medical image 110 is used as it is or edited.
The method of claim 11,
In the position calculation step, the position calculation in the basic model 210,
Automatic medical picture generation system, characterized in that represented by the left sign value in the medical image (110).
The method of claim 11,
In the size calculation step, the disease model 220,
An automatic medical picture generation system, characterized in that the relative size in the basic model 210 is calculated by measuring at least one ratio of length, size, and volume of a lesion with an organ and a disease in the medical image 110.
The method of claim 12,
The reliability application step,
An automatic medical picture generation system, characterized in that the brightness or saturation of the disease model 220 is differently generated in stages according to the occurrence time of the disease.
The method of claim 12,
By operating the GUI control according to the time of occurrence of the disease,
Automatic medical picture generation method, characterized in that switching the viewpoint of the medical picture implemented in the display step.

Images