KR20210050424A - Deep learning computer vision-based emergency patient shock signs discovery system - Google Patents

Abstract

Disclosed are a method, a device, and a computer program for detecting event occurrence through image analysis based on artificial intelligence. According to various embodiments of the present invention, in the method performed by a computing device, the method for detecting event occurrence through image analysis based on artificial intelligence comprises the steps of: collecting visualized biometric information data; using the visualized biometric information data to generate a biometric information image; analyzing the biometric information image; and detecting event occurrence for a user in accordance with an analysis result of the biometric information image.

Description

Deep Learning Computer Vision-based Emergency Patient Shock Signs Discovery System {DEEP LEARNING COMPUTER VISION-BASED EMERGENCY PATIENT SHOCK SIGNS DISCOVERY SYSTEM}

Various embodiments of the present invention relate to a deep learning computer vision-based emergency patient shock symptom detection system.

The intensive care unit has a great influence on the survival and death of patients, and the quality and efficiency of treatment, such as the lack of dedicated intensive care personnel, hospital/regional interdifferentiation, and high mortality during patient transfer, even in important health care institutions that account for 25% of domestic medical expenses. It is a very lagging situation in.

According to an article in the Hankook Ilbo on May 22, 2018, according to the '2014 (1st) intensive care unit adequacy evaluation result' released by the Health Insurance Review and Assessment Service in 2016, average beds in the intensive care unit per person dedicated to the intensive care unit in Korea The number of hospitals is as high as 44.7 beds, 40.4 beds in a general hospital and 48.9 beds in a general hospital. As the average number of beds per person of a dedicated specialist is high, there is a problem that it is difficult for patients admitted to the intensive care unit to even see a specialist's face.

As such, deaths due to not being able to detect sepsis at an early stage are also a successive reality, and according to the Simpyeongwon survey, the average mortality rate of adult patients admitted to the intensive care unit in Korea is 16.9%, 14.3% for senior hospitals and 17.4% for general hospitals.

The intensive care unit is for patients with very big problems such as breathing and heartbeat, which are essential functions for life maintenance, and is a place where intensive care is received 24 hours a day, 7 days a week, and changes in the state of intensive care patients that can change at any time. In order not to miss, the patient's biometric data must be measured and analyzed in real time.

However, there is a problem in that it is difficult to extract valuable information from large-scale, complex data, as the data generated by each medical device has many types and formats, and the amount of data generated is vast.

In addition, in the case of anaphylaxis shock, among various shocks that can occur in emergency patients, an allergic reaction occurs rapidly throughout the body, contracting the muscles around the bronchi, causing breathing difficulties, loosening peripheral blood vessels, and sharply lowering blood pressure. It is one of the very dangerous shocks that can cause you to lose weight and also put a strain on your heart, stopping the heart, and leading to death within minutes. Therefore, there is a need to develop a system capable of accurately predicting such a shock in advance.

In addition, attention is paid to the severity of false alarms of patient monitors occurring in emergency and hospital sites. The false-positive alarm sound, which refers to an alarm sound that occurs even though data indicating an exact physiological risk, has been a serious problem in responding to emergency patients for a long time in emergency and hospital sites around the world using modern medicine. Such false-positive alarm sounds can result in lowering the alertness of the medical staff to the alarm sounds, and can occur even in serious situations requiring immediate medical attention, which can be a major problem for patient safety. Therefore, there is a need to develop a technology capable of accurately detecting such abnormal symptoms in advance.

Korean Patent Registration No. 10-1070389 (2011.09.28)

The problem to be solved by the present invention is an artificial intelligence-based image that provides real-time information on changes in the patient's condition by analyzing a vast amount of data that measures patient's biometric information and detecting various events and events occurring in the patient. It is to provide a method, an apparatus, and a computer program for detecting event occurrence through analysis.

The problems to be solved by the present invention are not limited to the problems mentioned above, and other problems that are not mentioned will be clearly understood by those skilled in the art from the following description.

The method for detecting event occurrence through image analysis based on artificial intelligence according to an embodiment of the present invention for solving the above-described problem is a method performed by a computing device, comprising the steps of: collecting visualized biometric information data, the Generating a biometric information image using the visualized biometric information data, analyzing the biometric information image, and detecting the occurrence of an event for the user according to the analysis result of the biometric information image.

An apparatus for detecting event occurrence through image analysis based on artificial intelligence according to another embodiment of the present invention for solving the above-described problem includes a memory for storing one or more instructions and a processor for executing the one or more instructions stored in the memory. The processor may include, by executing the one or more instructions, collecting visualized biometric information data, generating a biometric information image using the visualized biometric information data, and analyzing the biometric information image. The method of detecting the occurrence of an event through image analysis based on artificial intelligence including the step of detecting the occurrence of an event for the user according to the step and the analysis result of the biometric information image may be performed.

The computer program for detecting event occurrence through image analysis based on artificial intelligence according to another embodiment of the present invention for solving the above-described problems is combined with a computer that is hardware to collect visualized biometric information data, the visualization Generating a biometric information image using the biometric information data, analyzing the biometric information image, and detecting the occurrence of an event for the user according to the analysis result of the biometric information image. It may be stored in a computer-readable recording medium so that the event occurrence detection method through image analysis can be performed.

Other specific details of the present invention are included in the detailed description and drawings.

The problem to be solved by the present invention has the advantage of providing real-time information on a change in a patient's state by analyzing a vast amount of data measuring biometric information of a patient and detecting various events and events occurring in the patient.

In addition, in order to accurately predict the patient's risk signs, the patient's condition is monitored in real time 24 hours a day, 365 days a year, and artificial intelligence-based prediction is performed. When abnormal signs appear, an alarm is immediately sent to the nurse in charge to respond in advance. It is equipped with a visualization function so that the progress can be grasped at a glance.

In addition, it is possible to contribute to the improvement of the quality of care for elderly critically ill patients who are relatively vulnerable by eliminating manual input of patient vitality information by nursing personnel and ensuring real-time processing of patient data.

In addition, as it is possible to achieve the effect of shortening the hospitalization period for severely ill patients, there is an advantage in that it is possible to reduce medical costs for severely ill patients throughout the country.

The effects of the present invention are not limited to the effects mentioned above, and other effects not mentioned will be clearly understood by those skilled in the art from the following description.

1 is a block diagram of an event occurrence detection system through image analysis based on artificial intelligence according to an embodiment of the present invention.
2 is a hardware configuration diagram of a device for detecting event occurrence through image analysis based on artificial intelligence according to another embodiment of the present invention.
3 is a flowchart of a method for detecting event occurrence through image analysis based on artificial intelligence according to another embodiment of the present invention.
FIG. 4 is a diagram illustrating a configuration in which an event occurrence detection device images biometric information data according to various embodiments of the present disclosure.
5 is a diagram illustrating a process in which an event detection device learns a first artificial intelligence model and a second artificial intelligence model, according to various embodiments.
6 is a flowchart of a method of detecting occurrence of an event using an artificial intelligence model in various embodiments.

Advantages and features of the present invention, and a method of achieving them will become apparent with reference to the embodiments described below in detail together with the accompanying drawings. However, the present invention is not limited to the embodiments disclosed below, but may be implemented in a variety of different forms. It is provided to fully inform the skilled person of the scope of the present invention, and the present invention is only defined by the scope of the claims.

The terms used in the present specification are for describing exemplary embodiments and are not intended to limit the present invention. In this specification, the singular form also includes the plural form unless specifically stated in the phrase. As used herein, “comprises” and/or “comprising” do not exclude the presence or addition of one or more other elements other than the mentioned elements. Throughout the specification, the same reference numerals refer to the same elements, and "and/or" includes each and all combinations of one or more of the mentioned elements. Although "first", "second", and the like are used to describe various elements, it goes without saying that these elements are not limited by these terms. These terms are only used to distinguish one component from another component. Therefore, it goes without saying that the first component mentioned below may be the second component within the technical idea of the present invention.

Unless otherwise defined, all terms (including technical and scientific terms) used in the present specification may be used with meanings that can be commonly understood by those of ordinary skill in the art to which the present invention belongs. In addition, terms defined in a commonly used dictionary are not interpreted ideally or excessively unless explicitly defined specifically.

The term "unit" or "module" used in the specification refers to a hardware component such as software, FPGA or ASIC, and the "unit" or "module" performs certain roles. However, "unit" or "module" is not meant to be limited to software or hardware. The “unit” or “module” may be configured to be in an addressable storage medium, or may be configured to reproduce one or more processors. Thus, as an example, "sub" or "module" refers to components such as software components, object-oriented software components, class components, and task components, processes, functions, properties, Includes procedures, subroutines, segments of program code, drivers, firmware, microcode, circuits, data, databases, data structures, tables, arrays and variables. Components and functions provided within "sub" or "modules" may be combined into a smaller number of components and "sub" or "modules" or into additional components and "sub" or "modules". Can be further separated.

Spatially relative terms "below", "beneath", "lower", "above", "upper", etc. It can be used to easily describe the correlation between a component and other components. Spatially relative terms should be understood as terms including different directions of components during use or operation in addition to the directions shown in the drawings. For example, if a component shown in a drawing is turned over, a component described as "below" or "beneath" of another component will be placed "above" the other component. I can. Accordingly, the exemplary term “below” may include both directions below and above. Components may be oriented in other directions, and thus spatially relative terms may be interpreted according to the orientation.

In the present specification, a computer means all kinds of hardware devices including at least one processor, and may be understood as encompassing a software configuration operating in the corresponding hardware device according to embodiments. For example, the computer may be understood as including all of a smartphone, a tablet PC, a desktop, a laptop, and a user client and application running on each device, and is not limited thereto.

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings.

Each of the steps described herein is described as being performed by a computer, but the subject of each step is not limited thereto, and at least some of the steps may be performed by different devices according to embodiments.

1 is a block diagram of an event occurrence detection system through image analysis based on artificial intelligence according to an embodiment of the present invention.

Referring to FIG. 1, an event occurrence detection system through image analysis based on artificial intelligence according to an embodiment of the present invention may include an event occurrence detection device 100, a user terminal 200, and an external server 300. have.

Here, the event occurrence detection system through artificial intelligence-based image analysis shown in FIG. 1 is according to an embodiment, and its components are not limited to the embodiment shown in FIG. It can be deleted.

In an embodiment, the event occurrence detection device 100 may collect biometric information data sensed from a user. For example, the event occurrence detection device 100 is installed on at least a part of the user's body, and the blood pressure from a plurality of sensors (eg, blood pressure sensor, body temperature sensor, pulse sensor, respiratory rate sensor, etc.) that detects the user's biometric information. Vital sign data including at least one of data, body temperature data, pulse data, and respiratory rate data may be collected.

In various embodiments, the event occurrence detection device 100 may collect biometric information data from the outside, but collect biometric information data visualized like a graph.

In various embodiments, the event occurrence detection device 100 may collect biometric information data in a scalar form from the outside. However, the present invention is not limited thereto, and various types of data representing the user's biometric information may be collected.

In an embodiment, the event occurrence detection device 100 may generate a biometric information image using the visualized biometric information data. For example, the event occurrence detection device 100 may generate a biometric information image by capturing biometric information data in the form of a graph collected in real time at each preset period.

In various embodiments, when the biometric information data collected from the outside is in a scalar form, the biometric information data in the scalar form may be visualized in a graph form, and the biometric information image may be generated by capturing the visualized biometric information data. However, it is not limited thereto.

In various embodiments, a biometric information image may be generated based on a predetermined form or rule of various biometric data acquired from a patient monitor. Patient monitoring is a method of displaying an analog signal on a monitor, and is an auxiliary tool that grasps the patient's condition from the shape of the waveform based on the expert knowledge of the medical staff.

In an embodiment, a biometric information image that expresses the biometric data acquired from the patient monitoring device as a graph in a predetermined form based on a predetermined rule may be generated, but the present invention is not limited thereto.

However, conventional patient monitoring devices only display analog voltage signals on the screen, and measurement values are displayed on the screen, but digitized data cannot be collected. Therefore, since the information disappears after each screen passes, there is a problem that medical personnel must visually check each time.

Accordingly, there is a disadvantage that it is impossible to check every 24 hours, and even if a brief crisis signal occurs, it can be missed. Therefore, it is required to introduce a technology that automatically digitizes the patient's condition using all information displayed by photographing the screen of a patient monitoring device, which is an analog device, with a camera.

In an embodiment, the biometric information image may be obtained based on an image photographed on the screen of the patient monitoring device. For example, an image of an analog screen used in a conventional patient monitoring device or a result of correcting and pre-processing the image may be used as a biometric information image. As another example, biometric data of a patient may be extracted from an image taken of an analog screen used in a conventional patient monitoring device. The extracted biometric data can be processed and used to generate biometric information images. For example, a periodic signal such as respiration or pulse may determine its amplitude and period, and information indicated as a number may convert the displayed value into a numerical value, but is not limited thereto.

For example, waveform analysis can be used by fusion of a general image processing technology and a Convolutional Neural Network (CNN) of an artificial neural network and a Fast Fourier Transform (FFT), which is a transformation of Fourier Transform. In addition, the CNN series among artificial neural networks may be used for image analysis, but the present invention is not limited thereto.

In addition, since there is a lot of unnecessary information in the captured screen image, it is possible to develop a filter that removes noise by introducing an AE (Auto Encoder) technique to remove this information. AE is divided into a process of reducing information and then restoring it. In the process of reducing, noise information is removed. If there is not much noise, this step may be omitted.

For character parts such as numbers and letters, OCR (Optical Character Recognition) is used, but if there is a fixed font, accuracy can be further improved based on this.

Also, in this process, a system that analyzes images in real time with deep learning computer vision can be applied. When photographing patient monitoring values through a camera, it may be necessary to set a period of photographing and a period of collecting data based thereon. In addition, information must be able to be extracted from images other than waveforms and texts on the screen being photographed, and it must be possible to determine when diagrams or texts are not always expressed in the same form at a fixed location, or if the form is different for each manufacturer.

In an embodiment, the event occurrence detection device 100 may determine the detection of an event occurrence for a user by analyzing the biometric information image. For example, the event occurrence detection device 100 may determine detection of an event occurrence for a user by analyzing a biometric information image using an image analysis model.

Here, the image analysis model may be an artificial intelligence model (eg, a first artificial intelligence model and a second artificial intelligence model) that has been previously trained using a specific image and information indicated by the specific image as training data. However, it is not limited thereto.

In addition, the event here means an abnormal state generated by the user. For example, the event may include any one or more of hypovolemic shock, psychogenic shock, obstructive shock, and distributive shock.

In an embodiment, the event occurrence detection device 100 may provide a user interface (UI) that outputs information related to an event, such as a user's biometric information data, a biometric information image, and whether an event has occurred.

In one embodiment, the user terminal 200 may include a display on at least a portion of the user terminal 200, and various information (eg, event occurrence) provided from the event occurrence detection device 100 through the network 400 Information on whether or not and information on the possibility of event occurrence) can be output. For example, the user terminal 200 may output a UI provided from the event occurrence detection device 100 (e.g., a UI that outputs user's biometric information and information related to event occurrence), and an event occurs through the UI. Various types of information provided from the sensing device 100 may be output.

In various embodiments, the user terminal 200 may include at least one of a smartphone, a tablet PC, a notebook desktop, and a kiosk. However, it is not limited thereto.

In one embodiment, the external server 300 may be wired or wirelessly connected to the event occurrence detection device 100 through the network 400, and various information (eg, information on whether an event occurs, and Information on the possibility of event occurrence) can be provided and stored.

Here, the event occurrence detection system through image analysis based on artificial intelligence illustrated in FIG. 1 is described in a form in which various data generated by the event occurrence detection device 100 are stored in the external server 300, but is not limited thereto. Instead, the event occurrence detection device 100 may include a separate storage device to store various types of data in a separately provided storage device.

In various embodiments, the external server 300 may store a variety of patient-related information (eg, patient's personal information, patient's disease information, etc.), and store any one or more of the stored patient-related data as an event occurrence detection device. It can be provided as (100). For example, the external server 300 may be a hospital server, and when receiving a request for information on a specific patient from the event occurrence detection device 100, the event occurrence detection device 100 selects various information related to a specific patient. ) Can be provided. Hereinafter, a hardware configuration of the event occurrence detection device 100 will be described with reference to FIG. 2.

2 is a hardware configuration diagram of a device for detecting event occurrence through image analysis based on artificial intelligence according to another embodiment of the present invention.

Referring to FIG. 2, an event occurrence detection device 100 (hereinafter, "computing device 100") according to another embodiment of the present invention may include a processor 110 and a memory 120. In various embodiments, the computing device 100 may further include a network interface (or communication interface) (not shown), a storage (not shown), and a bus (not shown).

In one embodiment, the processor 110 may control the overall operation of each component of the computing device 100. The processor 110 may be configured to include a central processing unit (CPU), a micro processor unit (MPU), a micro controller unit (MCU), or any type of processor well known in the art.

In various embodiments, the processor 110 may perform an operation on at least one application or program for executing the method according to the embodiments of the present invention. In various embodiments, the processor 110 includes one or more cores (not shown) and a graphic processing unit (not shown) and/or a connection path (eg, a bus) for transmitting and receiving signals with other components. can do.

In various embodiments, the processor 110 temporarily and/or permanently stores a signal (or data) processed inside the processor 110. -Only Memory, not shown) may be further included. In addition, the processor 110 may be implemented in the form of a system on chip (SoC) including at least one of a graphic processor, RAM, and ROM.

In one embodiment, the processor 110 executes one or more instructions stored in the memory 120, so that a method to be described with reference to FIGS. 3 to 8 (a method of detecting event occurrence through image analysis based on artificial intelligence) You can do it. For example, the processor 110 executes one or more instructions stored in the memory 120 to collect visualized biometric information data, an operation of generating a biometric information image using the visualized biometric information data, and biometric information. An operation of analyzing an image and an operation of detecting the occurrence of an event for a user may be performed according to an analysis result of the biometric information image.

In one embodiment, the memory 120 may store various types of data, commands, and/or information. The memory 120 may store programs (one or more instructions) for processing and controlling the processor 110. Programs stored in the memory 120 may be divided into a plurality of modules according to functions.

In various embodiments, steps of a method or algorithm described in connection with an embodiment of the present invention may be implemented directly in hardware, implemented as a software module executed by hardware, or a combination thereof. Software modules include Random Access Memory (RAM), Read Only Memory (ROM), Erasable Programmable ROM (EPROM), Electrically Erasable Programmable ROM (EEPROM), Flash Memory, hard disk, removable disk, CD-ROM, or It may reside on any type of computer-readable recording medium well known in the art to which the present invention pertains.

Components of the present invention may be implemented as a program (or application) and stored in a medium to be executed by being combined with a computer that is hardware. Components of the present invention may be implemented as software programming or software elements, and similarly, embodiments include various algorithms implemented with a combination of data structures, processes, routines or other programming elements, including C, C++. , Java, assembler, or the like may be implemented in a programming or scripting language. Functional aspects can be implemented with an algorithm running on one or more processors. Hereinafter, a method of detecting event occurrence through artificial intelligence-based image analysis performed by the computing device 100 will be described with reference to FIGS. 3 to 8.

3 is a flowchart of a method for detecting event occurrence through image analysis based on artificial intelligence according to another embodiment of the present invention.

Referring to FIG. 3, in step S110, the computing device 100 may collect biometric information data from the outside. For example, for example, the computing device 100 may include blood pressure data, body temperature data, and pulse data generated from a plurality of sensors (eg, blood pressure sensor, body temperature sensor, pulse sensor, and breathing sensor) that detect biometric information of a patient. And it is possible to collect vital sign data including at least one data of the respiratory rate data.

In various embodiments, the computing device 100 includes blood pressure data, body temperature data, pulse data, and respiratory rate generated from a plurality of sensors (eg, blood pressure sensor, body temperature sensor, pulse sensor, and respiration sensor) for detecting biometric information of a patient. Visualized vital sign data may be collected from a biometric information output device (eg, a bed-side monitoring device in an intensive care unit) that visualizes and outputs vital sign data including at least one of the data.

In various embodiments, the computing device 100 directly collects vital sign data including at least one of blood pressure data, body temperature data, pulse data, and respiratory rate data from a plurality of sensors for sensing biometric information of a patient, Vital sign data collected directly can be visualized in the form of a graph.

In various embodiments, the computing device 100 analyzes outliers on biometric information data collected from outside (e.g., when there is a standardized outlier pattern, removes outliers, and uses standard deviations such as Mahalanobis distance). Using a filter to remove noise) may be performed, and a biometric information image may be generated using only the biometric information data from which noise has been removed through an outlier analysis in step S120 to be described later.

In step S120, the computing device 100 may generate a biometric information image by using the visualized biometric information data collected in step S110.

In various embodiments, the computing device 100 generates a biometric information image using the visualized biometric information data, divides the biometric information image into a preset time unit, and provides a plurality of unit biometric information images for learning an image analysis model. Can be created. Hereinafter, a method of generating a biometric information image performed by the computing device 100 will be described with reference to FIG. 4.

FIG. 4 is a diagram illustrating a configuration in which an event occurrence detection device images biometric information data according to various embodiments of the present disclosure.

Referring to FIG. 4, in various embodiments, the computing 100 device may generate a biometric information image using biometric information data collected from the outside, and generate a plurality of unit biometric information images by dividing the biometric information image. can do.

First, the computing device 100 collects biometric information data (eg, vital sign data) visualized in a graph form from the outside, or collects biometric information data in a scalar form from the outside and graphs the collected biometric information data in a scalar form. It is possible to create visualized biometric information data by visualizing it in a form.

In various embodiments, the computing device 100 may collect the user's biometric information data every preset first period T.

In various embodiments, when the periods in which the biometric information data is detected from each of the plurality of sensors are different from each other, the computing device 100 is Information data can be collected. However, it is not limited thereto.

Thereafter, the computing device 100 may generate a biometric information image by using the biometric information data visualized in the form of a graph. For example, the computing device 100 may generate a biometric information image by capturing biometric information data in a graph form collected in real time at every preset second period.

Here, the computing device 100 may set the second period equal to or shorter than the first period, but is not limited thereto.

Thereafter, the computing device 100 may generate a plurality of unit biometric information images by dividing the captured biometric information image.

In various embodiments, the computing device 100 may generate a plurality of unit biometric information images by dividing the biometric information image based on a preset time unit.

In various embodiments, the computing device 100 may generate a plurality of unit biometric information images by dividing the biometric information images by a preset number.

In various embodiments, the computing device 100 may adjust a time unit of each of a plurality of unit biometric information images or combine two or more unit biometric information images into one based on whether an event has occurred and an event possibility.

Here, the computing device 100 may use a plurality of unit biometric information images as input values of image analysis performed in the image analysis model, and may be used as training data for learning the image analysis model after image analysis. However, it is not limited thereto.

In various embodiments, the computing device 100 generates a biometric information image using a plurality of biometric information data (eg, vital sign data in the form of a graph), but visualizes it in different shapes and patterns according to the type of biometric information data. Alternatively, biometric information images can be created by visualizing them in different colors. For example, when the biometric information data is blood pressure data, the computing device 100 may visualize it in the form of a bar graph indicating the minimum blood pressure value and the maximum blood pressure value, and when the biometric information data is body temperature data, the body temperature at the time of detection Values can be visualized in the form of a line graph.

In addition, the computing device 100 may visualize a plurality of biometric information in the form of a broken line graph, but set the blood pressure value to blue, the body temperature value to red, and the pulse rate value to orange. That is, the computing device By visualizing differently for each type of biometric data or by making a difference in color, (100) not only can check the change in numerical value and pattern for each biometric data, but also check the correlation between each biometric data. In this case, it can be used to determine whether an event has occurred and whether an event has occurred.

In various embodiments, the computing device 100 generates a biometric information image using a plurality of biometric information data (eg, vital sign data in the form of a graph), but individually uses each biometric information data. A plurality of biometric information images corresponding to the data may be generated.

Referring back to FIG. 3, in step S130, the computing device 100 may perform image analysis on the biometric information image generated in step S120.

In various embodiments, the computing device 100 may image-analyze the biometric information image itself, or perform image analysis on each of a plurality of unit biometric information images generated by dividing the biometric information image.

In various embodiments, the computing device 100 analyzes the biometric information image using a previously learned artificial intelligence model, but can selectively use the artificial intelligence model according to the type of analysis result to be derived by analyzing the biometric information image. have.

For example, when the type of the analysis result to be derived by analyzing the biometric information image is whether an event occurs, the computing device 100 uses a previously learned first artificial intelligence model to provide a biosignal image and a plurality of unit biometric information. Images can be image-analyzed. Here, the pre-trained first artificial intelligence model may refer to a model obtained by learning a plurality of biosignal images labeled as whether or not an event occurs as training data.

In various embodiments, when the type of the analysis result to be derived by analyzing the biometric information image is the likelihood of occurrence of an event, the computing device 100 uses a previously learned second artificial intelligence model to determine the biosignal image and a plurality of unit biometrics. The information image can be image-analyzed. Here, it may refer to a model in which the biometric information image at a second point in time before a predetermined time before the first point in time is learned as training data based on the first point in time when the event occurs.

Here, the first artificial intelligence model and the second artificial intelligence model are models that are trained using different learning data (e.g., a plurality of biosignal images labeled with the occurrence of an event, and biometric information images prior to the occurrence of the event). Although described, but is not limited to this, one artificial intelligence model (eg, CNN model, NN model, RNN model) learns all of the biosignal images labeled with the occurrence of an event, and biometric information images prior to the occurrence of the event. Thus, one artificial intelligence model can perform both the functions of the first artificial intelligence model and the second artificial intelligence model.

Here, the first artificial intelligence model and the second artificial intelligence model are models that are trained using different learning data (e.g., a plurality of biosignal images labeled with the occurrence of an event, and biometric information images prior to the occurrence of the event). Although described, but is not limited to this, one artificial intelligence model (eg, CNN model, NN model, RNN model) learns all of the biosignal images labeled with the occurrence of an event, and biometric information images prior to the occurrence of the event. Thus, one artificial intelligence model can perform both the functions of the first artificial intelligence model and the second artificial intelligence model. Hereinafter, a process of learning the first artificial intelligence model and the second artificial intelligence model by the computing device 100 will be described with reference to FIG. 5.

In various embodiments, when biometric information data is collected in a scalar form from the outside, the computing device 100 analyzes the biometric information data using a third artificial intelligence model that has been previously learned from the scalar biometric information data as training data. can do.

In various embodiments, the computing device 100 analyzes the biometric information image generated in step S120, but determines the type of the biometric information data based on the visualization form and color of the biometric information data included in the biometric information image, The image can be analyzed by applying different artificial intelligence models according to the determined type of biometric data.

5 is a diagram illustrating a process of learning a first artificial intelligence model and a second artificial intelligence model by an event detection device according to various embodiments.

In various embodiments, the computing device 100 may train the first artificial intelligence model and the second artificial intelligence model by using the biometric information image and a plurality of unit biometric information images.

First, the computing device 100 may generate a plurality of unit biometric information images by dividing the biometric information image by a preset time unit or by dividing the biometric information image by a preset number.

Thereafter, the computing device 100 may select a unit biometric information image to be used as learning data from among the plurality of unit biometric information images. For example, the computing device 100 includes a unit biometric information image including a time point at which an event is determined to have occurred through image analysis among a plurality of unit biometric information images and a time point at which the event bar is generated based on the time point at which the event occurs. The previous unit biometric information image can be selected as training data.

In various embodiments, the computing device 100 may provide a UI for selecting a biometric information image related to an event from among a plurality of unit biometric information images to the user terminal 200. An image of biometric information related to an event can be selected.

Thereafter, the computing device 100 may label the selected unit biometric information image as the training data. For example, the computing device 100 labels the correct answer of each of the selected unit biometric information images as learning data (eg, labels the unit biometric information image including the time when the event occurs, as class 1, and is not related to the occurrence of the event). The unit biometric information image can be labeled as class 0).

In various embodiments, the computing device 100 may directly receive labeling for an event-related biometric information image from a user through a UI through which an event-related biometric information image is selected. For example, the computing device 100 may provide a UI through which a biometric information image related to an event is selected by a terminal of a specialist, and the biometric information image related to an event is selected from among a plurality of unit biometric information images and is selected at the same time. Labeling information of the biometric information image may be input together.

In various embodiments, the computing device 100 may label a unit biometric information image including a time point at which the event occurs differently according to the cause of the event and the type of the event. However, it is not limited thereto.

In various embodiments, when it is determined that an event occurs based on the user's biometric information image, the computing device 100 automatically labels biometric information images before a predetermined time from the time when the event occurs. can do. In this case, the computing device 100 may also label information indicating how many hours before the event occurs biometric information. For example, the computing device 100 may label biometric information images 6 hours ago, 12 hours ago, and 24 hours before the event occurrence time in different classes.

Thereafter, the computing device 100 may supervised learning the first artificial intelligence model and the second artificial intelligence model using the labeled unit biometric information image. For example, a first artificial intelligence model and a second artificial intelligence model may be trained by supervising the computing device 100 using learning data (biometric information image related to an event) acquired according to a labeling result. However, the present invention is not limited thereto, and various methods of learning an artificial intelligence model using biometric information images may be applied.

Referring back to FIG. 3, in step S140, the computing device 100 may detect the occurrence of an event using an analysis result derived through image analysis performed in step S130. Hereinafter, a method of detecting occurrence of an event performed by the computing device 100 will be described with reference to FIG. 6.

6 is a flowchart of a method of detecting occurrence of an event using an artificial intelligence model in various embodiments.

Referring to FIG. 6, in step S210, the computing device 100 may determine whether an event has occurred using an analysis result derived through image analysis performed in step S130.

In various embodiments, the computing device 100 inputs a biometric information image for determining whether an event occurs, into a pre-learned first artificial intelligence model, a plurality of biosignal images labeled with the occurrence of an event, and It is possible to determine whether an event has occurred based on the analysis result derived using the first artificial intelligence model.

In various embodiments, the computing device 100 may determine whether an event occurs by comparing a biometric information image for determining whether an event occurs with a biometric information image detected when the event occurs.

In step S220, when it is determined that the event has occurred through step S210, the computing device 100 may determine the cause of the event.

In various embodiments, the computing device 100 may determine the cause of the event using the first artificial intelligence model.

For example, in response to determining that an event has occurred according to the analysis result derived through the first artificial intelligence model, the computing device 100 may analyze the biometric information image from the analysis result obtained by analyzing each of the plurality of unit biometric information images. The image pattern of can be detected.

Thereafter, the computing device 100 may determine the type and cause of the event generated by using the detected image pattern. For example, when the detected image pattern has a first pattern, the computing device 100 may determine that the generated event is a hypovolemic shock, and when it has a second pattern, the generated event is a psychogenic shock. It can be determined to be. However, it is not limited thereto.

In step S230, when it is determined that the event has not occurred through step S210, the computing device 100 may determine the possibility of occurrence of the event.

In various embodiments, the computing device 100 uses the biometric information image for determining the possibility of occurrence of the event, based on the first time point at which the event occurs, and the biometric information image at a second time point before the first time point. As training data, the possibility of occurrence of an event may be determined based on the analysis result analyzed using the second artificial intelligence model, and inputting it into the previously learned second artificial intelligence model. Here, steps S210 to S230 may correspond to steps S140 of FIG. 3.

In various embodiments, the computing device 100 determines the likelihood of occurrence of an event by analyzing a user's biometric information image, but when the determined likelihood of occurrence of an event is greater than or equal to a reference value, the computing device 100 may classify it into an event occurrence risk group. Thereafter, the computing device 100 may monitor the user's state more frequently by setting a short period of collecting biometric information data and generating biometric information images for the user classified as the risk group of event occurrence.

In various embodiments, the computing device 100 may train the first artificial intelligence model and the second artificial intelligence model by using the biometric information image of a user classified as a risk group and whether an actual event of the corresponding user occurs as learning data. have. Through this, the computing device 100 may determine whether an actual event occurs according to the likelihood of occurrence of the event using the artificial intelligence model.

In various embodiments, the computing device 100 may provide a user interface (UI) that visualizes user-related information, user's biometric information data, and whether an event has occurred for the user, and configures and outputs a single screen. I can. Through this, the computing device 100 provides a dashboard for viewing patient information and status information at a glance, so that a user who monitors the patient's condition more easily monitors the patient's condition, and according to the occurrence of an event. You can induce them to respond quickly.

In various embodiments, when an event occurs, the computing device 100 may output a guide message and a warning signal for guiding whether an event has occurred and information related to the generated event through a UI. For example, when it is determined that an event has occurred through image analysis of the biometric information image, the computing device 100 outputs a guide message for guiding information related to the event and a warning signal in the form of a voice through the UI. , It is possible to enable a user who monitors the patient's condition in real time to quickly recognize whether an event has occurred.

In various embodiments, the computing device 100 may set a form in which a warning signal is output based on the type of an event determined through the biometric information image. For example, the computing device 100 sets the type of voice data to be output according to the type of the event (e.g., when the event is a hypovolemic shock, it outputs the first voice data, and in the case of a psychogenic shock, the second voice data) Output) or set the output time of the audio data. However, it is not limited thereto.

In various embodiments, the computing device 100 may provide a UI for outputting information on a plurality of users, biometric information for each of the plurality of users, and information related to an event, and request information for a specific user through the UI. When this is input, only information about a specific user, biometric information about a specific user, and information related to an event may be configured as a single screen and output through the UI. For example, the computing device 100 may configure only information on a specific user, biometric information on a specific user, and information related to an event as a single screen and output in the form of a pop-up window. However, it is not limited thereto.

The above-described method of detecting event occurrence through image analysis based on artificial intelligence has been described with reference to the flowchart shown in the drawings. For the sake of simplicity, the method of detecting event occurrence through image analysis based on artificial intelligence has been described as a series of blocks, but the present invention is not limited to the order of the blocks, and some blocks are shown and described in this specification. It may be performed in a different order or may be performed simultaneously. In addition, new blocks not described in the specification and drawings may be added, or some blocks may be deleted or changed.

In the above, embodiments of the present invention have been described with reference to the accompanying drawings, but those skilled in the art to which the present invention pertains can be implemented in other specific forms without changing the technical spirit or essential features. You will be able to understand. Therefore, the embodiments described above are illustrative in all respects, and should be understood as non-limiting.

100: event occurrence detection device (computing device)
200: user terminal
300: external server
400: network

Claims (10)

In the method performed by the computing device,
Collecting visualized biometric information data;
Generating a biometric information image using the visualized biometric information data;
Analyzing the biometric information image; And
Including the step of detecting the occurrence of an event for the user according to the analysis result of the biometric information image,
A method of detecting event occurrence through image analysis based on artificial intelligence. The method of claim 1,
Generating the biometric information image,
Comprising the step of generating the biometric information image by capturing the biometric information data in the form of a graph collected in real time at each preset period,
A method of detecting event occurrence through image analysis based on artificial intelligence. The method of claim 1,
Analyzing the biometric information image,
Dividing the biometric information image into a predetermined time unit to generate a plurality of unit biometric information images; And
Comprising the step of performing image analysis on each of the plurality of unit biometric information images using an image analysis model,
A method of detecting event occurrence through image analysis based on artificial intelligence. The method of claim 3,
The step of determining whether the event has occurred,
In response to determining that the event has occurred according to the analysis result, detecting an image pattern of the biometric information image from an analysis result obtained by analyzing each of the plurality of unit biometric information images; And
Including the step of determining the type and cause of the generated event using the detected image pattern,
A method of detecting event occurrence through image analysis based on artificial intelligence. The method of claim 1,
Analyzing the biosignal image,
Analyzing the biosignal image using a first artificial intelligence model that has previously learned a plurality of biosignal images labeled as to whether or not the event has occurred, as learning data,
The step of determining whether the event has occurred,
Including the step of determining whether or not the event occurs and a cause of occurrence based on the analysis result analyzed using the first artificial intelligence model,
A method of detecting event occurrence through image analysis based on artificial intelligence. The method of claim 1,
Analyzing the biosignal image,
Based on the first time point at which the event occurs, the biosignal image is obtained by using a second artificial intelligence model that has previously learned the biometric information image at a second time point before a predetermined time before the first time point as training data. Comprising the step of analyzing,
The step of determining whether the event has occurred,
Including the step of determining the possibility of occurrence of an event for the user based on the analysis result analyzed using the second artificial intelligence model,
A method of detecting event occurrence through image analysis based on artificial intelligence. The method of claim 1,
Providing a user interface (UI) for visualizing the user-related information, the user's biometric information data, and whether an event for the user has occurred, and configuring and outputting a single screen,
Providing the UI,
When the event occurs, further comprising the step of outputting the warning signal and a guide message for guiding whether the event has occurred and information related to the generated event through the UI,
A method of detecting event occurrence through image analysis based on artificial intelligence. The method of claim 1,
The biometric information data,
Including vital sign data including at least one of the user's blood pressure data, body temperature data, pulse data, and respiratory rate data,
The step of determining whether the event has occurred,
Analyzing the imaged vital sign data to determine whether or not any one or more of hypovolemic shock, psychogenic shock, obstructive shock, and distributive shock to the user has occurred,
A method of detecting event occurrence through image analysis based on artificial intelligence. A memory for storing one or more instructions; And
A processor that executes the one or more instructions stored in the memory,
The processor executes the one or more instructions,
An apparatus for performing the method of claim 1. A computer program combined with a computer as hardware and stored in a recording medium readable by a computer to perform the method of claim 1.

Images