KR102097743B1 - Apparatus and Method for analyzing disease based on artificial intelligence - Google Patents

Abstract

According to one embodiment disclosed in the present document, an apparatus for reading a disease based on artificial intelligence comprises: a memory including at least one instruction; and a processor. Since the at least one instruction is executed, the processor may generate a feature map image as a medical image is encoded based on one nerve network, generate a restored image on the medical image by decoding the feature map image based on the nerve network, detect an abnormal area where a disease is suspected based on a difference between the medical image and the restored image, generate an abnormal area image by cutting the abnormal area from the feature map image, and determine position information on an affected area and disease information corresponding to the abnormal area image based on another nerve network. The apparatus may generate the feature map image related to the abnormal area based on unsupervised learning.

Description

Apparatus and Method for analyzing disease based on artificial intelligence}

Various embodiments disclosed in this document relate to artificial intelligence-based image reading technology.

Imaging specialists can read medical images (eg x-rays, CT images) to diagnose diseases and write out readings. However, the number of medical practitioners qualified as a specialist in a year is limited, and among them, the proportion of radiologists is extremely low. So, small and medium-sized hospitals that lack video specialists cannot immediately diagnose diseases, or the working environment of video specialists is getting worse. To improve these problems, the interest of artificial intelligence systems that can assist or replace imaging specialists is increasing.

AI systems mostly learn based on supervised learning. For example, when a person (e.g., an imaging specialist) gives artificial intelligence the correct answer, the AI system learns the characteristics of the data by comparing the correct answer with data (e.g., medical image). The process of making the correct answer is called labeling and the correct answer is called label.

However, since medical-related labels can be generated only by specialists, and the number of specialists capable of generating labels is small, the generation of labeling data (data associated with the correct answer) may take a long time and the amount of labor may be considerable. Therefore, it may be difficult for an imaging specialist to perform a large amount of labeling work for medical image learning.

Various embodiments disclosed in this document may provide an apparatus and method for reading AI disease based on unsupervised learning, capable of generating feature map images related to abnormal regions.

An apparatus for reading AI disease according to an embodiment disclosed in the present document may include a memory including at least one instruction; And a processor, wherein the processor generates a feature map image by encoding a medical image based on one neural network as the at least one instruction is executed, and the feature map based on the one neural network. Decoding an image generates a reconstructed image for the medical image, detects an abnormal region suspected of a disease based on a difference between the medical image and the reconstructed image, and cuts the abnormal region from the feature map image An abnormal region image may be generated according to the smell, and lesion location information and disease information corresponding to the abnormal region image may be determined based on another neural network.

In addition, the method for reading AI disease according to an embodiment disclosed in the present disclosure includes: generating a feature map image by encoding a medical image based on a neural network; Generating a reconstructed image by decoding the feature map image based on the neural network; Detecting an area having a difference as an abnormal area when an area having a difference of a threshold or more exists between the medical image and the reconstructed image; Generating an abnormal region image by cutting the abnormal region from the feature map image; And determining location information and disease information corresponding to the abnormal region image based on another neural network.

According to various embodiments disclosed in the present document, a feature map image related to an abnormal region may be generated based on unsupervised learning, so that the amount of labeling work for learning can be reduced. In addition, various effects that can be directly or indirectly identified through this document may be provided.

1 shows an implementation environment of an AI disease reading apparatus according to an embodiment.
2 is a block diagram of an AI disease reading apparatus according to an embodiment.
3 is a first flowchart of an AI disease reading method according to an embodiment.
4 is a second flowchart of an AI disease reading method according to an embodiment.
In connection with the description of the drawings, the same or similar reference numerals may be used for the same or similar components.

1 shows an implementation environment of an AI disease reading apparatus according to an embodiment.

According to an embodiment, the medical record storage device 100 is a computing device that acquires and stores a medical image, and may transmit the medical image to the AI disease reading device 200 through at least one network. For example, the medical record storage device 100 may transmit a request for generating a medical image and a reading statement for the medical image to the AI disease reading device 200. The medical record storage device 100 may be an imaging device including at least one of an x-ray imaging device, a computed tomography (CT) device, a magnetic resonance imaging device (MRI), an ultrasound imaging device, and an endoscopic imaging device. The medical record storage device 100 may include a medical record storage system (eg, picture archiving communication system (PACs), electronic medical record (EMR) system) of an institution such as a medical institution, a medical research institution, and a health insurance corporation. have. The medical image may include, for example, diagnostic radiation and images such as x-ray images, CT images, MRI images, ultrasound images, and endoscopy images. According to various embodiments, the medical record storage device 100 may transmit an electronic document related to the medical record to the AI disease reading device 200.

According to an embodiment, the AI disease reading device 200 obtains a medical image (one or more medical images) from the medical record storage device 100 and based on a plurality of neural networks, the disease related to the medical image Whether to read and transmit the result of reading the medical image to the medical record storage device 100.

The apparatus for reading AI disease 200 may detect an abnormal region (eg, a disease suspect region) included in a medical image obtained based on a first neural network (or an auto encoder algorithm). The first neural network is an unsupervised learning-based neural network, and may be determined based on, for example, a stacked auto encoder (AE) and a VAE (Variational Auto-Encoder) algorithm. The first neural network is normal learning while repeating a process of encoding a plurality of medical images (hereinafter referred to as "normal learning images") related to an animal (or human) without disease, and then restoring the encoded images. It may be determined to extract characteristics of an image. For example, the AI disease reading apparatus 200 reconstructs an encoded image after dimensionally encoding a medical image (input image) based on the first neural network, and an area having a difference of a threshold value or more between the input image and the reconstructed image Can be detected as an abnormal area. The AI disease reading apparatus 200 extracts (eg, crops) an area corresponding to the abnormal area from the feature map image generated in the encoding process of the medical image when detecting the abnormal region from the medical image based on the first neural network An abnormal region image can be generated.

The apparatus for reading AI disease 200 may determine the location information of the affected area corresponding to the abnormal region based on the second neural network. For example, the artificial intelligence disease reading apparatus 200 checks the degree of similarity between the feature map image related to the abnormal region and the plurality of first reference images based on the second neural network, thereby providing the affected part location information corresponding to the abnormal region. Can decide. The plurality of first reference images are feature map images (or the captured images) of images of a plurality of body parts having no disease, respectively, and a plurality of body part names (eg, head, chest, abdomen) It may be a related (eg, labeled) image. The affected part location information may include at least one of body part names, direction information such as upper, lower, left and right, for example.

The AI disease reading apparatus 200 may determine disease information corresponding to the abnormal region based on the third neural network. For example, the AI disease reading apparatus 200 may determine disease information corresponding to the abnormal region by checking the similarity between the abnormal region image and the plurality of second reference images based on the third neural network. The plurality of second reference images are, for example, feature map images (or the photographed images) of images of body parts (or body organs) having a plurality of diseases, respectively, and a plurality of disease names It may be a video related to. For another example, if the disease information cannot be identified based on the similarity between the plurality of second reference images and the abnormal region image, the AI disease reading apparatus 200 may output abnormal finding information.

The AI disease reading apparatus 200 may generate a reading statement (sentence) describing disease information corresponding to a medical image and location information on the affected area based on the fourth neural network. The fourth neural network may be determined by learning an electronic document related to medical records based on a deep learning-based recurrent neural network (LNN) or a long short term memory model (LSTM) utilizing a memory device type. The readout may include, for example, first information determined based on the first neural network (eg, detection / non-detection guidance information), and second information indicating the biohistological location of the affected area determined based on the second neural network (eg, affected area) It may be text including at least one of location information) and third information (eg, disease information or abnormal finding information) indicating presence or absence of a disease identified based on the third neural network.

The AI disease reading device 200 may provide analysis result data for a medical image to the medical record storage device 100. The analysis result data may output, for example, at least one of a readout, a masking image, a calculated score (eg, similarity probability), and a calculation result of each neural network (eg, various graphs). The masking image is superimposed on the medical image or the feature map image, and the abnormal region may be displayed, for example, in the form of a heat map or a point object.

When the analysis result data is received from the AI disease reading device 200, the medical record storage device 100 may store it in connection with a medical image. Thereafter, the analysis result data may be reviewed by an object (eg, a specialist) who needs analysis result data for a medical image, or used as labeling data for other machine learning.

In addition, according to the above-described embodiment, since the AI disease reading apparatus 200 can assist or replace the work of an imaging specialist, it is possible to close a gap between a large number of large hospitals and small hospitals where imaging specialists do not.

Furthermore, according to the above-described embodiment, the AI disease reading apparatus 200 can reduce the workload of an imaging specialist and expand the target of medical services, resulting in improved quality of medical services.

2 is a block diagram of an AI disease reading apparatus according to an embodiment.

Referring to FIG. 2, the apparatus for reading AI disease 200 according to an embodiment may include a communication circuit 210, a memory 220, and a processor 230. In one embodiment, the AI disease reading apparatus 200 may omit some components or further include additional components. In addition, some of the components of the AI disease reading apparatus 200 are combined to be composed of one individual, and the functions of the corresponding components before combining may be performed in the same way.

The communication circuit 210 may support the establishment of a communication channel or a wireless communication channel between the AI disease reading device 200 and another device (eg, the medical record storage device 100), and performing communication through the established communication channel. have. The communication circuit 210 may include a wireless communication module (eg, a cellular communication module, a short-range wireless communication module, or a global navigation satellite system (GNSS) communication module) or a wired communication module (eg, a local area network (LAN) communication module, or power line Communication module). The corresponding communication module among these communication modules may be a first network (eg, a short-range communication network such as Bluetooth, WiFi direct, or an infrared data association (IrDA)) or a second network (eg, a cellular network, the Internet, or a computer network (eg, a LAN). Or it may communicate with other devices (medical record storage device 100) through a telecommunication network (WAN).

The memory 220 may store various data used by at least one component (eg, the processor 230) of the AI disease reading device 200. The data can include, for example, input data or output data for software and related commands. For example, the memory 220 may store at least one instruction capable of detecting abnormal region, disease information, and lesion location information included in the medical image. The memory 220 may include volatile memory or nonvolatile memory.

The processor 230 may control at least one other component (eg, hardware or software component) of the AI disease reading apparatus 200 as executing the at least one instruction, and perform various data processing or calculations. Can be done. The processor 230 includes, for example, a central processing unit (CPU), a graphics processing unit (GPU), a microprocessor, an application processor, an application specific integrated circuit (ASIC), and field programmable gate arrays (FPGAs). )), And may have a plurality of cores.

The processor 230 may include an acquisition unit 231, a pre-processing unit 233, a reading unit 235, a reading statement generation unit 237, and a result writing unit 239. The acquisition unit 231, the pre-processing unit 233, the reading unit 235, the reading statement generation unit 237, and the result creation unit 239 are separate hardware modules, or software components implemented by at least one processor It can be an element. For example, functions performed by the acquiring unit 231, the pre-processing unit 233, the reading unit 235, the reading statement generating unit 237, and the result writing unit 239 are performed by one processor or separately. It can be performed by the processor.

According to an embodiment, the acquisition unit 231 acquires image data in a specified manner from a medical record storage device (eg, the medical record storage device 100 of FIG. 1) through the communication circuit 210, and the acquired image Data may be stored in the memory 220. For example, the acquisition unit 231 may acquire a binary file from the medical record storage device 100 in an asynchronous manner, and store the obtained binary file in a message queue of the memory 220 have. The acquisition unit 231 may convert the stored binary file into an image (eg, JPEG) of a specified format. In the above-described embodiment, the acquiring unit 231 may improve the transmission / reception speed of the image data and the processing load of the image data by transmitting and receiving the image data in a message queue method. The medical image may include, for example, diagnostic radiation and images such as x-ray images, CT images, MRI images, ultrasound images, and endoscopy images. The medical record storage device 100 may include an electronic medical record (EMR) system.

According to an embodiment, the pre-processing unit 233 may acquire a medical image from the memory 220 and pre-process the obtained medical image to convert it into its own format. The format conversion may be to maintain consistency for scale (size, resolution), contrast, brightness, color balance, and hue / saturation. The format conversion may be for removing noise generated during measurement (or a photographing process). The designated conversion may be to enhance contrast between the foreground and the background. For example, the pre-processing unit 233 may convert the resolution of the medical image to a specified resolution and the size of the medical image to a specified size. The pre-processing unit 233 may reduce the dimension of the image by converting the received 3D medical image into a 2D image. For another example, the pre-processing unit 233 analyzes the intensity (or intensity) value of the medical image to construct a histogram, and histogram equalization for the medical image based on the configured histogram. You can do In the uniformized image, contrast between an object and a background may be enhanced. Alternatively, the pre-processing unit 233 may perform a histogram equalization on the medical image so that the designated intensity value is emphasized among the intensity values included in the medical image. For example, the pre-processing unit 233 may perform histogram equalization such that an intensity value corresponding to an object to be used for classification among objects included in the medical image is emphasized (eg, increased). As another example, the pre-processor 233 may remove noise from the medical image based on statistical data. For example, the medical image includes noise due to movement of a patient or a measuring device (eg, CT imager, MRI imager), and noise generated in the image due to the motion has a certain pattern. Accordingly, the pre-processor 233 may remove noise (eg, bias correction) due to the motion included in the medical image based on the correction data stored in the memory 220. The correction data may be provided to remove a certain pattern of noise due to the movement.

The reader 235 may detect an abnormal region (eg, a suspected disease region) included in the normalized (or preprocessed) image based on the first neural network. The first neural network is an unsupervised learning-based neural network, and may be determined based on, for example, a stacked auto encoder (AE) and a VAE (Variational Auto-Encoder) algorithm. The first neural network may be configured by, for example, a combination of at least one neural network algorithm among Resnet (residual network), Desenet (densely connected convolutional networks), and Mobilenet. The first neural network encodes a plurality of medical images (hereinafter referred to as "normal learning images") related to a disease-free body (eg, a human or animal body) and restores the encoded image. It may be determined to be able to extract features of the normal learning image while repeating. The normal learning image may include a plurality of medical images (eg, x-ray images, CT images, MRI images) of a plurality of body parts (eg, brain, abdomen, and chest). For example, the reading unit 235 reconstructs the encoded image after dimensionally encoding the normalized image (input image) based on the first neural network, and abnormally detects an area having a difference of a threshold value or more between the input image and the reconstructed image. Can be detected as an area. Since the first neural network is determined based on the normal learning image, when encoding a medical image including an abnormal region based on the first neural network, the difference between the input image and the output image may be large. Accordingly, the reading unit 235 can easily detect an abnormal area suspected of being abnormal from the medical image using the difference. According to the above-described embodiment, since the reading unit 235 can detect feature points of the abnormal region from the medical image based on unsupervised learning without information about the disease, a neural network design (or, for detecting feature points of the abnormal region) Decision) It is possible to improve the problems of the conventional disease detection algorithm, which requires labeled learning data generated by checking each disease type at a time.

When the detection unit 235 detects an abnormal region from the normalized image based on the first neural network, the reader 235 may check an area corresponding to the abnormal region of the output image in the feature map image extracted in the process of encoding the normalized image. The reader 235 may generate an abnormal area image according to a crop of the identified area into a specified shape (eg, a square). As such, the abnormal region image may be a partial region image of the feature map image. According to various embodiments, the reading unit 235 may label feature points included in the identified region with feature points corresponding to the abnormal region. The reading unit 235 may generate detection guide information when the abnormal region is detected from the normalized image, and generate non-detection guide information when the abnormal region is not detected from the normalized image.

The reading unit 235 may check the location information of the affected area corresponding to the abnormal region based on the second neural network. The second neural network may be determined based on at least one of a high-speed computational algorithm, for example, a Mobilenet algorithm and an Xception algorithm. Since the feature map images are a combination of multiple layer feature map images (eg, merged into one image), to calculate the feature map images all at once, the processing time may be long. However, the second neural network may rapidly detect a body part from the feature map image by extracting feature points while reducing the dimension of the multi-layer feature map images based on at least one algorithm. The second neural network may be determined by learning a plurality of first reference images related to a plurality of body parts having no disease, for example. The plurality of first reference images may be, for example, a feature map image (or the photographed image) related to photographed images (body image) of a plurality of body parts without disease. The first reference images may be images associated with a plurality of body part names (eg, head, chest, and abdomen) (eg, labeled). For example, the reading unit 235 may check the similarity between the abnormal region image and the plurality of first reference images based on the second neural network. When the reader 235 checks the first reference image similar to the abnormality region image and the first designated similarity degree or higher, based on the label data related to the identified first reference image, the readout location information corresponding to the abnormal region may be generated. have. The affected part location information may include at least one of body part names, direction information such as upper, lower, left and right, for example. The direction information may be information indicating which of the upper, lower, left and right directions is located in the identified body part.

The reading unit 235 may determine disease information corresponding to the abnormal region based on the third neural network. The third neural network is, for example, a Resnet-based classifier, and may be generated as a learning result based on supervised learning to classify disease information. The third neural network may be generated by learning second reference images related to a plurality of body parts having a disease, for example. The plurality of second reference images may be, for example, a feature map image (or the captured image) related to a captured image of a plurality of body parts (or body organs) each having a disease. The plurality of second reference images may be associated with a plurality of disease names (eg, labeling). For example, if the reader 235 checks the similarity between the abnormal region image and the plurality of second reference images based on the third neural network, and if the second reference image is similar to the abnormal region image and the second designated similarity or higher, , Disease information corresponding to an abnormal region may be determined based on label data related to the identified second reference image. On the other hand, if the second reference image similar to the second designated similarity or higher than the abnormal region image among the plurality of second reference images cannot be identified, the reading unit 235 may generate abnormal finding information. In the above-described embodiment, the reading unit 235 may generate lesion location information and abnormal finding information in relation to the abnormal region image.

The reading statement generation unit 237 may generate a reading statement (sentence) describing disease information corresponding to a medical image and location information of a lesion based on the fourth neural network. The fourth neural network may be generated by learning electronic documents related to medical records based on a deep learning-based recurrent neural network (RNN) or a long short term memory model (LSTM) utilizing a memory device type. The readout may include, for example, first information determined based on the first neural network (eg, detection / non-detection guidance information), and second information indicating the biohistological location of the affected area determined based on the second neural network (eg, affected area) It may be text including at least one of location information) and third information (eg, disease information or abnormal finding information) indicating presence or absence of a disease identified based on the third neural network. For example, the readout generating unit 237 may generate a feature map of a readout statement describing the affected location information and the disease information, when confirming the affected location information and disease information generated based on the abnormal region image. For another example, if the undetected guide information generated based on the medical image is checked, the read statement generation unit 237 may generate a read statement to guide the normal state. As another example, the reading statement generation unit 237 may create a reading statement that guides abnormal finding information when it is not possible to determine the location information or the disease information based on the medical image.

 The reading statement generation unit 237 may generate the reading statement to be associated with at least one of a medical image, a feature map image, or an abnormal area image. For example, if any information (eg, second information) on the readout is selected, the readout generation unit 237 may generate a readout so that the abnormal region image is displayed. In the above-described embodiment, the prior art merely provided a simple form of reading text in which body parts and disease information are written in the template, but the reading text generating unit 237 learns electronic documents created by experts based on machine learning. By doing so, it is possible to generate a read sentence in the form of a compound sentence similar to the level of the electronic document.

The result creation unit 239 may generate analysis result data for a medical image. The analysis result data includes, for example, read statements, medical images, feature map images, masking images (eg, abnormal area images), computational scores of each neural network (eg, similarity probability), and computational results of each neural network (eg, various graphs) ). The masking image is superimposed on a medical image (or feature map image), and the abnormal region may be displayed, for example, in the form of a heat map or a point object. For example, when the result creation unit 239 displays a reading statement and selects a certain region of the reading statement (eg, a top portion of the reading portion, specific text), at least one of a medical image, a masking image, an associated score, and a calculation result of each neural network The analysis result data capable of selectively displaying one may be generated.

The result creation unit 239 may provide (eg, display) the analysis result data according to a designated form through the display of the AI disease reading device 200 or the medical record storage device 100. For example, the result creation unit 239 may display the analysis result data in the specified format through the designated web page on the medical record storage device 100.

According to the above-described embodiment, the artificial intelligence disease reading apparatus 200 may detect a feature map image corresponding to an abnormal region and an abnormal region from the medical image based on a neural network learning a medical image of the body without the disease. As described above, since the AI disease reading apparatus 200 is based on unsupervised learning, it is possible to improve a conventional problem that requires large-capacity labeling data to extract abnormal regions and feature points of abnormal regions from medical images.

In addition, the AI disease reading apparatus 200 may determine the affected part location information and the disease information corresponding to the abnormal region using only some regions (non-normal region images) of the feature map images extracted based on unsupervised learning, so It is possible to reduce the amount of computation for determining location and disease.

In addition, the AI disease reading apparatus 200 may output abnormal area information and abnormal finding information even if it is unable to detect the affected part location information or the disease information from the medical image. Therefore, an imaging specialist who had to check all medical images in the related art can support to check only the medical image in which the abnormal region has been detected, thereby greatly reducing the workload of the imaging specialist.

3 is a first flowchart of an AI disease reading method according to an embodiment.

Referring to FIG. 3, in operation 310, the apparatus for reading AI disease 200 may generate a feature map image by encoding a medical image based on the first neural network. The first neural network extracts features of a normal learning image by repeating a process of encoding and restoring a plurality of medical images (hereinafter referred to as "normal learning images") related to a disease-free body (eg, a person). It may have been determined to be possible. In operation 310, the medical image may be an image normalized to a designated format as it is preprocessed.

In operation 320, the AI disease reading apparatus 200 may generate a reconstructed image for the medical image by decoding the feature map image based on the first neural network.

In operation 330, the artificial intelligence disease reading apparatus 200 may detect an abnormal area in which a disease is suspected based on a difference between the medical image and the reconstructed image. For example, the AI disease reading apparatus 200 may detect the identified region as an abnormal region when there is an area having a difference of a threshold or more between the medical image and the reconstructed image.

In operation 340, the artificial intelligence disease reading apparatus 200 may generate an abnormal region image by cutting an area corresponding to the abnormal region of the reconstructed image from the feature map image. For example, the AI disease reading apparatus 200 may generate an abnormal region image according to a cropping of an abnormal region of the reconstructed image in a specified shape (eg, a rectangle) among the feature map images.

In operation 350, the artificial intelligence disease reading apparatus 200 inputs an abnormal region image into the second and third neural networks, and determines the affected area location information and the disease information corresponding to the abnormal region based on the second neural network and the third neural network. You can.

4 is a second flowchart of an AI disease reading method according to an embodiment.

Referring to FIG. 4, in operation 410, the AI disease reading device 200 may be another electronic device (eg, a medical record storage device (eg, the medical record storage device 100 of FIG. 1)) or its own memory (eg, FIG. 2, memory 220)). Another electronic device may be an imaging device including at least one of an x-ray imaging apparatus, a computed tomography (CT) apparatus, a magnetic resonance imaging apparatus (MRI), an ultrasound imaging apparatus, and an endoscopic imaging apparatus. The medical record storage device 100 may include a picture archiving communication system (PAC) and an electronic medical record (EMR) system. The medical image may include, for example, diagnostic radiation and images such as x-ray images, CT images, MRI images, ultrasound images, and endoscopy images.

In operation 420, the AI disease reading apparatus 200 may preprocess the obtained medical image and convert it into its own format. The format conversion may be, for example, to maintain consistency for scale (size, resolution), contrast, brightness, color balance, and hue / saturation. The format conversion may be for removing noise generated during measurement (or a photographing process). The designated conversion may be to enhance contrast between the foreground and the background.

In operation 430, the AI disease reading apparatus 200 may detect an abnormal region in the normalized image based on the first neural network. For example, the AI disease reading apparatus 200 may dimensionally encode the normalized image (input image) based on the first neural network determined by learning about at least one body without disease, and then restore the encoded image. You can. The AI disease reading apparatus 200 may detect the identified region as an abnormal region when a region having a difference of a threshold value or more between the input image and the reconstructed image is identified. The AI disease reading apparatus 200 may generate an abnormal region image by cutting an area corresponding to the abnormal region of the output image from the feature map image generated through the encoding. In operation 430, the artificial intelligence disease reading apparatus 200 may generate detection guidance information when the abnormal region is detected, and generate non-detection guidance information when the abnormal region is not detected.

In operation 440, the AI disease reading apparatus 200 may determine the affected part location information corresponding to the abnormal region based on the second neural network. The second neural network may be determined, for example, by learning a plurality of first reference images related to a plurality of body parts without disease. For example, the artificial intelligence disease reading apparatus 200 may generate lesion location information corresponding to the abnormal region by checking the similarity between the abnormal region image and the plurality of first reference images based on the second neural network. The affected part location information may include at least one of body part names, direction information such as upper, lower, left and right, for example.

In operation 450, the artificial intelligence disease reading apparatus 200 may determine disease information corresponding to the abnormal region based on the third neural network. The third neural network may be, for example, generated by learning second reference images related to a plurality of diseased body parts. For example, the AI disease reading apparatus 200 may determine disease information corresponding to the abnormal region by checking the similarity between the abnormal region image and the plurality of second reference images based on the third neural network. In operation 450, if the AI disease reading apparatus 200 cannot identify the second reference image similar to the abnormality region image and the second designated similarity or higher, the abnormality finding information may be generated.

In operation 460, the artificial intelligence disease reading apparatus 200 may generate a reading statement (sentence) describing disease information corresponding to the medical image and location information of the affected area based on the fourth neural network. The fourth neural network may be determined by learning electronic documents related to medical records. The readout may include, for example, first information determined based on the first neural network (eg, detection / non-detection guidance information), and second information indicating the biohistological location of the affected area determined based on the second neural network (eg, affected area) It may be text including at least one of location information) and third information (eg, disease information or abnormal finding information) indicating presence or absence of a disease identified based on the second neural network. The apparatus for reading AI disease 200 may generate the reading statement to be associated with at least one of a medical image, a feature map image, or an abnormal region image.

In operation 470, the AI disease reading device 200 may provide analysis result data for a medical image to another electronic device. The analysis result data may include, for example, at least one of a readout, a medical image, a masking image (eg, an abnormal region image), a calculation score of each neural network (eg, similarity probability), and a calculation result of each neural network (eg, various graphs). It may include. The masking image is superimposed on the medical image or the feature map image, and the abnormal region may be displayed, for example, in the form of a heat map or a point object. For example, the AI disease reading device 200 may provide analysis result data to the medical record storage device 100 in a designated format through a designated web page.

It should be understood that various embodiments of the document and terms used therein are not intended to limit the technical features described in this document to specific embodiments, and include various modifications, equivalents, or substitutes of the embodiment. In connection with the description of the drawings, similar reference numerals may be used for similar or related components. The singular form of a noun corresponding to an item may include one or more of the items, unless the context clearly indicates otherwise. In this document, "A or B", "at least one of A and B", "at least one of A or B", "A, B or C", "at least one of A, B and C" and "A, Each of the phrases such as “at least one of B or C” may include any of the items listed together in the corresponding phrase of the phrases, or any possible combination thereof. Terms such as “first”, “second”, or “first” or “second” can be used to simply distinguish a component from other components, and to separate components from other aspects (eg, importance or Order). Any (eg, first) component is “coupled” or “connected” to another (eg, second) component, with or without the term “functionally” or “communically” When mentioned, it means that any of the above components can be connected directly to the other components (eg, by wire), wirelessly, or through a third component.

As used herein, the term "module" may include units implemented in hardware, software, or firmware, and may be used interchangeably with terms such as, for example, logic, logic blocks, components, or circuits. The module may be an integrally configured component or a minimum unit of the component or a part thereof performing one or more functions. For example, according to an embodiment, the module may be implemented in the form of an application-specific integrated circuit (ASIC).

Various embodiments of the present document are one stored in a storage medium (eg, memory 220) (eg, internal memory or external memory) readable by a machine (eg, AI disease reading device). It may be implemented as software (eg, a program) including the above instructions. For example, a processor (eg, processor 230) of a device (eg, AI disease reading device 200) may call and execute at least one of one or more instructions stored from a storage medium. This enables the device to be operated to perform at least one function according to the at least one instruction called, wherein the one or more instructions may include code generated by a compiler or code executable by an interpreter. The storage medium readable by the device may be provided in the form of a non-transitory storage medium, where 'non-transitory' is a device in which the storage medium is tangible, and a signal ( For example, it does not include electromagnetic waves), and this term is used when data is stored semi-permanently and temporarily. Does not distinguish right.

According to one embodiment, a method according to various embodiments disclosed in this document may be provided as being included in a computer program product. Computer program products are commodities that can be traded between sellers and buyers. The computer program product is distributed in the form of a device-readable storage medium (eg compact disc read only memory (CD-ROM)), or through an application store (eg Play Store ^TM ) or two user devices ( It can be distributed (eg, downloaded or uploaded) directly or online between smartphones). In the case of online distribution, at least a portion of the computer program product may be stored at least temporarily in a storage medium readable by a device such as a memory of a manufacturer's server, an application store's server, or a relay server, or may be temporarily generated.

According to various embodiments, each component (eg, module or program) of the above-described components may include a singular or a plurality of entities. According to various embodiments, one or more components or operations of the above-described corresponding components may be omitted, or one or more other components or operations may be added. Alternatively or additionally, a plurality of components (eg, modules or programs) may be integrated into one component. In this case, the integrated component may perform one or more functions of each component of the plurality of components the same or similar to that performed by the corresponding component among the plurality of components prior to the integration. . According to various embodiments, operations performed by a module, program, or other component may be executed sequentially, in parallel, repeatedly, or heuristically, or one or more of the operations may be executed in a different order, or omitted Or, one or more other actions can be added.

Claims (8)

A memory including at least one instruction for detecting abnormal region, disease information, and lesion location information included in the medical image;
A feature map image is generated by encoding the medical image based on a first neural network based on non-supervised learning, and the feature map image is decoded based on the first neural network for the medical image. A reconstructed image is generated, an abnormal area in which a disease is suspected is detected based on a difference between the medical image and the reconstructed image, and an abnormal region image is generated by cutting the abnormal region from the feature map image. A reading unit for determining location information corresponding to the abnormal region based on a neural network and determining disease information corresponding to the abnormal region based on a third neural network; And
And a readout generating unit for generating a readout describing the disease information and the affected position information based on a fourth neural network generated by learning an electronic document related to a medical record,
The first neural network,
As a non-supervised learning-based neural network, it is determined based on a stacked auto encoder (AE) and a VAE (Variational Auto-Encoder) algorithm, and at least one neural network algorithm among Resnet (residual network), Desenet (densely connected convolutional networks) and Mobilenet It is composed of a combination, AI disease reading apparatus characterized in that determined by learning a plurality of medical images for at least one body without a disease. The method according to claim 1,
The reading unit,
On the basis of the second neural network, the similarity between the feature map image related to the abnormal region and a plurality of first reference images is checked to determine affected location information corresponding to the abnormal region,
The plurality of first reference images,
It is a feature map image of images of multiple body parts without disease,
The affected part location information,
And it characterized in that it comprises at least one of the body parts and direction designation information AI disease reading apparatus. The method according to claim 1,
The reading unit,
Determine the disease information corresponding to the abnormal region by checking the similarity between the abnormal region image and a plurality of second reference images based on the third neural network,
The plurality of second reference images,
AI disease reading device, characterized in that the feature map image of an image of a plurality of body parts with a disease. The method according to claim 1,
An acquiring unit acquiring the medical image in the form of a binary file from a medical record storage device in an asynchronous manner and storing the obtained binary file in a message queue of the memory; And
And preprocessing the acquired medical image in the form of a binary file to generate a normalized medical image. The method according to claim 4,
When selecting a certain area of the reading statement, writing a result of generating analysis result data capable of selectively displaying at least one of the medical image, the feature map image, the abnormal region image, the computational score of each neural network, and the computational results of each neural network Artificial intelligence disease reading device further comprising a wealth. delete delete Generating a feature map image by encoding a medical image based on a first neural network based on unsupervised learning by the reading unit ;
Generating a reconstructed image of the medical image by decoding the feature map image based on the first neural network by the reading unit ;
Detecting , by the reading unit, the region having the difference as an abnormal region when a region having a threshold difference or more exists between the medical image and the reconstructed image ;
Generating an abnormal area image by cutting the abnormal area from the feature map image by the reading unit ;
Determining, by the reading unit, lesion location information corresponding to the abnormal region based on a second neural network;
Determining, by the reading unit, disease information corresponding to the abnormal region based on a third neural network; And
And generating, by the readout generating unit, a readout describing the disease information and the affected area location information based on a fourth neural network generated by learning an electronic document related to a medical record,
The first neural network,
As a non-supervised learning-based neural network, it is determined based on a stacked auto encoder and a VAE algorithm, and consists of a combination of at least one neural network algorithm among Resnet, Desenet, and Mobilenet, and includes multiple medical images of at least one body without disease AI disease reading method, characterized in that determined by learning them.

Images