WO2023210893A1

WO2023210893A1 - Apparatus and method for analyzing ultrasound images

Info

Publication number: WO2023210893A1
Application number: PCT/KR2022/017348
Authority: WO
Inventors: 조정선; 박종원; 방혜진; 조유희
Original assignee: 가톨릭대학교 산학협력단; 한국기계연구원
Priority date: 2022-04-28
Filing date: 2022-11-07
Publication date: 2023-11-02
Also published as: KR20230153166A

Abstract

The present disclosure relates to an apparatus and method for analyzing ultrasound images and, particularly, can provide an apparatus and method for analyzing ultrasound images, wherein atherosclerotic plaque states and plaque areas can be predicted from transesophageal ultrasound image information. Specifically, an apparatus and method for analyzing ultrasound images can be provided wherein plaque states and plaque areas can be predicted by generating and applying a first prediction model for classifying plaque states and a second prediction model for extracting plaque areas.

Description

Ultrasound image analysis device and method

These embodiments provide an ultrasonic image analysis device and method.

As Korea enters an aging society, the incidence of stroke is increasing due to the increase in the elderly population. Cerebral infarction, an ischemic brain disease, is caused by narrowing of the inner walls of cerebral blood vessels due to arteriosclerosis, aortic atherosclerotic plaques fall off, or blood clots form in the heart. It can be divided into cases caused by embolic cerebral infarction and cases caused by decreased systemic blood flow. However, cerebral infarction of unknown cause has been reported in 25-40% of cases, and identifying the cause for treatment may be important in the patient's prognosis. Research has been ongoing for several years to show that not only embolic cerebral infarction, but also cerebral infarction of unknown cause can occur due to embolism of fatty deposits (atheroma or plaque) that occur in the aorta due to atherosclerosis, blocking cerebral blood vessels and causing stroke.

In particular, ultrasound, which is used to determine aortic atherosclerosis, is a type of elastic wave that propagates inside the human body, and depending on the physical characteristics of human tissue, it may be reflected or transmitted at the interface of the medium, and amplitude attenuation may occur due to absorption. Using these characteristics of ultrasound, images of internal tissues of the human body can be obtained and the size and characteristics of the tissues can be determined from these images, so ultrasound diagnostic devices are widely used in the medical industry. However, due to the nature of the transesophageal ultrasound examination, there may be an error in plaque measurement for each examiner, and there is a problem in that the examination requires rapidity as it is a method of inserting the examination equipment into the esophagus.

On the other hand, with the recent rapid development of machine learning theory, imaging medical diagnostic technology at home and abroad uses ultrasound images, computed tomography (CT), and magnetic resonance imaging (MRI) to diagnose high-risk diseases such as stroke. By applying artificial intelligence models to image data such as magnetic resonance imaging, it is used as an important means of shortening the time it takes for doctors to interpret images and deriving accurate results. Accordingly, there is a need for technology that can prevent and manage aortic atherosclerosis, which is emerging as a major cause of embolic stroke, in advance through artificial intelligence models to prevent it from developing into a high-risk disease.

Therefore, there is a need for technology that can accurately analyze plaques caused by atherosclerosis in the aorta from transesophageal ultrasound image information through an artificial intelligence model.

Against this background, the present embodiments provide an ultrasound image analysis device and method that can accurately analyze plaque from transesophageal ultrasound image information through an artificial intelligence model.

In order to achieve the above-described object, in one aspect, the present embodiment provides an ultrasound image analysis device, which acquires ultrasound image information from transesophageal echocardiography information and plaque information included in the blood vessel in the ultrasound image information. A learning data generator that generates learning data by labeling, a first prediction model that performs learning based on the generated learning data to classify the plaque state from ultrasound image information, and a second prediction model that extracts the plaque area. An ultrasound image analysis device is provided, comprising a model generation unit for generating a model and a plaque prediction unit for predicting the plaque state and plaque area from specific ultrasound image information by applying each generated prediction model.

In another aspect, the present embodiment is an ultrasound image analysis method, in which ultrasound image information is acquired from transesophageal echocardiography information, and plaque information contained in blood vessels in the ultrasound image information is labeled to provide learning data. A learning data generation step that generates a model generation step that performs learning based on the generated learning data to generate a first prediction model that classifies the plaque state from ultrasound image information and a second prediction model that extracts the plaque area, and the generated An ultrasound image analysis method is provided, which includes a plaque prediction step of predicting the plaque state and plaque area from specific ultrasound image information by applying each prediction model.

1 is a diagram illustrating the configuration of an ultrasound image analysis device according to an embodiment of the present disclosure.

FIG. 2 is a flowchart illustrating a prediction operation of an ultrasound image analysis device according to an embodiment of the present disclosure.

FIG. 3 is a diagram illustrating an example for explaining learning data of an ultrasound image analysis device according to an embodiment of the present disclosure.

FIG. 4 is a diagram illustrating an example for explaining a first prediction model of an ultrasound image analysis device according to an embodiment of the present disclosure.

FIG. 5 is a diagram illustrating an example for explaining a second prediction model of an ultrasound image analysis device according to an embodiment of the present disclosure.

FIG. 6 is a diagram illustrating an example for explaining the performance of a first prediction model in an ultrasound image analysis device according to an embodiment of the present disclosure.

FIG. 7 is a diagram illustrating an example for explaining the performance of a second prediction model in an ultrasound image analysis device according to an embodiment of the present disclosure.

8 is a flowchart of an ultrasound image analysis method according to an embodiment of the present disclosure.

Figure 9 is a block diagram of an ultrasound image analysis device according to an embodiment of the present disclosure.

This disclosure relates to an ultrasound image analysis device and method.

Hereinafter, some embodiments of the present disclosure will be described in detail through illustrative drawings. When adding reference numerals to components in each drawing, it should be noted that identical components are given the same reference numerals as much as possible even if they are shown in different drawings. Additionally, in describing the present disclosure, if it is determined that a detailed description of a related known configuration or function may obscure the gist of the present disclosure, the detailed description will be omitted.

Additionally, in describing the components of the present disclosure, terms such as first, second, A, B, (a), and (b) may be used. These terms are only used to distinguish the component from other components, and the nature, sequence, or order of the component is not limited by the term. When a component is described as being “connected,” “coupled,” or “connected” to another component, that component may be directly connected or connected to that other component, but there is another component between each component. It will be understood that elements may be “connected,” “combined,” or “connected.”

Hereinafter, the present disclosure will be described in detail with reference to the attached drawings.

Referring to FIG. 1, the ultrasound image analysis device 100 according to an embodiment of the present disclosure acquires ultrasound image information from transesophageal echocardiography information and acquires plaque information included in the blood vessel in the ultrasound image information. A learning data generator 110 that generates learning data by labeling, a first prediction model that performs learning based on the generated learning data to classify the plaque state from ultrasound image information, and a second that extracts the plaque area. Ultrasound image analysis comprising a model generator 120 that generates a prediction model and a plaque prediction unit 130 that predicts the plaque state and plaque area from specific ultrasound image information by applying each generated prediction model. A device 100 is provided.

The learning data generator 110 according to one embodiment may generate learning data by labeling ultrasound image information with plaque information. As an example, the learning data generator 110 may acquire ultrasound image information from transesophageal echocardiography information and generate learning data by labeling plaque information included in blood vessels in the ultrasound image information. there is. Ultrasound image information is the original image included in the information of Transesophageal Echocardiography (TEE), which can take the closest image of the aortic wall of aortic atherosclerotic plaque in patients with cerebral infarction or transient ischemic attack. It could be information. Additionally, the plaque information may be information including information about the plaque state or plaque area. For example, the learning data generator 110 may classify ultrasound image information into preset plaque states and label the plaque area in the ultrasound image information by displaying it in a specific color to contrast with the surrounding area. Here, the plaque state can be preset into two or three categories depending on the plaque shape and thickness. Specifically, the learning data generator 110 divides the plaque state into two categories: complex plaque and non-complex plaque, or three categories: normal, simple plaque, and complex plaque. They can be labeled separately. That is, the learning data generator 110 may label the ultrasound image information as at least one of the preset plaque states: normal, simple plaque, and complex plaque.

For another example, the learning data generator 110 may change the size of the ultrasound image information to a preset pixel and generate learning data by matching the ultrasound image information with label image information labeled with plaque information. Specifically, the learning data generator 110 may change the size of the ultrasound image information to 512 pixels horizontally and vertically, respectively, and generate image information matching the original ultrasound image information and the label image information as learning data. Additionally, the learning data can be divided into a training dataset, a validation dataset, and a validation dataset and used for learning.

The model generator 120 according to one embodiment may generate a first prediction model and a second prediction model that predict plaque information from ultrasound image information. As an example, the model generator 120 may perform learning based on the generated training data to generate a first prediction model that classifies the plaque state from ultrasound image information and a second prediction model that extracts the plaque area. For example, the model generator 120 may calculate the probability of each plaque state from ultrasound image information as a probability and generate a first prediction model that predicts the state predicted with the highest probability as the final plaque state. . The first prediction model consists of a structure in which a plurality of convolution layers, a flatten layer, and a dense layer are sequentially connected, and a softmax function (Softmax) is applied to the result output from the dense layer. function) can be applied to classify the plaque condition.

For another example, the model generator 120 may generate a second prediction model that extracts the plaque area from ultrasound image information. The second prediction model consists of a reduced encoding area and an enlarged decoding area based on a multiple convolutional layer structure, and the plaque area is created by mapping the feature information in the encoding area and concatenating the output result to the decoding area. It can be extracted. Specifically, the model generator 120 may generate a segmentation map by connecting the results output from each layer of the encoding area included in the second prediction model to the input of the same layer in the decoding area. Accordingly, the plaque area can be specifically extracted on a pixel basis from the generated segmentation map.

For another example, the model generator 120 may perform learning of a first prediction model and a second prediction model based on the generated training data. The model generator 120 divides the number of learning data into a training dataset, validation dataset, and test dataset according to a preset ratio to learn each prediction model. It can be done. Here, the preset ratio may be 6:2:2, but is not limited thereto.

The plaque prediction unit 130 according to one embodiment may predict the plaque state and plaque area from specific ultrasound image information by applying each generated prediction model. For example, the plaque prediction unit 130 calculates the probability of each plaque state by applying the first prediction model with specific ultrasound image information as input, and selects the plaque state with the highest probability among each plaque state as the final plaque state. It can be predicted by state. In addition, the plaque prediction unit 130 generates a segmentation map that minimizes the loss rate by applying a second prediction model to specific ultrasound image information as input, displays the plaque area extracted from the map, and outputs it as predicted image information. there is. In the predicted image information, the plaque area may be displayed in a specific color to contrast with the surrounding area.

Referring to FIG. 2, a process in which the ultrasound image analysis device 100 according to an embodiment of the present disclosure generates each prediction model and predicts the plaque state and plaque area through this can be explained. As an example, the learning data generator 110 may label ultrasound image information with plaque information (S210). For example, the learning data generator 110 may use the original ultrasound image information obtained from the transesophageal echocardiography information of a patient with cerebral infarction or transient ischemic attack and the label image information in which the plaque area is labeled in the ultrasound image information as learning data. You can. This image information may be image information in the form of Digital Imaging and Communications in Medicine (DICOM). Specifically, the learning data generator 110 may generate learning data by classifying original ultrasound image information for each plaque state for the first prediction model. Here, the plaque state can be classified into two preset types (No complex (0), Complex plaque (1)) or three types (Normal (0), Simple plaque (1), Complex plaque (2)). . Additionally, the learning data generator 110 may change the size of the ultrasound image information to a preset pixel and generate learning data using image information that matches the original ultrasound image information and label image information.

As an example, the model generator 120 may generate a first prediction model that classifies the plaque condition (S220). For example, the model generator 120 is a first prediction model that extracts features by repeating compression and decompression through an encoder and decoder to classify the plaque state using the original ultrasound image information of the aorta. can be created. In addition, the model generator 120 divides the number of learning data into a training data set, a validation data set, and a test data set according to a preset ratio to determine the number of the first prediction model. Learning can be done. Here, the ratio can be set to a value that can increase the accuracy of plaque status classification. Specifically, assuming that the preset ratio is 6:2:2 and the total number of training data is 1,536, the training data set, validation data set, and validation data set can be determined to be 922, 307, and 307, respectively. Additionally, the first prediction model can be trained in Google Colab's GPU environment. Details regarding the first prediction model will be described later with reference to FIG. 4 .

As an example, the model generator 120 may generate a second prediction model that extracts the plaque area (S230). For example, the model generator 120 may generate a segmentation map using original ultrasound image information of the aorta and create a second prediction model that extracts the plaque area on a pixel basis. In addition, the model generator 120 divides the number of learning data into a training dataset, a validation dataset, and a test dataset according to a preset ratio to form a second prediction model. Learning can be done. Here, the ratio can be set to a value that can increase the accuracy of the plaque area. Specifically, assuming that the preset ratio is 6:2:2 and that the total number of learning data matching the ultrasound original image information and label image information is 300, the training data set, validation data set, and verification data set are 180, respectively. It can be determined as 1, 60, and 60. Additionally, the second prediction model can be designed with a learning rate of 10 ^-3 , a batch size of 4, and an epoch of 100. The second prediction model can be trained in Google Colab to speed up the UNET process. Details regarding the second prediction model will be described later with reference to FIG. 5.

As an example, the plaque prediction unit 130 can predict the plaque state and plaque area from specific ultrasound image information (S240). For example, the plaque prediction unit 130 may predict the plaque state by calculating the probability of each plaque state by applying a first prediction model to specific ultrasound image information as input. Additionally, the plaque prediction unit 130 may predict the plaque area by using specific ultrasound image information as input and extracting the plaque area by applying a second prediction model.

Referring to FIG. 3, the learning data generator 110 of the ultrasound image analysis device according to an embodiment of the present disclosure may acquire ultrasound image information and generate learning data. As an example, the learning data generator 110 may classify ultrasound image information into preset plaque states and display the plaque area in the ultrasound image information in a specific color to contrast with the surrounding area to generate label image information. Additionally, the learning data generator 110 may generate learning data by matching ultrasound image information and label image information. For example, the learning data generator 110 may classify ultrasound image information by each plaque state (Normal, Simple plaque, Complex) and file type (Dicom, Mark, Nomark, Label) from transesophageal echocardiography information. You can. Additionally, the learning data generator 110 may generate learning data by labeling the original ultrasound image information of the aorta with the plaque status (0, 1, 2) and plaque area determined by the medical staff. In particular, image information matching the original ultrasound image information and label image information can be used as learning data for the second prediction model.

Referring to FIG. 4 , it can be explained how the model generator 120 of the ultrasound image analysis device generates the first prediction model. As an example, the model generator 120 is composed of a structure in which a plurality of convolution layers 410, a platen layer 420, and a dense layer 430 are sequentially connected, and the result output from the dense layer 430 The softmax function 440 can be applied to generate a first prediction model that classifies the plaque condition. For example, the first prediction model may use original ultrasound image information as input and learn a plurality of convolution layers 410 by approximating the output to the input. Additionally, the first prediction model may extract weights from the learned convolution layer 410. Here, the learned convolution layer may consist of four convolution layers, but is not limited to this. And, the first prediction model is sequentially applied to a flatten layer 420 that changes the output information of the convolution layer 410 to one dimension and a dense layer 430 that changes it back to n dimension. It can be applied. In addition, the first prediction model can input the n-dimensional vector, which is the output information of the dense layer 430, into the Softmax function 440 to estimate the probability of belonging to each class and classify the plaque state. The output of the softmax function 440 is a vector with dimensions equal to the number of plaque states to be classified, and the probability of each specific plaque state can be estimated as a value between 0 and 1. Here, the first prediction model may be implemented as a convolutional auto encoder model, but is not limited to this as long as it is an artificial neural network algorithm useful for learning spatial features or short temporal features.

Referring to FIG. 5 , it can be explained how the model generator 120 of the ultrasound image analysis device generates the second prediction model. As an example, the model generator 120 consists of a reduced encoding area and an enlarged decoding area based on a plurality of convolutional layer structures, and maps feature information in the encoding area and concatenates the output result to the decoding area. ) to create a second prediction model that extracts the plaque area. For example, the second prediction model can automatically extract the plaque area by segmenting ultrasound image information and labeling it on a pixel basis. Specifically, the second prediction model can be classified into an encoding area where the image is reduced on the left and a decoding area on the right where the image is enlarged based on the symmetrical center. The decoding area of the second prediction model may be configured as an up-sampling area by removing the pooling layer and adding an up-convolution filter instead. And, when each encoding area and decoding area are in a horizontal relationship, the second prediction model can concatenate the result output from the encoding area with the decoding area. This connection process is performed by overlapping two pieces of image information based on the channel dimension, and a segmentation map can be created. And, during this process, the second prediction model can specify values at the same location on a pixel basis and extract them as a plaque area. Here, the second prediction model may be implemented using U-Net, an image segmentation technique.

Additionally, the model generator 120 may calculate the number of specific pixels corresponding to specific RGB values in the plaque area output from the second prediction model and extract the number as the area of the plaque area. Additionally, the model generator 120 may calculate the plaque ratio through the ratio of the area of the extracted plaque area to the blood vessel area calculated by the number of pixels for the area displayed with a specific RGB value.

Referring to FIG. 6, the performance of the first prediction model for predicting the plaque state as described above can be explained. As an example, a receiver operating characteristic (ROC) curve 610 is displayed to determine the performance of the first prediction model in predicting the plaque status of the aorta in two types ((no complex(0), complex plaque(1))). In addition, it can be confirmed that the model's performance is superior as the AUC (area under the curve) value of the ROC curve is closer to 1. For example, the first prediction model is an angle for aortic plaque. The AUC for the class can be analyzed as 0.776 and 0.775, and the ACC, which is the accuracy of the ROC curve 610, can be analyzed as 0.88. That is, the AUC, which represents the classification performance of the first prediction model, is about It may be measured lower than the ACC accuracy of 0.77.

As another example, an ROC curve (620) is displayed to determine the performance of the first prediction model in predicting the aortic plaque status in three types (normal (0), simple plaque (1), complex plaque (2)). You can. For example, the second prediction model may measure AUC of 0.891, 0.894, and 0.885 for each class for aortic plaque, which is higher than the accuracy ACC of 0.724. Accordingly, when the first prediction model classified plaque status into two types, accuracy (ACC) was higher than AUC, and when it classified plaque status into three types, accuracy (ACC) was lower than the AUC value. can confirm.

As another example, the performance of the first prediction model in predicting the plaque status of the aorta in two forms ((no complex(0), complex plaque(1))) can be expressed in terms of precision, recall, and F1-score. For example, For example, the first prediction model can be analyzed as having a precision of 0.90 for the first indicator, the result predicting that it is not a complex plaque (class 0), and the precision of the result predicting it being a complex plaque (class 1) is 0.65. This means that 90% of the data predicted by the first prediction model to be not complex plaque (class 0) matched the actual results, and 65% of the data predicted to be complex plaque (class 1) matched the actual results. For another example, the first prediction model may analyze the second indicator, recall, as 0.96 and 0.43, respectively, which means that the first prediction model is not actually a complex plaque (class 0). This can mean that among the data, the proportion predicted as not complex plaque (class 0) is 96%, and among the data that is actually complex plaque (class 1), the proportion predicted as complex plaque (class 1) is 43%. Another For example, in the first prediction model, the F1-score, which is the weighted harmonic average of precision and recall, is 0.93 and 0.51 for each class, and the accuracy of predicting aortic plaque status in the two classes is predicted to be about 88%. Accordingly, the prediction performance of the first prediction model may be shown as Table 1.

Referring to FIG. 7, the performance of the second prediction model for predicting the plaque area can be explained as described above. As an example, the second prediction model may extract a plaque area caused by aortic atherosclerosis as a cause of cerebral infarction from ultrasound image information. For example, Figure 7 may be a result showing an unlabeled input image for each case according to plaque status, a labeled image indicating a plaque area in the input image, and an output image resulting from learning of the second prediction model. The second prediction model connects and learns the input image information and the label image information marked by the medical staff as the plaque area for each case to output predicted image information indicating the plaque area. Additionally, as a result of the test, the final loss rate of the second prediction model can be calculated as 0.1942.

As another example, the performance of the second prediction model in predicting the aortic plaque status in three types (normal (0), simple plaque (1), complex plaque (2)) can be expressed in terms of precision, recall, and F1-score. there is. For example, the first prediction model predicted an accuracy of about 72% when classifying plaques into the first branch type, which was lower than the accuracy of about 88% when classifying plaques into two types. Accordingly, the prediction performance of the second prediction model can be shown as Table 2.

Hereinafter, an ultrasonic image analysis method that can be performed by the ultrasonic image analysis device described with reference to FIGS. 1 to 7 will be described.

Referring to FIG. 8, the ultrasound image analysis method according to an embodiment of the present disclosure may include a learning data generation step of generating training data by labeling ultrasound image information with plaque information (S810). As an example, the ultrasound image analysis device may acquire ultrasound image information from transesophageal echocardiography information and generate learning data by labeling plaque information included in blood vessels in the ultrasound image information. Ultrasound image information is the original image included in the information of Transesophageal Echocardiography (TEE), which can take the closest image of the aortic wall of aortic atherosclerotic plaque in patients with cerebral infarction or transient ischemic attack. It could be information. For example, an ultrasound image analysis device can classify ultrasound image information into preset plaque states and label the plaque area in the acoustic image information by displaying it in a specific color to contrast with the surrounding area. Specifically, the ultrasound image analysis device can classify and label the plaque status into two categories: complex plaque and non-complex plaque, depending on the plaque shape and thickness. Alternatively, the ultrasound image analysis device can label the plaques into three categories: normal, simple plaque, and complex plaque.

For another example, the ultrasound image analysis device may change the size of ultrasound image information to a preset pixel and generate learning data by matching ultrasound image information with label image information labeled with plaque information. Specifically, the ultrasound image analysis device can change the size of the ultrasound image information to 512 pixels horizontally and vertically, respectively, and generate only image information matching the original ultrasound image information and the label image information as learning data.

The ultrasound image analysis method according to one embodiment may include a model generation step of generating a first prediction model and a second prediction model for predicting plaque information from ultrasound image information (S820). As an example, the ultrasound image analysis device may perform learning based on the generated training data to generate a first prediction model that classifies the plaque state and a second prediction model that extracts the plaque area from ultrasound image information. For example, the ultrasound image analysis device may calculate the probability of each plaque state from ultrasound image information as a probability and generate a first prediction model that predicts the state predicted with the highest probability as the final plaque state. The first prediction model consists of a structure in which a plurality of convolution layers, a flatten layer, and a dense layer are sequentially connected, and a softmax function (Softmax) is applied to the result output from the dense layer. function) can be applied to classify the plaque condition.

For another example, the ultrasound image analysis device may generate a second prediction model that extracts the plaque area from ultrasound image information. The second prediction model consists of a reduced encoding area and an enlarged decoding area based on a multiple convolutional layer structure, and the plaque area is created by mapping the feature information in the encoding area and concatenating the output result to the decoding area. It can be extracted.

As another example, the ultrasound image analysis device may perform learning of a first prediction model and a second prediction model based on generated learning data. The ultrasound image analysis device divides the number of learning data into a training dataset, validation dataset, and test dataset according to a preset ratio and performs learning of each prediction model. You can. Here, the preset ratio may be 6:2:2, but is not limited thereto.

The ultrasound image analysis method according to one embodiment may include a plaque prediction step of predicting the plaque state and plaque area from specific ultrasound image information by applying each generated prediction model (S830). For example, the ultrasound image analysis device calculates the probability of each plaque state by applying a first prediction model to specific ultrasound image information as input, and selects the plaque state with the highest probability among each plaque state as the final plaque state. It is predictable. Additionally, the ultrasound image analysis device may generate a segmentation map that minimizes the loss rate by applying a second prediction model to specific ultrasound image information as input, display the plaque area extracted from the map, and output it as predicted image information.

Referring to FIG. 9, the ultrasound image analysis device 100 according to one embodiment includes a communication interface 910 and a processor 920. The ultrasound image analysis device 100 may further include a memory 930. Each component, communication interface 910, processor 920, and memory 930 may be connected to each other through a communication bus. For example, a communication bus may include circuitry that connects components to each other and carries communications (e.g., control messages and/or data) between the components.

The communication interface 910 is a transesophageal echocardiography device that can acquire ultrasound image information of the carotid artery of a cardiovascular patient. Additionally, the communication interface 910 can perform communication with an external device through wireless or wired communication.

The processor 920 may perform at least one method or an algorithm corresponding to at least one method described above with reference to FIGS. 1 to 8 . The processor 920 may be a data processing device implemented in hardware that has a circuit with a physical structure for executing desired operations. For example, the intended operations may include code or instructions included in the program. For example, data processing devices implemented in hardware include microprocessors, central processing units, processor cores, multi-core processors, and multiprocessors. , may include a Neural Processing Unit (NPU), an Application-Specific Integrated Circuit (ASIC), and a Field Programmable Gate Array (FPGA).

Additionally, the processor 920 may execute a program and control the ultrasound image analysis device 100. Program code executed by the processor 920 may be stored in the memory 930.

Information about the neural network model according to one embodiment may be stored in the internal memory of the processor 920 or in external memory, that is, memory 930. For example, the memory 930 may store ultrasound image information of a cardiovascular patient obtained through the communication interface 910. Memory 930 may store artificial intelligence models including neural network models. Additionally, the memory 930 may store various information generated during processing of the processor 920 and output information extracted by the processor 920. The output information may be a neural network calculation result or a neural network test result. The memory 930 may store neural network learning results. The neural network learning results may be obtained from the ultrasound image analysis device 100 or may be obtained from an external device. The learning results of the neural network model may include weight and bias values. In addition, the memory 930 can store various data and programs. Memory 930 may include volatile memory or non-volatile memory. The memory 930 may be equipped with a high-capacity storage medium such as a hard disk to store various data.

The above-described ultrasound image analysis device 100 may be implemented in the form of a server or PC. Additionally, the ultrasound image analysis device 100 may be implemented as a portable mobile device.

In the above, even though all the components constituting the embodiments of the present disclosure have been described as being combined or operated in combination, the present disclosure is not necessarily limited to these embodiments. That is, within the scope of the purpose of the present disclosure, all of the components may be operated by selectively combining one or more of them. In addition, although all of the components may be implemented as a single independent hardware, a program module in which some or all of the components are selectively combined to perform some or all of the functions of one or more pieces of hardware. It may be implemented as a computer program having and recorded on a computer-readable medium. The computer-readable medium may include program instructions, data files, data structures, etc., singly or in combination. Program instructions recorded on the medium may be specially designed and configured for the embodiment or may be known and available to those skilled in the art of computer software. Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks, and magnetic tapes, optical media such as CD-ROMs and DVDs, and magnetic media such as floptical disks. -Includes optical media (magneto-optical media) and hardware devices specifically configured to store and execute program instructions, such as ROM, RAM, flash memory, etc. Examples of program instructions include machine language code, such as that produced by a compiler, as well as high-level language code that can be executed by a computer using an interpreter, etc. The hardware devices described above may be configured to operate as one or more software modules to perform the operations of the embodiments, and vice versa.

In addition, terms such as “include,” “comprise,” or “have” as used above mean that the corresponding component may be included, unless specifically stated to the contrary, and thus do not exclude other components. Rather, it should be interpreted as being able to include other components. All terms, including technical or scientific terms, unless otherwise defined, have the same meaning as commonly understood by a person of ordinary skill in the technical field to which this disclosure pertains. Commonly used terms, such as terms defined in a dictionary, should be interpreted as consistent with the context meaning of the related technology, and should not be interpreted in an idealized or overly formal sense unless explicitly defined in the present disclosure.

The above description is merely an illustrative explanation of the technical idea of the present disclosure, and those skilled in the art will be able to make various modifications and variations without departing from the essential characteristics of the present disclosure. Accordingly, the embodiments disclosed in the present disclosure are not intended to limit the technical idea of the present disclosure but are for illustrative purposes, and the scope of the technical idea of the present disclosure is not limited by these embodiments. The scope of protection of this disclosure should be interpreted in accordance with the claims below, and all technical ideas within the equivalent scope should be interpreted as being included in the scope of rights of this disclosure.

CROSS-REFERENCE TO RELATED APPLICATIONCROSS-REFERENCE TO RELATED APPLICATION

This patent application claims priority under Article 119(a) of the U.S. Patent Act (35 U.S.C § 119(a)) to Patent Application No. 10-2022-0053044 filed in Korea on April 28, 2022. All contents are hereby incorporated by reference into this patent application. In addition, if this patent application claims priority for a country other than the United States for the same reasons as above, the entire contents thereof will be incorporated into this patent application by reference.

Claims

a learning data generator that acquires ultrasound image information from transesophageal echocardiography information and generates learning data by labeling plaque information contained in blood vessels in the ultrasound image information;

a model generator that performs learning based on the generated training data to generate a first prediction model for classifying a plaque state and a second prediction model for extracting a plaque area from the ultrasound image information; and

An ultrasound image analysis device comprising a plaque prediction unit that predicts the plaque state and plaque area from specific ultrasound image information by applying each generated prediction model.
According to claim 1,

The learning data generator,

An ultrasound image analysis device that classifies the ultrasound image information into preset plaque states and labels the plaque area in the ultrasound image information by displaying it in a specific color to contrast with the surrounding area.
According to claim 1,

The learning data generator,

An ultrasound image analysis device that changes the size of the ultrasound image information to a preset pixel and generates learning data by matching the ultrasound image information with label image information labeled with plaque information.
According to claim 1,

The plaque condition is,

An ultrasound image analysis device characterized in that it is in at least one of normal, simple plaque, and complex plaque states.
According to claim 1,

The first prediction model is,

It consists of a structure in which a plurality of convolution layers, a flatten layer, and a dense layer are sequentially connected, and a softmax function is applied to the results output from the dense layer. An ultrasound image analysis device characterized by classifying the plaque condition.
According to claim 1,

The second prediction model is,

It consists of an encoding area that is reduced and a decoding area that is enlarged based on a plurality of convolutional layer structures, and the result output by mapping feature information in the encoding area is concatenated to the decoding area to extract the plaque area. Ultrasound image analysis device, characterized in that.
According to claim 1,

The model creation unit,

An ultrasound image analysis device characterized in that learning is performed by dividing the number of learning data into a training dataset, validation dataset, and test dataset according to a preset ratio.
A learning data generation step of obtaining ultrasound image information from transesophageal echocardiography information and generating learning data by labeling plaque information included in blood vessels in the ultrasound image information;

A model generation step of performing learning based on the generated training data to generate a first prediction model for classifying a plaque state and a second prediction model for extracting a plaque area from the ultrasound image information; and

An ultrasound image analysis method comprising a plaque prediction step of predicting the plaque state and plaque area from specific ultrasound image information by applying each generated prediction model.
According to claim 8,

The learning data generation step is,

An ultrasound image analysis method characterized by classifying the ultrasound image information into preset plaque states and labeling the plaque area in the ultrasound image information by marking it with a specific color to contrast with the surrounding area.
According to claim 8,

The learning data generation step is,

An ultrasound image analysis method characterized by changing the size of the ultrasound image information to a preset pixel and generating learning data by matching the ultrasound image information with label image information labeled with plaque information.
According to claim 8,

The plaque condition is,

An ultrasound image analysis method, characterized in that the state is at least one of normal, simple plaque, and complex plaque.
According to claim 8,

The first prediction model is,

It consists of a structure in which a plurality of convolution layers, a flatten layer, and a dense layer are sequentially connected, and a softmax function is applied to the results output from the dense layer. An ultrasound image analysis method characterized by classifying the plaque condition.
According to claim 8,

The second prediction model is,

It consists of an encoding area that is reduced and a decoding area that is enlarged based on a plurality of convolutional layer structures, and the result output by mapping feature information in the encoding area is concatenated to the decoding area to extract the plaque area. Ultrasound image analysis method, characterized in that.
According to claim 8,

The model creation step is,

An ultrasound image analysis method, characterized in that learning is performed by dividing the number of learning data into a training dataset, validation dataset, and test dataset according to a preset ratio.