WO2023195741A1 - Method and device for acquiring information from medical image expressing time-varying information - Google Patents

Abstract

The objective of the present invention is to acquire information from a medical image expressing time-varying information, and a method for acquiring information from a medical image may comprise the steps of: performing segmentation on one or more signals in one or more medical images; determining one or more measurement values by signal on the basis of an envelope of the one or more signals acquired through the segmentation; confirming one or more true signals from among the one or more signals; and determining one or more final measurement values on the basis of one or more measurement values for the one or more true signals.

Description

Method and device for obtaining information from medical images expressing time-varying information

The present invention relates to the processing of medical images, and in particular, to a method and device for obtaining information from medical images expressing time-varying information.

Disease refers to a condition that causes disorders in the human mind and body, impeding normal functioning. Depending on the disease, a person may suffer and may even be unable to sustain his or her life. Accordingly, various social systems and technologies for diagnosing, treating, and even preventing diseases have developed along with human history. In the diagnosis and treatment of diseases, various tools and methods have been developed in accordance with the remarkable advancement of technology, but the reality is that they are still ultimately dependent on the judgment of doctors.

Meanwhile, artificial intelligence (AI) technology has recently developed significantly and is attracting attention in various fields. In particular, due to the environment of vast amounts of accumulated medical data and image-oriented diagnostic data, various attempts and research are underway to apply artificial intelligence algorithms to the medical field. Specifically, various studies are being conducted to use artificial intelligence algorithms to solve tasks that have traditionally been limited to clinical judgment, such as diagnosing and predicting diseases. In addition, various studies are being conducted to solve the task of processing and analyzing medical data as an intermediate process for diagnosis, etc. using artificial intelligence algorithms.

The present invention is intended to provide a method and device for effectively obtaining information from medical images using an artificial intelligence (AI) algorithm.

The present invention is intended to provide a method and device for extracting information in real time from medical images expressing time-varying information.

The present invention is intended to provide a method and device for extracting and analyzing signals of each pattern from medical images with repetitive patterns.

The technical problems to be achieved in the present invention are not limited to the technical problems mentioned above, and other technical problems not mentioned will be clearly understood by those skilled in the art from the description below. You will be able to.

According to an embodiment of the present invention, a method for obtaining information from a medical image includes performing segmentation on at least one signal in at least one medical image, the at least one signal obtained through the segmentation. determining at least one measurement value for each signal based on an envelope, confirming at least one true signal among the at least one signals, and at least one measurement value for the at least one true signal. It may include determining at least one final measurement value based on the method.

According to an embodiment of the present invention, the at least one medical image may represent at least one medical image that represents a result of arranging time-varying information on the time axis.

According to one embodiment of the present invention, the at least one medical image may include at least one Doppler echocardiography image.

According to an embodiment of the present invention, the at least one measurement value is the maximum velocity value of blood flow, velocity time integral (VTI), deceleration time (DT), pressure half time (PHT), acceleration time (AT), and EDV. (end-diastolic velocity), DT(deceleration time), dP/dt(the rate of pressure change using the 4V2 formula over time during isovolumic contraction), S'sept(septal)(peak systolic mitral annular velocity at the septal part of mitral annulus), E'sept(peak early diastolic mitral annular velocity at the septal part of mitral annulus), A'sept(peak late diastolic mitral annular velocity at the septal part of mitral annulus), S'lat(lateral)(peak systolic mitral annular velocity at the lateral part of mitral annulus), E'lat(peak early diastolic mitral annular velocity at the lateral part of mitral annulus), A'lat(peak late diastolic mitral annular velocity at the lateral part of mitral annulus) It may include at least one of:

According to an embodiment of the present invention, the at least one true signal includes probability information for each pixel generated for the segmentation, the length of the envelope, the area of the area specified by the envelope, and the area specified by the envelope. It can be identified based on at least one of the shapes of .

According to an embodiment of the present invention, the at least one true signal may be identified based on the distribution of entropy values generated for the segmentation.

According to an embodiment of the present invention, the step of checking the at least one true signal may include a step of checking entropy values generated in the step of performing the segmentation.

According to one embodiment of the present invention, the segmentation may be performed based on multi-scale pyramid representations.

According to one embodiment of the present invention, the method includes extracting an electrocardiogram (ECG) signal from the at least one medical image, and at least one other signal based on the ECG signal and the at least one final measurement value. The step of determining the measured value may be further included.

According to one embodiment of the present invention, the at least one signal includes a first signal related to a first value and a second signal related to a second value, and the first signal and the second signal are electrocardiogram signals. may be classified based on, or may be classified based on the pattern of the at least one signal.

According to one embodiment of the present invention, the method includes grouping the at least one signal into pairs including two consecutive signals on the time axis, and the width of signals included in each of the pairs ( A step of classifying the signals based on width may be further included.

According to an embodiment of the present invention, a device for obtaining information from a medical image includes a storage unit that stores a set of instructions for operating the device, and at least one processor connected to the storage unit, wherein the at least One processor performs segmentation on at least one signal in at least one medical image, and determines at least one measurement value for each signal based on an envelope of the at least one signal obtained through the segmentation, Control may be performed to confirm at least one true signal among the at least one signals and determine at least one final measurement value based on at least one measurement value for the at least one true signal.

According to an embodiment of the present invention, a program stored in a medium can execute the above-described method when operated by a processor.

The features briefly summarized above with respect to the present invention are merely exemplary aspects of the detailed description of the present invention that follows, and do not limit the scope of the present invention.

According to the present invention, information can be effectively obtained from medical images using an artificial intelligence (AI) algorithm.

The effects that can be obtained from the present invention are not limited to the effects mentioned above, and other effects not mentioned can be clearly understood by those skilled in the art from the description below. will be.

1 shows a system according to one embodiment of the present invention.

Figure 2 shows the structure of a device according to an embodiment of the present invention.

Figure 3 shows an example of a perceptron constituting an artificial intelligence model applicable to the present invention.

Figure 4 shows an example of an artificial neural network constituting an artificial intelligence model applicable to the present invention.

Figures 5A to 5F show an example of an image processing process according to an embodiment of the present invention.

Figure 6 shows an example of the structure of an artificial intelligence model according to an embodiment of the present invention.

Figure 7 shows an example of the structure of an adaptive context module (ACM) in an artificial intelligence model according to an embodiment of the present invention.

Figure 8 shows an example of a procedure for obtaining information from a medical image according to an embodiment of the present invention.

Figure 9 shows an example of a procedure for classifying true signals and false signals according to an embodiment of the present invention.

FIGS. 10A to 10D show examples of information obtainable from a MV (Mitral Valve) inflow PW (pulsed-wave) view according to an embodiment of the present invention.

11A to 11D show examples of information obtainable from a TDI (tissue Doppler imaging) view according to an embodiment of the present invention.

Figures 12A to 12D show examples of information obtainable in an aortic regurgitation (AR) pressure half time (PHT) view according to an embodiment of the present invention.

FIGS. 13A to 13D show examples of information obtainable from a mitral valve (MS) PHT view according to an embodiment of the present invention.

Figures 14a and 14b show examples of information obtainable in a pulmonic valve (PV) PW view according to an embodiment of the present invention.

Figures 15a and 15b show examples of information obtainable in a mitral regurgitation (MR) PW view according to an embodiment of the present invention.

Figure 16 shows an example of a procedure for obtaining information from a medical image using electrocardiogram (ECG) extraction and segmentation according to an embodiment of the present invention.

Figure 17 shows an example of a medical image including an electrocardiogram according to an embodiment of the present invention.

Figure 18 shows examples of graphs representing electrocardiogram signals according to an embodiment of the present invention.

Figure 19 shows an example of a procedure for obtaining information from a medical image considering the presence or absence of an electrocardiogram signal according to an embodiment of the present invention.

Figure 20 shows an example of a medical image of a PW view of the MV inlet in a normal state according to an embodiment of the present invention.

Figure 21 shows an example of a two-path split network configuration, which is one of the semantic segmentation models applicable to the present invention.

Figure 22 shows an example of a method for aggregating output generated from detailed branches and semantic branches applicable to the present invention.

Figure 23 shows an example of the configuration of a three-path partition network, one of the semantic segmentation models according to an embodiment of the present invention.

Hereinafter, with reference to the attached drawings, embodiments of the present invention will be described in detail so that those skilled in the art can easily implement the present invention. However, the present invention may be implemented in many different forms and is not limited to the embodiments described herein.

In describing embodiments of the present invention, if it is determined that a detailed description of a known configuration or function may obscure the gist of the present invention, the detailed description thereof will be omitted. In addition, in the drawings, parts that are not related to the description of the present invention are omitted, and similar parts are given similar reference numerals.

1 shows a system according to one embodiment of the present invention.

Referring to FIG. 1, the system includes a service server 110, a data server 120, and at least one client device 130.

The service server 110 provides services based on artificial intelligence models. That is, the service server 110 performs learning and prediction operations using an artificial intelligence model. The service server 110 may communicate with the data server 120 or at least one client device 130 through a network. For example, the service server 110 may receive learning data for training an artificial intelligence model from the data server 120 and perform training. The service server 110 may receive data necessary for learning and prediction operations from at least one client device 130. Additionally, the service server 110 may transmit information about the prediction result to at least one client device 130.

The data server 120 provides learning data for training the artificial intelligence model stored in the service server 110. According to various embodiments, the data server 120 may provide public data that anyone can access or provide data that requires permission. If necessary, the learning data may be preprocessed by the data server 120 or the service server 120. According to another embodiment, the data server 120 may be omitted. In this case, the service server 110 may use an externally trained artificial intelligence model, or learning data may be provided to the service server 110 offline.

At least one client device 130 transmits and receives data related to an artificial intelligence model operated by the service server 110 with the service server 110. At least one client device 130 is equipment used by the user, and transmits information input by the user to the service server 110, and stores the information received from the service server 110 or provides it to the user (e.g. : mark) is possible. In some cases, a prediction operation may be performed based on data transmitted from one client, and information related to the result of the prediction may be provided to another client. At least one client device 130 may be various types of computing devices, such as desktop computers, laptop computers, smartphones, tablets, and wearable devices.

Although not shown in FIG. 1, the system may further include a management device for managing the service server 110. The management device is a device used by the entity that manages the service, and monitors the status of the service server 110 or controls settings of the service server 110. The management device may be connected to the service server 110 through a network or directly through a cable connection. According to the control of the management device, the service server 110 can set parameters for operation.

As described with reference to FIG. 1, the service server 110, the data server 120, at least one client device 130, a management device, etc. may be connected and interact through a network. Here, the network may include at least one of a wired network and a wireless network, and may be comprised of any one or a combination of two or more of a cellular network, a local area network, and a wide area network. For example, the network is based on at least one of LAN (local area network), WLAN (wireless LAN), Bluetooth, long term evolution (LTE), LTE-advanced (LTE-A), and 5th generation (5G). This can be implemented.

Figure 2 shows the structure of a device according to an embodiment of the present invention. The structure illustrated in FIG. 2 may be understood as the structure of the service server 110, data server 120, and at least one client device 130 of FIG. 1.

Referring to FIG. 2, the device includes a communication unit 210, a storage unit 220, and a control unit 230.

The communication unit 210 performs functions to connect to the network and communicate with other devices. The communication unit 210 may support at least one of wired communication and wireless communication. For communication, the communication unit 210 may include at least one of a radio frequency (RF) processing circuit and a digital data processing circuit. In some cases, the communication unit 210 may be understood as a component including a terminal for connecting a cable. Since the communication unit 210 is a component for transmitting and receiving data and signals, it may be referred to as a 'transceiver'.

The storage unit 220 stores data, programs, microcode, instruction sets, applications, etc. necessary for the operation of the device. The storage unit 220 may be implemented as a temporary or non-transitory storage medium. Additionally, the storage unit 220 may be fixed to the device or may be implemented in a detachable form. For example, the storage unit 220 may include compact flash (CF) cards, secure digital (SD) cards, memory sticks, solid-state drives (SSD), and micro It may be implemented as at least one of magnetic computer storage devices such as NAND flash memory such as an SD card and a hard disk drive (HDD).

The control unit 230 controls the overall operation of the device. To this end, the control unit 230 may include at least one processor, at least one microprocessor, etc. The control unit 230 can execute a program stored in the storage unit 220 and access the network through the communication unit 210. In particular, the control unit 230 may perform algorithms according to various embodiments described later and control the device to operate according to embodiments described later.

Based on the structure described with reference to FIGS. 1 and 2, services based on artificial intelligence algorithms can be provided according to various embodiments of the present invention. Here, an artificial intelligence model consisting of an artificial neural network can be used to implement the artificial intelligence algorithm. The concepts of perceptron, which is a structural unit of artificial neural network, and artificial neural network are as follows.

The perceptron is modeled after a biological nerve cell and has a structure that takes multiple signals as input and outputs a single signal. Figure 3 shows an example of a perceptron constituting an artificial intelligence model applicable to the present invention. Referring to Figure 3, the perceptron sets weights (302-1 to 302-n) (e.g., w _1j , w _2j , w) for each of the input values ₍ e.g., x ₁ , x 2, x ₃ , ..., x _n ). After multiplying by _3j , ..., w _nj ), the weighted input values are summed using a transfer function 304. During the summation process, a bias value (e.g., b _k ) may be added. The perceptron generates an output value (e.g., o _{j) by applying an activation function (306) to the net input value (e.g., net j )} , which is the output of the transformation function (304). In some cases, activation function 306 may operate based on a threshold (eg, θ _j ). Activation functions can be defined in various ways. The present invention is not limited to this, but for example, a step function, sigmoid, Relu, Tanh, etc. may be used as the activation function.

An artificial neural network can be designed by arranging perceptrons as shown in Figure 3 and forming layers. Figure 4 shows an example of an artificial neural network constituting an artificial intelligence model applicable to the present invention. In FIG. 4, each node represented by a circle can be understood as the perceptron of FIG. 3. Referring to FIG. 4, the artificial neural network includes an input layer 402, a plurality of hidden layers 404a and 404b, and an output layer 462.

When performing prediction, when input data is provided to each node of the input layer 402, the input data is weighted and transformed by the perceptrons that make up the input layer 402 and the hidden layers 404a and 404b. and forward propagation to the output layer 462 through activation function calculation, etc. Conversely, when training is performed, the error is calculated through backward propagation from the output layer 162 to the input layer 402, and the weight values defined in each perceptron can be updated according to the calculated error. there is.

A variety of medical images can be efficiently analyzed through AI technology. For example, Doppler echocardiography is a technology that records ultrasound images of the heart and magnifies two-dimensional echocardiography. Doppler echocardiography is a technology that measures blood velocity in the heart and great vessels and is a key technology in the evaluation of valvular heart disease and cardiac performance. am. However, to measure velocity-time-integral (VTI) and peak velocity from Doppler echocardiography, images can be acquired and then analyzed by manually tracing the Doppler envelope. A skilled technician is required.

Echocardiographers considering generating multiple measurements to obtain an average will have to define extra time expenditure or disinvest attention in other areas. Additionally, the fact that sonographers tend to select representative beats to consider as averages may contribute to the significant test-retest variability of Doppler measurements. Accordingly, the present disclosure proposes a technique for automated Doppler envelope quantification.

5A to 5F show an example of an image processing process according to an embodiment of the present invention. FIGS. 5A to 5F illustrate results obtained from each of the processing steps for analysis of a medical image representing time-varying information, according to various embodiments.

FIG. 5A shows an example of a raw image 510. In the raw image 510, the horizontal axis represents time and the vertical axis represents blood flow speed. That is, the raw image 510 lists the time-varying heart blood flow speed on the time axis. For example, when measuring blood flow velocity using Doppler ultrasound technology, the image can change in real time by adding new data and excluding past data over time, as shown in the raw image 510 of FIG. 5A. ) can be understood as capturing an image that changes in real time at a specific point in time. As shown in Figure 5a, the waveform has a certain pattern, and the pattern has periodicity. Hereinafter, a pattern with consistency is referred to as a 'signal'.

Figure 5b shows an example of a segmentation result for a signal. Referring to FIG. 5B, segmentation results 521, 522, and 523 for the three signals expressed in the image are generated. Segments 521, 522, and 523 include at least a portion of the envelope for the signals.

Figure 5c shows an example of detection results of maximum speed values 531a, 532a, 533a and time values 531b, 532b, 533b. Maximum speed values 531a, 532a, 533a and time values 531b, 532b, 533b are detected based on the segmentation results 521, 522, 523 illustrated in FIG. 5B. That is, in the segmentation results 521, 522, or 523 of each signal, the points with the largest size are selected as the maximum speed values 531a, 532a, and 533a, and the end points of each signal are selected as the maximum visual values 531b, 532b, and 533b. ) is selected as.

Figure 5d shows an example of the results of calculating various measurement values 541, 542, and 543. Referring to Figure 5d, for each of the three signals, maximum velocity (Vmax), VTI, deceleration time (DT), pressure half time (PHT), acceleration time (AT), end-diastolic velocity (EDV), dP/ Measured values 541, 542, and 543, including dt (the rate of pressure change using the 4V2 formula over time during isovolumic contraction), may be determined.

Figure 5e shows examples of entropies 551, 552, and 553 for segmentation. Entropy expresses the probability value for the segmentation area created in the process of performing segmentation. It can be understood that the clearer the entropy and the clearer the distinction from other areas, the higher the accuracy of the derived segmentation. In the case of FIG. 5E, the first entropy 551 and the second entropy 552 on the right are confirmed to be relatively clearer than the third entropy 553 on the left.

Figure 5f shows examples of classification results 561, 562, and 563 for true and false signals. True signals and false signals can be classified based on the entropies 551, 552, and 553 as shown in FIG. 5E. When segmentation is performed on a portion of the signal, clarity is lower than that of the other entropies 551 and 552, such as the third entropy 553 in FIG. 5E. Accordingly, true signals and false signals can be determined based on the sharpness or blurring of the entropy.

In the procedure described with reference to FIGS. 5A to 5F, segmentation is performed on the signal. Here, segmentation can be performed using an artificial intelligence model, and the artificial intelligence model for segmentation can be defined in various ways. According to one embodiment, the artificial intelligence model shown in FIGS. 6 and 7 below or an artificial intelligence model similar thereto may be used for segmentation of signals in an image.

Figure 6 shows an example of the structure of an artificial intelligence model according to an embodiment of the present invention. Figure 6 illustrates an artificial intelligence model that can be used to generate segmentation results like Figure 5b.

Referring to Figure 6, the artificial intelligence model includes a CNN (610), a feature map transformer (620), and multiple adaptive context modules (ACM) (630-1 to 630). -S), a concatenation unit 640, and a convolutional layer 650.

CNN 610 generates features of the input image. Here, the generated features include a dense 3-dimension convolution feature cube. A 3D convolutional feature cube (e.g., X) includes convolutional feature vectors (e.g., X _i ) that are local features at each position. For example, a 3D convolutional feature cube may have a 3D structure with a width w, a height h, and a number of channels c. For example, CNN 610 may be implemented based on ResNet or InceptionNet.

The feature map conversion unit 620 converts the feature map generated by the CNN 610 into multi-scale pyramid representations. In other words, the feature map converter 620 generates feature maps (eg, Y ^s ) with different scales by decomposing the 3D convolutional feature cube. For example, a 3D convolutional feature cube is divided into s x s sub-regions and combined at scales from 1 to S, so that S feature maps can be generated. Each of the multi-scale pyramid representations is input to a corresponding adaptive context module among the multiple adaptive context modules 630-1 to 630-S.

The multiple adaptive context modules 630-1 to 630-S are a set of adaptive context modules that process feature maps of different scales. For example, the first adaptive context module 630-1 processes a feature map of scale 1, and the second adaptive context module 630-2 processes a feature map of scale 2. Each of the multiple adaptive context modules 630-1 to 630-S generates at least one adaptive context vector from a feature map of the corresponding scale. That is, each of the multiple adaptive context modules 630-1 to 630-S determines a context vector for each local location by leveraging global-guided local affinity (GLA).

The connection unit 640 concatenates adaptive context vectors generated by multiple adaptive context modules 630-1 to 630-S. That is, the connection unit 640 connects adaptive context vectors having different scales.

The convolutional layer 650 generates output data based on the connected adaptive context vectors. Output data may include a semantic label for the input image. That is, the output data is a prediction result for each pixel of the input image and includes semantic labels.

Figure 7 shows an example of the structure of an adaptive context module in an artificial intelligence model according to an embodiment of the present invention. FIG. 7 is a structure applicable to each of the multiple adaptive context modules 630-1 to 630-S, and illustrates the structure of an adaptive context module with a scale of s.

Referring to Figure 7, the adaptive context module includes a convolutional layer 702, a global information extractor 704, a summation unit 706, a convolutional layer 708, and a reshape unit. ) 710, a pooling unit 712, a convolutional layer 714, a reshaping unit 716, a matrix multiplication unit 718, a reshaping unit 720, and a summing unit 722. .

The convolutional layer 702 converts the input feature map of scale s (e.g., X) into a reduced feature map (e.g., x). To this end, the convolutional layer 702 can perform a 1×1 convolution operation. Reduced feature maps are generated for each location and are used for computational efficiency. For example, the reduced feature map may have a three-dimensional structure with a size of h×w×512.

The global information extractor 704 generates a global information representation vector based on the reduced feature map. To this end, the global information extractor 704 may perform spatial global average pooling operations and convolution operations.

The summation unit 706 integrates by summing local features in the reduced feature map generated by the convolutional layer 702 and the global information representation vector generated by the global information extractor 704. ). Through this, local features at each local position are integrated with the global information representation vector.

The convolutional layer 708 performs a convolution operation on the local features integrated with the global information representation vector. Through this, the convolutional layer 708 can generate affinity vectors corresponding to each sub-region. For example, h×w relevance vectors with length s ² can be generated.

The reshaping unit 710 generates an affinity map by reshaping the affinity vectors. A relevance map may be referred to as a relevance matrix. For example, the relevance map may have a two-dimensional structure with a size of h·w×s ² . The elements included in the relevance map correspond to the affinity coefficient, and the degree to which each sub-region (e.g. Y ^s _j ) is used in estimating the semantic label of the local feature (e.g. X _i ) of the feature map. of how) Indicates whether it contributes.

The pooling unit 712 performs an average pooling operation on the input feature map (eg, X) of scale s. Through this, the pooling unit 712 summarizes the content of each sub-region included in the feature map of scale s into a feature vector (eg, y ^s _j ). The convolutional layer 714 performs a 1×1 convolution operation. Through the operations of the pooling unit 712 and the convolutional layer 714, a feature vector corresponding to each sub-region may be generated. In other words, the pooling unit 712 and the convolutional layer 714 summarize the sub-region into one feature vector by performing an average pooling operation and a convolution operation. For example, a feature vector is a single-scale representation and may have a three-dimensional structure with a size of s×s×512.

The reshaping unit 716 reshapes the feature vector so that it can be multiplied by the relevance map. Through this, the feature vector is transformed into a two-dimensional feature vector of size s ² × 512 (e.g. y ^s ). The matrix multiplier 718 multiplies the two-dimensional feature vector and the relevance map. Through this, an adaptive context matrix with a two-dimensional structure is created. For example, an adaptive context matrix with a two-dimensional structure may have a size of h·w×512. The reshaping unit 720 reshapes the adaptive context matrix into a three-dimensional structure. For example, an adaptive context matrix (e.g., z ^s ) with a three-dimensional structure may have a size of h×w×512. The summation unit 722 sums the adaptive context matrix of the three-dimensional structure and the reduced feature map (eg, x).

Figure 8 shows an example of a procedure for obtaining information from a medical image according to an embodiment of the present invention. FIG. 8 illustrates a method of operating a device with computing capabilities (eg, the service server 110 of FIG. 1).

Referring to FIG. 8, in step S801, the device performs segmentation on the signal. In other words, the device performs segmentation on at least one signal having a certain pattern in a medical image representing time-varying information. The device can perform segmentation using an artificial intelligence algorithm. By performing segmentation, the device can obtain an envelope for at least one signal included in the medical image. According to one embodiment, the artificial intelligence model with the structure described with reference to FIGS. 6 and 7 may be used for segmentation.

In step S803, the device determines measurement values. The device may determine various measurement values related to a diagnosed subject (e.g., heart blood flow rate) to obtain a medical image. For example, when the blood flow rate at a specific location in the heart is the diagnostic target, measurement values for items such as maximum velocity, signal extinction time, VTI, DT, and PHT may be determined. Here, measurement values can be determined for each segmented signal. To determine measurement values, the device may search for feature points by analyzing the segmentation results and calculate measurement values according to predefined rules based on time axis values and velocity axis values corresponding to the searched feature points. For example, feature points may be defined based on the slope of the boundary line determined by segmentation, coordinate values, etc. Specifically, the feature point corresponding to the maximum speed may be a point with the maximum absolute value on the speed axis among the points forming the boundary line. According to one embodiment, a separate artificial intelligence model may be used to search for feature points.

In step S805, the device identifies true signals and false signals. In other words, according to this, the device distinguishes whether each of the at least one signal segmented in step S801 is a true signal or a false signal. Here, true signals and false signals are distinguished depending on whether the signal is completely captured in the medical image. For example, when the entire signal is captured, as in the first segmentation 521 or the second segmentation 522 of FIG. 5B, the signal may be treated as a true signal. As another example, when only a part of the signal is captured, such as the third segmentation 523 of FIG. 5B, the signal may be treated as a false signal. In other words, a true signal can be understood as a complete signal captured in its entirety, and a false signal can be understood as an incomplete signal captured only in part. To identify true signals and false signals, the device uses pixel-specific probability information generated during the segmentation process or the envelope obtained through segmentation (e.g., length of the envelope, area of the area specified by the envelope, shape, etc.) can be used.

In step S807, the device determines the final measurement value. Specifically, the device determines the final measurement value based on measurement values obtained from at least one true signal. For at least some of the items measured for each signal, the final measurement value may be determined by combining (eg, averaging) a plurality of signals. At this time, at least one signal classified as a false signal may be excluded from combining. That is, the device can generate final measurement values for each item by compiling measurement values obtained from at least one true signal for each item.

Afterwards, although not shown in FIG. 8, the device may output final measurement values. For example, the device may display final output values through a display means provided in the device, or may transmit data including the final measurement values to another device through a communication network. Furthermore, the device can record final measurement values by uploading them to a database that manages medical information.

In the embodiment described with reference to FIG. 8, final measurement values may be determined using at least one true signal. For this, at least one true signal must be captured in the medical image. That is, if all captured signals are false signals, final measurement values cannot be determined. Therefore, according to another embodiment, after step S805, the device determines whether a true signal exists, and if no true signal exists, step S807 may be omitted.

The procedure illustrated in FIG. 8 can be performed on the result of capturing an image that changes in real time at a specific point in time. Accordingly, the above-described procedure may be performed repeatedly according to changes in the image, and the repetition period may vary depending on specific embodiments. For example, the repetition period may be determined based on the rate at which new data is added to the image.

Because the image changes as new data is added, sequentially captured images may contain the same signal. In this case, segmentation and measurement values for the same signal analyzed in a previously captured image can be reused when analyzing the captured image later. Accordingly, at each iteration, the above-described procedure may be performed on only a portion of the captured images.

Additionally, because the image changes as new data is added, the number of true signals acquired may accumulate over time. Accordingly, the operation of determining the final measurement value in step S807 in the procedure of FIG. 8 may be performed using a plurality of medical images, rather than a single captured medical image. That is, the device can generate final measurement values based on measurement values of signals segmented from a plurality of medical images captured at different times.

Figure 9 shows an example of a procedure for classifying true signals and false signals according to an embodiment of the present invention. FIG. 9 is a procedure for classifying signals based on entropy, and illustrates a method of operating a device with computational capabilities (eg, the service server 110 of FIG. 1).

Referring to FIG. 9, in step S901, the device determines entropy for segmented signals. Entropy is information expressing the probability value for the segmentation area generated during the process of performing segmentation. Entropy is a measure of uncertainty for pixels and is determined on a pixel-by-pixel basis for a given class. In the case of the present invention, the entropy of the envelope of the signal is determined. For example, entropy can be expressed as shown in Figure 5e. Since the operation of calculating entropy is part of segmentation, this step can be understood as an operation of checking entropy values generated during the segmentation operation.

In step S903, the device classifies the signal based on entropy. Specifically, the device can classify the signal as a true signal or a false signal by dividing the entropy values for each signal and analyzing the distribution of the entropy values for each signal. The clearer the entropy and the clearer the distinction from other areas, the higher the accuracy of the derived segmentation. Therefore, for the entropy values for each signal, the device can generate an unsharpness index that indicates the clarity of distinction between large values and small values, and classify the signal based on the unsharpness index. For example, the unsharpness index may be determined to be higher as more values fall in the middle region between the maximum and minimum entropy values. According to one embodiment, the sharpness index may be defined as a statistical value (e.g., average value, variance value) of entropy values of pixels belonging to an area specified by the segmentation result. According to another embodiment, the sharpness index is a statistical value (e.g., average value, variance value) of the entropy values of pixels that fall within a certain distance inside and a certain distance outside from the boundary line of the area created by segmentation. can be defined. In the case of Figure 5e, the first entropy 551 and the second entropy 552 on the right are confirmed to be relatively clearer than the third entropy 553 on the left, so the first entropy 551 and the second entropy (552) 552) can be classified as true signals, and the third entropy 553 can be classified as false signals.

According to the various embodiments described above, measurement values for signals segmented from a medical image may be determined. Depending on the medical image used, and specifically, depending on the view of the Doppler echocardiography, the items of the acquired measurement values may vary. For example, the items that can be measured depending on the Doppler echocardiography view may be as shown in [Table 1] below.

Type	Doppler view	measurement
MV (Mitral Valve)	MV inflow PW	MV E vel, MV A vel, MV dt
	MV(MS) CW	MV Vmax, MV VTI, MV PHT
	MV(MR) CW	MR Vmax, MR VTI, dp/dt
	Septal annulus TDI (tissue Doppler imaging)	E' sept, A' sept, S' sept
	Lateral annulus TDI	E'lat, A'lat, S'lat
AV (Aortic Valve)	AV(LVOT) PW(pulsed-wave)	LVOT(left ventricular outflow tract) Vmax, LVOT VTI
	LVOT obstruction CW(continuous-wave)	LVOT obstruction Vmax
	AV(AS(aortic stenosis)) CW	AV Vmax, AV VTI
	AV(AR) CW	AR(aortic regurgitation) Vmax, AR PHT
PV (Pulmonic Valve)	PV(RVOT) PW	RVOT(right ventricular outflow tract) Vmax, RVOT VTI, RVOT at
	PV(PS) CW	PV Vmax, PV VTI
	PV(PR) CW	PR Vmax, PR EDV
TV (Tricuspid Valve)	TV (TR) CW	TR Vmax, TR VTI
	TV (TS) CW	TV Vmax, TV VTI
Pulmonary Vein	Pulmonary Vein	S, D, A

The items that can be measured according to the above-mentioned Doppler echocardiography view are examined with reference to the image as follows.

FIGS. 10A to 10D show examples of information obtainable from a MV (Mitral Valve) inflow PW (pulsed-waved) view according to an embodiment of the present invention. Referring to FIGS. 10A to 10D, from the medical image of the MV inlet PW view, MV E velocity (early diastolic inflow velocity) (1002), MV A velocity (late diastolic inflow velocity) (1004), and MV dt (deceleration time) )(1006) can be measured. MV dt (1006) means the time from MV E speed (1002) to the lowest point that appears after MV E speed (1002).

11A to 11D show examples of information obtainable from a TDI (tissue Doppler imaging) view according to an embodiment of the present invention. Figures 11A, 11B, and 11D illustrate the septal annulus TDI view, and Figure 11C illustrates the lateral annulus TDI view. Referring to FIGS. 11A to 11D, from the medical image of the TDI view, S'sept (septal) (peak systolic mitral annular velocity at the septal part of mitral annulus) 1102, E'sept (peak early diastolic mitral annular velocity) at the septal part of mitral annulus)(1103), A'sept(peak late diastolic mitral annular velocity at the septal part of mitral annulus)(1104), S'lat(lateral)(1106), E'lat(peak systolic) mitral annular velocity at the lateral part of mitral annulus) (1107) and A'lat (peak late diastolic mitral annular velocity at the lateral part of mitral annulus) (1108) can be measured.

Figures 12A to 12D show examples of information obtainable in an aortic regurgitation (AR) pressure half time (PHT) view according to an embodiment of the present invention. Referring to FIGS. 12A to 12D, AR Vmax (1202) and AR PHT (1204) can be measured. AR PHT (1204) means the time from AR Vmax (1202) to the point where the speed value rapidly decreases after AR Vmax (1202).

FIGS. 13A to 13D show examples of information obtainable from a mitral valve (MS) PHT view according to an embodiment of the present invention. Referring to FIGS. 13A to 13D, MV PHT 1302 can be measured.

Figures 14a and 14b show examples of information obtainable in a pulmonic valve (PV) PW view according to an embodiment of the present invention. Referring to FIGS. 14A and 14B, RVOT Vmax (1402), RVOT at (acceleration time) 1404, and RVOT VTI (1406) can be measured.

Figures 15a and 15b show examples of information obtainable in a mitral regurgitation (MR) PW view according to an embodiment of the present invention. Referring to FIGS. 15A and 15B, MR Vmax (1502), MR dp/dt (1504), and MR VTI (1506) can be measured.

According to the various embodiments described above, measurement values for signals segmented from a medical image may be determined. The above-mentioned measurement values are examples of information directly confirmed from the segmentation results. In other words, the above-described measured values are examples of information that can be obtained by reading the value of a specific point in the segmentation result or calculating the length of the section.

Additionally, secondary measurement values may be obtained through a calculation formula based on at least one measurement value directly obtained from the segmentation result. Examples of secondary measurement values are shown in [Table 2] below.

Type	Doppler view	Measurement from formula
MV	MV inflow PW	E/A ratio
	MV(MS) CW	MV maxPG, MVmeanPG, MVA by PHT
	MR CW	MR maxPG
	Septal annulus TDI	E/E'sept
AV (Aortic Valve)	AV(LVOT) PW	LVOT maxPG, LVOT meanPG, LV stroke volume, AVA continuity Equation
	LVOT obstruction CW	LVOT obstruction maxPG
	AV(AS(aortic stenosis)) CW	AV maxPG, AV meanPG
	AV(AR) CW	AR maxPG
PV (Pulmonic Valve)	PV(RVOT) PW	RVOT maxPG, RVOT meanPG, RV stroke volume, QP/PS, meanPAP
	PV(PS) CW	PV maxPG, PV meanPG
	PV(PR) CW	PR maxPG, meanPAP
TV (Tricuspid Valve)	TV (TR) CW	TR maxPG, meanPAP, PVSP
	TV (TS) CW	TV maxPG, TV maxPG

Figure 16 shows an example of a procedure for obtaining information from a medical image using electrocardiogram (ECG) extraction and segmentation according to an embodiment of the present invention. FIG. 16 illustrates a method of operating a device with computing capabilities (eg, the service server 160 of FIG. 1).

Referring to FIG. 16, in step S1601, the device determines at least one measurement value based on segmentation of the signal. At least one measurement value may be determined based on a segmentation result for at least one true signal among signals included in the medical image. According to one embodiment, the device may determine at least one measurement value according to at least one of the procedures described with reference to FIGS. 5A to 5F, FIG. 8, or FIG. 9.

In step S1603, the device extracts the electrocardiogram signal. The device extracts electrocardiogram signals from the medical images used in the S1601. For this purpose, the device can use artificial intelligence models. For example, in a medical image such as that shown in FIG. 17, a signal 1702 may be extracted as an electrocardiogram signal.

In step S1605, the device performs ED (end-diastolic)/ES (end-systolic) segmentation from the extracted ECG signal. Here, ED/ES segmentation refers to the operation of segmenting the section from ED to ES or the section from ES to ED. In other words, the device can segment the ED-ES region or ES-ED region in the ECG signal. For this purpose, the device can use artificial intelligence models. For example, the ECG signal extracted in step S1603 may be obtained with baseline wandering due to noise, etc., as shown in graph 1802 of FIG. 18. Accordingly, the device can obtain an ECG signal in a stable state as shown in graph 1804 by performing filtering and then perform ED and ES segmentation.

In step S1607, the device generates information based on the measurement values and ED/ES. For example, the device may classify at least one measurement value obtained in step S1601 based on the timing of ED and ES. For example, if the medical image is from the MV inflow PW view, the device determines E-related values (e.g., MV E velocity) and A-related values (e.g., MV A velocity) based on the viewpoints in the ED and ES. ) can be classified. In other words, the ED/ES segmentation result can be used as a standard for classifying the measurement value determined based on the signal segmentation result.

Figure 19 shows an example of a procedure for obtaining information from a medical image considering the presence or absence of an electrocardiogram signal according to an embodiment of the present invention. FIG. 19 illustrates a method of operating a device with computing capabilities (eg, the service server 160 of FIG. 1).

Referring to FIG. 19, in step S1901, the device determines a measurement value based on segmentation of the signal. At least one measurement value may be determined based on a segmentation result for at least one true signal among signals included in the medical image. According to one embodiment, the device may determine at least one measurement value according to at least one of the procedures described with reference to FIGS. 5A to 5F, FIG. 8, or FIG. 9. Here, the measured values may include E-related values (e.g., MV E velocity) and A-related values (e.g., MV A velocity).

In step S1903, the device determines whether an electrocardiogram signal exists in the medical image. The presence of an electrocardiogram signal may be determined based on analysis of a medical image, or may be determined by a separate input.

If an ECG signal exists, in step S1905, the device performs E/A classification based on the ECG signal. In other words, the device can classify E-related values (e.g., MV E rate) and A-related values (e.g., MV A rate) based on the electrocardiogram signal. Specifically, the device segments the ED-ES region or ES-ED region in the electrocardiogram signal and, based on the ED/ES segmentation results, generates E-related values (e.g., MV E rate) and A-related values (e.g., MV A rate). can be classified.

If the ECG signal does not exist, in step S1907, the device performs E/A classification based on the signal pattern. In medical images, a signal representing an E-related value (hereinafter referred to as an 'E signal') and a signal representing an A-related value (hereinafter referred to as an 'A signal') are repeatedly observed as a pair. For example, referring to Figure 20, the E-related value (2002) and the A-related value (2004) are sequentially confirmed on the time axis, and it is confirmed that the pair of related E signals and A signals is repeated. At this time, referring to FIG. 20, it is confirmed that the width (2012) of the E signal is relatively larger than the width (2014) of the A signal. Therefore, the device can distinguish between the E signal and the A signal by pairing two consecutive signals in the segmentation result and comparing the widths of the signals included in the pair. Accordingly, the device can also distinguish between E-related values and A-related values.

In step S1909, the device checks whether abnormal E/A distribution has occurred. If the heart of a subject involved in a medical image is in a normal state, the distribution of pairs of E and A signals is uniform, as shown in FIG. 20. However, in case of E/A summation or arrhythmia, a different pattern may be observed. For example, in the case of arrhythmia, a characteristic in which the interval of the E signal is non-uniform may be observed. For example, in the case of EA summation, a characteristic that the cardiac cycle identified in E-mode echocardiography and the period of the E signal do not match may be observed. By checking whether features such as uneven E signal spacing and discrepancy with E-mode echocardiography are observed, the device can determine whether there is an abnormal E/A distribution.

If abnormal E/A distribution is confirmed, in step S1911, the device outputs a warning message. That is, the device outputs the results of E/A classification based on the signal pattern and may further output a warning message notifying that an abnormal E/A distribution has been confirmed. For example, the warning message may further include information about suspected abnormal conditions (e.g., E/A summation, arrhythmia, etc.). On the other hand, if abnormal E/A distribution is not confirmed, the device may output only the E/A classification results without a warning message.

According to the various embodiments described above, various measurement values can be obtained based on segmentation of Doppler echocardiography images. At this time, as an artificial intelligence model for segmentation, an artificial intelligence model such as that shown in FIGS. 6 and 7 or an artificial intelligence model similar thereto may be used. Alternatively, according to another embodiment, an artificial intelligence model different from the artificial intelligence model shown in FIGS. 6 and 7 may be used for segmentation. For example, various artificial intelligence models can be used, considering computing power, time required for learning, accuracy, etc.

Figure 21 shows an example of the configuration of a two-path partition network, which is one of the semantic segmentation models applicable to the present invention. The two-path segmentation network performs semantic segmentation tasks by using low-level details and high-level semantics. 2 The path segmentation network improves the accuracy and efficiency of the semantic segmentation model by separating spatial details and categorical meaning.

Referring to FIG. 21, the two-path split network includes a two pathway backbone (2110), an aggregation layer (2150), and a booster part (2140).

The two-path backbone 2110 again includes a detail branch 2120 and a semantic branch 2130. The detail branch 2120 and the semantic branch 2130 include at least one stage, and at least one operation is performed within each stage. Computation modules used for computation tasks may be Conv2d, Stem, GE, CE, etc. [Table 3] below is an example of a case where the detailed branch 2120 has three stages.

Stage	opr	k	c	s	r	output size
Input						512×1024
S1		Conv2d	3	64	2	One	256×512
	Conv2d	3	64	One	One	256×512
S2		Conv2d	3	64	2	One	128×256
	Conv2d	3	64	One	2	128×256
S3		Conv2d	3	128	2	One	64×128
	Conv2d	3	128	One	2	64×128

In [Table 3], opr refers to the operation module, k refers to the kernel size, c refers to the number of output channels, s refers to the stride, and r refers to the number of processing repetitions.

The detail branch 2120 is responsible for spatial details and is low-level detail information. Therefore, the detail branch 2120 requires abundant channel capacity to encode spatial details. Meanwhile, since the detailed branch 2120 focuses only on low-level details, the detailed branch 2120 can be designed as a thin structure with small strides. The core concept of detail branch 2120 is to use wide channels and shallow layers for spatial details.

[Table 4] below is an example of a case where the semantic branch 2130 has five stages.

Stage	opr	k	c	e	s	r	output size
Input							512×1024
S1		Stem	3	16	-	4	One	256×512 256×512
S2								128×256
								128×256
S3		GE	3	32	6	2	One	64×128
	GE	3	32	6	One	One	64×128
S4		GE	3	64	6	2	One	32×64
	GE	3	64	6	One	One	32×64
S5		GE	3	128	6	2	One	16×32
	GE	3	128	-	One	3	16×32
	C.E.	3	128	-	One	One	16×32

In [Table 4], opr refers to the operation module, k refers to the kernel size, c refers to the number of output channels, e refers to the expansion coefficient, s refers to the stride, and r refers to the number of processing repetitions. do.

The semantic branch 2130 is configured in parallel with the detailed branch 2120. The semantic branch 2130 is designed to obtain high-level semantics. Because spatial details can be provided by the detail branch 2120, the channel capacity of the semantic branch 2130 can be set low. The semantic branch 2130 can be designed by selecting any one of the lightweight convolution models. The semantic branch 2130 adopts a fast downsampling strategy to improve the level of feature representation and quickly increase the receptive field. Therefore, for high level semantics, a large receptive field is needed. Semantic branch 2130 embeds the global contextual response using global average pooling.

The aggregation layer 2150 is a layer for merging the output generated from the detailed branch 2120 and the semantic branch 2130. The feature representations of the detail branch 2120 and the semantic branch 2130 are complementary, and one branch does not recognize information from the other branch. Therefore, the aggregation layer 2150 is designed to merge two types of feature representations. Because of the fast downsampling strategy, the spatial dimensions of the output generated from the semantic branch 2130 are smaller than the spatial dimensions of the output generated from the detailed branch 2120. Upsampling is necessary to match the feature map of the output generated from the semantic branch 2130 with the output of the detailed branch 2120. Methods in which the aggregation layer 2150 aggregates the outputs of the detailed branch 2120 and the semantic branch 2130 may be implemented in various ways.

Figure 22 shows an example of a method for aggregating output generated from the detailed branch 2120 and the semantic branch 2130 applicable to the present invention. In Figure 22, DW conv means depth-wise convolution, APooling means average pooling, BN means batch normalization, Upsample means bilinear interpolation, and Sigmoid means It refers to the sigmoid activation function, Sum refers to the addition part, m×m refers to the kernel size, H×W×C refers to the tensor shape, and N refers to element-wise multiplication. . Through the calculation procedure shown in FIG. 22, the aggregation layer 2150 fuses the output of the detailed branch 2120 and the output generated from the semantic branch 2130. The aggregation layer 2150 using the calculation procedure shown in FIG. 22 is called a guided aggregation layer (GAL).

Semantic segmentation is completed by performing atrous spatial pyramid pooling (ASPP) based on the high-dimensional feature map that passed through the aggregation layer 2150.

The booster part 2140 is where the auxiliary segmentation head is extracted to further improve semantic segmentation accuracy. The segmentation head is the result of predicting which class each pixel belongs to, and is used for artificial intelligence learning. The main segmentation head is extracted as the final result of the semantic segmentation model. Since the output value of the process of extracting the main segmentation head is used, the performance of semantic segmentation can be improved by adding a few calculation procedures. The booster part 2140 can determine where to extract the auxiliary segmentation head from different locations in the semantic branch 2130. The booster part 2140 is used during the artificial intelligence learning process, but the booster part 2140 may not be used when testing or utilizing the learned artificial intelligence. By appropriately selecting the weights of the auxiliary segmentation head and the main segmentation head, more efficient artificial intelligence learning can be performed.

Figure 23 shows an example of the configuration of a three-path partition network, one of the semantic segmentation models according to an embodiment of the present invention. 3 Path segmentation networks further improve semantic segmentation by using low-level details, high-level details, and shapes. This model achieves high accuracy and efficiency in real-time semantic segmentation by separating not only spatial details and categorical meanings, but also geometric meanings.

The three-path split network includes a two pathway backbone (2310), an aggregation layer (2350), and a booster part (2390). The three-path backbone 2310 includes a detail branch 2320, a semantic branch 2330, and a shape branch 2340. The detailed branch 2320, semantic branch 2330, and booster part 2390 can be configured in the same way as in the embodiment using FIG. 22.

The shape branch 2340 obtains shape information based on the output generated at each stage of the detail branch 2320 and the semantic branch 2330. The shape branch 2340 processes each feature obtained from the detail branch 2320 and the semantic branch 2330 and generates a semantic boundary as output based on the image gradient. A gated convolutional layer (GCL) 2380 is used to facilitate the flow of the output generated in the detail branch 2320 and the output generated in the semantic branch 2330. Shape branch 2340 includes at least one guided aggregation layer. Outputs generated for each stage are selected from the detailed branch (2320) and the semantic branch (2330) as many as the number of guided aggregation layers (2360 and 2370) and input to each guided aggregation layer. When there are a plurality of guided aggregation layers 2360 and 2370, the result of convolution of all the results calculated from each guided aggregation layer 2360 and 2370 is used. If the tensor shapes of the outputs calculated in the aggregation layers 2360 and 2370 are different, 1×1 convolution may be performed first. The gated convolution layer (2380) convolutions all the results calculated from each guided aggregation layer (2360 and 2370) with the image gradient (▽I) and shapes it using the sigmoid function. Extract information about

Additionally, semantic segmentation is performed by performing atrous spatial pyramid pooling (ASPP) based on the high-dimensional feature map calculated in the guided aggregation layer 2350 and the gated convolution layer 2380. Learning about semantic segmentation is performed based on segmentation loss, edge loss, and dual task loss.

The semantic segmentation model that the present invention can use is not limited to the embodiment described using FIG. 21 and the embodiment using FIG. 23. Specifically, DNN (deep neural networks) such as DenseNet-121, U-net, VGG net, DenseNet, FCN (fully convolutional network) with encoder-decoder structure, SegNet, DeconvNet, DeepLAB V3+, Lawin+, SegFormer, Swin, and It can be replaced with the same Transformer, SqueezeNet, Alexnet, ResNet18, MobileNet-v2, GoogLeNet, Resnet-v2, Resnet50, RetinaNet, Resnet101, Inception-v3, HRNet, ResNeXt, EfficientNet, etc.

Exemplary methods of the present invention are expressed as a series of operations for clarity of explanation, but this is not intended to limit the order in which the steps are performed, and each step may be performed simultaneously or in a different order, if necessary. In order to implement the method according to the present invention, other steps may be included in addition to the exemplified steps, some steps may be excluded and the remaining steps may be included, or some steps may be excluded and additional other steps may be included.

The various embodiments of the present invention do not list all possible combinations, but are intended to explain representative aspects of the present invention, and matters described in the various embodiments may be applied independently or in combination of two or more.

Additionally, various embodiments of the present invention may be implemented by hardware, firmware, software, or a combination thereof. For hardware implementation, one or more ASICs (Application Specific Integrated Circuits), DSPs (Digital Signal Processors), DSPDs (Digital Signal Processing Devices), PLDs (Programmable Logic Devices), FPGAs (Field Programmable Gate Arrays), general purpose It can be implemented by a processor (general processor), controller, microcontroller, microprocessor, etc.

The scope of the present invention includes software or machine-executable instructions (e.g., operating systems, applications, firmware, programs, etc.) that enable operations according to the methods of various embodiments to be executed on a device or computer, and such software or It includes non-transitory computer-readable medium in which instructions, etc. are stored and can be executed on a device or computer.

Claims (15)

1. In a method for obtaining information from medical images,

   performing segmentation on at least one signal in at least one medical image;

   determining at least one measurement value for each signal based on an envelope of the at least one signal obtained through the segmentation;

   confirming at least one true signal among the at least one signal; and

   A method comprising determining at least one final measurement value based on at least one measurement value for the at least one true signal.
2. In claim 1,

   A method of expressing at least one medical image, wherein the at least one medical image represents a result of arranging time-varying information on a time axis.
3. In claim 1,

   The method wherein the at least one medical image includes at least one Doppler echocardiography image.
4. In claim 1,

   The at least one measurement value is the maximum velocity value of blood flow, velocity time integral (VTI), deceleration time (DT), pressure half time (PHT), acceleration time (AT), older-diastolic velocity (EDV), and DT ( deceleration time), dP/dt(the rate of pressure change using the 4V2 formula over time during isovolumic contraction), S'Sept(septal)(peak systolic mitral annular velocity at the septal part of mitral annulus), E'Sept(peak) early diastolic mitral annular velocity at the septal part of mitral annular), A'Sept(peak late diastolic mitral annular velocity at the septal part of mitral annular), S'lat(lateral)(peak systolic mitral annular velocity at the lateral part of A method comprising at least one of (mitral annulus), E'lat (peak early diastolic mitral annular velocity at the lateral part of mitral annulus), and A'lat (peak late diastolic mitral annular velocity at the lateral part of mitral annulus).
5. In claim 1,

   The at least one true signal is confirmed based on at least one of pixel-specific probability information generated for the segmentation, the length of the envelope, the area of the area specified by the envelope, and the shape of the area specified by the envelope. How to become.
6. In claim 1,

   The method wherein the at least one true signal is identified based on a distribution of entropy values generated for the segmentation.
7. In claim 6,

   The step of confirming the at least one true signal includes:

   A method comprising checking entropy values generated in the step of performing the segmentation.
8. In claim 1,

   A method wherein the segmentation is performed based on multi-scale pyramid representations.
9. In claim 1,

   extracting an electrocardiogram (ECG) signal from the at least one medical image; and

   The method further comprising determining at least one other measurement value based on the electrocardiogram signal and the at least one final measurement value.
10. In claim 1,

    The at least one signal includes a first signal related to a first value and a second signal related to a second value,

    A method in which the first signal and the second signal are classified based on an electrocardiogram signal, or based on a pattern of the at least one signal.
11. In claim 10,

    grouping the at least one signal into pairs including two consecutive signals on the time axis; and

    The method further includes classifying the signals based on the width of the signals included in each of the pairs.
12. The method of claim 1, wherein performing the segmentation includes:

    determining spatial details of the signal;

    determining semantics of the signal;

    generating aggregated data by aggregating the spatial details and the semantics;

    determining a shape using the aggregated data; and

    A method comprising performing segmentation based on the spatial information, the semantics and the shape.
13. The method of claim 1, wherein performing the segmentation includes:

    DenseNet-121, U-net, VGG net, DenseNet, FCN (fully convolutional network) with encoder-decoder structure, SegNet, DeconvNet, DNN (deep neural network) such as DeepLAB V3+, Transformer such as Lawin+, SegFormer, and Swin , a method comprising performing segmentation using one of the following models: SqueezeNet, Alexnet, ResNet18, MobileNet-v2, GoogLeNet, Resnet-v2, Resnet50, RetinaNet, Resnet101, Inception-v3, HRNet, ResNeXt, and EfficientNet.
14. In a device for obtaining information from medical images,

    a storage unit that stores a set of instructions for operating the device; and

    It includes at least one processor connected to the storage unit,

    The at least one processor,

    Perform segmentation on at least one signal in at least one medical image,

    Determining at least one measurement value for each signal based on the envelope of the at least one signal obtained through the segmentation,

    Confirming at least one true signal among the at least one signal,

    A device for controlling to determine at least one final measurement value based on at least one measurement value for the at least one true signal.
15. A program stored in a medium for executing the method according to any one of claims 1 to 13 when operated by a processor.

Images