KR20220122540A - Method and apparatus for detecting medical indices in medical image - Google Patents

Abstract

The present invention is to detect medical indices in a medical image. A method for acquiring information from the medical image may include: a step of performing segmentation for sections of a heart in the medical image; a step of generating at least one reference line based on the segmentation; and a step of determining at least one medical index based on the at least one reference line.

Description

Method and apparatus for detecting clinical indicators in medical images

The present invention relates to processing of a medical image, and more particularly, to a method and an apparatus for detecting medical indices in a medical image.

A disease refers to a condition in which a normal function is impaired by causing a disorder in the mind and body of a human being, and depending on the disease, a person may suffer and even be unable to sustain life. Accordingly, various social systems and technologies for diagnosing, treating, and further preventing diseases have been developed along with the history of mankind. In the diagnosis and treatment of diseases, various tools and methods have been developed according to the remarkable development of technology, but it is a reality that ultimately depends on the judgment of a doctor.

On the other hand, as artificial intelligence (AI) technology has developed significantly in recent years, it is attracting attention in various fields. In particular, due to the vast amount of accumulated medical data and the environment of image-oriented diagnostic data, various attempts and studies to apply artificial intelligence algorithms to the medical field are underway. Specifically, various studies are being conducted to solve tasks that have remained in the conventional clinical judgment, such as diagnosing and predicting a disease, using an artificial intelligence algorithm. In addition, as an intermediate process for diagnosis, etc., various studies are being conducted to solve the task of processing and analyzing medical data using artificial intelligence algorithms.

An object of the present invention is to provide a method and apparatus for effectively acquiring information from a medical image using an artificial intelligence (AI) algorithm.

An object of the present invention is to provide a method and apparatus for automatically detecting clinical indices from a medical image.

An object of the present invention is to provide a method and apparatus for detecting a clinical indicator based on segmentation of a medical image.

The technical problems to be achieved in the present invention are not limited to the technical problems mentioned above, and other technical problems not mentioned will be clearly understood by those of ordinary skill in the art to which the present invention belongs from the description below. will be able

According to an embodiment of the present invention, a method for obtaining information from a medical image includes: performing segmentation on regions of the heart in the medical image; generating at least one reference line based on the segmentation; The method may include determining at least one clinical index based on the at least one baseline.

According to an embodiment of the present invention, the at least one clinical indicator may include at least one of a length of the at least one baseline and a value determined based on the length of the at least one baseline.

According to an embodiment of the present invention, the method may further include generating at least one reference point based on the segmentation.

According to an embodiment of the present invention, the at least one reference point may be generated based on at least one central point among the regions.

According to an embodiment of the present invention, the zones may include at least one of a first area, a second area, a third area, a fourth area, a fifth area, and a sixth area.

According to an embodiment of the present disclosure, the first area is located between the second area and the third area, the second area is located between the fourth area and the first area, or the The fourth region is located at the uppermost end, the fifth region is adjacent to the right side of the first region, or the sixth region is adjacent to the right side of the first region and is located at the lower end of the fifth region can be located

According to an embodiment of the present invention, in the generating of the at least one reference line, a transverse line passing through a reference point of the first area and a center of a left boundary line of the first area and crossing the first area is identified. and identifying a first orthogonal line passing through the reference point and orthogonal to the transverse line, and generating a first reference line to include at least a portion of the first orthogonal line.

According to an embodiment of the present invention, the generating of the at least one reference line includes passing a junction between the first reference line and a segmentation contour of the second region and orthogonal to the long axis of the second region. It may include identifying a second orthogonal line, and generating a second reference line to include at least a portion of the second orthogonal line.

According to an embodiment of the present invention, the generating of the at least one reference line includes a third orthogonal line passing through a junction between the first reference line and the segmentation closed curve of the third area and orthogonal to the long axis of the third area. and generating a third reference line to include at least a portion of the third orthogonal line.

According to an embodiment of the present invention, the generating of the at least one reference line may include identifying a center line of a short axis of an area corresponding to the second area, the center line and a boundary line of the area meeting Identifying a first vertical line perpendicular to a boundary line of a second region at a point, and generating a first reference line to include at least a portion of the first vertical line, wherein the first reference line includes the first It can be used to measure the diameter of an area.

According to an embodiment of the present invention, in the generating of the at least one reference line, a second vertical line perpendicular to the boundary line of the third area is identified at a point where the boundary line of the first reference line and the boundary line of the third area meet. and generating a second reference line to include at least a portion of the second vertical line, wherein the second reference line may be used to measure the thickness of the third region.

According to an embodiment of the present invention, the generating of the at least one reference line includes: identifying a point closest to the fourth area among points on the boundary line of the fifth area; , identifying a parallel line parallel to a junction of the fifth region and the first region, and generating a third reference line to include at least a portion of the parallel line.

According to an embodiment of the present invention, the generating of the at least one reference line may include identifying a sinus point on the boundary line of the fifth area, and on the boundary line of the area corresponding to the sixth area. identifying a vertical line that includes a point closest to the sinus point, is parallel to a vertical axis of the medical image, and passes through an inside of an area corresponding to the sixth area, and a second line to include at least a part of the vertical line 4 generating a baseline.

According to an embodiment of the present invention, the generating of the at least one reference line may include identifying a sinus point on a boundary line of the fifth area, and a boundary line of an area corresponding to the sixth area. identifying a vertical line including a point closest to the sinus point on the image, perpendicular to the long axis of the area corresponding to the sixth area, and penetrating the inside of the area corresponding to the sixth area, at least of the vertical line determining a fourth baseline to include a portion.

According to an embodiment of the present invention, the generating of the at least one reference line includes a point on the joint of the fifth region and the first region, and includes a vertical line perpendicular to the center line of the long axis of the fourth region. and generating a fifth reference line to include at least a portion of the vertical line.

According to an embodiment of the present invention, the generating of the at least one reference line includes a point on the joint of the fifth region and the first region, and identifies a vertical line parallel to the vertical axis of the medical image. and generating a fifth reference line to include at least a portion of the vertical line.

According to an embodiment of the present invention, the zones are one of right ventricle (RV), interventricular septum (IVS), aorta, left ventricle (LV), LV posterior wall (LVPW), or left atrium (LA). It may include at least one.

According to an embodiment of the present invention, an apparatus for obtaining information from a medical image includes a storage unit for storing an instruction set for operation of the apparatus, and at least one processor connected to the storage unit, wherein the at least The one processor performs segmentation on regions of the heart in the medical image, generates at least one baseline based on the segmentation, and at least one clinical index based on the at least one baseline. can be controlled to determine

According to an embodiment of the present invention, when the program stored in the medium is operated by the processor, the above-described method may be executed.

The features briefly summarized above with respect to the invention are merely exemplary aspects of the detailed description of the invention that follows, and do not limit the scope of the invention.

According to the present invention, information can be effectively obtained from a medical image using an artificial intelligence (AI) algorithm.

The effects obtainable in the present invention are not limited to the above-mentioned effects, and other effects not mentioned may be clearly understood by those of ordinary skill in the art from the following description. will be.

1 shows a system according to an embodiment of the present invention.
2 shows the structure of an apparatus according to an embodiment of the present invention.
3 shows an example of a perceptron constituting an artificial intelligence model applicable to the present invention.
4 shows an example of an artificial neural network constituting an artificial intelligence model applicable to the present invention.
5 illustrates a mechanism for determining a landmark detection method using a regression model according to an embodiment of the present invention.
6A to 6C show examples of detection results for each landmark detection method.
7 illustrates an example of a procedure for detecting a clinical indicator based on segmentation according to an embodiment of the present invention.
8 illustrates an example of a segmentation result for a medical image according to an embodiment of the present invention.
9 illustrates an example of a procedure for determining a reference line according to an embodiment of the present invention.
10 illustrates another example of a procedure for determining a baseline according to an embodiment of the present invention.
11 illustrates another example of a procedure for determining a baseline according to an embodiment of the present invention.
12 illustrates another example of a procedure for determining a baseline according to an embodiment of the present invention.
13A illustrates an example of reference lines determined from an image of an end-diastole (ED) state according to an embodiment of the present invention.
13B illustrates an example of reference lines determined from an image of an end-systole (ES) state according to an embodiment of the present invention.
14 illustrates an example of reference points and reference lines determined based on a segmentation result according to an embodiment of the present invention.
15A and 15B show an example of a long short-term memory (LSTM) network applicable to the present invention.
16 shows an example of a training procedure for segmentation according to an embodiment of the present invention.
17 illustrates a heart image and a propagated heart image according to an embodiment of the present invention.
18 shows an example of a segmentation procedure according to an embodiment of the present invention.
19A to 19C show specific examples of data augmentation according to an embodiment of the present invention.
20 shows a specific implementation example of an artificial intelligence model for segmentation according to an embodiment of the present invention.
21 shows another specific implementation example of an artificial intelligence model for segmentation according to an embodiment of the present invention.

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings so that those of ordinary skill in the art to which the present invention pertains can easily implement them. However, the present invention may be embodied in various different forms and is not limited to the embodiments described herein.

In describing an embodiment of the present invention, if it is determined that a detailed description of a well-known configuration or function may obscure the gist of the present invention, a detailed description thereof will be omitted. And, in the drawings, parts not related to the description of the present invention are omitted, and similar reference numerals are attached to similar parts.

1 shows a system according to an embodiment of the present invention.

Referring to FIG. 1 , the system includes a service server 110 , a data server 120 , and at least one client device 130 .

The service server 110 provides an artificial intelligence model-based service. That is, the service server 110 performs learning and prediction operations using the artificial intelligence model. The service server 110 may communicate with the data server 120 or at least one client device 130 through a network. For example, the service server 110 may receive training data for training the artificial intelligence model from the data server 120 and perform training. The service server 110 may receive data required for learning and prediction operations from at least one client device 130 . Also, the service server 110 may transmit information on the prediction result to at least one client device 130 .

The data server 120 provides learning data for training the artificial intelligence model stored in the service server 110 . According to various embodiments, the data server 120 may provide public data that anyone can access or data requiring permission. If necessary, the training data may be pre-processed by the data server 120 or the service server 120 . According to another embodiment, the data server 120 may be omitted. In this case, the service server 110 may use an externally trained artificial intelligence model or may provide the learning data offline to the service server 110 .

At least one client device 130 transmits and receives data related to the artificial intelligence model operated by the service server 110 with the service server 110 . At least one client device 130 is equipment used by a user, transmits information input by the user to the service server 110, stores information received from the service server 110, or provides it to the user (eg : can be displayed. In some cases, a prediction operation may be performed based on data transmitted from one client, and information related to the prediction result may be provided to another client. The at least one client device 130 may be a computing device of various types, such as a desktop computer, a laptop computer, a smart phone, a tablet, and a wearable device.

Although not shown in FIG. 1 , the system may further include a management device for managing the service server 110 . The management device is a device used by a subject that manages a service, and monitors the status of the service server 110 or controls settings of the service server 110 . The management device may be connected to the service server 110 through a network or may be directly connected through a cable connection. According to the control of the management device, the service server 110 may set parameters for operation.

As described with reference to FIG. 1 , the service server 110 , the data server 120 , at least one client device 130 , a management device, etc. may be connected through a network and interact with each other. Here, the network may include at least one of a wired network and a wireless network, and may be formed of any one or a combination of two or more of a cellular network, a local area network, and a wide area network. For example, the network is based on at least one of a local area network (LAN), wireless LAN (WLAN), Bluetooth (bluetooth), long term evolution (LTE), LTE-advanced (LTE-A), and 5th generation (5G). can be implemented.

2 shows the structure of an apparatus according to an embodiment of the present invention. The structure illustrated in FIG. 2 may be understood as a structure of the service server 110 , the data server 120 , and at least one client device 130 of FIG. 1 .

Referring to FIG. 2 , the device includes a communication unit 210 , a storage unit 220 , and a control unit 230 .

The communication unit 210 performs a function for accessing a network and performing communication with other devices. The communication unit 210 may support at least one of wired communication and wireless communication. For communication, the communication unit 210 may include at least one of a radio frequency (RF) processing circuit and a digital data processing circuit. In some cases, the communication unit 210 may be understood as a component including a terminal for connecting a cable. Since the communication unit 210 is a component for transmitting and receiving data and signals, it may be referred to as a 'transceiver'.

The storage unit 220 stores data, programs, microcodes, instruction sets, applications, etc. necessary for the operation of the device. The storage unit 220 may be implemented as a temporary or non-transitory storage medium. Also, the storage unit 220 may be fixed to the device or implemented in a detachable form. For example, the storage unit 220 may include a compact flash (CF) card, a secure digital (SD) card, a memory stick, a solid-state drive (SSD), and a microSD card. It may be implemented as at least one of a NAND flash memory such as an SD card and a magnetic computer storage device such as a hard disk drive (HDD).

The controller 230 controls the overall operation of the device. To this end, the controller 230 may include at least one processor, at least one microprocessor, and the like. The control unit 230 may execute a program stored in the storage unit 220 , and may access a network through the communication unit 210 . In particular, the controller 230 may perform algorithms according to various embodiments to be described later, and control the device to operate according to embodiments to be described later.

Based on the structure described with reference to FIGS. 1 and 2 , an artificial intelligence algorithm-based service according to various embodiments of the present disclosure may be provided. Here, an artificial intelligence model consisting of an artificial neural network may be used to implement an artificial intelligence algorithm. The concept of a perceptron, a structural unit of an artificial neural network, and an artificial neural network is as follows.

The perceptron is a model of the nerve cell of an organism, and has a structure that outputs a single signal by taking multiple signals as input. 3 shows an example of a perceptron constituting an artificial intelligence model applicable to the present invention. Referring to FIG. 3 , the perceptron uses weights 302-1 to 302-n (eg, w _1j , w _2j , w for each input value (eg, x ₁ , x ₂ , x ₃ , …, x _n )) _3j , ..., w _nj ), and then the weighted input values are summed using a transfer function 304 . During the summing process, a bias value (eg, b _k ) may be added. The perceptron generates an output value (eg o _j ) by applying an activation function 406 to a net input value (eg, net _j ) that is an output of the transformation function 304 . In some cases, the activation function 406 may operate based on a threshold (eg, θ _j ). The activation function can be defined in various ways. Although the present invention is not limited thereto, for example, as the activation function, a step function, a sigmoid, Relu, Tanh, or the like may be used.

As perceptrons as shown in FIG. 3 are arranged and layered, an artificial neural network can be designed. 4 shows an example of an artificial neural network constituting an artificial intelligence model applicable to the present invention. In FIG. 4 , each node represented by a circle may be understood as a perceptron of FIG. 3 . Referring to FIG. 4 , the artificial neural network includes an input layer 402 , a plurality of hidden layers 404a and 404b , and an output layer 406 .

When prediction is performed, when input data is provided to each node of the input layer 402 , the input data is weighted by the perceptrons constituting the input layer 402 and the hidden layers 404a and 404b, and a transform function operation is performed. And it is forward propagated to the output layer 406 through an activation function operation and the like. Conversely, when training is performed, an error is calculated through backward propagation from the output layer 406 toward the input layer 402, and the weight values defined in each perceptron may be updated according to the calculated error. have.

The present invention proposes a technique for detecting a clinical indicator from a medical image. Specifically, the present invention provides more robust segmentation of multiple structures (eg, left atrium, left ventricle, right atrium, right ventricle, inner wall, etc.) in a 2D ultrasound image used to evaluate or diagnose a patient's heart shape, structure, or function. ), and by detecting major landmarks (eg, Apex, Annulus, etc.), we propose a technique for deriving clinically important indicators.

Echocardiography is a routine, primary, and essential test, which is non-invasive and can be performed at a low cost. Echocardiography enables structural and functional evaluation of the heart, and if additional tests are needed, secondary tests may be performed. Due to the above-described characteristics, it is relatively easier to construct data from an echocardiogram image than a method based on other medical equipment. However, since the manpower cost and time cost of a medical professional occurring during the analysis process are large, the echocardiogram image is data that has the potential to further increase its usefulness.

In the case of echocardiography data, a subdivided name is determined based on a location at which the heart is photographed according to the criteria recommended by the American Heart Association, and an echocardiography analysis method is also different according to the subdivided name. Therefore, an echocardiographic analysis expert is essential to quantify the echocardiography data. Recently, in order to solve this problem, studies for automating echocardiography analysis have been conducted, and studies of segmenting the structural position of the left ventricle using artificial intelligence techniques have been reported.

Although many techniques for segmenting specific cardiac structures using artificial intelligence have been introduced, the techniques for detecting actual clinically meaningful landmarks are much less compared to segmentation techniques, and specific image views (eg, A2CH, A3CH, A4CH) , PSAX, PLAX (parasternal long axis), etc.). For this reason, it is not possible to consider each image view that looks different from each other even with the same structure, which may result in performance degradation.

Even if the same cardiac structure is included in the image, it has different characteristics depending on the image view. For example, A2CH includes only the left atrium and left ventricle, and A4CH includes not only the left atrium and left ventricle, but also the right atrium and right ventricle. As such, A2CH and A4CH express the same cardiac structure, but in the image itself for a view given through a regression model as shown in FIG. 5 for more robust landmark detection from each of the views representing different organ organs. It may be determined whether to detect the landmark or whether to detect the landmark from a result of performing segmentation.

5 illustrates a mechanism for determining a landmark detection method using a regression model according to an embodiment of the present invention. Referring to FIG. 5 , a regression model 510 is input to images taken from views such as A2CH 502 , A4CH 504 , and PSAX 506 . One of image-based detection 522 , segmentation-based detection 524 , complement detection 526 may be selected by regression model 510 . can As described above, when an image of a specific view is provided as an input of the regression model 510, which view it is can be predicted, and which approach to use can be determined through this. Since a specific structure visible to the naked eye is different for each view, one of image-based detection 522 , segmentation-based detection 524 , and complementary detection 526 is used to process an image more efficiently.

6A to 6C show examples of detection results for each landmark detection method. Referring to FIG. 6A , A2CH 602 may be analyzed by image-based detection 622 . In the case of the image-based detection 622 , landmarks (eg, Apex 632 and Annulus 634a and 634b) may be detected using only the ultrasound image. Since the image-based detection 622 does not perform segmentation and detects a landmark from only an image, it has the advantage of being able to detect the landmark more quickly.

Referring to FIG. 6B , A2CH 602 may be analyzed by segmentation-based detection 624 . In the case of the segmentation-based detection 624 , after segmentation is performed on the ultrasound image, a landmark is detected from the segmentation result. Accordingly, it takes a relatively long time compared to the image-based detection 622 , but recently, the time required has been greatly reduced through artificial intelligence. In addition, since the artificial intelligence algorithm derives image segmentation results at a level acceptable to clinicians, landmarks (e.g., Apex (642), Annulus (644a, 644b)) can be detected.

Referring to FIG. 6C , A2CH 602 may be analyzed by complementary detection 626 . In the case of the complementary detection 626 , it may be used when it is difficult to detect a landmark by only one of the image-based detection 622 or the segmentation-based detection 624 . Complementary detection 626 includes a probability determined by image-based detection 622 (eg, Apex 632, Annulus 634a, 634b) and a probability determined by segmentation-based detection 624 (eg, Apex 642). ), Annulus (644a, 644b)) can be refined and supplemented by the Bayesian method, and landmarks can be detected.

Guidelines for measuring various clinical indicators in echocardiography are defined by the American Society of Echocardiography (ASE). In order to measure clinical indicators according to the guidelines of the ASE, clinically specific landmarks defined in the ASE are required. In this case, in order to detect specific landmarks in the echocardiography image of a patient requiring analysis, human intervention is inevitable, which consumes time and money. Accordingly, the present invention proposes a technique for measuring clinical indicators without human intervention. Through the technology proposed in the present invention, specific landmarks can be automatically detected in the echocardiography image of a patient requiring analysis, and it is expected that the time and cost requirements will be greatly reduced compared to the existing ones.

7 illustrates an example of a procedure for detecting a clinical indicator based on segmentation according to an embodiment of the present invention. 7 exemplifies an operation method of a device having a computing capability (eg, the service server 110 of FIG. 1 ).

Referring to FIG. 7 , in step S701 , the device performs segmentation on an image into regions. The device is designed for segmentation and may perform segmentation using a trained AI model. For example, the image is an echocardiogram image, and may be a PLAX view image. Zones mean the spaces, muscles, blood vessels, etc. that make up the heart. For example, as shown in Figure 8, by segmentation, RV (right ventricle) 802, IVS (interventricular septum) 804, aorta (aorta) 806, LV (left ventricle) (808), LVPW ( An LV posterior wall 810 and a left atrium (LA) 812 may be identified.

In step S703, the device generates at least one reference line based on the segmentation result. A baseline may be generated based on the regions obtained by segmentation. For example, the reference line may be determined based on a relative position between regions, a point based on a positional relationship between regions, a boundary line of each region, and the like.

In step S705 , the device determines at least one clinical indicator based on the at least one baseline. For example, the at least one clinical indicator may include a length of at least one generated baseline, a value determined based on the at least one baseline, and the like.

Although not shown in FIG. 7 , at least one clinical indicator determined based on at least one baseline may be provided to the user. For example, the device may display at least one clinical indicator through the display device. In this case, the at least one clinical indicator may be displayed to overlap the medical image or may be displayed through a separate interface. When displayed to be superimposed on a medical image, each clinical indicator may be disposed in an area within a threshold distance from a corresponding baseline. Additionally, an indicator (eg, a connecting line, etc.) indicating a relationship between each clinical indicator and a corresponding baseline may be displayed together. As another example, the device may transmit data indicative of at least one clinical indicator to another device via a network.

9 illustrates an example of a procedure for determining a reference line according to an embodiment of the present invention. 9 exemplifies an operation method of a device having computing capability (eg, the service server 110 of FIG. 1 ).

Referring to FIG. 9 , in step S901 , the device detects a center line of the IVS. The device detects a line crossing the short axis of the region corresponding to the IVS. For example, the centerline of the IVS is line 1314a in FIG. 13A or line 1314b in FIG. 13B.

In step S903, the device detects a vertical line to the IVS. The device detects a line perpendicular to the boundary line of the IVS at the point where the center line of the IVS and the boundary line of the IVS meet. In other words, the device detects a line perpendicular to the boundary line of the IVS at the point where the center line detected in step S901 and the boundary line of the region corresponding to the IVS meet. Here, the vertical line may penetrate the interior of the region corresponding to the LV. For example, the vertical line for IVS is line 1316 of FIG. 13A or line 1326 of FIG. 13B .

In step S905, the device detects the diameter of the LV. That is, the device may detect the diameter of the LV by measuring the length of the vertical line detected in step S903. That is, the vertical line detected in step S903 is one of the clinical indicators and may be used to measure the diameter of the LV.

In step S907, the device detects a vertical line in the LVPW. That is, the device detects a line perpendicular to the boundary line at the point where the vertical line detected in step S903 and the boundary line of the LVPW meet. The vertical line may penetrate the interior of the region corresponding to the LVPW. For example, the vertical line within LVPW is line 1314b of FIG. 13A or line 1324b of FIG. 13B. Here, the length of the vertical line is one of the clinical indicators to be measured, and may be used to measure the thickness of the LVPW.

10 illustrates another example of a procedure for determining a baseline according to an embodiment of the present invention. 10 exemplifies an operation method of a device having a computing capability (eg, the service server 110 of FIG. 1 ).

Referring to FIG. 10 , in step S1001, the device detects the closest point between the RV and the aorta. For example, the proximal point may be a point closest to the RV among points on the borderline of the aorta. Here, the proximity point may be a sinus point. The sinus point means a point having a predefined characteristic among points on the borderline of the aorta. For example, the sinus point may mean a high point of a convex portion from the top of the boundary line of the aorta determined in the echocardiography image of the PLAX view.

In step S1003, the device detects a parallel line to the junction of the aorta and the LV. The joint of the aorta and the LV means a line or at least one point on which the boundary line of the aorta and the boundary line of the LV obtained by segmentation overlap. The parallel line is a line that is parallel to the overlapping line, that is, a line corresponding to the joint, and includes the adjacent point identified in step S1001. That is, the parallel line starts at the proximal point and penetrates the interior of the region corresponding to the aorta. For example, the parallel line may be line 1318 of FIG. 13A .

In step S1005, the device determines the aortic line. Here, the aortic line means a parallel line detected in step S1003. The length of the aortic line may be one of the clinical indicators to be measured.

11 illustrates another example of a procedure for determining a baseline according to an embodiment of the present invention. 11 exemplifies an operation method of a device having a computing capability (eg, the service server 110 of FIG. 1 ).

In step S1101, the device detects a sinus point in the aorta. That is, the device detects a sinus point on the borderline of the aorta. The sinus point means a point having a predefined characteristic among points on the borderline of the aorta. For example, the sinus point may mean a high point or a bottom point of a convex portion from the top to the top or from the bottom to the bottom of the boundary line of the aorta determined in the echocardiography image of the PLAX view.

In step S1103, the device determines a vertical line passing through the LA. One end of the vertical line coincides with the sinus point detected in step S1101. According to an embodiment, the vertical line may be a line perpendicular to LA at the sinus point and passing through the interior of the area corresponding to LA. According to another embodiment, the vertical line may be a line that includes a point closest to the sinus point on the boundary line of the area corresponding to LA, is perpendicular to the center line of the long axis of LA, and passes through the inside of the area corresponding to LA. . According to another embodiment, the vertical line may be a line that includes a point closest to the sinus point on the boundary line of the area corresponding to LA, is parallel to the vertical axis of the medical image, and passes through the inside of the area corresponding to LA. . For example, the vertical line may be line 1328 of FIG. 13B . The vertical line length may be one of the clinical indicators to be measured.

12 illustrates another example of a procedure for determining a baseline according to an embodiment of the present invention. 12 exemplifies an operation method of a device having a computing capability (eg, the service server 110 of FIG. 1 ).

In step S1201, the device detects the junction of the IVS and the aorta. The junction of the IVS and the aorta means a line or at least one point where the boundary line of the IVS obtained by segmentation and the boundary line of the aorta overlap.

In step S1203, the device determines a vertical line passing through the RV. One end of the vertical line coincides with a point on the joint detected in step S1201. According to an embodiment, the vertical line may be a line parallel to the vertical axis of the image. According to another embodiment, the vertical line may be a line perpendicular to a center line of a long axis of the RV. For example, the vertical line may be line 1312 of FIG. 13A . The vertical line length may be one of the clinical indicators to be measured.

In the embodiment described with reference to FIG. 12 , the junction of the IVS and the aorta is used. According to another embodiment, instead of the joint of the IVS and the aorta, a point where a line extending from the joint of the LV and the aorta and the boundary of the region corresponding to the RV meet may be used.

According to various embodiments described above, at least one clinical indicator may be determined based on a segmentation result of a medical image. For example, examples of clinical indicators that can be measured by echocardiography of the PLAX view are shown in [Table 1] below.

Measurement Pearson correlation coefficient (r) LV septum diameter　at the diastole (unit: mm) 0.85 LV septum diameter　at the systole (unit: mm) 0.84 LV internal diameter at the diastole (unit: mm) 0.84 LV internal diameter at the systole (unit: mm) 0.86 LV posterior wall diameter at the diastole (unit: mm) 0.86 LV posterior wall diameter at the systole (unit: mm) 0.88 LA diameter (unit: mm) 0.91 Aorta diameter (unit: mm) 0.84 RV diameter (unit: mm) 0.81

The clinical indicators exemplified in [Table 1] are examples of measured values that can be obtained directly from the length of the generated at least one baseline. Additionally, based on the measured values obtained from the length of at least one baseline, at least one of clinical indicators as shown in Table 2 below may be further obtained.

Item formula Explanation EDV
(End diastolic volume) [7/ (2.4 + LVIDd]]] × LVIDd ³ Capacity inside the ventricles at the end of diastole of the heart. ESV
(End systolic volume) [7/ (2.4 + LVIDs)]] * LVIDs ³ Capacity inside the ventricles at the end of the systolic phase of the heart. EF
(Ejection fraction) (LVIDd ² - LVIDs ² / LVIDd ² )× 100(%) The most important and popular measure of left ventricular systolic function. It is the ratio of stroke volume to end diastolic volume, expressed as percentage. LVmass 0.8 * 1.04 *[(IVS + LVID + LVPW) ³ ] + 0.6 Left ventricular mass index.

As described above, at least one clinical indicator may be determined from the echocardiography image. The above-described embodiments of determining a clinical indicator based on the segmentation result, ie, segmentation contours of regions, may be applied to other types of images. In other words, if segmentation can be performed on a given image, a clinical indicator may be derived based on at least one reference point and at least one reference line. An example of the at least one reference point and the at least one reference line is shown in FIG. 14 .

14 illustrates an example of reference points and reference lines determined based on a segmentation result according to an embodiment of the present invention. 14 illustrates a segmentation result for a given image. Referring to FIG. 14 , by segmentation, a first region 1410 , a second region 1420 , a third region 1430 , a fourth region 1440 , a fifth region 1450 , and a sixth region 1460 . ) is partitioned. The first area 1410 is located between the second area 1420 and the third area 1430 , and the second area 1420 is located between the fourth area 1440 and the first area 1410 , The fourth area 1440 is located at the uppermost end, the fifth area 1450 is adjacent to the right side of the first area 1410 , and the sixth area 1460 is adjacent to the right side of the first area 1410 . while being positioned at the lower end of the fifth area 1450 .

The point a 1411 is the center point of the first region 1410 . Here, the central point means central moments in the corresponding region. The point b 1412 is a point that bisects the left boundary of the first region 1410 . Here, one end of the left boundary line is a second left point 1413 at which the boundary between the first area 1410 and the second area 1420 is separated from the left side of the first area 1410 , and the other end of the left boundary line is the first area. A first left point 1414 at which the boundary of 1410 and the third region 1430 is separated. A transverse line 1415 passing through the point b 1414 and crossing the first region 1410 may be drawn, and an orthogonal line 1416 orthogonal to the transversal line 1415 may be drawn. Here, at least a portion of the orthogonal line 1416 may be used as a reference line.

Point c 1421 is a junction between the orthogonal line 1416 and the segmentation closed curve of the second region 1420 . An orthogonal line 1423 passing through the point c 1421 and perpendicular to the long axis 1423 of the second region 1420 may be drawn, and at least a portion of the orthogonal line 1423 may be used as a reference line. That is, the reference line in the corresponding area (eg, the second area 1420 ) may be determined in a manner extending from the reference line generated in the adjacent area (eg, the first area 1410 ).

The point d 1431 is a junction between the orthogonal line 1416 and the segmentation closed curve of the third region 1430 . An orthogonal line 1432 passing through the point d 1431 and perpendicular to the long axis 1433 of the third region 1430 may be drawn, and at least a portion of the orthogonal line 1432 may be used as a reference line. That is, the reference line in the corresponding region (eg, the third region 1430 ) may be determined in a manner extending from the reference line generated in the adjacent region (eg, the first region 1410 ).

According to the above-described various embodiments, clinical indicators may be automatically extracted and provided based on a segmentation result of a medical image. To this end, it is required to perform segmentation on a given medical image. Segmentation of an image is a type of image analysis operation, and refers to an operation of detecting or extracting a region (eg, a set of pixels or voxels) expressing an interest target among objects expressed in an image. do. In general, since an image acquired through photographing expresses other objects in the vicinity other than the object of interest, it is required to distinguish the object of interest from the other objects through image segmentation. Image segmentation may be performed on a single image (eg, using a model including LSTM and U-net as shown in FIG. 21 ), but when a time-series image set exists, several images before or after the image to be segmented There is a possibility that the accuracy of image segmentation may be improved by further using them. Accordingly, the present invention will describe various embodiments of performing image segmentation using a time-series image set. In particular, in the present invention, when a label representing a segmentation result is added to only some images of a time-series image set, a labeled image corresponding to the remaining surrounding images is generated, and the generated labeled image Various embodiments using them will be described.

First, as an artificial intelligence model that can be used for segmentation, the structure of a long short-term memory (LSTM) network is as follows. A recurrent neural network (RNN) is an artificial neural network that expresses a structure for judging a current state using information input in the past. RNN uses the iterative structure to continuously use the information obtained in the previous step. As a type of RNN, an LSTM network has been proposed. The LSTM network has been proposed to control long-term dependencies, and has an iterative structure like RNN. The structure of the LSTM network is shown in FIGS. 15A and 15B below.

15A and 15B show an example of an LSTM network applicable to the present invention.

를 곱셈 연산자(1506b)를 통해 덧셈 연산자에게 제공한다. 덧셈 연산자는 입력 받은 값들을 적용하여 h_t를 결과값으로 출력한다.Referring to FIG. 15A , the LSTM network provides the current state h to be transmitted to the next network through h of the previous state (t-1) and the input (x _t ) of the current state (t). The LSTM network calculates the previous state in time series through the state of r, which determines whether to use the previous state information internally. The hidden network includes sigmoid networks 1502a , 502b , tanh networks 1504 , multiplication operators 1506a , 506b , 506c , and an addition operator 1508 . Each of the sigmoid networks 1502a, 502b has a weight and a bias, and uses the sigmoid function as an activation function. The tanh network 1504 has weights and biases, and uses a sigmoid tanh function as the activation function. After applying the weighted sum of h _t-1 and x _t , the sigmoid network 1502a provides a result value r _t to the multiplication operator 1506a. After applying the weighted sum of h _t-1 and x _t , the sigmoid network 1502b provides the resulting value z _t to the multiplication operators 1506b and 506c. The sigmoid network 1502b provides the result value to the multiplication operator 1506b via the 1-operator. Multiplication operator 1506b applies a multiplication to h _t-1 and the value provided through 1-operator from sigmoid network 1502b and provides it to the addition operator. The tanh network 1504 applies the tanh function to the input x _t at the current time t, and then

is provided to the addition operator through the multiplication operator 1506b. The addition operator applies the input values and outputs h _t as the result value.

Referring to FIG. 15B , the LSTM network has a structure in which hidden networks 1510-1 to 510-3 between an input layer and an output layer are repeated. Accordingly, when inputs x _t-1 , x _t , x _t+1 , etc. according to time are provided, the output from the hidden network 1510-1 for the input x _t- 1 at time t-1 is The hidden state value is input to the hidden network 1510-2 for the next time t together with the input x _t at the next time t. The hidden network 1510 - 2 includes sigmoid networks 1512a , 512b , 512c , tanh networks 1514a , 514b , multiplication operators 1516a , 516b , 516c , and an addition operator 1518 . Each of the sigmoid networks 1512a, 512b, 512c has a weight and a bias, and uses a sigmoid function as an activation function. Each of the tanh networks 1514a and 514b has a weight and a bias, and uses a sigmoid tanh function as an activation function.

The sigmoid network 1512a functions as a forget gate. The sigmoid network 1512a applies the sigmoid function to the weighted sum of the hidden state value h _t-1 of the hidden layer at the previous time and the input x _t at the current time, and then converts the result value to the multiplication operator 1516a. to provide. The result value of the sigmoid function is multiplied with the cell memory value C _t-1 of the previous time by the multiplication operator 1516a. Through this, the LSTM network can determine whether to forget the memory value of the previous point in time. That is, the output value of the sigmoid network 1512a indicates how long to maintain the cell memory value C _t-1 of the previous time.

를 곱셈 연산자(1516b)로 제공한다. 시그모이드 네트워크(1512b)의 결과 값 i_t 및 tanh 네트워크(1514)의 결과 값

은 곱셈 연산자(1516b)에 의해 곱해진 후, 덧셈 연산자(1510)에 제공된다. 이를 통해, LSTM 네트워크는 현재 시점의 셀 메모리 값 C_t에 현재 시점의 입력 x_t를 얼마나 반영할지를 결정하고, 결정에 따라 스케일링(scaling)할 수 있다. 덧셈 연산자(1510)에 의해, 망각 계수와 곱해진 이전 시점의 셀 메모리 값 C_t-1·f_t 및

가 합산된다. 이를 통해, LSTM 네트워크는 현재 시점의 셀 메모리 값 C_t를 결정할 수 있다.Sigmoid network 1512b and tanh network 1514 serve as input gates. The sigmoid network 1512b applies the sigmoid function to the weighted sum of the hidden state value h _{t-1 at the previous time point t-1} and the input x _t at the current time point t, and then applies the result value i _t to the multiplication operator ( 1516b). The tanh network 1514 applies the tanh function to the weighted sum of the hidden state value h _{t-1 at the previous time point t-1} and the input x _t at the current time point t, and then the resulting value

is provided as the multiplication operator 1516b. The resulting value i _t of the sigmoid network 1512b and the resulting value of the tanh network 1514 .

is provided to the addition operator 1510 after being multiplied by the multiplication operator 1516b. Through this, the LSTM network may determine how much to reflect the input x _t of the current time to the cell memory value C _t of the current time, and may perform scaling according to the determination. By the addition operator 1510, the cell memory value C _t-1 ·f _t at the previous point in time multiplied by the forgetting coefficient, and

is summed up Through this, the LSTM network may determine the cell memory value C _t of the current time.

The sigmoid network 1512c, the tanh network 1514b, and the multiplication operator 1516c serve as output gates. The output gate outputs a filtered value based on the cell state at the current time. The sigmoid network 1512c applies the sigmoid function to the weighted sum of the hidden state value h _{t-1 at the previous time point t-1} and the input x _t at the current time point t, and then applies the result value o _t to the multiplication operator ( 1516b). The tanh network 1514b applies the tanh function to the cell memory value C _t of the current time t, and then provides the result value to the multiplication operator 1516c. The multiplication operator 1516c generates the hidden state value h _t of the current time t by multiplying the result value of the tanh network 1514b and the result value of the sigmoid network 1512c. Through this, the LSTM network can control how long the cell memory value of the current time is maintained in the hidden layer.

The LSTM model has various deformation models. 15B shows an example of an LSTM deformation model. The LSTM model used in the present invention is not limited to the above-described embodiment, and various modified models may be used.

16 shows an example of a training procedure for segmentation according to an embodiment of the present invention. 16 exemplifies an operation method of a device having a computing capability (eg, the service server 170 of FIG. 1 ).

Referring to FIG. 16 , in step S1601 , the device acquires a heart image and a label. The cardiac image is a part of time-series images, and the cardiac images may have periodicity. The cardiac image may be a reference image. The label may indicate a boundary of at least one chamber of the cardiac image. That is, the label may indicate a boundary of at least one of the left ventricle, the left atrium, the right ventricle, and the right atrium of the heart image. Labels may optionally be expressed as annotations, masks, ground truth labels, and the like.

As another example, the device may acquire an ultrasound image and a label for the lung. Ultrasound images of the lungs are part of time-series images and may have periodicity of exhalation and inspiration. The label may indicate at least one boundary of the lung ultrasound image. The present invention may be applied to segmentation of time-series images having periodicity, and is not limited to the above-described embodiment. Lung ultrasound is a diagnostic method with high sensitivity. As an example, the device may distinguish pneumonia, atelectasis, tumors, diaphragm elevation, and pleural effusion through lung ultrasound, unlike chest X-rays taken at the patient's bedside. Chest X-rays are routinely performed to evaluate the occurrence of pneumothorax or hydrothorax after operations such as central venous catheterization. However, the sensitivity of chest X-rays is very low to detect a small amount of pneumothorax or hydrothorax that occurs after such a procedure. In contrast, pulmonary ultrasound can diagnose very small amounts of pneumothorax, consolidation, pulmonary edema, atelectasis, pneumonia, and pleural effusion.

In operation S1603 , the device may acquire a propagated heart image and a propagated label based on the acquired heart image. The device may acquire a propagated heart image and a propagated label using the heart image acquired in step S1601 and the image having a time-series relationship with the heart image. Specifically, the device may estimate the motion vector field of the cardiac image acquired in step S1601 and the image having a time-series relationship with the cardiac image. For example, the device may estimate a motion vector field based on an artificial intelligence model. Here, the device may estimate the motion vector field using CNN. Alternatively, the device may measure the motion based on an optical flow. As an example, the device may measure motion based on LiteflowNet. The device may obtain a propagated heart image and a propagated label based on the motion vector field. That is, the device may augment the cardiac image. Accordingly, the device may augment images included in the cycle based on a reference image that is a part of images having a cardiac cycle. The device may acquire a dataset sufficient to perform time series data modeling.

In step S1605, the device may learn an artificial intelligence model including time series data modeling. Specifically, the device may learn the artificial intelligence model based on the acquired heart image and label, and the propagated heart image and the propagated label. For example, an artificial intelligence model may be based on LSTM.

According to the embodiment described with reference to FIG. 16 , the device may acquire not only the cardiac image at time t, but also the cardiac image at time t-1 and the image at time t+1, which are adjacent thereto. The device may learn to perform prediction at time t based on the acquired image. That is, the device performs a prediction on the cardiac image at time t based on the cardiac image at time t-1, the cardiac image at time t, and the cardiac image at time t+1 to perform prediction. ) can be extracted. As an example, the device may extract features based on ResNet. Also, the device may learn the extracted features based on the LSTM. In addition, the device can use ConvLSTM for time-series cardiac image modeling. Also, the device may use 3D Conv to model time series data using adjacent frames. The feature extracted from the cardiac image may be related to a boundary of at least one of a left ventricle, a left atrium, a right ventricle, and a right atrium of the cardiac image. The device may extract features based on a plurality of images, and the number of the above-described embodiments is not limited. Artificial intelligence models related to various time series data modeling may be used, and the present invention is not limited to the above-described embodiment.

According to the embodiments described with reference to FIG. 16 , an image of the heart may be augmented. An example of the augmented images is shown in FIG. 17 below. 17 illustrates a cardiac image and a propagated cardiac image according to an embodiment of the present invention. Cardiac images may have a cardiac cycle. The device is a mid-diastole 1702, an End-Diastole 1704, a Mid-Systole 1706, and an End-Systole (End-Systole) as reference images among cardiac images. 1708), at least one image may be acquired. That is, the reference image may include an image captured in at least one state of a cardiac cycle, and in this case, the heart state may be determined based on the size of the left ventricle. As a specific example, the device may acquire images of the end-diastolic 1704 and the end-systolic 1708 as reference images. The reference image refers to an image including a ground truth (GT) label. The images have a time series relationship with each other and are suitable for use in time series data modeling. However, it may be insufficient for the device to learn the artificial intelligence of the time series data modeling structure just by acquiring the reference images from chapters 2 to 4 in one cardiac cycle. The device may propagate the image and label from the reference image based on the motion vector field as described above. Accordingly, the device may acquire a plurality of pseudo GTs 1722a to 1722h. The pseudo GTs 1722a to 1722h may constitute a cardiac cycle together with a reference image as a propagated image. When calculating the loss value by the loss function during training of the artificial intelligence model, the weight applied to the prediction result related to the pseudo GTs 1722a to 1722h may be set lower than the reference image.

Meanwhile, the pseudo image and the pseudo GT generated based on the motion estimation value of the image may be different from the shape of the actual heart. Accordingly, the weight may be used for updating a parameter weight value through network learning in an actual image and an actual GT. The generated pseudo image and pseudo GT may be distorted as the motion of the actual image and the actual GT increases. That is, the greater the time difference within one cardiac cycle, the more difficult it is to estimate the cardiac motion. Accordingly, the weight may decrease as the time difference r from the image serving as the actual image generation standard increases. That is, the weight may decrease as the distance from the reference image increases. Here, the weight may be determined as an experimental value. In addition, the heart cycle may be, for example, a normal heart rate of 60 to 100 beats per minute. In addition, in the case of a bradycardia patient with a slow heart rate, the pulse rate per minute may be 60 or less. In addition, in the case of a tachycardia patient with a rapid heartbeat, the pulse rate per minute may be 100 or more.

18 shows an example of a segmentation procedure according to an embodiment of the present invention. 18 exemplifies an operation method of a device having a computing capability (eg, the service server 170 of FIG. 1 ).

Referring to FIG. 18 , in step S1801 , the device acquires continuously photographed heart images. That is, the device may acquire a plurality of heart images. The plurality of acquired images may have a time-series relationship. For example, the heart images may be provided in real time during imaging by an ultrasound device.

In step S1803 , the device may segment the acquired heart image using an artificial intelligence model including time series data modeling. According to an embodiment, the device uses time-series images (eg, n images from time t to time t+n−1) as input data, and uses one of time t to time t+n−1 as input data. A segmentation result may be generated for an image at a time, a time after time t+n-1, or a time before time t. Here, the artificial intelligence model may be in a trained state based on a propagated heart image, a propagated label, and a weight, as in the procedure described with reference to FIG. 16 .

19A to 19C show specific examples of data augmentation according to an embodiment of the present invention. Specifically, FIG. 19 shows an example of an A2CH echocardiography view. 19A shows an original image and an annotation mask. 19B shows an original image and a propagated mask. 19C shows a propagated image and a propagated mask. The upper part of each picture shows the visualization of the image and the mask. The lower part of each picture represents the magnified of view, and the green line represents the LV boundary. A label and a mask may be used interchangeably.

Cardiac images are 2D images that change over time. To efficiently augment cardiac images and annotated masks, the device may measure a cardiac motion vector field and propagate reference data. Here, propagate may be a term indicating an example of an image conversion method, unlike propagate on a neural network. For example, the device may measure the cardiac motion vector field based on a convolutional neural network (CNN).

The device may generate pairwise data composed of a label and an image. As an example, the device may generate pairwise data including an image without a propagated label and a ground truth label. The device may augment this data. An image without a ground true label at a viewpoint separated by m (frame) from the reference image may be expressed as I ^i+m . The mth augmented propagated label may be expressed as L _P ^i+m .

The image without the augmented ground true label and the augmented propagated label may not match the boundaries. For example, an image without an augmented label and an augmented and propagated label may have non-consistent boundaries based on erroneous motion vector fields. In addition, as the propagation step increases, the boundary error may increase in the image without the augmented label and the augmented and propagated label. Referring to FIG. 19B , when the propagation step size of the non-propagated image and the propagated label is 2, there is an error between the image and the label. Referring to FIG. 19C , it can be seen that an error exists when the propagation step size of the non-propagated image and the propagated label is 6, and the error is larger than that of the case where the step size is 2. Sequential propagation of estimating a motion vector field using a past image and a current image may cause an error in the motion vector field and may distort the original image.

The device can reduce the error in the motion vector field by propagating the image as well as the label. In addition, the apparatus can augment accurate data by making the propagation step of the label the same as the propagation step of the image.

20D shows a specific implementation example of an artificial intelligence model for segmentation according to an embodiment of the present invention. Referring to FIG. 20 , the artificial intelligence model includes ResNet (2010a, 2010b) and LSTM (2020a, 2020b). In FIG. 20, ResNet (2010a, 2010b) is expressed as a plurality of networks, but this represents that the feature extraction operation by ResNet (2010a, 2010b) is performed for each time point, and the artificial intelligence model may include one ResNet. have. Similarly, although the LSTMs 2020a and 2020b are represented by a plurality of networks, the AI model may include one LSTM having a recursive structure in which an output is fed back as an input.

Referring to FIG. 20 , the apparatus performs prediction on the image at time t using not only the image at time t, but also an image at time t-1 and an image at time t+1 close to the image at time t. ResNet (2010a) can extract features of the image at time t-1, the image at time t, and the image at time t+1. The extracted features are input to the LSTM 2020a for implementing time series data modeling. The LSTM 2020a generates a prediction result 2030a based on the features, and transmits a hidden state corresponding to the prediction result 2030a to the LSTM 2020b. ResNet (2010b) extracts the features of the image at time t, the image at time t+1, and the image at time t+2, and the LSTM 2020b extracts the extracted features and the hidden state provided from the LSTM 2020a. A prediction result 2030b is generated based on .

Exemplary methods of the present invention are expressed as a series of actions for clarity of description, but this is not intended to limit the order in which the steps are performed, and each step may be performed simultaneously or in a different order if necessary. In order to implement the method according to the present invention, other steps may be included in addition to the illustrated steps, other steps may be included after excluding some steps, or additional other steps may be included except for some steps.

Various embodiments of the present invention do not list all possible combinations, but are intended to describe representative aspects of the present invention, and the details described in various embodiments may be applied independently or in combination of two or more.

In addition, various embodiments of the present invention may be implemented by hardware, firmware, software, or a combination thereof. For implementation by hardware, one or more Application Specific Integrated Circuits (ASICs), Digital Signal Processors (DSPs), Digital Signal Processing Devices (DSPDs), Programmable Logic Devices (PLDs), Field Programmable Gate Arrays (FPGAs), general purpose It may be implemented by a processor (general processor), a controller, a microcontroller, a microprocessor, and the like.

The scope of the present invention includes software or machine-executable instructions (eg, operating system, application, firmware, program, etc.) that cause operation according to the method of various embodiments to be executed on a device or computer, and such software or and non-transitory computer-readable media in which instructions and the like are stored and executed on a device or computer.

Claims (19)

A method for obtaining information from a medical image, comprising:
performing segmentation on regions of the heart in the medical image;
generating at least one reference line based on the segmentation;
determining at least one medical index based on the at least one baseline.
The method according to claim 1,
The at least one clinical indicator includes at least one of a length of the at least one baseline and a value determined based on the length of the at least one baseline.
The method according to claim 1,
The method further comprising generating at least one reference point based on the segmentation.
The method according to claim 3,
The at least one reference point is generated based on a center point of at least one of the zones.
The method according to claim 1,
wherein the zones include at least one of a first region, a second region, a third region, a fourth region, a fifth region, and a sixth region.
6. The method of claim 5,
The first area is located between the second area and the third area,
The second area is located between the fourth area and the first area,
The fourth area is located at the top or
The fifth area is adjacent to the right side of the first area, or
The sixth area is adjacent to the right side of the first area and is located at a lower end of the fifth area.
6. The method of claim 5,
The generating of the at least one reference line comprises:
identifying a transverse line passing through a reference point of the first area and a center of a left boundary line of the first area and crossing the first area;
identifying a first orthogonal line passing through the reference point and orthogonal to the transverse line; and
and generating a first baseline to include at least a portion of the first orthogonal line.
8. The method of claim 7,
The generating of the at least one reference line comprises:
identifying a second orthogonal line that passes through a junction between the first reference line and a segmentation contour of the second region and is orthogonal to the long axis of the second region; and
and generating a second reference line to include at least a portion of the second orthogonal line.
8. The method of claim 7,
The generating of the at least one reference line comprises:
identifying a third orthogonal line passing through a contact point between the first reference line and the segmentation closed curve of the third area and perpendicular to the long axis of the third area; and
and generating a third reference line to include at least a portion of the third orthogonal line.
8. The method of claim 7,
The generating of the at least one reference line comprises:
identifying a center line of a short axis of an area corresponding to the second area;
identifying a first vertical line perpendicular to the boundary line of the second region at a point where the center line and the boundary line of the region meet; and
generating a first reference line to include at least a portion of the first vertical line;
The first reference line is a method used to measure a diameter of the first region.
8. The method of claim 7,
The generating of the at least one reference line comprises:
identifying a second vertical line perpendicular to the boundary line of the third area at a point where the first reference line and the boundary line of the third area meet; and
generating a second reference line to include at least a portion of the second vertical line;
The second reference line is a method used to measure the thickness of the third region.
6. The method of claim 5,
The generating of the at least one reference line comprises:
identifying a point closest to the fourth area among points on the boundary line of the fifth area;
identifying a parallel line that includes the proximity point and is parallel to a junction of the fifth region and the first region; and
and generating a third reference line to include at least a portion of the parallel lines.
6. The method of claim 5,
The generating of the at least one reference line comprises:
identifying a sinus point on the boundary line of the fifth area;
identifying a vertical line that includes a point closest to the sinus point on the boundary line of the area corresponding to the sixth area, is parallel to the vertical axis of the medical image, and passes through the inside of the area corresponding to the sixth area; ; and
and generating a fourth reference line to include at least a portion of the vertical line.
6. The method of claim 5,
The generating of the at least one reference line comprises:
identifying a sinus point on the boundary line of the fifth area; and
On the boundary line of the region corresponding to the sixth region, including a point closest to the sinus point, perpendicular to the long axis of the region corresponding to the sixth region, and penetrating the interior of the region corresponding to the sixth region checking the vertical line;
and determining a fourth reference line to include at least a portion of the vertical line.
6. The method of claim 5,
The generating of the at least one reference line comprises:
identifying a vertical line including a point on the joint of the fifth region and the first region and perpendicular to the center line of the long axis of the fourth region; and
and generating a fifth reference line to include at least a portion of the vertical line.
6. The method of claim 5,
The generating of the at least one reference line comprises:
identifying a vertical line including a point on the joint of the fifth region and the first region and parallel to the vertical axis of the medical image; and
and generating a fifth reference line to include at least a portion of the vertical line.
The method according to claim 1,
wherein the zones include at least one of a right ventricle (RV), an interventricular septum (IVS), an aorta, a left ventricle (LV), an LV posterior wall (LVPW), or a left atrium (LA).
An apparatus for obtaining information from a medical image, comprising:
a storage unit for storing a set of instructions for operation of the device; and
It includes at least one processor connected to the storage unit,
the at least one processor,
performing segmentation on regions of the heart in the medical image;
generating at least one baseline based on the segmentation;
A device for controlling to determine at least one clinical index based on the at least one baseline.
A program stored on a medium for executing the method according to any one of claims 1 to 17 when operated by a processor.

Images