KR20210002197A - Method for automatic measurement of amniotic fluid volume based on artificial intelligence model - Google Patents

Abstract

The present invention relates to a method for automatically measuring the amount of uterine amniotic fluid based on an artificial intelligence model, which comprises the steps of: selecting and pre-processing a plurality of pre-acquired ultrasound images to obtain a plurality of learning images in which an intrauterine amniotic fluid region is labeled; learning an artificial intelligence model for predicting the amniotic fluid region through the learning images; predicting the amniotic fluid region in a patient image through the artificial intelligence model when a patient image obtained by ultrasonically imaging a patient uterus is input from an ultrasound device; extracting a region of interest by horizontally connecting a left edge to a right edge of the amniotic fluid region; extracting the longest straight line length secured with a predetermined width while sequentially scanning the region of interest in a horizontal direction; and converting the longest straight line length into the amount of amniotic fluid in accordance with a preset amniotic fluid amount conversion ratio. Accordingly, more objective and reliable amniotic fluid measuring operations can be performed.

Description

{Method for automatic measurement of amniotic fluid volume based on artificial intelligence model}

The present invention relates to a method for automatically measuring uterine amniotic fluid, and in particular, to a method for automatically measuring amniotic fluid in uterus based on an artificial intelligence model that enables more objective and reliable measurement of amniotic fluid in uterus.

Amniotic fluid plays an important role in the growth and development of the fetus in the womb. Amniotic fluid acts as a buffer to protect the fetus from external shocks, enables the development of the musculoskeletal system, plays an antibacterial role from infection, maintains fetal body temperature, develops the gastrointestinal tract and respiratory system, and contains moisture and nutrients necessary for the growth and development of the fetus. It serves as a source of supply, etc.

Abnormal amounts of amniotic fluid are associated with fetal malformations or various fetal diseases, and even in the case of a normal fetus, abnormal amniotic fluid increases the perinatal prevalence and mortality. Therefore, measurement of amniotic fluid during pregnancy is an important item for evaluating the well-being of the fetus.

In order to accurately measure the amount of amniotic fluid, it is necessary to measure it by the pigment dilution method through amniocentesis, but it is difficult to perform clinically, so it is generally measured through a semi-quantitative method using ultrasound.

In general, it is measured through a single maximum positive pocket measurement or positive index measurement. The maximum amniotic fluid pocket measurement is to measure the depth by finding the deepest amniotic fluid pocket, and amniotic fluid index measurement is to divide the uterus into four parts, measure the deepest part of each part, and then add the value.

However, in order to measure the amount of amniotic fluid through ultrasound, the measurer must visually search for the portion where the amniotic fluid is the most visible through the ultrasound image, and through this, manually select the deepest area to measure the vertical linear distance.

Therefore, 1) problems when selecting the part with the largest amount of water, 2) problems when selecting the deepest visible part of a section, and 3) problems when measuring the amount of water by drawing a straight line, and the positive water according to the subject of the measurer There is a problem that the amount of measurement varies.

Accordingly, as to solve the above problems, the present invention is a method for automatically measuring uterine amniotic fluid volume based on an artificial intelligence model that detects the deepest amniotic fluid region from an ultrasound image using an artificial intelligence module and performs an amniotic fluid volume measurement operation based thereon. Want to provide.

In addition, it is intended to provide a method for automatically measuring uterine amniotic fluid volume based on an artificial intelligence model that enables both ultrasound imaging of pregnant women and amniotic fluid calculation operations to be performed simultaneously.

The object of the present invention is not limited to the object mentioned above, and other objects not mentioned will be clearly understood by those of ordinary skill in the art from the following description.

As a means for solving the above problem, according to an embodiment of the present invention, by selecting and preprocessing a plurality of previously acquired ultrasound images, obtaining a plurality of learning images labeled with an intrauterine amniotic fluid region; Training an artificial intelligence model for predicting a positive region through the plurality of training images; When an ultrasound image of the patient's uterus is input from an ultrasound device, predicting a positive region in the patient image through the artificial intelligence model; Extracting a region of interest by connecting horizontally from a left edge to a right edge of the positive region; Extracting the longest straight line length having a predetermined width while sequentially scanning the region of interest in a horizontal direction; And converting and outputting the longest straight line length into a positive amount according to a preset conversion ratio of the amount of amniotic fluid. It provides a method for automatically measuring uterine amniotic fluid amount based on an artificial intelligence model.

The method of automatically measuring the amount of uterine amniotic fluid may further include determining and notifying the final amniotic fluid amount by summing all the amniotic fluid amounts of the N areas when the ultrasound imaging site of the patient is N (N is a natural number of 2 or more).

The artificial intelligence model is characterized in that it is implemented as a U-Net model.

The acquiring of the training image may include selecting an ultrasound image to be used for training of an artificial intelligence model, performing binarization based on a threshold value, and leaving only pixels having a pixel value of 1; Labeling after connecting and grouping adjacent pixels, and determining an ultrasound region based on a distance between a center of the labels and an image center value; Connecting boundary values of the ultrasound region, filling the inside, and performing a multiplication operation with the ultrasound image to obtain a training image; Unifying the size of the training image to a preset size; And detecting a positive region in the training image, and displaying and outputting a positive region detection result on the learning image.

The detecting of the longest positive depth may include generating a scanning line vertically crossing the region of interest; Generating a plurality of straight lines based on an intersection point between the scanning line and a positive area outline while horizontally moving the scanning line at a predetermined pixel interval; And extracting the longest straight line having a predetermined width interval based on an outline of the positive number region among the plurality of straight lines, and converting the length of the longest straight line to the longest positive depth, and outputting the same. .

In addition, in the method of automatically measuring the amount of uterine amniotic fluid, when an ultrasound imaging angle of an ultrasound device is additionally input, the display angle of the ROI is corrected according to the ultrasound imaging angle, and the scanning line crosses the ROI by the camera imaging angle. It characterized in that it further comprises a step.

At this time, the display angle of the region of interest is calculated by calculating a correction angle according to the equation of "ultrasonic photographing angle-90°", and if the correction angle is-value, the region of interest is rotated clockwise by the correction angle, In the case of the correction angle + value, it is characterized in that the region of interest is rotated counterclockwise by the correction angle.

The present invention detects the deepest amniotic fluid region from an ultrasound image using an artificial intelligence module, and then calculates the amniotic fluid amount based on this, so that subjective intervention by the measurer is not required, and a more objective and reliable amniotic fluid amount measurement operation is possible. To lose.

In addition, by simultaneously performing ultrasound imaging of pregnant women and calculating the amount of amniotic fluid, it is possible to secure fast response.

1 is a view for explaining a method for automatically measuring uterine amniotic fluid according to an embodiment of the present invention.
2 is a diagram for explaining a step of obtaining a learning image according to an embodiment of the present invention.
3 is a diagram for explaining an artificial intelligence model learning step according to an embodiment of the present invention.
4 and 5 are diagrams for explaining a step of extracting and correcting an ROI according to an embodiment of the present invention.
6 to 9 are diagrams for explaining a method of scanning an ROI according to an embodiment of the present invention.

Objects and effects of the present invention, and technical configurations for achieving them will become apparent with reference to the embodiments described below in detail together with the accompanying drawings. In describing the present invention, when it is determined that a detailed description of a known function or configuration may unnecessarily obscure the subject matter of the present invention, a detailed description thereof will be omitted.

In addition, terms to be described later are terms defined in consideration of functions in the present invention, which may vary depending on the intention or custom of users or operators.

However, the present invention is not limited to the embodiments disclosed below, and may be implemented in various different forms. These embodiments are provided only to make the disclosure of the present invention complete, and to fully inform the scope of the invention to those skilled in the art to which the present invention pertains, and the present invention is defined by the scope of the claims. It just becomes. Therefore, the definition should be made based on the contents throughout this specification.

1 is a diagram showing a system for automatically measuring a quantity of water according to an embodiment of the present invention.

Referring to FIG. 1, the system for automatically measuring the amount of amniotic fluid of the present invention includes an ultrasound device 100 that acquires and provides an ultrasound image by photographing a patient's abdomen through an ultrasound probe 110, and interlocks with the ultrasound device 100, And a pumping quantity measuring device 300 for performing an automatic pumping quantity measurement operation.

The pumping quantity measuring apparatus 300 of the present invention pre-trains the artificial intelligence model for predicting the positive water region by using the obtained ultrasonic images, and then inputs the ultrasonic image obtained through the ultrasonic device 100 into the artificial intelligence model. , Let the artificial intelligence model automatically detect the positive region in the ultrasound image. In addition, the deepest part of the amniotic fluid area is automatically searched, and the amniotic fluid amount is calculated based on this.

In addition, if necessary, the ultrasonic device 100 may include a rotation angle sensor 120 coupled to the ultrasonic probe 110, and additionally sense and output an ultrasonic photographing angle using this. In this case, the pumping amount measuring device 300 of the present invention additionally performs the operation of calculating the amount of water by reflecting the ultrasound photographing angle through the rotation angle sensor 210, so that the measurer must vertically mount the ultrasonic probe 110 on the mother's abdomen. Do not have to.

2 is a view for explaining a method for automatically measuring the amount of uterine amniotic fluid according to an embodiment of the present invention.

Referring to FIG. 2, the method of the present invention selects and preprocesses a plurality of previously acquired ultrasound images to obtain a plurality of learning images labeled with an intrauterine amniotic fluid region (S1), through the plurality of training images. Learning an artificial intelligence model for predicting an amniotic fluid area (S2), when an ultrasound image of the patient's uterus is input from an ultrasound device (S3), predicting an amniotic fluid area in the patient image through the artificial intelligence model (S4), extracting a region of interest by horizontally connecting from the left edge to the right edge of the positive region (S5), extracting the longest straight line length while sequentially scanning the region of interest in the horizontal direction (S6), A method for automatically measuring uterine amniotic fluid amount based on an artificial intelligence model comprising the step (S7) of converting and outputting the longest straight line length to amniotic fluid according to a set amniotic fluid conversion ratio. And the like.

In particular, in the present invention, steps S3 to S7 may be performed while ultrasound imaging of a pregnant woman is being performed, and as a result, an operation of measuring an amount of amniotic fluid can be performed simultaneously with ultrasound imaging of a pregnant woman.

In addition, this amniotic fluid measurement operation can be performed on one site basis, but if necessary, after dividing the uterus into N areas, the amniotic fluid level measurement operation is repeatedly performed on each of the N areas, and all of these amniotic fluid measurement results are considered (e.g. For example, by adding up or averaging), the final quantity of water can be calculated and reported.

Hereinafter, the method of the present invention will be described in more detail with reference to FIGS. 3 to 7.

3 is a view for explaining a learning image acquisition step according to an embodiment of the present invention.

The size of the ultrasound image and the location of the ultrasound display area vary according to the type of ultrasound device. Accordingly, in the present invention, an ultrasound image size and an ultrasound display area provided by various types of ultrasound devices are unified through an image preprocessing process, and then provided to the artificial intelligence model as a learning image.

To this end, one ultrasound image to be used for learning an artificial intelligence model is selected and loaded from among a plurality of ultrasound images previously stored in the medical database (S11).

After the loaded ultrasound image is binarized based on a threshold, only pixels having a pixel value of 1 are left (S12). Then, adjacent pixels are connected and grouped using the "Connected Component With Stats function" and then labeled (S13).

Labels whose value is less than a preset value (e.g., 220) by calculating the distance between the centroid of the labels and the center value of the ultrasound image, that is, the distance between the center and the center of the ultrasound image is less than the preset value. It is determined that it is an ultrasonic region (S14).

After connecting the boundary values of the ultrasound area using the "Convex_hull function", the area to be learned is acquired by filling the inside (S15). In FIG. 2, the values of pixels in the yellow portion are 1, and all other pixels are 0.

In order for the artificial intelligence model to learn only the learning target area of step S15, the ultrasound image of step S11 is multiplied by the ultrasound area of step S15 to obtain a training image (S16). During the multiplication operation, the values of pixels at the same position are multiplied. Since the pixel values in step S15 are displayed as 1 and 0 only, the portion of the image in step S11 where the pixel value is 1 remains as it is. The part with a value of 0 becomes 0 and disappears.

In addition, considering that each ultrasound device has different image sizes, the size of the training image is unified to a preset size (for example, 1152 mm × 872 mm) (S17). If the size of the training image is smaller than the preset size value, pixels with a value of 0 are added to the right and lower portions of the image to match the image size.

Finally, the label image is contoured to detect the positive region in the training image, and the positive region detection result is additionally displayed on the learning image to be output (S18).

4 is a diagram for explaining an artificial intelligence model learning step according to an embodiment of the present invention.

In the present invention, the artificial intelligence model is implemented as a U-Net model mainly used for medical image segmentation.

As shown in FIG. 3, the U-net model is divided into an encoder that reduces an image and a decoder that grows an image, which is driven as follows.

The encoder is repeated 6 times in the order of "Convolution -> Dropout -> Convolution -> MaxPooling", and the last nth time, MaxPooling is not performed. -Executed once.

The kernel used in convolution is 3*3, and ReLu (Rectified Linear Unit) is used as the activation function. By giving'same' as the padding value, the input and output of the convolution have the same image size. Each time convolution is performed, the number of channels increases, and the number of channels is currently 2 ^indexes . Also,'he_normal' is used as the weight initialization method. Dropout is to prevent overfitting, and some networks are omitted. Omit 0.1 networks in the current index. Maxpooling allows you to extract the largest feature using 2*2 Maxpooling, and when calculating Maxpooling, the size of the image is reduced by 1/2 compared to the existing size.

The decoder is repeated five times in the order of "ConvoltionTranspose -> concatenate -> convolution -> dropout -> convolution).

There are two types of convolutions: convolution and convolution transfer. Convolution is the same as the convolution of the encoder, and convolution transfer is the point that the size of the kernel used is 2*2, and the size of the output image is twice the size of the input image due to the inverse operation of convolution (kernel = 2*2). There is a difference in that it increases as ).

Conquednate is a process of combining the decoder and the encoder of the current index of the decoder, and corresponds to the gray horizontal line in FIG. 3. The dropout is the same as the encoder dropout and omits the network as much as 0.1 of the current index. After completing the decoder, convolutional transfer is applied once more, and the model is completed.

At this time, the convolutional transpose gives 1 as the value of the filters, so that one channel, that is, one image, is output. 'Sigmoid' is applied as the activation function, and'glorot_normal' is used as the weight initialization method.

5 is a diagram for describing a step of extracting and correcting an ROI according to an embodiment of the present invention.

When an ultrasound image of the patient's uterus is acquired through step S3, the patient image is input to the artificial intelligence model to predict the amniotic fluid region. Then, the size of the patient image is unified to a preset size value (eg, 1152 mm × 872 mm) (S41).

Acquires and stores the coordinates of the most end points of the left, right, top, and bottom based on the outline of the positive area. In FIG. 5, the leftmost coordinates are displayed in red, the rightmost coordinates are yellow green, the uppermost coordinates are blue, and the lowermost coordinates are displayed in light blue (S42).

A region of interest horizontally connected from the left edge to the right edge of the positive region is extracted based on the leftmost and rightmost coordinates of the positive region (S43).

6 is a diagram illustrating a method of scanning an ROI according to an embodiment of the present invention.

In the present invention, a scanning line vertically crossing the region of interest is generated (S51).

Then, the unit of the x-axis of the linear equation of the scanning line is changed from the unit of a decimal point (float) of 0.1 to the unit of an integer type (S52) so as to enable comparison for each pixel of the image (S52).

First, redundant values and values out of the range (1152, 872) among the pixels included in the scanning line are deleted, and stored in one line (S53).

And by setting a bool variable named'flag_line', it calculates whether or not the line starts. The values of the straight lines are viewed one by one, and the start and end values of the straight lines are calculated according to whether or not the corresponding pixel overlaps the predicted positive region and the value of the flag_line variable, and these are stored in one array (line_arr) (S54). The algorithm for each situation is shown in Table 1 below.

Predicted positive domain (predict) flag_line Do predict False The current pixel becomes the starting pixel of the straight line.
Change the value of the flag_line variable to True. Not predict True The current pixel becomes the end pixel of the straight line.
Change the value of the flag_line variable to False.
Store the start and end points of the line in line_arr.

Steps S53 and S54 are repeatedly performed while horizontally moving the scanning line at a predetermined pixel interval on the region of interest until scanning of the region of interest is completed (S55) (S56).

When the scanning of the region of interest is completed, the longest straight line length is found based on several pairs of points (start of straight line, end of straight line) stored in line_arr (S55).

In this process, in order to increase the accuracy of the pumping quantity measurement and to minimize the influence on the pumping wall, the length of a straight line that secures a certain amount of width is searched based on the outline of the pumping area. 9 is a diagram for explaining a method of finding a line having a certain width based on an outline of a positive number area in a process of finding the longest length. That is, in the present invention, during the search process for the deepest place, an area in which a certain width is secured based on the positive wall can be measured.

Finally, after converting the unit of the length of the longest straight line from pixels to mm, it is obtained and output as the longest positive depth (S56).

The ratio between the pixel and mm for converting the unit of the length of the longest straight line for performing step S56 from pixel to mm may be obtained and stored in advance, but if necessary, it may be obtained directly from the ultrasound image through the method of FIG. May be.

For reference, length indices for guiding the image size scale are displayed in a line in the ultrasound image. Accordingly, after selecting an index display area in which length indices are displayed in a row (eg, 100 columns on the left of the image), the number of pixels having a value other than 0 is counted in column units (S56-1).

However, when a color palette exists in a partial area of a color image, the pixel counting operation is performed after excluding the color palette display area. That is, when all pixel values in a column are examined one by one, and when pixels having a pixel value (brightness) of 50 or more are connected to 100 or more cells, exception processing is performed by excluding the column (S56-2).

In addition, among the columns included in the index display area, the column having the largest counted number of pixels is selected as the reference column (S56-3).

All values with a pixel value of 1 or more in the column are collected in the index_arr array. From the last pixel value (final) of index_arr, the index_arr value is examined one by one. If the value is not connected to the last pixel value (final), the corresponding value is determined to be the next cell, and this is called the next pixel value (beside). The value of final-beside means the interval between indexes (for example, 10mm), and the ratio between pixels and mm = length of the interval between indexes/(final-beside)mm is derived from this (S56-4).

In addition, if the ultrasound apparatus 100 of the present invention provides an ultrasound imaging angle in addition to an ultrasound image, as shown in FIG. 8, after correcting the display angle and the scanning angle of the ROI according to the ultrasound imaging angle, the largest positive depth is searched. It also allows you to perform actions.

In other words, after calculating the correction angle according to the equation of "Ultrasonic shooting angle-90°", if the correction angle is-value, the ROI is rotated clockwise by the correction angle, and if the correction angle + value, the ROI is By rotating clockwise by the correction angle, the display angle of the region of interest is corrected. And after correcting the angle of the scanning line according to the ultrasound imaging angle (i.e., after making the scanning line cross the region of interest by the camera shooting angle), horizontally moving it at preset pixel intervals, the largest positive depth can be searched. Make it possible.

At this time, the ultrasound photographing angle is set as the reference angle (eg, 90°) with the ultrasound probe 110 vertically mounted on the mother's abdomen, and then the amount of change in the yaw axis angle relative to the reference angle according to the movement of the ultrasound probe (90°). -X°).

The above description is merely illustrative of the technical idea of the present invention, and those of ordinary skill in the art to which the present invention pertains will be able to make various modifications and variations without departing from the essential characteristics of the present invention. Accordingly, the embodiments disclosed in the present invention are not intended to limit the technical idea of the present invention, but to explain the technical idea, and the scope of the technical idea of the present invention is not limited by these embodiments. The scope of protection of the present invention should be interpreted by the following claims, and all technical ideas within the scope equivalent thereto should be interpreted as being included in the scope of the present invention.

Claims (7)

Selecting and pre-processing a plurality of previously acquired ultrasound images to obtain a plurality of learning images labeled with intrauterine amniotic fluid regions;
Training an artificial intelligence model for predicting a positive region through the plurality of training images;
When an ultrasound image of the patient's uterus is input from an ultrasound device, predicting a positive region in the patient image through the artificial intelligence model;
Extracting a region of interest by connecting horizontally from a left edge to a right edge of the positive region;
Extracting a longest straight line length having a predetermined width while sequentially scanning the region of interest in a horizontal direction; And
A method for automatically measuring uterine amniotic fluid amount based on an artificial intelligence model, comprising converting and outputting the longest straight line length into amniotic fluid according to a preset amniotic fluid conversion ratio. The method of claim 1, wherein the artificial intelligence model
A method for automatically measuring uterine amniotic fluid based on an artificial intelligence model, which is implemented as a U-NET model. The method of claim 1, wherein obtaining the training image
Selecting an ultrasound image to be used for training an artificial intelligence model, performing binarization based on a threshold value, and leaving only pixels having a pixel value of 1;
Labeling after connecting and grouping adjacent pixels, and determining an ultrasound region based on a distance between a center of the labels and an image center value;
Connecting boundary values of the ultrasound region, filling the inside, and performing a multiplication operation with the ultrasound image to obtain a training image;
Unifying the size of the training image to a preset size; And
Detecting an amniotic fluid area in the training image, and displaying and outputting the amniotic fluid area detection result on the training image. The method of claim 1, wherein detecting the longest positive depth
Generating a scanning line crossing the region of interest in a vertical direction;
Generating a plurality of straight lines based on an intersection point between the scanning line and an outline of a positive region while horizontally moving the scanning line at a predetermined pixel interval;
Extracting the longest straight line having a predetermined width interval based on an outer line of the positive number region among the plurality of straight lines; And
And extracting the longest straight line among the plurality of straight lines, converting the length of the longest straight line into the longest amniotic fluid depth, and outputting the output. The method of claim 4,
When an ultrasound imaging angle is additionally input from an ultrasound device, the display angle of the ROI is corrected according to the ultrasound imaging angle, and the scanning line further comprises the step of making the ROI cross the ROI by a camera imaging angle. A method of automatically measuring uterine amniotic fluid based on an artificial intelligence model. The method of claim 5, wherein the display angle of the region of interest is
After calculating the correction angle according to the equation of "ultrasonic shooting angle-90°", if the correction angle is-value, the region of interest is rotated clockwise by the correction angle, and if the correction angle is + value, the interest The method of automatically measuring uterine amniotic fluid amount based on an artificial intelligence model, characterized in that the area is corrected in a manner that is rotated by the correction angle in a counterclockwise direction. The method of claim 1,
When the patient's ultrasound imaging area is N (N is a natural number of 2 or more), the uterine amniotic fluid amount based on an artificial intelligence model further comprises the step of determining and notifying the final amniotic fluid in consideration of all the amniotic fluid amounts of the N areas How to measure.

Images