WO2021129323A1

WO2021129323A1 - Ultrasound image lesion describing method and apparatus, computer device, and storage medium

Info

Publication number: WO2021129323A1
Application number: PCT/CN2020/133026
Authority: WO
Inventors: 李焰驹
Original assignee: 飞依诺科技(苏州)有限公司
Priority date: 2019-12-25
Filing date: 2020-12-01
Publication date: 2021-07-01
Also published as: CN113034426B; CN113034426A

Abstract

An ultrasound image lesion describing method and apparatus, a computer device, and a storage medium. The method comprises: recognizing a lesion in an ultrasound image to obtain a corresponding region of interest (202); detecting the region of interest by means of an image segmentation algorithm to obtain a lesion contour (204); and generating lesion description information according to a geometric topology structure of the lesion contour and gray-scale features of the region of interest (206). According to the method, the lesion in the ultrasound image is recognized to position the corresponding region of interest, then the region of interest is detected by means of the image segmentation algorithm to obtain the lesion contour, and the lesion description information is automatically generated according to the geometric topology structure of the lesion contour and the gray-scale features of the region of interest, thereby effectively reducing the workload of a doctor, and improving diagnosis efficiency.

Description

Ultrasound image lesion description method, device, computer equipment and storage medium

Technical field

This application relates to the technical field of medical image processing, and in particular to a method, device, computer equipment, and storage medium for describing an ultrasound image lesion.

Background technique

Breast cancer is a common malignant tumor in female diseases and has become one of the diseases that seriously threaten women's health. Early detection, early diagnosis, and early treatment are the basic principles currently adopted in medicine for the prevention and treatment of breast cancer. Ultrasound imaging has become one of the main methods of clinical diagnosis of breast tumors due to its advantages of non-invasiveness, non-radiation, and low cost.

However, due to the influence of imaging equipment, ultrasound images often have large noise, low contrast, uneven grayscale, varying degrees of attenuation and infiltration effects, etc., making the surface of breast tumors more similar to the surrounding normal tissues, that is, ultrasound The image has a weak ability to express the morphology of human organs, or the presentation of organs in the image is blurry and abstract; in addition, breast tumors vary greatly among individuals. Therefore, the judgment and reading of the lesion area in breast ultrasound images requires clinicians to have a higher professional level and rich experience. It is difficult for general doctors to accurately and quickly compare the breast tumor area with the normal surroundings in the ultrasound image. Distinguish the tissues and make a description of the lesion. Especially when it is necessary to perform lesion analysis on a large number of patients, the doctor must not only analyze the lesion, but also manually fill in the results of the lesion analysis. When the number of patients is large, the workload of doctors will increase dramatically.

Summary of the invention

Based on this, it is necessary to provide a method, device, computer equipment, and storage medium for describing a lesion in an ultrasound image in response to the above-mentioned problem of a large workload when a doctor analyzes a lesion in an ultrasound image.

In order to achieve the foregoing objective, on the one hand, an embodiment of the present application provides a method for describing a lesion in an ultrasound image, and the method includes:

Identify the lesion in the ultrasound image to obtain the corresponding region of interest;

Detect the region of interest through the image segmentation algorithm to obtain the outline of the lesion;

The description information of the lesion is generated according to the geometric topology structure of the outline of the lesion and the gray-scale features of the region of interest.

On the other hand, an embodiment of the present application also provides an ultrasound image lesion description device, including: a region of interest recognition module for identifying a lesion in the ultrasound image to obtain a corresponding region of interest; a lesion segmentation module for The region of interest is detected by the image segmentation algorithm to obtain the lesion outline; the lesion description information generation module is used to generate the lesion description information according to the geometric topology structure of the lesion outline and the gray-scale features of the region of interest.

In another aspect, an embodiment of the present application also provides a computer device, including a memory and a processor, the memory stores a computer program, and the processor implements the steps of the above method when the computer program is executed.

In another aspect, the embodiments of the present application also provide a computer-readable storage medium on which a computer program is stored, and when the computer program is executed by a processor, the steps of the method described above are implemented.

The above-mentioned ultrasound image lesion description method, device, computer equipment and storage medium identify the lesion in the ultrasound image to locate the corresponding region of interest, and then detect the region of interest through the image segmentation algorithm to obtain the outline of the lesion. And according to the geometric topological structure of the lesion contour and the gray-scale features of the region of interest, the lesion description information is automatically generated, thereby effectively reducing the workload of the doctor and improving the diagnosis efficiency.

Description of the drawings

Fig. 1 is an application environment diagram of an ultrasound image lesion description method in an embodiment;

2 is a schematic flowchart of a method for describing a lesion in an ultrasound image in an embodiment;

Figure 3 is a schematic diagram of an ultrasound image in an embodiment;

Fig. 4 is a schematic diagram of a region of interest obtained by performing target detection on Fig. 3;

FIG. 5 is a schematic diagram of the outline of the lesion obtained after segmenting the image segmentation algorithm in FIG. 4;

Figure 6 is a schematic diagram showing the outline of the lesion on the original image;

FIG. 7 is a schematic flowchart of the step of obtaining the contour of a lesion through an image segmentation algorithm in an embodiment;

Fig. 8A is a schematic diagram of an ultrasound image in another embodiment;

FIG. 8B is a schematic diagram of a region of interest obtained by performing target detection on FIG. 8A;

FIG. 8C is a schematic diagram of the initial binary image obtained after threshold segmentation of FIG. 8B;

FIG. 9 is a schematic diagram of a new binary image obtained after performing a morphological opening operation on FIG. 8C;

Figure 10 is a schematic diagram of roughly estimating the lesion area after analyzing Figure 9;

Fig. 11 is a schematic diagram of a created binary image with the same size as Fig. 8B;

FIG. 12 is a schematic diagram of a lesion outline obtained after segmentation based on distance regularization;

FIG. 13 is a schematic flowchart of the step of obtaining the contour of a lesion through an image segmentation algorithm in another embodiment;

Fig. 14A is a schematic diagram of an ultrasound image in another embodiment;

Fig. 14B is a schematic diagram of a region of interest obtained by performing target detection on Fig. 14A;

Fig. 15 is a schematic diagram of a binarized image corresponding to an initial zero level set function;

FIG. 16 is a schematic diagram of pixel point x and its neighboring pixel y;

FIG. 17 is a schematic diagram of a binary image of the target zero level set corresponding to the target level set function;

Fig. 18 is a schematic diagram of a new target zero level set binary image obtained after processing Fig. 17;

Figure 19 is a schematic diagram of the lesion area determined after analyzing Figure 18;

Figure 20A is a schematic diagram of a low echo grayscale histogram;

Fig. 20B is a schematic diagram of an echoless grayscale histogram;

Figure 21 is a schematic diagram of a complex plane of the mapping;

Fig. 22A is a schematic diagram of fitting the ellipse in Fig. 6;

22B is a schematic diagram of the angle between the major axis of the ellipse and the horizontal coordinate axis in FIG. 22A;

Figure 23 is a schematic flow chart of the steps of describing whether a lesion is calcified in an embodiment;

Fig. 24A is a schematic diagram of an ultrasound image in still another embodiment;

FIG. 24B is a schematic diagram of a region of interest obtained by performing target detection on FIG. 24A;

FIG. 24C is a schematic diagram of a binary image of the contour of the lesion obtained after segmentation of FIG. 24B using the LBF lesion segmentation algorithm;

FIG. 24D is a schematic diagram obtained based on FIG. 24B and FIG. 24C;

Figure 24E is a schematic diagram obtained after performing morphological closing operations on Figure 24D;

Fig. 24F is a schematic diagram of a binary image obtained after seed growth of Fig. 24E;

25 is a structural block diagram of an ultrasound image lesion description device in an embodiment;

Fig. 26 is a diagram of the internal structure of a computer device in an embodiment.

Detailed ways

In order to make the purpose, technical solutions, and advantages of this application clearer and clearer, the following further describes the application in detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application, and are not used to limit the present application.

The ultrasound image lesion description method provided in this application can be applied to the application environment as shown in FIG. 1. Wherein, the terminal 102 and the server 104 communicate through a network. In this embodiment, the terminal 102 may be a device with an ultrasound image collection function, or a device that stores the collected ultrasound images, and the server 104 can be an independent server. Or it can be realized by a server cluster composed of multiple servers. Specifically, the terminal 102 is used to collect or store ultrasound images, and send the collected or stored ultrasound images to the server 104 through the network, and the server 104 recognizes the lesions in the ultrasound images to locate the corresponding region of interest, and then pass The image segmentation algorithm detects the region of interest to obtain the outline of the lesion, and automatically generates description information of the lesion according to the geometric topology of the outline of the lesion and the gray-scale features of the region of interest, which effectively reduces the workload of the doctor and improves Diagnostic efficiency.

In one embodiment, as shown in FIG. 2, a method for describing lesions in ultrasound images is provided. Taking the method applied to the server in FIG. 1 as an example for description, the method includes the following steps:

Step 202: Identify the lesion in the ultrasound image to obtain a corresponding region of interest.

Among them, ultrasound images are clinically related to breast, thyroid, liver, kidney, spleen, etc. ultrasound images. A region of interest (region of interest, ROI for short) is an image region selected from an ultrasound image that needs to be processed, and this region is the focus of image analysis. Specifically, the lesion in the ultrasound image can be automatically identified and located through target detection, so as to obtain the corresponding region of interest in the ultrasound image. Among them, target detection may be automatic target detection based on image recognition algorithms, or manual target detection based on manual frame selection. As shown in FIG. 3, it is a schematic diagram of the acquired ultrasound image, and FIG. 4 is a schematic diagram of the region of interest obtained from the target detection in FIG. 3.

Step 204: Detect the region of interest through an image segmentation algorithm to obtain the contour of the lesion.

Among them, the image segmentation algorithm includes a lesion segmentation algorithm based on Distance Regularized Level Set Evolution (DRLSE) and a lesion segmentation algorithm based on Local Binary Fitting (LBF) evolution. Specifically, the above-mentioned image segmentation algorithm is used to detect the region of interest shown in FIG. 4 to obtain the corresponding segmentation result, that is, the binary image shown in FIG. 5, and the white area of the binary image is the segmentation The area of the lesion is the result of image segmentation. The medical image term in Figure 3 is a mask, which represents the boundary contour of the segmented object, so the segmentation result is a collection of the coordinates of a series of image points. The coordinates of these points are presented in the original image (ie the original ultrasound image as shown in Figure 3) to obtain the outline of the lesion, as shown in Figure 6. The rectangular box in Figure 6 represents the ROI (ie the region of interest), which is irregular The curved box represents the outline of the lesion, and the inside of the irregular curved box is the lesion area.

Step 206: Generate lesion description information according to the geometric topological structure of the outline of the lesion and the gray level features of the region of interest.

Among them, the lesion description information includes the growth direction of the lesion, the echo type, whether there is calcification, and the shape, smoothness, and clarity of the outline of the lesion. In this embodiment, based on the lesion outline and the region of interest obtained in the above steps, the lesion description information is automatically generated according to the geometric topological structure of the lesion outline and the gray level characteristics of the region of interest.

The above-mentioned ultrasound image lesion description method uses the identification of the lesion in the ultrasound image to locate the corresponding region of interest, and then detects the region of interest through the image segmentation algorithm to obtain the outline of the lesion, and according to the geometric topology of the outline of the lesion And the gray-scale features of the region of interest automatically generate lesion description information, thereby effectively reducing the workload of the doctor and improving the diagnosis efficiency.

In an embodiment, as shown in FIG. 7, taking the image segmentation algorithm as a lesion segmentation algorithm based on distance regularization level set evolution as an example, the specific implementation method for detecting the region of interest to obtain the outline of the lesion is described, including:

Step 702: Perform threshold segmentation on the region of interest to obtain a corresponding initial binary image, and estimate the centroid of the lesion area based on the initial binary image.

Specifically, perform target detection on the original ultrasound image (Figure 8A) to obtain the corresponding region of interest (Figure 8B), and perform threshold segmentation on the region of interest based on the Otsu method (OTSU) to obtain the corresponding initial binary image (As shown in FIG. 8C), the initial binary image is morphologically opened by using the set mask size to obtain a new binary image after the operation (as shown in FIG. 9). According to the neighborhood connectivity criterion, 8 neighborhoods are used to extract the connected components of each neighborhood in the new binary image. At this time, there are 8 connected components in Figure 9, that is, 8 independent contours (each white area is a contour ). Then calculate the area of each connected component area (that is, the number of pixels occupied by the connected component area). The area where the connected component with the largest area is located among the areas where each connected component is located is roughly estimated as the lesion area (as shown in FIG. 10). The image moment algorithm is used to calculate the center of mass of the lesion area. The image moment algorithm can specifically use the Hu moment algorithm, that is, the center of mass coordinate O(X0,Y0) of the lesion area is obtained through the Hu moment algorithm.

Step 704: Construct an initial zero level set function according to the centroid of the region of interest and the lesion region.

Specifically, an initial image with the same size as the ROI is created according to the size of the ROI, and then based on the centroid of the lesion area and the minimum side length of the region of interest, the initial area of the level set is determined in the initial image. That is, a circular area with a pixel gray value of -2 (displayed as black) is constructed in the initial image, the gray value of pixels outside the circular area is 2 (displayed as white), and the center coordinates of the circular area are The center of mass of the lesion area, the radius is the side length of the smaller side of the ROI divided by 5. The circular boundary of the circular area is the initial contour of the level set evolution, which is the initial area of the level set.

Since the pixel gray value of the initial area of the level set in the above initial image is -2, and the pixel gray value of other areas is 2, the binary image of the initial image is obtained, that is, the image shown in Figure 11. The valued image is the initial zero-level set image, and it is also the initial object of the evolutionary algorithm. Also, because digital images can be represented by a binary function, the binary image shown in Figure 11 can be represented by the following function:

Among them, x, y are the horizontal and vertical coordinates of the image, and R ₀ represents the ROI image area.

Step 706: Use the distance regularized level set evolution algorithm to iteratively calculate the initial zero level set function based on the region of interest to obtain the target level set function at the end of the evolution.

Specifically, in the DRLSE algorithm, the energy functional function of the image information is defined as:

E(φ)=μR _p (φ)+E _ext (φ)(1.2), where μ>0 is a constant, E _ext (φ) is the external energy functional, which makes the zero level set evolve toward the target boundary, R _p (φ) is the regularization term of the target level set function.

specific,

In the formula, α is an arbitrary real number, that is, a constant, and λ is a positive real number, which are the weights of the length term and the area term on the right side of the formula (1.3). In this embodiment, λ=4 and α=3 can be taken.

It is the gradient of the target level set function φ(x,y), x,y are the horizontal and vertical coordinates of the image; δ _ε (·) and H _ε (·) are the one-dimensional regularized Dirac function and Heaviside function. Then there are:

g is the edge stop function, which defines

Among them, G _σ represents a function with a standard deviation of σ, that is, the function corresponding to the region of interest as shown in FIG. 8B, I represents the binary image as shown in FIG. 11, and * represents a convolution operator. In this embodiment, the edge stop function in the DRLSE algorithm can be rewritten as:

In formula (1.2), the definition

Among them, p is the potential energy, defined as follows:

Substituting formula (1.3) and (1.6) into formula (1.2), the following formula can be obtained:

This formula is the energy functional to be solved. Obviously, the independent variable of the energy functional E(φ) is φ, and φ is a function. Therefore, the equation (1.8) is differentiated at the same time, and the following equation ( 1.9):

The formula (1.9) is solved by the gradient descent method to achieve the purpose of solving the minimum value of the energy functional. Approximately transforming the partial differential equation in formula (1.9) into a discrete finite difference form, the level set evolution equation, that is, the DRLSE model, is obtained:

Among them, in formula (1.10)

It is the expression on the right side of the equation in equation (1.9), k is the number of iterations iter, and Δt is the step size step. Specifically, the step size and the number of iterations can be determined according to the echo type of the lesion. In this embodiment, when the echo type is determined to be anechoic, the corresponding number of iterations is 120 to 260, and the time step is 1.0; When the type is low echo, the corresponding number of iterations is 650-950, and the time step is 1.5. Substitute the set step size Δt and the number of iterations k into equation (1.10) for calculation, so as to obtain the target level set function φ(x,y) at the end of the evolution.

Step 708: Obtain a corresponding binary image of the target zero level set based on the target level set function.

Since a digital image can be understood as a binary function, a binary function can be expressed as a continuous surface in the Cartesian three-dimensional coordinate system. Therefore, in this embodiment, the corresponding target zero level set binary image (as shown in Figure 12) can be obtained based on the target level set function, where the boundary pixels between the foreground and the background in the target zero level set binary image are Contour data of the lesion.

In one embodiment, as shown in FIG. 13, taking the image segmentation algorithm as a lesion segmentation algorithm based on local binary fitting evolution as an example, the specific implementation method for detecting the region of interest to obtain the outline of the lesion is described, including:

Step 1302: Construct an initial zero level set function according to the region of interest.

Specifically, target detection is performed on the original ultrasound image (Figure 14A) to obtain the corresponding region of interest (Figure 14B). Since a digital image can be understood as a binary function, in this embodiment, based on interest The coordinates of the vertices of the area are used to obtain the initial contour of the lesion area. For example, the coordinates of the vertices of the region of interest are determined as reference points, and the reference points are translated according to the set translation amount to obtain new vertices relative to each reference point. The coordinates, the initial contour of the lesion area is obtained according to the new vertex coordinates.

Furthermore, the corresponding initial zero level set function is determined from the initial contours of the region of interest and the lesion area. In this embodiment, the initial zero level set function is used to represent the initial contour of the lesion area, which is also the initial object of the evolution algorithm. Among them, the main idea of the level set method is to embed the curve as a zero level set on a higher one-dimensional surface, and obtain the evolution equation of the function through the evolution equation of the surface. Specifically, an initial image with the same size as the ROI is created according to the initial contour of the region of interest (ROI) and the lesion area (as shown in Figure 15), where the black area shown in Figure 15 represents the interior of the initial contour , Which is the initial area of the breast tumor lesion, and its pixel value is set to -2; the white area represents the outside of the initial contour, and its pixel value is set to 2.

Since the pixel gray value of the initial area of the level set in the above initial image is -2, and the pixel gray value of the other areas is 2, the initial image is a binarized image, and the binarized image is the initial zero level The set of images is also the initial object of the evolutionary algorithm. Also, because digital images can be represented by a binary function, the binary image shown in Figure 15 can be represented by the following function (ie, the initial zero level set function):

Where r is the row coordinate of any pixel in the image, c is the column coordinate, and R0 represents the initial area of the lesion.

In step 1304, the energy functional is defined by the local binary fitting evolution algorithm based on the initial zero level set function.

Specifically, suppose x is any pixel in the original image, and y is any pixel adjacent to pixel x (called the neighborhood pixel of x), as shown in Figure 16, where x and y Both are two-dimensional vectors, which can be expressed as x(c,r),y(c,r). Then define the energy functional as:

F(φ,f ₁ ,f ₂ )=E ^LBF (φ,f ₁ ,f ₂ )+μΡ(φ)+υL(φ) (2.2)

Among them, the first term on the right side of the equation is the subject term of the energy functional, the P in the second term is the penalty term, the L in the third term is the length of the zero-level curve of the level set function, and μ,υ are Normal number.

Step 1306: Solve the minimum value of the energy functional by the gradient descent method to obtain the target level set function at the end of the evolution.

In this embodiment, the energy functional is minimized by the gradient descent method, and the level set active contour evolution equation is obtained.

Specifically, in the above formula (2.2):

Among them, H in equation (2.3) is the Heaviside function. In this paper, the regularized Heaviside function H _{ε is used} to approximate the Heaviside function. K _σ (x) is the Gaussian kernel function with standard deviation σ (this function is based on the region of interest Obtained after Gaussian processing). I(y) represents the pixel gray value of the neighboring pixel y of any pixel x in the binarized image as shown in Figure 15. λ ₁ and λ ₂ are positive constants, which are the weights of the corresponding integral terms. In this implementation In the example, λ ₁ =1 (that is, it is always 1), and λ ₂ can be determined according to the echo type of the region of interest. In formula (2.4)

It is to find the gradient of the zero level set function φ(c,r). The Dirac function δ in formula (2.5) is the first derivative of the Heaviside function. The regularized Dirac function is expressed as δ _ε . Then there are:

Regularizing the first and third terms on the right side of equation (2.2) can be approximately expressed as:

F _ε (φ,f ₁ ,f ₂ )=E _ε ^LBF (φ,f ₁ ,f ₂ )+μΡ(φ)+υL _ε (φ) (2.9)

In formula (9), f ₁ (x) and f ₂ (x) are always greater than zero, where:

Calculate the minimum value of the energy functional by the gradient descent method. Specifically, keep f ₁ and f ₂ fixed, and use the standard gradient descent method to minimize _{the energy functional F ε} (φ,f ₁ ,f _{2) about φ} In order to obtain the level set active contour evolution equation:

In formula (2.12),

e ₁ (x)=∫ _Ω K _σ (yx)|I(x)-f ₁ (y)| ² dy (2.13)

e ₂ (x)=∫ _Ω K _σ (yx)|I(x)-f ₂ (y)| ² dy (2.14)

The active contour evolution equation based on the level set adopts the set step size and iteration number to calculate iteratively to obtain the target level set function at the end of the evolution. Specifically, the partial differential equation in equation (2.12) is approximately transformed into a discrete finite difference form:

In the formula

It is the expression on the right side of the equation in equation (2.12). Iteratively calculates equation (2.15) with the set step size Δt and the number of iterations k to obtain the target level set function at the end of the evolution, that is, the current energy The zero level set contour of the corresponding level set function φ when the functional F _ε (φ, f ₁ , f _{2) takes the minimum value is the final result.} Among them, the step size and the number of iterations can be determined according to the echo type of the lesion. In this embodiment, when the echo type is determined to be anechoic, the corresponding number of iterations is 80-260, and the time step is 0.1-1.0, and _{The value of λ 2} in the corresponding formula (2.3) is 2.0～3.3, and the value of V in the corresponding formula (2.1) is 0.003*255*255～0.008*255*255; if the echo type is low echo, the corresponding The number of iterations is 280-320, the time step is 0.1-1.0, and the _{value of λ 2} in the corresponding formula (2.3) is 1.5-2.2, and the value of V in the corresponding formula (2.1) is 10-8. The calculation is performed based on the echo type and the corresponding parameters are substituted to obtain the target level set function at the end of the evolution, which corresponds to the binary image of the target zero level set as shown in FIG. 17.

Step 1308: Obtain the corresponding target zero level set binary image based on the target level set function, and perform post-processing on the target zero level set binary image to obtain the target contour of the lesion area.

Specifically, the target zero level set binary image as shown in FIG. 17 is subjected to inverse color processing to obtain a plurality of foreground regions to be screened after the inverse color processing, wherein the pixel grayscale of the foreground area after the inverse color processing is The value is 255 (that is, white), and the gray value of the background pixel is 0 (that is, black). Based on the multiple foreground regions to be screened after the above inverted color processing, the multiple foreground regions to be screened are filled with holes, so as to obtain a new binary image of the target zero level set after filling, as shown in FIG. 18.

Then according to the neighborhood connectivity criterion, 8 neighborhoods are used to extract the connected components of each neighborhood in the new target zero-level set binary image. At this time, there are 4 connected components in Figure 18. Obviously, each white area in Figure 18 There is no connection with other white areas, that is, 4 independent white areas. Then calculate the area of each connected component area (that is, the number of pixels occupied by the connected component area). The area of the connected component with the largest area among the areas where the connected components is located is determined as the focus area. As shown in Figure 19, the boundary pixels of the focus area are the corresponding target contours, that is, the boundary pixels between the foreground and the background in Figure 19 are The target contour of the lesion area.

In one embodiment, the lesion description information includes the echo type of the lesion. Generally, the echo type includes hypoechoic and anechoic, as shown in FIG. 20A and FIG. 20B, which represent the gray histograms of hypoechoic and anechoic respectively. The horizontal axis represents a total of 256 intervals from 0 to 255 (that is, the possible gray values), and the vertical axis represents the frequency of each gray value. Obviously, the frequency of the most frequent gray values in the echo-free histogram is very different from the average frequency of other gray values. Therefore, the distribution feature of the histogram can be used to distinguish the echo type of the lesion. Specifically, the following formula can be used to quantitatively express: ratio=maxFrequency/mean_num, where maxFrequency is the frequency of the gray value that appears most frequently in the histogram, and mean_num is the average frequency of other gray values.

Therefore, in this embodiment, the corresponding histogram is obtained according to the region of interest, and then the above formula is used to calculate the ratio of the frequency of the most frequently occurring gray value in the histogram to the average frequency of other gray values. The degree value refers to the gray value in the histogram except the gray value with the most frequent occurrence. The echo type of the lesion is generated based on the size of the ratio. The larger the ratio, the more likely it is an anechoic lesion. Specifically, when the ratio is greater than 7, it can be determined that the lesion is an anechoic lesion, and when the ratio is less than 7, it can be determined that the lesion is a hypoechoic lesion.

In an embodiment, the lesion description information further includes the shape of the lesion, which usually includes an ellipse, a circle-like shape, and an irregular shape. According to the geometric topological structure of the outline of the lesion and the gray-scale features of the region of interest, the lesion description information is generated, including:

The least squares fitting ellipse algorithm proposed by Fitzgibbon is used to fit the contour point set of the lesion to an ellipse (Figure 22A), compare the similarity between the contour of the lesion and the ellipse, and then generate the shape description of the lesion based on the similarity between the two. Such as the description of the generated lesions as oval, round or irregular.

Specifically, the circularity, convexity, and compactness of the contour of the lesion are calculated respectively, and the calculation formula is as follows:

Circularity formula:

In the formula, S is the area enclosed by the contour of the lesion, and L is the circumference of the contour. The closer the circularity value is to 1, the more regular the shape.

Convex hull degree formula:

In the formula, S1 is the area enclosed by the contour of the lesion, and S2 is the area enclosed by the convex hull of the contour of the lesion. The closer the convex hull degree is to 1, the more regular the shape.

Firmness formula:

In the formula, L is the circumference of the lesion outline, and S is the area enclosed by the lesion outline. The closer the compactness is to 0, the more regular the shape. Considering roundness, convex hull, and compactness, the following formula is obtained:

In the formula, 0<metric<1. In this embodiment, the metric is called a geometric measurement. The smaller the value, the more regular the contour of the lesion. Specifically, the geometric measurement threshold can be set to 0.4, that is, if the metric is less than 0.4, the contour of the lesion is considered to be relatively regular, otherwise the contour of the lesion is considered to be irregular.

Then calculate the Fourier Descriptors of the lesion contour, that is, map the contour point set in the image plane coordinate system to the complex plane. As shown in Figure 21, suppose that there are K points in the contour of the lesion. When traveling on the contour counterclockwise, the pixels on the contour can be regarded as a coordinate pair (x ₀ , y ₀ ), (x ₁ , y ₁ ),(x ₂ ,y ₂ ),...,(x _K-1 ,y _K-1 ). The coordinates of these points may be represented as _{x (k) = x k,} y (k) = the form y _k, the focus contour may be represented as a coordinate sequence s (k) = [x ( k), y (k)], k=0,1,...,K-1. Each coordinate pair can be treated as a complex number:

s(k)=x(k)+jy(k), k=0,1,2,...,K-1

Then the discrete Fourier transform of S(k) is:

Among them, a(u) is the Fourier descriptor. In order to maintain the invariance of the translation, rotation, and scaling of the contour, it is normalized to:

Among them, a _norm (u) is the Fourier descriptor of each point on the contour, which can also be represented by a complex number:

a _norm (k)=x(k)+jy(k), k=0,1,2,...,K-1

First calculate the Fourier descriptors of the lesion contour and the ellipse contour separately (the number of these two contour points is generally different, so it is necessary to perform uniform sampling in the two contour points respectively, that is, the interval between every two sampling points is equal, So that K points are collected in each of the two contours, and then the Euclidean distance is used to calculate the distance d between the two contours:

In the formula, x and y are the _{real and imaginary parts of the complex number a norm} , and the distance d can be used to judge the similarity between the contour of the lesion and the ellipse. If the similarity threshold is set to 0.4, the specific judgment steps are: if d is less than Equal to the threshold, the lesion contour is "regular"; if d is greater than the threshold, it is further judged by geometric measurement; if the geometric test metric<=0.4, the lesion contour is "regular", if metric>0.4, the lesion contour is " Irregular shape".

If it is judged to be an irregular shape, the shape description of the irregular shape can be directly generated. If it is judged to be a "regular shape", then the specific shape is further judged (such as ellipse and round shape are regular shapes). Specifically, calculate the ratio of the long axis to the short axis of the fitted ellipse ratio=w/h, if ratio>=1.5, the shape of the lesion is determined to be "elliptical", if ratio<1.5, then the shape of the lesion is determined The shape is "quasi-circular".

In one embodiment, the lesion description information includes the growth direction of the lesion, and generally, the growth direction includes parallel and non-parallel. Then, according to the geometric topology structure of the lesion outline and the gray-scale features of the region of interest, the lesion description information is generated, including: fitting the lesion outline point set to an ellipse using the least squares fitting ellipse algorithm proposed by Fitzgibbon (Figure 22A), Calculate the angle θ between the major axis of the ellipse and the horizontal axis of the image, where line 1 in Figure 22B represents the direction of the horizontal axis of the image coordinate system, line 2 represents the direction of the major axis of the ellipse, and the angle between line 1 and line 2 That is θ, the growth direction of the lesion is generated according to the size of the included angle θ. Specifically, when 0°≤θ≤20°, the growth direction of the lesion is “parallel”, and when θ>20°, the growth direction of the lesion is “non-parallel”.

In an embodiment, the lesion description information includes the smoothness of the outline of the lesion. Generally, the smoothness includes smoothness and non-smoothness. Then, the lesion description information is generated according to the geometric topological structure of the lesion contour and the gray-scale features of the region of interest, including: measuring the smoothness of the lesion contour by the mean value of the included angle of the contour points and the boundary roughness.

Specifically, let the contour point set of the contour of the lesion be n pixels, the calculation method of the mean value of the included angle of the contour point: Calculate the sum of the angles of each pixel on the contour of the lesion and its neighboring pixels, and then divide by The number of pixels on the contour can be calculated by the following formula:

Where θ _i,j is the angle between the pixel point i and the pixel point j.

Since the distance from the center of mass of the contour to a certain point on the contour is called the radial length d(i) of the point, in the following formula (2), let the coordinates of the center of mass be (x _c , y _c ), the coordinates of a certain point on the contour Is (x _i ,y _i ). The denominator max d(i) represents the maximum value of the radial length, and the normalized radial length is d _norm (i), then:

The calculation method of boundary roughness is: calculate the difference of the normalized radial length of adjacent pixels on the contour of the lesion in a clockwise direction, then sum all the differences, and then take the average, which can be calculated by the following formula :

The smaller the value of (1) and (3), the smoother the contour. Using the mean value of the included angle of the contour points and the weighted value of the boundary roughness as the measure of judging the contour smoothness, the following formula can be used for calculation:

val=val _θ +val _rough (4)

Generate the smoothness of the lesion contour based on the calculation result of formula (4). Specifically, a threshold can be set to compare with the value val of formula (4). If val is less than the threshold, the lesion contour is determined to be "smooth" , On the contrary, it is determined that the contour of the lesion is "unsmoothed", where the threshold can be set to 3.6, and when val is less than 3.6, the contour of the lesion is determined to be smooth.

In an embodiment, the lesion description information further includes the clarity of the outline of the lesion, specifically including clarity and unclearness. Then, generating the lesion description information according to the geometric topological structure of the lesion contour and the gray-scale feature of the region of interest includes: judging the sharpness of the lesion contour according to the sharpness value of the edge point of the lesion contour.

Specifically, for the pixel points of the contour of the lesion, 8 neighborhood points are selected in turn to calculate the difference, the weight is set according to the distance of the 8 points, and the weight sum is calculated, for example, the weights of 45 and 135 degrees are set to

Finally, calculate the average value of the weights of all pixels of the contour to obtain the sharpness value of the edge points of the lesion contour, as shown in the following formula (5):

Among them, m and n represent the width and height of the image, df represents the change in the gray value of the pixel, dx represents the distance between the pixels, and df/dx is the magnitude of the gray change.

Furthermore, the sharpness of the lesion contour is generated based on the size of the sharpness value of the edge point of the lesion contour, since the smaller the sharpness value of the edge point, the more blurred the edge. Therefore, you can set a threshold and compare the above-mentioned calculated edge point sharpness value with the threshold. If the edge point sharpness value is less than the threshold, the lesion contour is determined to be "unclear", otherwise, the lesion contour is determined to be "clear" . The threshold can be set to 130. If the edge point sharpness value is less than 130, it means that the contour of the lesion is not clear, and if the edge point sharpness value is greater than 130, it means that the contour of the lesion is clear.

In one embodiment, as shown in FIG. 23, the lesion description information also includes whether the lesion is calcified, then generating the lesion description information according to the geometric topology structure of the lesion contour and the gray-scale features of the region of interest includes the following steps:

Step 2302: Perform image processing on the region of interest according to the contour of the lesion to obtain the processed region of interest.

Specifically, assuming that the original ultrasound image is FIG. 24A, FIG. 24B is the ROI image obtained after target detection in FIG. 24A, and FIG. 24C is the binary image of the lesion contour obtained after segmentation using the above-mentioned LBF lesion segmentation algorithm. Using the white area in Fig. 24C as a template (ie, the inner area of the lesion outline), select the pixels corresponding to the coordinates of the white area in Fig. 24C in Fig. 24B, and set the pixel gray values of the remaining coordinates to 0 (ie black), such as Shown in Figure 24D. The purpose is to reduce the interference outside the lesion area in the ROI image, reduce the amount of calculation, and increase the calculation speed. Performing the morphological closing operation on Fig. 24D, and obtaining Fig. 24E (that is, the processed region of interest), at this time, the pixel gray level of the lesion area becomes uniform, which is beneficial to the subsequent processing.

Step 2304, traverse the gray values of all pixels in the region of interest after processing, and determine the coordinates of the pixel with the largest gray value and the corresponding gray value.

Specifically, for Fig. 24E, first traverse all pixels with non-zero gray values in the figure, and select the pixel with the largest gray value by statistics, including the gray value val _{max of the} pixel and its pixel coordinates p(x ₀ , y ₀ ).

In step 2306, when the gray value is greater than the set threshold, image segmentation is performed on the processed region of interest based on the coordinates of the pixel with the largest gray value and a region growing algorithm is used to obtain a corresponding binary image.

Specifically, it is determined whether val _max is greater than the threshold 110. It should be noted that the threshold value 110 in this embodiment is an empirical value. Since the pixel gray scale of the calcified area is generally higher, this embodiment selects the 8-bit image with a gray scale range of 0-255. In this embodiment, if val _max <110, it means that the lesion is "no calcification", if val _max ≥110, it means that the lesion may have calcification, and further treatment is required at this time. That is, taking the coordinates p (x ₀ , y ₀ ) of the pixel with the largest gray value as the seed point, the seed growth (also known as region growth) algorithm is used for image segmentation, so as to obtain the corresponding binary image, as shown in FIG. 24F.

Step 2308: Calculate the number of pixels in the suspected calcification area in the binary image, and determine whether the lesion is calcified based on the number of pixels in the suspected calcification area.

Calculate the area S (number of pixels) of the white area (ie, the suspected calcification area) in the binary image in Figure 24F, and determine whether the lesion is calcified based on the number of pixels in the suspected calcification area. Specifically, if S>=9, the area is a calcified area, so that the lesion is “calcified”; if S<9, the area is not a calcified area, so it is determined that the lesion is “no calcification”. In this case, it may be The area generated due to high-gray noise pixels in the image as seed points.

It should be understood that although the various steps in the flowcharts of FIGS. 1-24F are displayed in sequence as indicated by the arrows, these steps are not necessarily performed in sequence in the order indicated by the arrows. Unless specifically stated in this article, the execution of these steps is not strictly limited in order, and these steps can be executed in other orders. Moreover, at least part of the steps in FIGS. 1-24F may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily executed at the same time, but can be executed at different times. These sub-steps or stages The execution order of is not necessarily performed sequentially, but may be performed alternately or alternately with at least a part of other steps or sub-steps or stages of other steps.

In one embodiment, as shown in FIG. 25, an ultrasound image lesion description device is provided, which includes: a region of interest recognition module 2501, a lesion segmentation module 2502, and a lesion description information generating module 2503, wherein:

The region of interest identification module 2501 is used to identify the lesion in the ultrasound image to obtain the corresponding region of interest;

The lesion segmentation module 2502 is used to detect the region of interest through an image segmentation algorithm to obtain the outline of the lesion;

The lesion description information generating module 2503 is configured to generate lesion description information according to the geometric topological structure of the outline of the lesion and the gray-scale features of the region of interest.

In one embodiment, the lesion description information includes the growth direction of the lesion; the lesion description information generating module 2503 is specifically configured to: use the least squares fitting ellipse algorithm to fit the contour point set of the lesion contour into an ellipse; and calculate the length of the ellipse The angle between the axis and the horizontal coordinate axis generates the growth direction of the lesion according to the size of the angle.

In one embodiment, the lesion description information includes the echo type of the lesion; the lesion description information generating module 2503 is specifically configured to: obtain the corresponding histogram according to the region of interest; calculate the frequency and the frequency of the gray value with the most frequent occurrence in the histogram The ratio of the average frequency of other gray values. Other gray values refer to gray values other than the gray values that appear most frequently in the histogram; the echo type of the lesion is generated based on the magnitude of the ratio.

In one embodiment, the lesion description information includes the smoothness of the lesion contour; the lesion description information generating module 2503 is specifically configured to: calculate the mean value of the included angle and the boundary roughness of the contour points based on the contour point set of the lesion contour; The mean value of the included angle of the points and the weighted value of the boundary roughness generate the smoothness of the lesion contour.

In one embodiment, the lesion description information includes the definition of the lesion contour; the lesion description information generating module 2503 is specifically configured to: calculate the weight sum of each contour point relative to the neighboring point based on the contour point set of the lesion contour; The weight of the contour points relative to the neighboring points and the edge point sharpness value of the lesion contour are calculated; the sharpness of the lesion contour is generated based on the size of the edge point sharpness value of the lesion contour.

In one embodiment, the lesion description information includes whether the lesion is calcified; the lesion description information generating module 2503 is specifically configured to: perform image processing on the region of interest according to the outline of the lesion to obtain the processed region of interest; and traverse the processed interest The gray value of all pixels in the area, determine the coordinate of the pixel with the largest gray value and the corresponding gray value; when the gray value is greater than the set threshold, the coordinate of the pixel with the largest gray value is used and the area growth algorithm is adopted Perform image segmentation on the processed region of interest to obtain the corresponding binary image; calculate the number of pixels in the suspected calcification area in the binary image, and determine whether the lesion is calcified based on the number of pixels in the suspected calcification area.

In one embodiment, the lesion description information also includes the shape of the lesion; the lesion description information generating module 2503 is specifically configured to: use the least squares fitting ellipse algorithm to fit the contour point set of the lesion contour into an ellipse; The similarity of the ellipse generates a description of the shape of the lesion.

In one embodiment, the image segmentation algorithm includes a lesion segmentation algorithm based on distance regularization level set evolution; the lesion segmentation module 2502 is specifically configured to: threshold the region of interest to obtain the corresponding initial binary image, and according to Estimate the centroid of the lesion area from the initial binary image; construct the initial zero level set function based on the area of interest and the centroid of the lesion area; use the distance regularized level set evolution algorithm to iteratively calculate the initial zero level set function based on the area of interest, and get The target level set function at the end of the evolution; based on the target level set function, the corresponding target zero level set binary image is obtained, where the boundary pixels between the foreground and the background in the target zero level set binary image are the lesion contours.

In one embodiment, the image segmentation algorithm includes a lesion segmentation algorithm based on local binary fitting evolution; the lesion segmentation module 2502 is specifically configured to: construct an initial zero level set function according to the region of interest, where the initial zero level set function Represents the initial contour of the lesion area; the energy functional is defined by the local binary fitting evolution algorithm based on the initial zero level set function; the minimum value of the energy functional is solved by the gradient descent method to obtain the target level set function at the end of the evolution; The target level set function obtains the corresponding target zero level set binary image, and post-processes the target zero level set binary image to obtain the target contour of the lesion area.

For the specific definition of the ultrasound image lesion description device, please refer to the above definition of the ultrasound image lesion description method, which will not be repeated here. The various modules in the above-mentioned ultrasound image lesion description device can be implemented in whole or in part by software, hardware, and a combination thereof. The above-mentioned modules may be embedded in the form of hardware or independent of the processor in the computer equipment, or may be stored in the memory of the computer equipment in the form of software, so that the processor can call and execute the operations corresponding to the above-mentioned modules.

In one embodiment, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as shown in FIG. 26. The computer equipment includes a processor, a memory, a network interface, and a database connected through a system bus. Among them, the processor of the computer device is used to provide calculation and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program, and a database. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage medium. The database of the computer equipment is used to store ultrasound image data. The network interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a method for describing a lesion in an ultrasound image is realized.

Those skilled in the art can understand that the structure shown in FIG. 26 is only a block diagram of part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may Including more or fewer parts than shown in the figure, or combining some parts, or having a different arrangement of parts.

In one embodiment, a computer device is provided, including a memory and a processor, a computer program is stored in the memory, and the processor implements the following steps when the processor executes the computer program:

In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored, and when the computer program is executed by a processor, the following steps are implemented:

A person of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be implemented by instructing relevant hardware through a computer program. The computer program can be stored in a non-volatile computer readable storage. In the medium, when the computer program is executed, it may include the processes of the above-mentioned method embodiments. Wherein, any reference to memory, storage, database, or other media used in the embodiments provided in this application may include non-volatile and/or volatile memory. Non-volatile memory may include read only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. As an illustration and not a limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous chain Channel (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

The technical features of the above embodiments can be combined arbitrarily. In order to make the description concise, all possible combinations of the technical features in the above embodiments are not described. However, as long as there is no contradiction in the combination of these technical features, they should be It is considered as the range described in this specification.

The above-mentioned embodiments only express several implementation manners of the present application, and the description is relatively specific and detailed, but it should not be understood as a limitation on the scope of the invention patent. It should be noted that for those of ordinary skill in the art, without departing from the concept of this application, several modifications and improvements can be made, and these all fall within the protection scope of this application. Therefore, the scope of protection of the patent of this application shall be subject to the appended claims.

Claims

An ultrasound image lesion description method, characterized in that the method includes:

Identify the lesion in the ultrasound image to obtain the corresponding region of interest;

Detect the region of interest through an image segmentation algorithm to obtain the contour of the lesion;

The lesion description information is generated according to the geometric topological structure of the outline of the lesion and the gray-scale feature of the region of interest.
The method according to claim 1, wherein the lesion description information includes the growth direction of the lesion; generating the lesion description information according to the geometric topological structure of the lesion outline and the gray-scale feature of the region of interest includes :

Fitting the contour point set of the lesion contour to an ellipse by using a least squares fitting ellipse algorithm;

The angle between the long axis of the ellipse and the horizontal coordinate axis is calculated, and the growth direction of the lesion is generated according to the size of the angle.
The method according to claim 1, wherein the lesion description information includes the shape of the lesion; generating the lesion description information according to the geometric topological structure of the outline of the lesion and the gray-scale feature of the region of interest includes:

Fitting the contour point set of the lesion contour to an ellipse by using a least squares fitting ellipse algorithm;

The shape description of the lesion is generated based on the similarity between the outline of the lesion and the ellipse.
The method according to claim 1, wherein the lesion description information includes the echo type of the lesion; generating the lesion description information according to the geometric topological structure of the lesion contour and the gray-scale feature of the region of interest includes :

Obtaining a corresponding histogram according to the region of interest;

Calculate the ratio of the frequency of the most frequently occurring gray value in the histogram to the average frequency of other gray values, where the other gray values refer to the gray values other than the most frequently occurring gray values in the histogram. Degree value

The echo type of the lesion is generated based on the magnitude of the ratio.
The method according to claim 1, wherein the lesion description information includes the smoothness of the lesion outline; the lesion description information is generated according to the geometric topology structure of the lesion outline and the gray-scale feature of the region of interest ,include:

Calculating the mean value of the included angle and the boundary roughness of the contour points based on the contour point set of the contour of the lesion;

The smoothness of the lesion contour is generated according to the mean value of the included angle of the contour point and the weighted value of the boundary roughness.
The method according to claim 1, wherein the lesion description information includes the definition of the outline of the lesion; then the lesion description information is generated according to the geometric topology structure of the lesion outline and the gray-scale feature of the region of interest, include:

Respectively calculating the weight sum of each contour point relative to the neighboring points based on the contour point set of the lesion contour;

Calculating the sharpness value of the edge point of the lesion contour according to the weight of each contour point relative to the neighboring point;

The sharpness of the lesion contour is generated based on the magnitude of the sharpness value of the edge point of the lesion contour.
The method according to claim 1, wherein the lesion description information includes whether the lesion is calcified; generating the lesion description information according to the geometric topology of the outline of the lesion and the gray-scale features of the region of interest, comprising:

Performing image processing on the region of interest according to the contour of the lesion to obtain a processed region of interest;

Traverse the gray values of all pixels in the processed region of interest, and determine the coordinates of the pixel with the largest gray value and the corresponding gray value;

When the gray value is greater than the set threshold, image segmentation is performed on the processed region of interest based on the coordinates of the pixel with the largest gray value and a region growing algorithm is used to obtain a corresponding binary image;

Calculate the number of pixels in the suspected calcification area in the binary image, and determine whether the lesion is calcified based on the number of pixels in the suspected calcification area.
The method according to any one of claims 1 to 7, wherein the image segmentation algorithm comprises a lesion segmentation algorithm based on distance regularization level set evolution; then the region of interest is detected by an image segmentation algorithm, Get the outline of the lesion, including:

Performing threshold segmentation on the region of interest to obtain a corresponding initial binary image, and estimating the centroid of the lesion area according to the initial binary image;

Construct an initial zero level set function according to the center of mass of the region of interest and the lesion region;

Using a distance regularized level set evolution algorithm to iteratively calculate the initial zero level set function based on the region of interest to obtain the target level set function at the end of evolution;

A corresponding target zero level set binary image is obtained based on the target level set function, and the boundary pixel between the foreground and the background in the target zero level set binary image is the contour of the lesion.
The method according to any one of claims 1 to 7, wherein the image segmentation algorithm comprises a lesion segmentation algorithm based on local binary fitting evolution; then the region of interest is detected by an image segmentation algorithm, Get the outline of the lesion, including:

Constructing an initial zero level set function according to the region of interest, where the initial zero level set function represents the initial contour of the lesion area;

Using a local binary fitting evolution algorithm to define an energy functional based on the initial zero level set function;

Solving the minimum value of the energy functional by a gradient descent method to obtain the target level set function at the end of the evolution;

A corresponding target zero level set binary image is obtained based on the target level set function, and post-processing is performed on the target zero level set binary image to obtain a target contour of the lesion area.
An ultrasound image lesion description device, characterized in that the device comprises:

The region of interest recognition module is used to identify the lesion in the ultrasound image to obtain the corresponding region of interest;

The lesion segmentation module is used to detect the region of interest through an image segmentation algorithm to obtain the outline of the lesion;

The lesion description information generating module is configured to generate lesion description information according to the geometric topology structure of the lesion contour and the gray-scale feature of the region of interest.
A computer device comprising a memory and a processor, the memory storing a computer program, wherein the processor implements the steps of the method according to any one of claims 1 to 9 when the computer program is executed.
A computer-readable storage medium having a computer program stored thereon, wherein the computer program implements the steps of any one of claims 1 to 9 when the computer program is executed by a processor.