CN111899273A

CN111899273A - Image segmentation method, computer device and storage medium

Info

Publication number: CN111899273A
Application number: CN202010523530.XA
Authority: CN
Inventors: 车占全; 高耀宗; 姚广
Original assignee: Shanghai United Imaging Intelligent Healthcare Co Ltd
Current assignee: Shanghai United Imaging Intelligent Healthcare Co Ltd
Priority date: 2020-06-10
Filing date: 2020-06-10
Publication date: 2020-11-06

Abstract

The present application relates to an image segmentation method, a computer device, and a storage medium. The method comprises the following steps: acquiring a medical image of a lymph node to be segmented; inputting the lymph node medical image into a neural network model to obtain a lymph node mask image and a lymph node partition line mask image; the neural network model is obtained by training based on a plurality of lymph node training images and a gold standard lymph node mask image and a gold standard lymph node partition line mask image corresponding to each lymph node training image, wherein the lymph node mask image comprises at least two lymph nodes; and carrying out segmentation processing on the lymph nodes in the lymph node mask image based on the lymph node segmentation line mask image to obtain segmentation images of the lymph nodes. The lymph node segmentation efficiency can be improved by adopting the method.

Description

Image segmentation method, computer device and storage medium

Technical Field

The present application relates to the field of image processing, and in particular, to an image segmentation method, a computer device, and a storage medium.

Background

Lymph nodes are the most important immune organs of a human body, are generally distributed at the neck, the armpit and the inner side of the thigh, have an early warning effect on many diseases of the human body, and therefore, the detection of the lymph nodes of the human body is particularly important, and before the lymph nodes are detected, the lymph nodes are firstly separated from an image shot by the human body.

In the related art, when segmenting a lymph node on a medical image of a patient, a number of stuck lymph nodes are inevitably encountered, and in this case, it is common that a doctor manually performs annotation on the medical image of the patient and then segments the lymph node according to the annotation result.

However, the above technique has a problem of low segmentation efficiency.

Disclosure of Invention

In view of the above, it is necessary to provide an image segmentation method, an image segmentation apparatus, a computer device, and a storage medium capable of improving image segmentation efficiency.

A method of image segmentation, the method comprising:

acquiring a medical image of a lymph node to be segmented;

inputting the lymph node medical image into a neural network model to obtain a lymph node mask image and a lymph node partition line mask image; the neural network model is obtained by training based on a plurality of lymph node training images and a gold standard lymph node mask image and a gold standard lymph node partition line mask image corresponding to each lymph node training image, wherein the lymph node mask image comprises at least two lymph nodes;

the lymph nodes in the lymph node mask image are segmented based on the lymph node segmentation line mask image, and segmentation images of the lymph nodes are obtained.

In one embodiment, the obtaining of the segmented image of each lymph node by performing the segmentation process on the lymph node in the lymph node mask image based on the lymph node partition line mask image includes:

acquiring coordinates of each first pixel point on a lymph node segmentation line and acquiring coordinates of each second pixel point on a lymph node in a lymph node mask image;

determining a target lymph node partition line corresponding to the lymph node partition line based on the coordinates of each first pixel point and the coordinates of each second pixel point; the end point of the target lymph node partition line is on the outline of the lymph node in the lymph node mask image;

and (4) adopting a target lymph node segmentation line to segment the lymph nodes in the lymph node mask image to obtain segmentation images of all the lymph nodes.

In one embodiment, the determining a target lymph node partition line corresponding to the lymph node partition line based on the coordinates of each first pixel point and the coordinates of each second pixel point includes:

acquiring coordinates of first pixel points at two ends of a lymph node partition line from the coordinates of the first pixel points, and acquiring coordinates of second pixel points on the lymph node outline from the coordinates of the second pixel points;

and respectively matching the coordinates of the first pixel points at the two ends of the lymph node partition line with the coordinates of each second pixel point on the lymph node outline, and determining the target lymph node partition line according to the matching result.

In one embodiment, the determining the target lymph node partition line according to the matching result includes:

and if the coordinates of the first pixel points at the two ends of the lymph node partition line are the same as the coordinates of any two second pixel points on the lymph node outline, determining the lymph node partition line as the target lymph node partition line.

if the coordinates of the first pixel points at the two ends of the lymph node partition line are the same as the coordinates of any second pixel point on the lymph node outline, or if the coordinates of the first pixel points at the two ends of the lymph node partition line exceed the range of the coordinates of the second pixel points on the lymph node outline, acquiring the coordinates of two target first pixel points from the coordinates of the first pixel points; the coordinates of the two target first pixel points are the same as the coordinates of any two second pixel points on the lymph node outline;

and intercepting the lymph node parting line based on the coordinates of the first pixel points of the two targets to obtain a target lymph node parting line.

and if the coordinates of the first pixel points at the two ends of the lymph node partition line do not exceed the range of the coordinates of each second pixel point on the lymph node outline, or if any one of the coordinates of the first pixel points at the two ends of the lymph node partition line exceeds the range of the coordinates of each second pixel point on the lymph node outline, performing interpolation processing on each first pixel point on the lymph node partition line to obtain a target lymph partition line.

In one embodiment, the method further includes:

setting a first image value for all the points of the lymph node on one side of the target lymph node partition line in the lymph node mask image and setting a second image value for all the points of the lymph node on the other side of the target lymph node partition line in the lymph node mask image by taking the target lymph node partition line as a boundary, and displaying;

and/or setting a first image value for a contour point of a lymph node located on one side of the target lymph node partition line in the lymph node mask image and a second image value for a contour point of a lymph node located on the other side of the target lymph node partition line in the lymph node mask image with the target lymph node partition line as a boundary, and displaying the first image value and the second image value.

In one embodiment, the training method of the neural network model includes:

inputting the lymph node training image into an initial neural network model to obtain a predicted lymph node mask image and a predicted lymph node partition line mask image;

calculating a first loss between the predicted lymph node mask image and the gold standard lymph node mask image, and calculating a second loss between the predicted lymph node partition line mask image and the gold standard lymph node partition line mask image;

and training the initial neural network model based on the first loss and the second loss to obtain the neural network model.

An image segmentation apparatus, the apparatus comprising:

the acquisition module is used for acquiring a medical image of the lymph node to be segmented;

the determining module is used for inputting the lymph node medical image into the neural network model to obtain a lymph node mask image and a lymph node partition line mask image; the neural network model is obtained by training based on a plurality of lymph node training images and a gold standard lymph node mask image and a gold standard lymph node partition line mask image corresponding to each lymph node training image, wherein the lymph node mask image comprises at least two lymph nodes;

and the segmentation module is used for carrying out segmentation processing on the lymph nodes in the lymph node mask image based on the lymph node segmentation line mask image to obtain segmentation images of all the lymph nodes.

A computer device comprising a memory and a processor, the memory storing a computer program, the processor implementing the following steps when executing the computer program:

acquiring a medical image of a lymph node to be segmented;

A computer-readable storage medium, on which a computer program is stored which, when executed by a processor, carries out the steps of:

acquiring a medical image of a lymph node to be segmented;

According to the image segmentation method, the image segmentation device, the computer equipment and the storage medium, the medical image of the lymph node to be segmented can be processed through the neural network model to obtain the corresponding lymph node mask image and the lymph node segmentation line mask image, and the lymph node in the lymph node mask image is segmented based on the lymph node segmentation line mask image to obtain the segmentation image of each lymph node, wherein the lymph node mask image comprises at least two adhered lymph nodes. In the method, because the lymph nodes adhered in the lymph node mask image can be segmented by adopting the lymph node segmentation line mask image without manually and manually separating the adhered lymph nodes, the method has higher speed when the adhered lymph nodes are segmented, namely the efficiency of lymph node segmentation can be improved; in addition, when the adherent lymph nodes are segmented, the lymph node segmentation line mask image obtained by the neural network model is used for segmentation, and the segmentation is not required to be performed by the experience of a doctor, so that the segmentation standards are uniform and can not be different from person to person, and the finally obtained lymph node segmentation result is accurate.

Drawings

FIG. 1 is a diagram illustrating an internal structure of a computer device according to an embodiment;

FIG. 2 is a flow diagram illustrating a method for image segmentation in one embodiment;

FIG. 3 is a flowchart illustrating an image segmentation method according to another embodiment;

FIG. 4 is a flowchart illustrating an image segmentation method according to another embodiment;

FIG. 4a is a diagram illustrating an example of a lymph node partition line just divided in another embodiment;

FIG. 4b is a diagram of an example of a lymph node segmentation line beyond segmentation in another embodiment;

FIG. 4c is a diagram of an example of insufficient segmentation of a lymph node segmentation line in another embodiment;

FIG. 5 is a flowchart illustrating an image segmentation method according to another embodiment;

FIG. 6 is a block diagram showing an example of the structure of an image segmentation apparatus.

Detailed Description

In order to make the objects, technical solutions and advantages of the present application more apparent, the present application is described in further detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the present application and are not intended to limit the present application.

Lymph nodes have the function of early warning on a plurality of diseases, when inflammation, tumor and other diseases appear in a human body, the lymph nodes at corresponding positions can give out health early warning, but the image values of the lymph nodes and surrounding tissues on a CT image are relatively similar, so that the lymph nodes are difficult to distinguish. At present, some schemes for segmenting lymph nodes by adopting a deep learning technology can segment all lymph nodes in an original image, but doctors usually need information of each lymph node during reading, so that doctors need to spend a lot of time and energy to analyze whether a segmented larger lymph node is a lymph node or a plurality of adhesive lymph nodes. Therefore, the present application provides an image segmentation method, apparatus, computer device and storage medium, which can solve the above technical problems.

The image segmentation method provided by the application can be applied to computer equipment, and the computer equipment can be a terminal or a server. Taking a computer device as an example, the internal structure diagram thereof can be as shown in fig. 1. The computer device includes a processor, a memory, a communication interface, a display screen, and an input device connected by a system bus. Wherein the processor of the computer device is configured to provide computing and control capabilities. The memory of the computer device comprises a nonvolatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of an operating system and computer programs in the non-volatile storage medium. The communication interface of the computer device is used for carrying out wired or wireless communication with an external terminal, and the wireless communication can be realized through WIFI, an operator network, NFC (near field communication) or other technologies. The computer program is executed by a processor to implement an image segmentation method. The display screen of the computer equipment can be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer equipment can be a touch layer covered on the display screen, a key, a track ball or a touch pad arranged on the shell of the computer equipment, an external keyboard, a touch pad or a mouse and the like.

Those skilled in the art will appreciate that the architecture shown in fig. 1 is merely a block diagram of some of the structures associated with the disclosed aspects and is not intended to limit the computing devices to which the disclosed aspects apply, as particular computing devices may include more or less components than those shown, or may combine certain components, or have a different arrangement of components.

The execution subject of the embodiment of the present application may be a computer device, or may be an image segmentation apparatus, and the following description will be given taking a computer device as an execution subject.

In one embodiment, an image segmentation method is provided, and the embodiment relates to a specific process of obtaining a lymph node segmentation line mask image and a lymph node mask image and segmenting a lymph node adhered in the lymph node mask image by using the lymph node segmentation line mask image. As shown in fig. 2, the method may include the steps of:

s202, acquiring a medical image of the lymph node to be segmented.

Here, the medical image of the lymph node to be segmented may be an image of any modality, such as a CT (computed tomography) image, an MR (magnetic resonance) image, a PET (positron emission tomography) image, an X-ray image, and the like, and the CT image is mainly used in the present embodiment. In addition, the medical image of the lymph node to be segmented may be a two-dimensional image, a three-dimensional image, or the like. Here the medical image of a lymph node to be segmented comprises at least two lymph nodes which are cohered together, but may of course also comprise other lymph nodes which are not cohered.

Specifically, the computer device may scan a lymph node of any body part of the object to be detected through the scanning device connected to the computer device, to obtain a medical image of the lymph node to be segmented, or obtain the medical image of the lymph node to be segmented from a server, a database, a cloud, and the like, in which the medical image of the lymph node to be segmented is stored in advance, or obtain other obtaining manners. Here, the object to be detected is generally a human body.

S204, inputting the lymph node medical image into the neural network model to obtain a lymph node mask image and a lymph node partition line mask image; the neural network model is obtained by training based on a plurality of lymph node training images and a gold standard lymph node mask image and a gold standard lymph node partition line mask image corresponding to each lymph node training image, wherein the lymph node mask image comprises at least two lymph nodes.

In this step, before the lymph node medical image is input into the neural network model, the lymph node medical image may be resampled and normalized, and the resampled and normalized lymph node medical image may be input into the neural network model.

Here, the lymph node mask image may be an image in which the foreground is a lymph node and the background pixel value is 0, and the lymph node partition line mask image may be an image in which the foreground is a lymph node partition line and the background pixel value is 0. Here too, the lymph node mask image typically includes at least two lymph nodes that are cohered together, and other, non-cohered, individual lymph nodes may be included; here, the lymph node division line included in the lymph node division line mask image may be one division line, two division lines, or three division lines, or even more division lines, where the number of specific division lines may be determined according to the number of the adhered lymph nodes, for example, if two lymph nodes are adhered to one lymph node mask image, one division line is included in the lymph node division line mask image, if three lymph nodes are adhered to one lymph node mask image, two division lines are included in the lymph node division line mask image, and the other number may be the same.

In addition, before the neural network model is used, the neural network model may be trained, during training, a plurality of lymph node training images may be obtained, where each of the plurality of lymph node training images includes at least two lymph nodes that are adhered together, the plurality of lymph node training images may be obtained by scanning lymph nodes at the same part of different objects to be detected, of course, the plurality of lymph node training images may also be obtained by scanning lymph nodes at different body parts of the same object to be detected, and of course, other situations may also be included; here, after each lymph node training image is obtained, a lymph node partition line and an adherent lymph node corresponding to the lymph node partition line may be drawn on the lymph node training image by means of manual labeling by a doctor or the like, the drawn lymph node partition line and the drawn adherent lymph node are placed on different labels or pixel values on the lymph node training image, and the obtained images are used as a gold standard lymph node partition line mask image and a gold standard lymph node mask image. After the training images and the two gold standard mask images corresponding to each training image are obtained, the neural network model can be trained, and the trained neural network model is obtained. It should be noted that the neural network model in this step is preferably a convolutional neural network uet network model.

After the neural network model is trained, the lymph node medical image to be segmented can be input into the neural network model, and in the neural network model, the lymph node medical image can be subjected to convolution, pooling and other processing by the neural network model, so that a lymph node mask image and a lymph node segmentation line mask image corresponding to the lymph node segmentation image to be segmented are finally obtained.

And S206, carrying out segmentation processing on the lymph nodes in the lymph node mask image based on the lymph node segmentation line mask image to obtain segmentation images of the lymph nodes.

After the lymph node division line mask image and the lymph node mask image are obtained, the lymph node division line mask image can be superposed on the lymph node mask image, the adhered lymph nodes are divided by taking the lymph node division line as a boundary according to the overlapping condition of the lymph node division line in the lymph node division line mask image and the lymph node in the lymph node mask image, a mask image corresponding to each lymph node is obtained, then the mask image corresponding to each lymph node is corresponding to the medical image of the lymph node to be divided, the lymph node division image corresponding to each lymph node can be obtained, and therefore the lymph nodes adhered together can be distinguished, and a doctor can perform more accurate analysis and judgment according to the division image of each lymph node subsequently.

It should be noted that the neural network model of this embodiment is also trained using training images including individual, non-adhesive lymph nodes during the training process, and then the neural network model trained using this embodiment may also be finally segmented for individual, non-adhesive lymph nodes.

In the image segmentation method, the medical image of the lymph node to be segmented can be processed through the neural network model to obtain the corresponding lymph node mask image and the lymph node segmentation line mask image, and the lymph node in the lymph node mask image is segmented based on the lymph node segmentation line mask image to obtain the segmentation image of each lymph node, wherein the lymph node mask image comprises at least two adhered lymph nodes. In the method, because the lymph nodes adhered in the lymph node mask image can be segmented by adopting the lymph node segmentation line mask image without manually and manually separating the adhered lymph nodes, the method has higher speed when the adhered lymph nodes are segmented, namely the efficiency of lymph node segmentation can be improved; in addition, when the adherent lymph nodes are segmented, the lymph node segmentation line mask image obtained by the neural network model is used for segmentation, and the segmentation is not required to be performed by the experience of a doctor, so that the segmentation standards are uniform and can not be different from person to person, and the finally obtained lymph node segmentation result is accurate.

In another embodiment, another image segmentation method is provided, and this embodiment relates to a specific process of how to segment a lymph node mask image based on a lymph node segmentation line mask image including a lymph node segmentation line. On the basis of the above embodiment, as shown in fig. 3, the above S206 may include the following steps:

s302, obtaining the coordinates of each first pixel point on the lymph node segmentation line, and obtaining the coordinates of each second pixel point on the lymph node in the lymph node mask image.

In this step, the lymph node partition line mask image includes at least one lymph node partition line, which may be one, two, three, etc., and this embodiment mainly describes two lymph nodes adhered together, including one lymph node partition line, as an example.

The lymph node division line mask image and the lymph node mask image are placed in the same known coordinate system, so that the coordinates of each pixel point on the lymph node division line in the lymph node division line mask image can be obtained and recorded as the coordinates of the first pixel point, and the coordinates of each pixel point on the adhered lymph node in the lymph node mask image can be obtained and recorded as the coordinates of the second pixel point.

S304, determining a target lymph node partition line corresponding to the lymph node partition line based on the coordinates of each first pixel point and the coordinates of each second pixel point; the end points of the target lymph node partition line are on the outline of the lymph node in the lymph node mask image.

In this step, since the node partition line obtained by the neural network model may exceed the lymph node, or may not sufficiently partition the lymph node, and certainly, the lymph node may be just enough to be partitioned, it is necessary to determine a target node partition line that is just capable of partitioning the lymph node, so as to completely partition the lymph node, and further time is not spent on useless partition.

Here, the target lymph node partition line refers to a partition line capable of completely partitioning a lymph node having both end points just above the contour of the lymph node, and does not exceed the contour of the lymph node, or fall short, i.e., inside the contour of the lymph node.

When a target lymph node partition line is determined, judging from a first pixel point of any endpoint on the lymph node partition line to the direction of the other endpoint in sequence, judging whether the coordinates of a second pixel point (the surrounding neighborhood can be 4 neighborhood, 8 neighborhood and the like) exist in the coordinate range of the surrounding neighborhood of each first pixel point, if the coordinates of the second pixel point exist, then partitioning the adhered lymph node along the direction until the coordinates of the second pixel point do not exist in the coordinate range of the surrounding neighborhood of the first pixel point, at the moment, the first pixel points critical at two ends can be used as two endpoint pixel points of the target lymph node partition line, the first pixel points critical at two ends and the first pixel points included in the middle of the first pixel points form the target lymph node partition line together, here, the first pixel point critical at both ends refers to the first pixel point having the coordinates of the second pixel point in the immediate surrounding neighborhood.

Certainly, when the target lymph node partition line is determined, the coordinates of the first pixel points at the two ends of the lymph node partition line may be matched with each second pixel point on the lymph node contour, and the target lymph node partition line may be determined according to the matching result.

And S306, carrying out segmentation treatment on the lymph nodes in the lymph node mask image by adopting the target lymph node segmentation line to obtain segmentation images of all the lymph nodes.

In this step, after the target lymph node partition line is obtained, the target lymph node partition line may be superimposed on a lymph node in the lymph node mask image, and the adhered lymph nodes are completely divided with the target lymph node partition line as a boundary, where lymph nodes on both sides of the target lymph node partition line may be set to different image values to divide two lymph nodes adhered together to obtain divided images of the respective lymph nodes; the two lymph nodes stuck together may be directly segmented into two images by the target lymph node segmentation line, and then the two images are mapped to the medical image of the lymph node to be segmented to obtain a segmented image of each lymph node, but other segmentation means may be used.

In the image segmentation method provided in this embodiment, when the mask image of the lymph node partition line includes a lymph node partition line, a target lymph node partition line may be obtained by coordinates of a pixel point on the lymph node partition line and coordinates of a pixel point on a lymph node in the lymph node mask image, and a segmentation image of each lymph node is obtained by segmenting an adhered lymph node with the target lymph node partition line, where an end point of the target lymph node partition line is on an outline of the lymph node in the lymph node mask image. In this embodiment, since the end point of the determined target lymph node partition line is located right above the contour of the lymph node, when the target lymph node partition line is used to partition the adhered lymph nodes, the adhered lymph nodes can be completely partitioned, so that the problem that the adhered lymph nodes cannot be completely partitioned and the obtained lymph node partition result is inaccurate can be avoided.

In another embodiment, another image segmentation method is provided, and the embodiment relates to a specific process of how to determine the target lymph node segmentation line based on the coordinates of each first pixel point and the coordinates of each second pixel point. On the basis of the above embodiment, as shown in fig. 4, the above S304 may include the following steps:

s402, obtaining the coordinates of the first pixel points at two ends of the lymph node partition line from the coordinates of the first pixel points, and obtaining the coordinates of the second pixel points on the lymph node outline from the coordinates of the second pixel points.

In this step, the coordinates of the smallest first pixel point and the largest first pixel point can be obtained by comparing the coordinates of the first pixel points, and the coordinates are used as the coordinates of the two endpoint pixel points of the lymph node partition line; the coordinates of each second pixel point on the lymph node contour can also be selected by comparing the coordinates of each second pixel point, and certainly, the coordinates of each second pixel point on the lymph node contour can also be artificially determined from each second pixel point.

S404, the coordinates of the first pixel points at the two ends of the lymph node partition line are respectively matched with the coordinates of each second pixel point on the lymph node outline, and the target lymph node partition line is determined according to the matching result.

In this step, there are three cases of overlapping of the lymph node partition line and the lymph node obtained by the neural network model, namely, just partition, insufficient partition, and over partition, respectively, so there are three cases correspondingly when matching the pixel points at the two ends of the lymph node partition line with the pixel points on the lymph node contour, and just partition, insufficient partition, and over partition, which will be described below.

First, in the case of the right segmentation, as shown in fig. 4a, the dotted line in the figure is a lymph node segmentation line, the curve is two lymph nodes that are adhered, and optionally, if the coordinates of the first pixel points at the two ends of the lymph node segmentation line are the same as the coordinates of any two second pixel points on the lymph node outline, the lymph node segmentation line is determined to be the target lymph node segmentation line. That is to say, the pixel points at the two ends of the lymph node partition line are just on the contour of the adhered lymph nodes, and at this time, the lymph node partition line can completely and completely partition the adhered lymph nodes, so that the lymph node partition line can be directly used as the target lymph node partition line.

Secondly, when the conditions exceed the segmentation conditions, as shown in fig. 4b, the dotted line in the figure is a lymph node segmentation line, and the curve is two lymph nodes which are adhered, optionally, if the coordinates of the first pixel points at the two ends of the lymph node segmentation line are the same as the coordinates of any second pixel point on the lymph node outline, or if the coordinates of the first pixel points at the two ends of the lymph node segmentation line exceed the range of the coordinates of each second pixel point on the lymph node outline, the coordinates of two target first pixel points are obtained from the coordinates of each first pixel point; the coordinates of the two target first pixel points are the same as the coordinates of any two second pixel points on the lymph node outline; and intercepting the lymph node parting line based on the coordinates of the first pixel points of the two targets to obtain a target lymph node parting line.

That is, when one end point of the lymph node dividing line is on the adherent lymph node contour and the other end point thereof exceeds the adherent lymph node contour, or when both end points of the lymph node dividing line exceed the adherent lymph node contour, it can be considered that the lymph node dividing line is beyond the range of the lymph node contour. Aiming at the condition that one endpoint of the lymph node partition line is on the adhered lymph node contour and the other endpoint exceeds the adhered lymph node contour, an endpoint pixel point on the adhered lymph node contour can be used as a target first pixel point, and a pixel point with the same coordinate as the pixel point on the lymph node contour is found from the coordinates of other first pixel points and used as another target first pixel point; aiming at the condition that two endpoints of a lymph node partition line exceed the adhered lymph node outline, two pixel points with the same pixel point coordinate as the lymph node outline can be found out from the coordinate of each first pixel point and are used as two target first pixel points, the two target first pixel points are the two endpoints of the target lymph node partition line, then the two target first pixel points are used as the two side endpoints of the target lymph node partition line, the original lymph node partition line is intercepted, and the target lymph node partition line can be obtained.

Thirdly, under the condition of insufficient segmentation, as shown in fig. 4c, the dotted line in the figure is a lymph node segmentation line, the curve is two lymph nodes which are adhered, optionally, if the coordinates of the first pixel points at the two ends of the lymph node segmentation line do not exceed the range of the coordinates of each second pixel point on the lymph node outline, or if any one of the coordinates of the first pixel points at the two ends of the lymph node segmentation line exceeds the range of the coordinates of each second pixel point on the lymph node outline, performing interpolation processing on each first pixel point on the lymph node segmentation line to obtain a target lymph segmentation line. That is, when both end points of the lymph node partition line are within the adhered lymph node contour, or when one end point of the lymph node partition line exceeds the adhered lymph node contour and the other end point is within the adhered lymph node contour (that is, any one of the coordinates of the first pixel points at both ends of the lymph node partition line exceeds the range of the coordinates of the second pixel points on the lymph node contour, and the other coordinate does not exceed the range of the coordinates of the second pixel points on the lymph node contour), the lymph node partition line can be considered as being insufficient for partitioning the lymph node. Aiming at the condition that two endpoints of the lymph node parting line are both in the adhered lymph node outline, performing curve fitting on the coordinates of each first pixel point on the lymph node parting line through spline interpolation to obtain a fitted lymph node parting line, and precisely parting the lymph node parting line when the fitted lymph node parting line is used as a target lymph node parting line; and aiming at the condition that one end point of the lymph node partition line exceeds the adhered lymph node outline and the other end point of the lymph node partition line is in the adhered lymph node outline, performing curve fitting on the coordinates of each first pixel point on the lymph node partition line through spline interpolation to obtain the fitted lymph node partition line, and then selecting a target lymph node partition line which can just partition the lymph node from the fitted lymph node partition line.

The image segmentation method provided in this embodiment may match the coordinates of the first pixel points at the two ends of the lymph node segmentation line with the coordinates of each second pixel point on the lymph node contour, and determine the final target lymph node segmentation line according to the matching result. In this embodiment, since the process of coordinate matching is relatively simple and fast, the method can determine the target lymph node partition line relatively simply and fast, so that when the target lymph node partition line is used to partition the adherent lymph nodes, the efficiency of partition of the adherent lymph nodes can be further improved.

In another embodiment, another image segmentation method is provided, and the embodiment relates to a specific process that different pixel values can be set to display each lymph node after the adherent lymph node is segmented. On the basis of the above embodiment, the method may further include the steps of:

setting a first image value for all the points of the lymph node on one side of the target lymph node partition line in the lymph node mask image and setting a second image value for all the points of the lymph node on the other side of the target lymph node partition line in the lymph node mask image by taking the target lymph node partition line as a boundary, and displaying; and/or setting a first image value for a contour point of a lymph node located on one side of the target lymph node partition line in the lymph node mask image and a second image value for a contour point of a lymph node located on the other side of the target lymph node partition line in the lymph node mask image with the target lymph node partition line as a boundary, and displaying the first image value and the second image value.

In this step, the image values may be hu (hounsfiled unit) values reflecting the degree of absorption of X-rays by the tissue, commonly referred to as hounsfield units, which may also be referred to as CT values; of course, the image values here may also be pixel values, voxel values, etc.

After the target lymph node partition line is obtained, the target lymph node partition line may be superimposed on lymph nodes of the lymph node mask image, and all pixel points on the lymph node mask image on both sides of the target lymph node partition line are placed in different image values, from left to right and from top to bottom, with the target lymph node partition line as a boundary. Of course, after the target lymph node partition line is obtained, the target lymph node partition line may be superimposed on lymph nodes of the lymph node mask image, and the contour pixel points on the lymph node mask image on both sides of the target lymph node partition line are placed in different image values from left to right and from top to bottom with the target lymph node partition line as a boundary. In consideration of the possibility that two or more lymph nodes may stick together, in setting the image values, the first image value and the second image value are generally different, that is, the first image value may be gradually increased from the second image value; it is of course also possible that the first image values are decreasing to the second image values.

Further, in the present step, when setting image values for three or more lymph nodes that are stuck together, it can be explained for each target lymph node partition line, and then different image values can be set for lymph nodes on both sides of each target lymph node partition line according to the method of the present embodiment. Of course, the same image value may be set for three or more lymph nodes that are adhered together, as long as two adjacent lymph nodes that are adhered together can be distinguished.

For example, assuming that the target lymph node partition line is a partition line that partitions three stuck lymph nodes from left to right, all pixel points on a first lymph node on the left side of the lymph node mask image may be set as a first image value, all pixel points on a second lymph node from left to right may be set as a second image value, and all pixel points on a third lymph node may be set as a third image value, where the first image value is smaller than the second image value, and the second image value is smaller than the third image value, and of course, the first image value and the third image value may be equal. Pixel points on the lymph node contour are similar.

After the points on the respective lymph nodes are set to different image values, the set images can be displayed to a doctor so that the doctor can better observe the respective lymph nodes and obtain more accurate analysis results on the lymph nodes.

Of course, in addition to the above-mentioned differentiation of different lymph nodes by setting different image values, different colors or pixel values may be set for the lymph nodes that are stuck together, for example, in the case of three lymph nodes that are stuck together, when different colors are set, a red color may be set for the first lymph node, a green color may be set for the second lymph node, and a red color may be set for the third lymph node, although the third lymph node may be set to be a different color from the red and green colors, for example, yellow; when different pixel values are set, a first pixel value may be set for a first lymph node, a second lymph node may be set for a second lymph node, and a third lymph node may be set for the first pixel value.

The image segmentation method provided by the embodiment can use the target lymph node segmentation line as a boundary, and different image values are placed on lymph nodes positioned on different sides of the target lymph node segmentation line in the lymph node mask image and are displayed, so that doctors can observe each lymph node better to obtain a more accurate analysis result about the lymph node.

In another embodiment, another image segmentation method is provided, and the embodiment relates to a specific process of how to train the neural network model. On the basis of the above embodiment, as shown in fig. 5, the training process of the neural network model may include the following steps:

and S502, inputting the lymph node training image into the initial neural network model to obtain a predicted lymph node mask image and a predicted lymph node partition line mask image.

In this step, before each lymph node training image is input into the initial neural network model, resampling and normalization processing may be performed on each lymph node training image, where the normalization processing may be normalization of HU values on each lymph node training image, or normalization of gray scale, and the resolution of resampling may be [2mm,2mm,2mm ], for example. Through resampling and normalization, rapid convergence of a subsequent neural network model in a training process can be accelerated.

After resampling and normalizing each lymph node training image, randomly sampling image blocks on each lymph node training image, wherein the size of the image block can be [128,128,128], and inputting the sampled image blocks into an initial neural network model to perform convolution, pooling and other processing to obtain a predicted lymph node mask image and a predicted lymph node partition line mask image corresponding to each lymph node training image. The image is divided into image blocks for Processing, the problem of limited memory of a Graphics Processing Unit (GPU) can be fully considered, and the Processing performance of the GPU can be improved.

S504, a first loss between the predicted lymph node mask image and the gold standard lymph node mask image is calculated, and a second loss between the predicted lymph node partition line mask image and the gold standard lymph node partition line mask image is calculated.

In this step, the first loss may be a loss between each pixel point on the lymph node mask image and each pixel point on the gold standard lymph node mask image, a loss between a lymph node volume on the lymph node mask image and a lymph node volume on the gold standard lymph node mask image, a loss between a lymph node area on the lymph node mask image and a lymph node area on the gold standard lymph node mask image, or the like, or may be any combination of these losses.

The second loss may be a loss between each pixel point on the mask image of the lymph node partition line and each pixel point on the mask image of the gold standard lymph node partition line, a loss between the length of the lymph node partition line on the mask image of the lymph node partition line and the length of the lymph node partition line on the mask image of the gold standard lymph node partition line, or the like, or may be any combination of these losses.

The first loss and the second loss may be calculated by using a loss function such as a Dice loss function or a focal loss function.

S506, training the initial neural network model based on the first loss and the second loss to obtain the neural network model.

Specifically, after the first loss and the second loss are obtained, the initial neural network model may be trained by respectively adopting the first loss or the second loss, or the initial neural network model may be trained by adopting a sum of the first loss and the second loss, in this embodiment, the initial neural network model is trained by taking as an example the sum of the first loss and the second loss, when the sum of the first loss and the second loss of the neural network model is smaller than a preset threshold value, or when the sum of the first loss and the second loss is substantially stable, it may be determined that the neural network model has been trained, otherwise, the training is continued, and when the training is completed, the parameters of the neural network model may be fixed, so as to facilitate the next use. In addition, when the neural network model is trained, the model can be iteratively updated by adopting the batch length of 4.

The image segmentation method provided by this embodiment may input a plurality of lymph node training images into the initial neural network model, and train the initial neural network model through the loss of the lymph node mask image and the loss of the lymph node segmentation line mask image, so as to obtain a trained neural network model. In this embodiment, since the neural network model is obtained by training a plurality of lymph node training images and corresponding gold standard mask images, the obtained neural network model is relatively accurate, and thus when a lymph node is segmented by using the accurate neural network model, the obtained lymph node mask image and lymph node segmentation line mask image are relatively accurate.

In another embodiment, in order to facilitate a more detailed description of the technical solution of the present application, the following description is given in conjunction with a more detailed embodiment, and the method may include the following steps S1-S9:

s1, a medical image of the lymph node to be segmented is acquired.

And S2, inputting the lymph node medical image into the neural network model to obtain a lymph node mask image and a lymph node division line mask image, wherein the lymph node division line mask image comprises lymph node division lines, and the lymph node mask image comprises at least two lymph nodes.

And S3, acquiring the coordinates of each first pixel point on the lymph node segmentation line and acquiring the coordinates of each second pixel point on the lymph node in the lymph node mask image.

And S4, acquiring the coordinates of the first pixel points at the two ends of the lymph node partition line from the coordinates of the first pixel points, and acquiring the coordinates of the second pixel points on the lymph node outline from the coordinates of the second pixel points.

And S5, matching the coordinates of the first pixel points at the two ends of the lymph node partition line with the coordinates of each second pixel point on the lymph node outline.

And S6, if the coordinates of the first pixel points at the two ends of the lymph node partition line are the same as the coordinates of any two second pixel points on the lymph node outline, determining the lymph node partition line as the target lymph node partition line.

S7, if the coordinates of the first pixel points at the two ends of the lymph node partition line are the same as the coordinates of any second pixel point on the lymph node outline, or if the coordinates of the first pixel points at the two ends of the lymph node partition line exceed the range of the coordinates of each second pixel point on the lymph node outline, acquiring the coordinates of two target first pixel points from the coordinates of each first pixel point; the coordinates of the two target first pixel points are the same as the coordinates of any two second pixel points on the lymph node outline; and intercepting the lymph node parting line based on the coordinates of the first pixel points of the two targets to obtain a target lymph node parting line.

And S8, if the coordinates of the first pixel points at the two ends of the lymph node partition line do not exceed the range of the coordinates of each second pixel point on the lymph node outline, or if any one of the coordinates of the first pixel points at the two ends of the lymph node partition line exceeds the range of the coordinates of each second pixel point on the lymph node outline, performing interpolation processing on each first pixel point on the lymph node partition line to obtain a target lymph partition line.

S9, setting a first image value for all the lymph nodes on one side of the target lymph node partition line in the lymph node mask image, and setting a second image value for all the lymph nodes on the other side of the target lymph node partition line in the lymph node mask image, with the target lymph node partition line as a boundary, and displaying them; and/or setting a first image value for a contour point of a lymph node located on one side of the target lymph node partition line in the lymph node mask image and a second image value for a contour point of a lymph node located on the other side of the target lymph node partition line in the lymph node mask image with the target lymph node partition line as a boundary, and displaying the first image value and the second image value.

It should be understood that although the various steps in the flow charts of fig. 2-5 are shown in order as indicated by the arrows, the steps are not necessarily performed in order as indicated by the arrows. The steps are not performed in the exact order shown and described, and may be performed in other orders, unless explicitly stated otherwise. Moreover, at least some of the steps in fig. 2-5 may include multiple steps or multiple stages, which are not necessarily performed at the same time, but may be performed at different times, which are not necessarily performed in sequence, but may be performed in turn or alternately with other steps or at least some of the other steps.

In one embodiment, as shown in fig. 6, there is provided an image segmentation apparatus including: an obtaining module 10, a determining module 11 and a segmenting module 12, wherein:

an obtaining module 10, configured to obtain a medical image of a lymph node to be segmented;

the determining module 11 is configured to input the lymph node medical image into the neural network model to obtain a lymph node mask image and a lymph node partition line mask image; the neural network model is obtained by training based on a plurality of lymph node training images and a gold standard lymph node mask image and a gold standard lymph node partition line mask image corresponding to each lymph node training image, wherein the lymph node mask image comprises at least two lymph nodes;

and the segmentation module 12 is configured to perform segmentation processing on lymph nodes in the lymph node mask image based on the lymph node segmentation line mask image to obtain segmentation images of the lymph nodes.

For specific limitations of the image segmentation apparatus, reference may be made to the above limitations of the image segmentation method, which are not described herein again.

In another embodiment, another image segmentation apparatus is provided, and based on the above embodiment, the lymph node segmentation line mask image includes lymph node segmentation lines, and the segmentation module 12 may include an obtaining unit, a determining unit, and a segmenting unit, where:

the acquisition unit is used for acquiring the coordinates of each first pixel point on the lymph node segmentation line and acquiring the coordinates of each second pixel point on a lymph node in the lymph node mask image;

the determining unit is used for determining a target lymph node partition line corresponding to the lymph node partition line based on the coordinates of each first pixel point and the coordinates of each second pixel point; the end point of the target lymph node partition line is on the outline of the lymph node in the lymph node mask image;

and the segmentation unit is used for segmenting the lymph nodes in the lymph node mask image by adopting the target lymph node segmentation line to obtain segmentation images of all the lymph nodes.

In another embodiment, another image segmentation apparatus is provided, and on the basis of the above embodiment, the determining unit may include an acquiring subunit and a matching subunit, where:

the acquisition subunit is used for acquiring the coordinates of the first pixel points at the two ends of the lymph node partition line from the coordinates of the first pixel points and acquiring the coordinates of the second pixel points on the lymph node outline from the coordinates of the second pixel points;

and the matching subunit is used for matching the coordinates of the first pixel points at the two ends of the lymph node partition line with the coordinates of each second pixel point on the lymph node outline respectively, and determining the target lymph node partition line according to the matching result.

Optionally, the matching subunit is further configured to determine the lymph node partition line as the target lymph node partition line when the coordinates of the first pixel points at the two ends of the lymph node partition line are the same as the coordinates of any two second pixel points on the lymph node contour.

Optionally, the matching subunit is further configured to obtain coordinates of two target first pixel points from the coordinates of the first pixel points when the coordinates of the first pixel points at the two ends of the lymph node partition line are the same as the coordinates of any one second pixel point on the lymph node contour, or when the coordinates of the first pixel points at the two ends of the lymph node partition line exceed the range of the coordinates of each second pixel point on the lymph node contour; the coordinates of the two target first pixel points are the same as the coordinates of any two second pixel points on the lymph node outline; and intercepting the lymph node parting line based on the coordinates of the first pixel points of the two targets to obtain a target lymph node parting line.

Optionally, the matching subunit is further configured to perform interpolation processing on each first pixel point on the lymph node partition line to obtain the target lymph partition line when the coordinates of the first pixel points at the two ends of the lymph node partition line do not exceed the range of the coordinates of each second pixel point on the lymph node outline, or when any one of the coordinates of the first pixel points at the two ends of the lymph node partition line exceeds the range of the coordinates of each second pixel point on the lymph node outline.

In another embodiment, another image segmentation apparatus is provided, which may further include a setting display module configured to set and display a first image value for all points of a lymph node located on one side of the target lymph node partition line in the lymph node mask image and a second image value for all points of a lymph node located on the other side of the target lymph node partition line in the lymph node mask image, with the target lymph node partition line as a boundary; and/or setting a first image value for a contour point of a lymph node located on one side of the target lymph node partition line in the lymph node mask image and a second image value for a contour point of a lymph node located on the other side of the target lymph node partition line in the lymph node mask image with the target lymph node partition line as a boundary, and displaying the first image value and the second image value.

In another embodiment, another image segmentation apparatus is provided, and on the basis of the above embodiment, the apparatus may further include a training module, configured to input a lymph node training image into the initial neural network model, to obtain a predicted lymph node mask image and a predicted lymph node partition line mask image; calculating a first loss between the predicted lymph node mask image and the gold standard lymph node mask image, and calculating a second loss between the predicted lymph node partition line mask image and the gold standard lymph node partition line mask image; and training the initial neural network model based on the first loss and the second loss to obtain the neural network model.

The respective modules in the image segmentation apparatus can be wholly or partially implemented by software, hardware and a combination thereof. The modules can be embedded in a hardware form or independent from a processor in the computer device, and can also be stored in a memory in the computer device in a software form, so that the processor can call and execute operations corresponding to the modules.

In one embodiment, a computer device is provided, comprising a memory and a processor, the memory having a computer program stored therein, the processor implementing the following steps when executing the computer program:

acquiring a medical image of a lymph node to be segmented;

In one embodiment, the processor, when executing the computer program, further performs the steps of:

acquiring coordinates of each first pixel point on a lymph node segmentation line and acquiring coordinates of each second pixel point on a lymph node in a lymph node mask image; determining a target lymph node partition line corresponding to the lymph node partition line based on the coordinates of each first pixel point and the coordinates of each second pixel point; the end point of the target lymph node partition line is on the outline of the lymph node in the lymph node mask image; and (4) adopting a target lymph node segmentation line to segment the lymph nodes in the lymph node mask image to obtain segmentation images of all the lymph nodes.

acquiring coordinates of first pixel points at two ends of a lymph node partition line from the coordinates of the first pixel points, and acquiring coordinates of second pixel points on the lymph node outline from the coordinates of the second pixel points; and respectively matching the coordinates of the first pixel points at the two ends of the lymph node partition line with the coordinates of each second pixel point on the lymph node outline, and determining the target lymph node partition line according to the matching result.

if the coordinates of the first pixel points at the two ends of the lymph node partition line are the same as the coordinates of any second pixel point on the lymph node outline, or if the coordinates of the first pixel points at the two ends of the lymph node partition line exceed the range of the coordinates of the second pixel points on the lymph node outline, acquiring the coordinates of two target first pixel points from the coordinates of the first pixel points; the coordinates of the two target first pixel points are the same as the coordinates of any two second pixel points on the lymph node outline; and intercepting the lymph node parting line based on the coordinates of the first pixel points of the two targets to obtain a target lymph node parting line.

calculating a first loss between the predicted lymph node mask image and the gold standard lymph node mask image, and calculating a second loss between the predicted lymph node partition line mask image and the gold standard lymph node partition line mask image; and training the initial neural network model based on the first loss and the second loss to obtain the neural network model.

In one embodiment, a computer-readable storage medium is provided, having a computer program stored thereon, which when executed by a processor, performs the steps of:

acquiring a medical image of a lymph node to be segmented;

In one embodiment, the computer program when executed by the processor further performs the steps of:

It will be understood by those skilled in the art that all or part of the processes of the methods of the embodiments described above can be implemented by hardware instructions of a computer program, which can be stored in a non-volatile computer-readable storage medium, and when executed, can include the processes of the embodiments of the methods described above. Any reference to memory, storage, database or other medium used in the embodiments provided herein can include at least one of non-volatile and volatile memory. Non-volatile Memory may include Read-Only Memory (ROM), magnetic tape, floppy disk, flash Memory, optical storage, or the like. Volatile Memory can include Random Access Memory (RAM) or external cache Memory. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM), among others.

The technical features of the above embodiments can be arbitrarily combined, and for the sake of brevity, all possible combinations of the technical features in the above embodiments are not described, but should be considered as the scope of the present specification as long as there is no contradiction between the combinations of the technical features.

The above-mentioned embodiments only express several embodiments of the present application, and the description thereof is more specific and detailed, but not construed as limiting the scope of the invention. It should be noted that, for a person skilled in the art, several variations and modifications can be made without departing from the concept of the present application, which falls within the scope of protection of the present application. Therefore, the protection scope of the present patent shall be subject to the appended claims.

Claims

1. A method of image segmentation, the method comprising:

acquiring a medical image of a lymph node to be segmented;

and carrying out segmentation processing on the lymph nodes in the lymph node mask image based on the lymph node segmentation line mask image to obtain segmentation images of the lymph nodes.

2. The method according to claim 1, wherein the lymph node partition line mask image includes lymph node partition lines, and the process of performing partition processing on lymph nodes in the lymph node mask image based on the lymph node partition line mask image to obtain partition images of respective lymph nodes includes:

acquiring coordinates of each first pixel point on the lymph node segmentation line and acquiring coordinates of each second pixel point on a lymph node in the lymph node mask image;

determining a target lymph node partition line corresponding to the lymph node partition line based on the coordinates of each first pixel point and the coordinates of each second pixel point; the end point of the target lymph node segmentation line is on the outline of the lymph node in the lymph node mask image;

and adopting the target lymph node segmentation line to perform segmentation processing on the lymph nodes in the lymph node mask image to obtain segmentation images of all the lymph nodes.

3. The method according to claim 2, wherein the determining a target lymph node partition line corresponding to the lymph node partition line based on the coordinates of each first pixel point and the coordinates of each second pixel point comprises:

acquiring coordinates of first pixel points at two ends of the lymph node partition line from the coordinates of the first pixel points, and acquiring coordinates of second pixel points on the lymph node outline from the coordinates of the second pixel points;

4. The method of claim 3, wherein the determining the target lymph node partition line based on the matching result comprises:

5. The method of claim 3, wherein the determining the target lymph node partition line based on the matching result comprises:

if the coordinates of the first pixel points at the two ends of the lymph node partition line are the same as the coordinates of any second pixel point on the lymph node outline, or if the coordinates of the first pixel points at the two ends of the lymph node partition line exceed the range of the coordinates of each second pixel point on the lymph node outline, acquiring the coordinates of two target first pixel points from the coordinates of each first pixel point; the coordinates of the two target first pixel points are the same as the coordinates of any two second pixel points on the lymph node outline;

and intercepting the lymph node partition line based on the coordinates of the two target first pixel points to obtain the target lymph node partition line.

6. The method of claim 3, wherein the determining the target lymph node partition line based on the matching result comprises:

and if the coordinates of the first pixel points at the two ends of the lymph node partition line do not exceed the range of the coordinates of the second pixel points on the lymph node outline, or if any one of the coordinates of the first pixel points at the two ends of the lymph node partition line exceeds the range of the coordinates of the second pixel points on the lymph node outline, performing interpolation processing on each first pixel point on the lymph node partition line to obtain the target lymph partition line.

7. The method according to any one of claims 2-6, further comprising:

setting a first image value for all the points of the lymph node located on one side of the target lymph node partition line in the lymph node mask image and setting a second image value for all the points of the lymph node located on the other side of the target lymph node partition line in the lymph node mask image with the target lymph node partition line as a boundary, and displaying the first image value and the second image value;

8. The method of claim 1, wherein the training method of the neural network model comprises:

9. A computer device comprising a memory and a processor, the memory storing a computer program, characterized in that the processor, when executing the computer program, implements the steps of the method of any of claims 1 to 8.

10. A computer-readable storage medium, on which a computer program is stored, which, when being executed by a processor, carries out the steps of the method of any one of claims 1 to 8.