CN110363774B

CN110363774B - Image segmentation method and device, computer equipment and storage medium

Info

Publication number: CN110363774B
Application number: CN201910521640.XA
Authority: CN
Inventors: 邵影; 高耀宗
Original assignee: Shanghai United Imaging Intelligent Healthcare Co Ltd
Current assignee: Shanghai United Imaging Intelligent Healthcare Co Ltd
Priority date: 2019-06-17
Filing date: 2019-06-17
Publication date: 2021-12-21
Anticipated expiration: 2039-06-17
Also published as: CN110363774A

Abstract

According to the image segmentation method, the image segmentation device, the computer equipment and the storage medium, the first segmentation model is used for preliminarily segmenting the input first sampling image to obtain a first segmentation image comprising a target morphological structure and an adjacent morphological structure; and further obtaining a second image to be segmented according to the first segmented image and the second sampling image, and then segmenting the input second image to be segmented again by adopting a second segmentation model to obtain a segmented image containing a target morphological structure. The image segmentation method can overcome the problem of inaccurate segmentation caused by over-segmentation or under-segmentation due to unclear boundaries of the target morphological structure and the adjacent morphological structures in the traditional segmentation method, and improves the segmentation precision of the segmented target morphological structure.

Description

Image segmentation method and device, computer equipment and storage medium

Technical Field

The present application relates to the field of image recognition technologies, and in particular, to an image segmentation method and apparatus, a computer device, and a storage medium.

Background

With the continuous development of various medical imaging devices, many diseases are detected by the medical imaging devices which are required to perform projection imaging inside the body, and then the images of the projection imaging are analyzed by the computer devices, so that the diseases are detected. In the detection process of various organ diseases, how to accurately segment a target organ in a projection imaging image becomes an important link in the detection process of the organ diseases at present.

At present, the segmentation method for the target organ is mainly a traditional segmentation method, that is, the method includes: firstly, preprocessing an imaging image to be segmented to eliminate the influence caused by imaging equipment and environmental conditions, and then further inputting the preprocessed image into a trained segmentation model to directly obtain a segmented image, wherein the position and the size of a target organ are displayed on the segmented image.

However, the above-described segmentation method has a problem of low accuracy of segmentation.

Disclosure of Invention

In view of the above, it is necessary to provide an image segmentation method, an apparatus, a computer device, and a storage medium capable of effectively improving the accuracy of segmenting an image in view of the above technical problems.

In a first aspect, a method of image segmentation, the method comprising:

sampling a first image to be segmented to obtain a first sampling image and a second sampling image; the first image to be segmented comprises at least two morphological structures; the resolution of the first sampled image is less than the resolution of the second sampled image;

inputting the first sampling image into a preset first segmentation model to obtain a first segmentation image; the first segmentation image comprises a target morphological structure and an adjacent morphological structure of the target morphological structure;

obtaining a second image to be segmented according to the first segmented image and the second sampling image; the second image to be segmented comprises a target morphological structure and a part of adjacent morphological structures;

and inputting the second image to be segmented into a preset second segmentation model to obtain a segmented image of the target morphological structure.

In one embodiment, obtaining a second image to be segmented according to the first segmented image and the second sampled image includes:

removing adjacent morphological structures in the first segmentation image to obtain a first intermediate segmentation image;

and obtaining a second image to be segmented according to the first intermediate segmentation image and the second sampling image.

In one embodiment, obtaining a second image to be segmented according to the first intermediate segmentation image and the second sampling image includes:

determining a bounding box of the target morphological structure according to the first intermediate segmentation image;

and according to the boundary frame, performing cutting processing on the second sampling image to obtain a second image to be segmented.

In one embodiment, inputting a second image to be segmented into a preset second segmentation model to obtain a segmented image of the target morphological structure includes:

inputting the second image to be segmented into a second segmentation model to obtain a second segmentation image; the second segmentation image comprises the target morphological structure and a portion of the neighboring morphological structure;

removing part of adjacent morphological structures in the second segmentation image to obtain a second intermediate segmentation image;

performing interpolation processing on the second intermediate segmentation image to obtain a segmentation image of a target morphological structure; the size and resolution of the segmented image of the target morphological structure are the same as the first image to be segmented.

In one embodiment, the method further comprises training the first segmentation model and the second segmentation model, including:

acquiring a plurality of first sample images and corresponding first gold standard images, and a plurality of second sample images and corresponding second gold standard images; the first gold standard image and the second gold standard image comprise a marked target morphological structure and an adjacent morphological structure of the target morphological structure; respectively taking the first gold standard image and the second gold standard image as training target images, and correspondingly training a first segmentation model and a second segmentation model; wherein the resolution of the second sample image is greater than the resolution of the first sample image.

In one embodiment, acquiring a first sample image comprises:

randomly selecting a central point on an original sample image, and sampling the original sample image at a preset first sampling interval to obtain a first sampling sample image;

carrying out gray level normalization processing on the first sampling sample image to obtain a first normalized image;

the first normalized image is taken as the first sample image.

In one embodiment, acquiring a second sample image comprises:

randomly selecting a central point on a target structure on the original sample image, and sampling the original sample image at a preset second sampling interval to obtain a second sampling sample image, wherein the second sampling interval is smaller than the first sampling interval;

carrying out gray level normalization on the second sampling sample image to obtain a second normalized image;

and taking a second normalized image as the second sample image.

In a second aspect, an image segmentation apparatus, the apparatus comprising:

the sampling module is used for sampling the first image to be segmented to obtain a first sampling image and a second sampling image; the first image to be segmented comprises at least two morphological structures; the resolution of the first sampled image is less than the resolution of the second sampled image;

the first segmentation module is used for inputting the first sampling image into a preset first segmentation model to obtain a first segmentation image; the first segmented image comprises a target morphological structure and neighboring structures of the target morphological structure;

the processing module is used for obtaining a second image to be segmented according to the first segmented image and the second sampling image; the second image to be segmented comprises the target morphological structure and part of the adjacent morphological structure;

and the second segmentation module is used for inputting the second image to be segmented to a preset second segmentation model to obtain a segmented image of the target morphological structure.

In a third aspect, a computer device includes a memory and a processor, the memory stores a computer program, and the processor implements the image segmentation method according to any one of the embodiments of the first aspect when executing the computer program.

In a fourth aspect, a computer-readable storage medium has stored thereon a computer program which, when executed by a processor, implements the image segmentation method of any of the embodiments of the first aspect.

According to the image segmentation method, the image segmentation device, the computer equipment and the storage medium, the first segmentation model is used for preliminarily segmenting the input first sampling image to obtain a first segmentation image comprising a target morphological structure and an adjacent morphological structure; and further obtaining a second image to be segmented according to the first segmented image and the second sampling image, and then segmenting the input second image to be segmented again by adopting a second segmentation model to obtain a segmented image containing a target morphological structure. In the two successive segmentation processes, because the resolution of the first sampling image is smaller than that of the second sampling image, which is equivalent to the first segmentation process being a rough segmentation process for the first image to be segmented, and the second segmentation process being a fine segmentation process for the first image to be segmented, the accuracy of the segmented image can be greatly improved by two successive segmentation processes with different degrees. In addition, because the target morphological structure is positioned and restricted and referenced by using the adjacent morphological structures in the segmentation process, the problem that the target morphological structure is not accurately segmented due to inaccurate positioning caused by different structural morphologies in the traditional segmentation process of the target morphological structure is solved.

Drawings

FIG. 1 is a schematic diagram illustrating an internal structure of a computer device according to an embodiment;

FIG. 2 is a flow diagram of an image segmentation method according to an embodiment;

FIG. 3 is a flowchart of one implementation of the embodiment S103 of FIG. 2;

FIG. 4 is a flowchart of one implementation of the embodiment S202 of FIG. 3;

FIG. 5 is a flowchart of one implementation of S104 of the embodiment of FIG. 2;

FIG. 6 is a flow diagram of a training method provided by an embodiment;

FIG. 7 is a flowchart of one implementation of S501 in the embodiment of FIG. 6;

FIG. 8 is a flowchart of another implementation of the embodiment S501 in FIG. 6;

FIG. 9 is a diagram illustrating an exemplary training network according to an embodiment;

FIG. 10 is a schematic diagram of an alternative training network according to an embodiment;

fig. 11 is a schematic structural diagram of a split network according to an embodiment;

FIG. 12 is a diagram illustrating a segmented image, according to an embodiment;

FIG. 13 is a diagram illustrating a segmented image, according to an exemplary embodiment;

fig. 14 is a schematic structural diagram of an image segmentation apparatus according to an embodiment;

fig. 15 is a schematic structural diagram of an image segmentation apparatus according to an embodiment.

Detailed Description

In order to make the objects, technical solutions and advantages of the present application more apparent, the present application is described in further detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the present application and are not intended to limit the present application.

The image segmentation method provided by the application can be applied to the computer device shown in fig. 1, the computer device can be a terminal, and the internal structure diagram of the computer device can be shown in fig. 1. The computer device includes a processor, a memory, a network interface, a display screen, and an input device connected by a system bus. Wherein the processor of the computer device is configured to provide computing and control capabilities. The memory of the computer device comprises a nonvolatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of an operating system and computer programs in the non-volatile storage medium. The network interface of the computer device is used for communicating with an external terminal through a network connection. The computer program is executed by a processor to implement an image segmentation method. The display screen of the computer equipment can be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer equipment can be a touch layer covered on the display screen, a key, a track ball or a touch pad arranged on the shell of the computer equipment, an external keyboard, a touch pad or a mouse and the like.

Those skilled in the art will appreciate that the architecture shown in fig. 1 is merely a block diagram of some of the structures associated with the disclosed aspects and is not intended to limit the computing devices to which the disclosed aspects apply, as particular computing devices may include more or less components than those shown, or may combine certain components, or have a different arrangement of components.

The following describes in detail the technical solutions of the present application and how the technical solutions of the present application solve the above technical problems by embodiments and with reference to the drawings. The following several specific embodiments may be combined with each other, and details of the same or similar concepts or processes may not be repeated in some embodiments.

Fig. 2 is a flowchart of an image segmentation method according to an embodiment. The execution subject of the embodiment is the computer device shown in fig. 1, and the embodiment relates to a specific process of the computer device for segmenting the first image to be segmented by using the segmentation model. As shown in fig. 2, the method includes:

s101, sampling a first image to be segmented to obtain a first sampling image and a second sampling image; the first image to be segmented comprises at least two morphological structures; the resolution of the first sampled image is less than the resolution of the second sampled image.

Wherein, the morphological structure can be various tissues, organs, tumors, etc. on the human body. The first image to be segmented represents an image to be segmented, and is an image that may contain various morphological structures, for example, it may specifically include various morphological structures such as stomach, liver, spleen, large intestine, heart, etc. Specifically, the first image to be segmented may simultaneously include at least two types of morphological structures having adjacent positional relationships, for example, two types of adjacent morphological structures such as a stomach and a liver, a large intestine, and a small intestine. The first image to be segmented may include, but is not limited to, a conventional CT image, an MRI image, a PET-MRI image, and the like, which is not limited in this embodiment. In practical application, the computer device may be connected to the scanning device to scan various morphological structures of the human body to obtain the first image to be segmented. Optionally, the computer device may also obtain the first image to be segmented including various adjacent morphological structures directly from the database or by downloading from the internet, which is not limited in this embodiment. The first sampling image is an image obtained after the first image to be segmented is sampled at a first sampling interval, the second sampling image is an image obtained after the first image to be segmented is sampled at a second sampling interval, and the first sampling interval is larger than the second sampling interval, so that the resolution of the first sampling image is smaller than that of the second sampling image.

In this embodiment, when the computer device acquires the first image to be segmented, sampling processing at different sampling intervals is performed on the first image to be segmented from different directions, so as to obtain sampling images with different resolutions. For example, taking a three-dimensional coordinate system as an example, the computer device may sample image data in x, y, and z directions of the first image to be segmented at a first sampling interval to obtain a first sampled image, and may also sample image data in x, y, and z directions of the first image to be segmented at a second sampling interval to obtain a second sampled image, where when the first sampling interval is greater than the second sampling interval, a resolution of the obtained first sampled image is smaller than a resolution of the second sampled image. It should be noted that specific values of the first sampling interval and the second sampling interval may be determined according to practical application requirements, as long as the requirement that the resolution of the obtained first sampled image is smaller than the resolution of the second sampled image is met, for example, the first sampling interval may specifically be 6, and the corresponding second sampling interval may be 1, which is not limited in this embodiment.

S102, inputting the first sampling image into a preset first segmentation model to obtain a first segmentation image; the first segmented image includes the target morphological structure and neighboring morphological structures of the target morphological structure.

The first segmentation model is a network model for segmenting the image, and a convolutional neural network model can be selected, and is specifically used for segmenting a target morphological structure and an adjacent morphological structure of the target morphological structure of the first sampled image to obtain a segmented first segmented image. The target morphological structure is a morphological structure that is to be segmented, and may be any type of morphological structure, for example, morphological structures of organs, tissues, or tumors such as stomach, spleen, liver, heart, etc., and accordingly, adjacent morphological structures of the target morphological structure may be any type of morphological structure as long as the target morphological structure is located adjacent to the target morphological structure. The first segmented image includes two types of morphological structures having an adjacent positional relationship, and specifically includes a target morphological structure and adjacent morphological structures of the target morphological structure, such as the stomach and the liver, the left atrium and the right atrium.

In this embodiment, when the computer device acquires the first sample image according to the step of S101, the first sample image may be input into a first segmentation network trained in advance, and the first segmentation network may be used to perform segmentation processing on the target morphological structure and the adjacent morphological structure on the first sample image to obtain a first segmentation image, so as to complete the first segmentation of this embodiment and segment the target morphological structure and the adjacent morphological structure of the target morphological structure.

S103, obtaining a second image to be segmented according to the first segmented image and the second sampling image; the second image to be segmented comprises a target morphological structure and a part of adjacent morphological structures.

The second image to be segmented represents an image that needs to be segmented for the second time, and is an image including a target morphological structure and a part of adjacent morphological structures, and may specifically include various adjacent morphological structures such as a stomach and a part of a liver, a large intestine and a part of a small intestine, a spleen and a part of a stomach, for example.

The embodiment relates to a specific process of processing a first segmentation image by computer equipment to obtain a second image to be segmented, which specifically includes: when the computer device acquires the second sampling image according to the method in S101 and the second image to be segmented according to the method in S103, the computer device may first perform positioning processing on the target morphological structure on the first segmented image to obtain positioning information of the target morphological structure, and then further perform extraction or cropping processing on the second sampling image according to the positioning information of the target morphological structure to obtain the second image to be segmented including the target morphological structure and a part of adjacent morphological structures.

And S104, inputting the second image to be segmented into a preset second segmentation model to obtain a segmented image of the target morphological structure.

The second segmentation model is a network model for segmenting the image, and a convolutional neural network model can be selected, and is specifically used for segmenting the second image to be segmented by a target morphological structure and a part of adjacent morphological structures so as to obtain a segmented image containing the target morphological structure and the part of adjacent morphological structures. The segmented image of the target morphological structure is a segmented image including only the target morphological structure.

In this embodiment, when the computer device acquires the second image to be segmented according to the method in S103, the second image to be segmented may be input into a second segmentation model trained in advance, the second segmentation model is used to perform segmentation processing on the second image to be segmented on the target morphological structure and part of the target morphological structure, so as to obtain a segmented image including the target morphological structure and part of the adjacent morphological structures, and then the corresponding image processing method may be adopted to perform processing on the segmented image to remove part of the adjacent morphological structures, so as to finally obtain a segmented image of the target morphological structure. As is apparent from the above description, comparing the first division model and the second division model, the resolution of the image processed by the first division model is smaller than the resolution of the image processed by the second division model, and therefore, the first division model is used to implement the coarse division process on the first image to be divided, and the second division model is used to implement the fine division process on the second image to be divided.

In the image segmentation method provided in the above embodiment, the first segmentation model is used to perform preliminary segmentation on the input first sample image to obtain a first segmented image including a target morphological structure and an adjacent morphological structure; and further obtaining a second image to be segmented according to the first segmented image and the second sampling image, and then performing segmentation processing on the input second image to be segmented by adopting a second segmentation model to obtain a segmented image containing a target morphological structure. In the two successive segmentation processes, because the resolution of the first sampling image is smaller than that of the second sampling image, which is equivalent to the first segmentation process being a rough segmentation process for the first image to be segmented, and the second segmentation process being a fine segmentation process for the first image to be segmented, the accuracy of the segmented image can be greatly improved by two successive segmentation processes with different degrees. In addition, because the target morphological structure is positioned and restricted and referenced by using the adjacent morphological structures in the segmentation process, the problem that the target morphological structure is not accurately segmented due to inaccurate positioning caused by different structural morphologies in the traditional segmentation process of the target morphological structure is solved.

Fig. 3 is a flowchart of an implementation manner of the embodiment S103 in fig. 2. The embodiment relates to a specific process of processing a first segmentation image by computer equipment to obtain a second image to be segmented, as shown in fig. 3, the process includes:

s201, removing adjacent morphological structures in the first segmentation image to obtain a first intermediate segmentation image.

In this embodiment, when the computer device acquires the first divided image, a corresponding image removing method may be further used to perform removing processing on the target image in the first divided image, and in particular, the object of this embodiment is to remove adjacent morphological structures in the first divided image, so as to obtain an image only including the target morphological structure, that is, the first intermediate divided image in this embodiment. In the image removing method according to the present invention, the target morphological structure and the adjacent morphological structure included in the first divided image are divided and labeled with different labels, and therefore, the adjacent morphological structure may be removed according to different division labels. For example, if the target morphological structure in the first segmented image is labeled with red, the adjacent morphological structure is labeled with blue, and the background image is labeled with black, the computer device may change the color of the label of the adjacent morphological structure, specifically, change the color of the label of blue to the color (black) of the background image, when performing the image removal of the adjacent morphological structure, so as to achieve the goal of removing the image of the adjacent morphological structure. Of course, the computer device may also adopt other image removing methods as long as the adjacent morphological structure can be removed from the first segmented image, and the embodiment is not limited thereto.

S202, obtaining a second image to be segmented according to the first middle segmentation image and the second sampling image.

When the computer device acquires the first intermediate segmented image based on S201, because the first intermediate segmented image only includes the target morphological structure, the computer device may further accurately obtain the position information of the target morphological structure according to the first intermediate segmented image, then the computer device may find the target morphological structure in the second sampled image according to the position information, and finally extract the target morphological structure from the second sampled image to obtain the second image to be segmented, so as to perform segmentation processing on the second image to be segmented later.

Based on the above embodiments, further, the present application also provides a specific implementation manner of S202. Fig. 4 is a flowchart of an implementation manner of the embodiment S202 in fig. 3. As shown in fig. 4, the process includes:

s301, determining a boundary frame of the target morphological structure according to the first intermediate segmentation image.

The bounding box is used for marking the position of the target morphological structure in the first segmented image, and may be a square frame or a rectangular frame. In practical applications, when the computer device acquires the first intermediate segmented image only including the target morphological structure, a bounding box may be specifically adopted to frame the target morphological structure in the first intermediate segmented image, so that the computer device may determine the position of the target morphological structure in the second sample image according to the position of the bounding box.

And S302, cutting the second sampling image according to the boundary frame to obtain a second image to be segmented.

In this embodiment, after the computer device determines the boundary frame of the target morphological structure, the second sampled image may be cropped by using the boundary frame as a boundary, and the morphological structure included in the boundary frame in the second sampled image is cropped to obtain a second image to be segmented. Optionally, after the computer device determines the bounding box of the target morphological structure, the bounding box may be reduced or enlarged first, then the second sampled image is cropped by using the bounding box as a boundary, and the morphological structure included in the reduced or enlarged bounding box in the second sampled image is cropped to obtain a second image to be segmented. In practical applications, the shape structure included in the bounding box may include a target shape structure, or include a part of the target shape structure, or include both the target shape structure and a part of the neighboring shape structure, which may be determined by the size of the bounding box. In addition, when the boundary frame is reduced or enlarged, the reduced size or the enlarged size may be determined according to the actual application requirements, for example, the reduced size or the enlarged size may be 10mm, 20mm, 30mm, and the like, which is not limited in this embodiment.

In an exemplary manner, when the size of the bounding box acquired by the computer device is 100mm × 100mm, and the size of the target morphological structure is 110mm × 110mm, the computer device performs cropping processing on the first sample image along the bounding box, and then the morphological structure included in the obtained bounding box is a partial target morphological structure; assuming that the size of the boundary box acquired by the computer device is 100mm × 100mm, and the size of the target morphological structure is 100mm × 100mm, the computer device performs cropping processing on the first sample image along the boundary box, and then the morphological structure included in the acquired boundary box is the target morphological structure; if the size of the bounding box acquired by the computer device is 100mm × 100mm and the size of the target morphological structure is 90mm × 90mm, the morphological structure included in the bounding box obtained by the computer device performing the cropping processing on the first sample image along the bounding box is the target morphological structure and the morphological structure partially adjacent to the target morphological structure.

In the above embodiment, the computer device first performs processing of removing adjacent morphological structures on the first segmented image to obtain a first intermediate segmented image; and then, determining a boundary frame indicating the position of the target morphological structure according to the target morphological structure contained in the first intermediate segmentation image, and then extracting the morphological structure contained in the boundary frame from the second sampling image by using the boundary frame to obtain a second image to be segmented. In the above process, since the boundary frame may be expanded and then the morphological structure in the expanded boundary frame may be extracted from the second sampled image to obtain the second image to be segmented, such a method may enable the expanded boundary frame to completely contain the target morphological structure and a small portion of the adjacent morphological structure, and then if such a second image to be segmented is input into the second segmentation model to perform the segmentation processing of the target morphological structure and the portion of the adjacent morphological structure, a second segmentation image completely containing the target morphological structure and a small portion of the adjacent morphological structure may be obtained, and then an image of the target morphological structure is further extracted from the second segmentation image to obtain an image finally containing only the target morphological structure, such an image segmentation method may overcome the problem of inaccurate segmentation caused by over-segmentation or under-segmentation due to unclear boundaries between the target morphological structure and the adjacent morphological structure in the conventional segmentation method, the segmentation precision of the segmentation target morphological structure provided by the application is improved.

Fig. 5 is a flowchart of an implementation manner of the embodiment S104 in fig. 2. The embodiment relates to a specific process of segmenting a second image to be segmented by computer equipment, as shown in fig. 5, the process includes:

s401, inputting a second image to be segmented into a second segmentation model to obtain a second segmentation image; the second segmentation image comprises the target morphological structure and a portion of the neighboring morphological structure.

In this embodiment, when the computer device acquires the second image to be segmented according to the foregoing embodiment, the second image to be segmented may be input into a second segmentation model trained in advance, and the second segmentation model is used to perform segmentation processing on the target morphological structure and part of the target morphological structure on the second image to be segmented, so as to obtain a second segmented image including the target morphological structure and part of the adjacent morphological structures, so that the computer device performs further processing on the second segmented image.

S402, removing part of adjacent morphological structures in the second segmentation image to obtain a second intermediate segmentation image.

The embodiment relates to a specific method for removing a part of adjacent morphological structures in a second segmented image by using an image removing method, which is the same as the removing method described in the foregoing S201, and specific contents may refer to the foregoing description, and redundant description is not repeated here. In practical applications, after the computer device removes part of the neighboring morphological structure in the second segmented image, a second intermediate segmented image containing only the target morphological structure can be obtained, so that the computer device can process the second intermediate segmented image.

And S403, performing interpolation processing on the second intermediate segmentation image to obtain a segmentation image of the target morphological structure, wherein the size and the resolution of the segmentation image of the target morphological structure are the same as those of the first image to be segmented.

The embodiment further relates to a step of performing post-processing on the second intermediate segmented image, that is, when the computer device acquires the second intermediate segmented image, the second intermediate segmented image may be further subjected to interpolation processing to change the size and the resolution of the second intermediate segmented image, so that the segmented image of the target morphological structure obtained by performing interpolation processing on the second intermediate segmented image has the same size and resolution as the first image to be segmented.

In the above embodiment, after the computer device obtains the second divided image, since the second divided image includes the target adjacent structure and the partial adjacent morphological structure, and then the corresponding image removing method is used to remove the partial adjacent structure, the divided image including only the target morphological structure can be obtained.

As can be seen from the foregoing description, the first segmentation model and the second segmentation model are network models that are trained by a computer device in advance, so the present application further provides a method for training the first segmentation model and the second segmentation model, fig. 6 is a flowchart of a training method provided by an embodiment, where the embodiment relates to a process of training the first segmentation model and the second segmentation model to be trained by the computer device according to the first sample image and the second sample image and using the first gold standard image and the second gold standard image as training target images, and as shown in fig. 6, the process includes:

s501, acquiring a first sample image and a corresponding first gold standard image, and acquiring a second sample image and a corresponding second gold standard image; the first gold standard image and the second gold standard image comprise marked target morphological structures and adjacent structures of the target morphological structures; wherein the resolution of the second sample image is greater than the resolution of the first sample image.

The first sample image represents an image used when the first segmentation model needs to be trained currently, and is an image obtained by performing first sampling processing on an original sample image by a computer device, and is the same as the type of the first sampling image described in the foregoing S101, the second sample image represents an image used when the second segmentation model needs to be trained currently, and is an image obtained by performing second sampling processing on an original sample image by a computer device, and is the same as the type of the second sampling image described in the foregoing S101, and specific contents may refer to the foregoing description, and redundant description is not repeated here.

The first gold standard image corresponds to the gold standard obtained by sampling the first sample image (randomly sampling on the marked original sample image). The second gold standard image corresponds to the gold standard obtained by sampling the second sample image (randomly sampling the target structure in the marked original sample image). The first gold standard image and the second gold standard image include at least two morphological structures having an adjacent relationship, and if two adjacent morphological structures are included, the first gold standard image and the second gold standard image may specifically include a complete target morphological structure and a complete adjacent target morphological structure, may include a partial target morphological structure and a partial adjacent target morphological structure, and may also include a complete target morphological structure and a partial adjacent target morphological structure.

It should be noted that the original sample image may be marked in various ways. For example, assuming that the target morphological structure and the adjacent morphological structures included in the original sample image are a stomach and a liver, when labeling, the region where the stomach is located may be specifically labeled as 1, the region where the liver is located may be labeled as 2, and the background region may be labeled as 0. For another example, when marking, the area where the stomach is located can be marked as red, the area where the liver is located can be marked as green, and the background area can be marked as black or other colors. The specific way to label can be determined according to the actual application requirements, and this embodiment is not limited.

The first gold standard image corresponds to the first sample image, so that when the computer device acquires the first gold standard image, the target morphological structure and the adjacent morphological structure in the original sample image can be marked according to the method, then the marked original sample image is further subjected to sampling processing at a first sampling interval, and the image obtained after sampling at this time is taken as the first gold standard image. Similarly, the second gold standard image corresponds to the second sample image, so that when the computer device acquires the second gold standard image, the marked original sample image is sampled at a second sampling interval, and the image obtained after sampling at this time is used as the second gold standard image. Since the first sampling interval is different from the second sampling interval, the first gold standard image and the second gold standard image are images obtained by different sampling methods. In practical application, the computer device can scan various morphological structures of a human body by connecting with the scanning device to obtain an original sample image. Optionally, the computer device may also obtain the original sample image containing various adjacent morphological structures directly from the database or from internet download, which is not limited in this embodiment.

S502, correspondingly training a first segmentation model and a second segmentation model by respectively taking the first gold standard image and the second gold standard image as training target images.

The embodiment relates to a process for training a first segmentation model and a second segmentation model, which specifically comprises the following steps: when the computer device obtains the first sample image and the corresponding first golden standard based on the step of S501, the first sample image and the corresponding first golden standard image are input into the first segmentation model, the segmentation image corresponding to the first sample image is output, and then parameters of the first segmentation model are adjusted according to a difference between the output segmentation model and the first golden standard image for training until a loss function of the first segmentation model to be trained converges, so as to obtain the trained first segmentation model, so as to be used in the embodiment of fig. 2. Accordingly, when the computer device acquires the second sample image and the corresponding second golden standard based on the step of S501, the second sample image and the corresponding second golden standard image are input into the second segmentation model, the segmentation image corresponding to the second sample image is output, and then parameters of the second segmentation model are adjusted according to a difference between the output segmentation image and the second golden standard image for training until a loss function of the second segmentation model to be trained converges, so as to obtain the trained second segmentation model, so as to be used in the above embodiment of fig. 2.

In the process of training the first segmentation model and the second segmentation model, morphological structures with adjacent relations are marked in the first gold standard image and the second gold standard image, namely, in the training process, the adjacent morphological structures are added as position reference bases of the target morphological structures, so that the problem of inaccurate segmentation caused by unclear boundaries between the adjacent morphological structures is solved.

Fig. 7 is a flowchart of an implementation manner of the embodiment S501 in fig. 6. The embodiment relates to a specific process for acquiring a first sample image by a computer device, as shown in fig. 7, the process includes:

s601, randomly selecting a central point on the original sample image, and sampling the original sample image at a preset first sampling interval to obtain a first sampling sample image.

Wherein the original sample image represents an image initially acquired by the computer device. For example, a computer device may directly acquire an image of an original sample through a CT scanning device. The first sampling interval may be determined according to practical application requirements, and this embodiment is not limited thereto. The embodiment relates to a step of preprocessing an original sample image before segmenting the original sample image by computer equipment, and specifically comprises the following steps: when the computer device obtains the original sample image, a point can be further randomly selected from any area in the original sample image, the point is taken as a central point, the size of the preset sampling image and the first sampling interval are combined, a sampling starting point is determined according to the central point, the original sample image is sampled in different directions, and the first sampling sample image with the size consistent with that of the preset sampling image is obtained. It should be noted that the size of the preset sampling image may be defined by the computer device in advance according to the actual application requirement, for example, the size of the preset sampling image may be set to 96 × 96, and the present embodiment is not limited thereto.

S602, carrying out gray level normalization processing on the first sampling sample image to obtain a first normalized image.

The embodiment relates to a step of preprocessing a first sample image before segmenting the first sample image by computer equipment, and specifically includes: when the computer device obtains the first sampling sample image, the gray normalization processing may be further performed on the first sampling sample image by using a corresponding gray value normalization method, and specifically, the gray value of each pixel point in the first sampling sample image may be set within a specific range to obtain a first normalized image, so that the first normalized image may be used to perform more accurate image segmentation.

And S603, taking the first normalized image as a first sample image.

After the computer device performs normalization processing on the first sampling image to obtain a first normalized image, the first normalized image can be directly used as a first sample image, and then the first sample image is conveniently segmented.

Fig. 8 is a flowchart of another implementation manner of the embodiment S501 in fig. 6. The embodiment relates to a specific process for acquiring a second sample image by a computer device, as shown in fig. 8, the process includes:

s701, randomly selecting a central point on a target structure on an original sample image, and sampling the original sample image at a preset second sampling interval to obtain a second sampling sample image; the second sampling interval is less than the first sampling interval.

The content of the present embodiment is substantially the same as the content described in the foregoing S601, except that the process of selecting the center point in the area where the target structure is located is different when the center point is selected on the original sample image, other steps are substantially the same, and the detailed content refers to the description of the foregoing S601, and repeated and redundant description is not provided herein. In addition, the second sampling interval in this embodiment is smaller than the first sampling interval, so that the resolution of the obtained second sampling image is smaller than that of the first sampling image, so as to meet the requirement of training the first segmentation model and the second segmentation model in practical application. It should be noted that, in this embodiment, when the central point is selected, the central point is selected in the area where the target structure is located, such a method enables the second sampling sample image to include only the complete target morphological structure and the complete adjacent morphological structure or a part of the adjacent morphological structure, and performs segmentation training on such second sampling sample image, so that the training model learns the difference between the target structure and the adjacent structure with emphasis, so that the target morphological structure and the adjacent morphological structure included in the second sampling sample image can be accurately segmented, and the segmentation accuracy when the trained second segmentation model is used to perform segmentation on the target morphological structure and the adjacent morphological structure is further improved.

And S702, carrying out gray level normalization on the second sampling sample image to obtain a second normalized image.

The present embodiment relates to a step of preprocessing a second sampled image, and the content of the description of the present embodiment is substantially the same as the content described in the foregoing S602, and please refer to the description of the foregoing S602 for details, which will not be described repeatedly.

And S703, taking the second normalized image as a second sample image.

The description of the present embodiment is substantially the same as the description of the foregoing S603, and please refer to the description of the foregoing S603 for detailed description, and redundant description will not be repeated here.

In combination with the embodiments of fig. 6-8, the present application further provides a method for training a first segmentation model and a second segmentation model respectively, and the following embodiments briefly describe the above two processes.

A first application scenario, training a first segmentation model to be trained by using a training network as shown in fig. 9, where the training network includes: the image segmentation method comprises a first sampling module, a first normalization module and a first segmentation model to be trained, wherein the first sampling module is used for randomly selecting a central point in any region of an original sample image, determining a sampling starting point according to the central point by combining the size (96 x 96 in the figure) of a preset sampling image and a first sampling interval (6 in the figure), and sampling the original sample image in different directions (x, y and z directions in a three-dimensional coordinate system in the figure) to obtain a first sampling sample image; in practical application, the first sampling module is further configured to randomly select a central point from any marked region on the marked original sample image, determine a sampling starting point according to the central point by combining a preset size (96 × 96 in the drawing) of the sampled image and a first sampling interval (6 in the drawing), and sample the marked original sample image in different directions (x, y, and z directions in a three-dimensional coordinate system in the drawing) to obtain a first gold standard image.

The first normalization module is configured to perform gray value normalization processing on the first sampling sample image to obtain a first normalized sample image, that is, a first sample image.

The process of training the first segmentation model to be trained by using the training network comprises the following steps:

s801, acquiring an original sample image and a marked original sample image.

S802, inputting the original sample image into a first sampling module for sampling processing to obtain a first sampling sample image; and inputting the marked original sample image into a first sampling module to perform sampling processing by using the same sampling central point to obtain a first gold standard image.

And S803, inputting the first sampling sample image into a first normalization module to perform gray value normalization processing, so as to obtain a first sample image.

S804, inputting the first sample image and the first gold standard image into a first segmentation model to be trained for model training, and obtaining the trained first segmentation model.

In a second application scenario, a second segmentation model to be trained is trained by using a training network as shown in fig. 10, where the training network includes: the second sampling module is used for randomly selecting a central point on a target structure of an original sample image, determining a sampling starting point according to the central point by combining the size (96 x 96 in the figure) of a preset sampling image and a second sampling interval (1 in the figure), and sampling the original sample image in different directions (x, y and z directions in a three-dimensional coordinate system in the figure) to obtain a second sampling sample image; in practical application, the second sampling module is further configured to randomly select a central point from any marked region on the marked original sample image, determine a sampling starting point according to the central point by combining a preset size (96 × 96 in the drawing) of the sampled image and a second sampling interval (1 in the drawing), and sample the marked original sample image in different directions (x, y, and z directions in a three-dimensional coordinate system in the drawing) to obtain a second sampled gold standard image.

The second normalization module is configured to perform gray-value normalization processing on the second sampling sample image to obtain a second normalized sample image, that is, a second sample image.

The process of training the second segmentation model to be trained by using the training network comprises the following steps:

and S901, acquiring an original sample image and a marked original sample image.

S902, inputting the original sample image into a second sampling module for sampling processing to obtain a second sampling sample image; and inputting the marked original sample image into a second sampling module to perform sampling processing by using the same sampling central point to obtain a second gold standard image.

And S903, inputting the second sampling sample image to a second normalization module for gray value normalization processing to obtain a second sample image.

And S904, inputting the second sample image and the second gold standard image into a second segmentation model to be trained for model training to obtain the trained second segmentation model.

The above embodiment is a process of training the first segmentation model and the second segmentation model, and the resolution of the first sample image obtained in the above process is smaller than the resolution of the second sample image, so that the trained first segmentation model can realize segmentation of an image with the same resolution as the first sample image, and can recognize a target morphological structure and an adjacent morphological structure in the input image. The trained second segmentation model can realize the segmentation of the image with the same resolution of the second sample image, and can identify the target morphological structure and part of the adjacent morphological structures in the input image.

According to the embodiment, the image segmentation training method can adopt two segmentation models to successively perform segmentation training on the corresponding images of the same original sample image at different resolutions, perform segmentation training on the image with the higher resolution firstly, and then perform segmentation training on the image with the lower resolution, so that the accuracy of the segmented image can be further improved by a layer-by-layer progressive training method.

By integrating the contents described in all the embodiments, the present application further provides an image segmentation method, which is applicable to the segmentation network shown in fig. 11, where the segmentation network includes a first sampling module, a first normalization module, a first segmentation model, a first image removal module, a bounding box determination module, a second sampling module, an image cropping module, a second normalization module, a second segmentation model, a second image removal module, and an image processing module.

The first sampling module is used for sampling an input original image at a sampling interval of 6 to obtain a first sampling image; the second sampling module is used for sampling the input original image at a sampling interval of 1 to obtain a second sampling image; the first normalization module is used for performing gray value normalization processing on the input first sampling image to obtain a first normalization image; the first segmentation model is used for segmenting the input first normalized image to obtain a first segmented image; the first image removing module is used for removing adjacent morphological structures of a target morphological structure in the input first segmentation image to obtain a first image after removal processing; the boundary frame determining module is configured to determine, according to the input first image after the removal processing, a position of a region where the target morphological structure is located on the first image, and specifically, frame out the target morphological structure by using a boundary frame, so as to obtain the position of the boundary frame, that is, the position of the region where the target morphological structure is located. And the image clipping module is used for clipping the second sampling image according to the position information indicated by the boundary box to obtain a second image to be segmented.

The second normalization module is used for performing gray value normalization processing on the input second image to be segmented to obtain a second normalized image; the second segmentation model is used for segmenting the input second normalized image to obtain a second segmented image; the second image removing module is used for removing part of adjacent morphological structures in the input second segmentation image to obtain a second image after removal processing; the image processing module is used for carrying out image interpolation processing on the input second image after the removal processing to obtain a segmented image which is the same as the original image in size and resolution, and the segmented image only comprises a target morphological structure.

Briefly describing the process of segmenting the input original image using the network structure shown above: firstly, inputting an acquired original image into a first sampling module by computer equipment for sampling at a sampling interval of 6 to obtain a first sampling image; the computer equipment further performs grey value normalization processing on the first sampling image to obtain a first normalized image; then the computer equipment inputs the first normalized image into a first segmentation model to carry out segmentation of a target morphological structure and an adjacent morphological structure to obtain a first segmentation image; then, the computer equipment inputs the first divided image into a first image removing module to remove adjacent morphological structures, so as to obtain a first image after removal processing, wherein the first image only comprises a target morphological structure; and then the computer equipment inputs the first image into a boundary box determining module to determine the position of the target morphological structure, so as to obtain the position information of the boundary box. Inputting the obtained original image into a second sampling module to perform sampling processing with a sampling interval of 1 to obtain a second sampled image, and then further inputting the second sampled image and the position information of the boundary frame into an image cutting module by computer equipment to perform cutting processing to obtain a cut image, namely a second image to be cut; then the computer equipment inputs the second image to be segmented into a second normalization module to carry out grey value normalization processing to obtain a second normalized image, and inputs the second normalized image into a second segmentation model to carry out segmentation processing to obtain a second segmentation image; then, the computer equipment inputs the second divided image into a second image removing module to remove part of adjacent morphological structures, so as to obtain a second image after removal processing, wherein the second image only contains a target morphological structure; and finally, inputting the second image into an image processing module by the computer equipment for image interpolation processing, so that the size and the resolution of the obtained segmented image are the same as those of the original image.

In the following embodiments, the segmentation method described above is applied to show the segmentation result by taking the target morphological structure as a stomach organ and the adjacent morphological structure as a liver organ. Fig. 12(a) shows a stomach organ segmentation image obtained without the segmentation method of the present embodiment, and fig. 12 (b) shows a stomach organ segmentation image obtained with the segmentation method of the present embodiment. Fig. 13(a) shows another segmented image of a stomach organ obtained without the segmentation method according to the present embodiment, and fig. 13 (b) shows another segmented image of a stomach organ obtained by the segmentation method according to the present embodiment. Comparing the two segmentation results in fig. 12, it can be seen that the segmentation of the stomach organ in fig. 12(a) over-segments onto the liver organ. Comparing the two segmentation results in fig. 13, it can be seen that the segmentation of the stomach organ in fig. 13(a) is completely over-segmented onto the liver organ.

It should be understood that although the various steps in the flow charts of fig. 2-8 are shown in order as indicated by the arrows, the steps are not necessarily performed in order as indicated by the arrows. The steps are not performed in the exact order shown and described, and may be performed in other orders, unless explicitly stated otherwise. Moreover, at least some of the steps of fig. 2-8 may include multiple sub-steps or multiple stages that are not necessarily performed at the same time, but may be performed at different times, and the order of performance of the sub-steps or stages is not necessarily sequential.

In one embodiment, as shown in fig. 14, there is provided an image segmentation apparatus including: a sampling module 11, a first segmentation module 12, a processing module 13, and a second segmentation module 14, wherein:

the sampling module 11 is configured to sample a first image to be segmented to obtain a first sampled image and a second sampled image; the first image to be segmented comprises at least two morphological structures; the resolution of the first sampled image is less than the resolution of the second sampled image;

the first segmentation module 12 is configured to input the first sampling image to a preset first segmentation model to obtain a first segmentation image; the first segmented image includes a target morphological structure and an adjacent structure of the target morphological structure;

the processing module 13 is configured to obtain a second image to be segmented according to the first segmented image and the second sampled image; the second image to be segmented comprises a target morphological structure and a part of adjacent morphological structures;

and the second segmentation module 14 is configured to input the second image to be segmented to a preset second segmentation model, so as to obtain a segmented image of the target morphological structure.

In one embodiment, as shown in fig. 15, the image segmentation apparatus further includes: an acquisition module 15 and a training module 16, wherein:

an obtaining module 15, configured to obtain a first sample image and a corresponding first gold standard image, and a second sample image and a corresponding second gold standard image; the first gold standard image and the second gold standard image comprise a marked target morphological structure and an adjacent morphological structure of the target morphological structure; wherein the resolution of the second sample image is greater than the resolution of the first sample image;

and the training module 16 is used for correspondingly training the first segmentation model and the second segmentation model by respectively taking the first gold standard image and the second gold standard image as training target images.

For specific limitations of the image segmentation apparatus, reference may be made to the above limitations of an image segmentation method, which are not described herein again. The respective modules in the image segmentation apparatus can be wholly or partially implemented by software, hardware and a combination thereof. The modules can be embedded in a hardware form or independent from a processor in the computer device, and can also be stored in a memory in the computer device in a software form, so that the processor can call and execute operations corresponding to the modules.

In one embodiment, a computer device is provided, comprising a memory and a processor, the memory having a computer program stored therein, the processor implementing the following steps when executing the computer program:

The implementation principle and technical effect of the computer device provided by the above embodiment are similar to those of the above method embodiment, and are not described herein again.

In one embodiment, a computer-readable storage medium is provided, having a computer program stored thereon, the computer program, when executed by a processor, further implementing the steps of:

The implementation principle and technical effect of the computer-readable storage medium provided by the above embodiments are similar to those of the above method embodiments, and are not described herein again.

It will be understood by those skilled in the art that all or part of the processes of the methods of the embodiments described above can be implemented by hardware instructions of a computer program, which can be stored in a non-volatile computer-readable storage medium, and when executed, can include the processes of the embodiments of the methods described above. Any reference to memory, storage, database, or other medium used in the embodiments provided herein may include non-volatile and/or volatile memory, among others. Non-volatile memory can include read-only memory (ROM), Programmable ROM (PROM), Electrically Programmable ROM (EPROM), Electrically Erasable Programmable ROM (EEPROM), or flash memory. Volatile memory can include Random Access Memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in a variety of forms such as Static RAM (SRAM), Dynamic RAM (DRAM), Synchronous DRAM (SDRAM), Double Data Rate SDRAM (DDRSDRAM), Enhanced SDRAM (ESDRAM), synchronous Link (Synchlink) DRAM (SLDRAM), Rambus Direct RAM (RDRAM), direct bus dynamic RAM (DRDRAM), and bus dynamic RAM (RDRAM).

The technical features of the above embodiments can be arbitrarily combined, and for the sake of brevity, all possible combinations of the technical features in the above embodiments are not described, but should be considered as the scope of the present specification as long as there is no contradiction between the combinations of the technical features.

The above-mentioned embodiments only express several embodiments of the present invention, and the description thereof is more specific and detailed, but not construed as limiting the scope of the invention. It should be noted that, for a person skilled in the art, several variations and modifications can be made without departing from the inventive concept, which falls within the scope of the present invention. Therefore, the protection scope of the present patent shall be subject to the appended claims.

Claims

1. A method of image segmentation, the method comprising:

sampling the first image to be segmented at different sampling intervals to obtain a first sampling image and a second sampling image; the first image to be segmented comprises at least two types of morphological structures with adjacent position relation; the resolution of the first sampled image is less than the resolution of the second sampled image;

inputting the first sampling image into a preset first segmentation model to obtain a first segmentation image; the first segmented image comprises a target morphological structure and an adjacent morphological structure of the target morphological structure; the target morphological structure and the adjacent morphological structure are segmented and labeled by different labels;

obtaining a second image to be segmented according to the first segmented image and the second sampling image; the second image to be segmented comprises the target morphological structure and part of the adjacent morphological structure;

inputting the second image to be segmented into a preset second segmentation model to obtain a second segmentation image; the second segmentation image comprises the target morphological structure and a portion of the neighboring morphological structure;

and removing the part of the adjacent morphological structure in the second segmentation image to obtain the segmentation image of the target morphological structure.

2. The method according to claim 1, wherein the deriving a second image to be segmented from the first segmented image and the second sampled image comprises:

and obtaining the second image to be segmented according to the first intermediate segmentation image and the second sampling image.

3. The method according to claim 2, wherein the deriving the second image to be segmented from the first intermediate segmented image and the second sampled image comprises:

determining a bounding box of the target morphological structure according to the first intermediate segmented image;

and according to the boundary frame, performing clipping processing on the second sampling image to obtain the second image to be segmented.

4. The method according to any of claims 1-3, wherein said removing said portion of said neighboring morphological structure in said second segmented image to obtain a segmented image of said target morphological structure comprises:

removing the part of the adjacent morphological structure in the second segmentation image to obtain a second intermediate segmentation image;

performing interpolation processing on the second intermediate segmentation image to obtain a segmentation image of the target morphological structure; and the size and the resolution of the segmented image of the target morphological structure are the same as those of the first image to be segmented.

5. The method of claim 1, further comprising, prior to the method, training the first and second segmentation models, comprising:

acquiring a first sample image and a corresponding first gold standard image, and acquiring a second sample image and a corresponding second gold standard image; the first gold standard image and the second gold standard image comprise a marked target morphological structure and an adjacent morphological structure of the target morphological structure;

correspondingly training the first segmentation model and the second segmentation model by respectively taking the first gold standard image and the second gold standard image as training target images;

wherein a resolution of the second sample image is greater than a resolution of the first sample image.

6. The method of claim 5, wherein said acquiring a first sample image comprises:

taking the first normalized image as the first sample image.

7. The method of claim 6, wherein acquiring a second sample image comprises:

randomly selecting a central point on a target structure on the original sample image, and sampling the original sample image at a preset second sampling interval to obtain a second sampling sample image; the second sampling interval is less than the first sampling interval;

taking the second normalized image as the second sample image.

8. An image segmentation apparatus, characterized in that the apparatus comprises:

the sampling module is used for sampling the first image to be segmented at different sampling intervals to obtain a first sampling image and a second sampling image; the first image to be segmented comprises at least two types of morphological structures with adjacent position relation; the resolution of the first sampled image is less than the resolution of the second sampled image;

the first segmentation module is used for inputting the first sampling image into a preset first segmentation model to obtain a first segmentation image; the first segmented image comprises a target morphological structure and neighboring structures of the target morphological structure; the target morphological structure and the adjacent morphological structure are segmented and labeled by different labels;

the second segmentation module is used for inputting the second image to be segmented to a preset second segmentation model to obtain a second segmentation image; the second segmentation image comprises the target morphological structure and a portion of the neighboring morphological structure; and removing the part of the adjacent morphological structure in the second segmentation image to obtain the segmentation image of the target morphological structure.

9. A computer device comprising a memory and a processor, the memory storing a computer program, wherein the processor implements the steps of the method of any one of claims 1 to 7 when executing the computer program.

10. A computer-readable storage medium, on which a computer program is stored, which, when being executed by a processor, carries out the steps of the method according to any one of claims 1 to 7.