CN111507950A - Image segmentation method and device, electronic equipment and computer-readable storage medium - Google Patents

Abstract

The application discloses a method and a device for image segmentation, an electronic device and a computer readable storage medium, wherein the method for image segmentation comprises the following steps: acquiring a first feature layer based on the image; obtaining a second characteristic layer through a neural network based on a residual error structure according to the first characteristic layer; and acquiring a segmentation result of the image according to the second characteristic layer. According to the technical scheme, even under the condition of being influenced by image quality, the accuracy of the image segmentation result can be improved.

Description

Image segmentation method and device, electronic equipment and computer-readable storage medium

Technical Field

The present application relates to the field of deep learning technologies, and in particular, to a method and an apparatus for image segmentation, an electronic device, and a computer-readable storage medium.

Background

The image segmentation technology can divide an image into a plurality of specific areas with unique properties, so that an object which is interested by a user can be segmented from the background, and the technology has wide application prospects in various fields of driving, pedestrian detection, medical treatment and the like. The existing image segmentation method has low accuracy, and particularly under the influence of image quality, the robustness of image segmentation is difficult to guarantee.

Disclosure of Invention

In view of the above, embodiments of the present application are directed to providing an image segmentation method and apparatus, an electronic device, and a computer-readable storage medium, which can improve the accuracy of a segmentation result of an image even under the influence of image quality.

According to a first aspect of embodiments of the present application, there is provided an image segmentation method, including: acquiring a first feature layer based on the image; obtaining a second characteristic layer through a neural network based on a residual error structure according to the first characteristic layer; and acquiring a segmentation result of the image according to the second characteristic layer.

In one embodiment, the obtaining, according to the first feature layer, a second feature layer through a neural network based on a residual structure includes: obtaining a third feature layer through a first neural network according to the first feature layer; obtaining a fourth feature layer through a second neural network according to the first feature layer; and acquiring the second characteristic layer according to the first characteristic layer, the third characteristic layer and the fourth characteristic layer.

In one embodiment, said obtaining the second feature layer according to the first feature layer, the third feature layer, and the fourth feature layer includes: and performing feature fusion on the first feature layer, the third feature layer and the fourth feature layer to obtain the second feature layer.

In one embodiment, said obtaining a first feature layer based on said image comprises: and obtaining the first characteristic layer through a dense convolution network according to the image.

In one embodiment, the obtaining a segmentation result of the image according to the second feature layer includes: and performing convolution operation on the second characteristic layer to obtain a segmentation result of the image.

In one embodiment, the obtaining a segmentation result of the image according to the second feature layer includes: obtaining an edge refining feature layer through a third neural network according to the second feature layer; and carrying out convolution operation on the edge thinning characteristic layer to obtain a segmentation result of the image.

In one embodiment, the method further comprises: obtaining a fifth characteristic layer through coding operation according to the first characteristic layer; according to the fifth characteristic layer, a sixth characteristic layer is obtained through the neural network based on the residual error structure; and obtaining a seventh characteristic layer through decoding operation according to the sixth characteristic layer.

In one embodiment, the obtaining a fifth feature layer through an encoding operation according to the first feature layer includes: obtaining a ninth feature layer through a dense convolution network according to the first feature layer; and performing convolution operation and downsampling operation on the ninth feature layer to obtain the fifth feature layer.

In one embodiment, before obtaining a seventh feature layer through a decoding operation according to the sixth feature layer, the method further includes: and obtaining a sixth characteristic layer after edge refinement through a third neural network according to the sixth characteristic layer.

In one embodiment, obtaining a seventh feature layer through a decoding operation according to the sixth feature layer includes: and obtaining the seventh characteristic layer through the decoding operation according to the sixth characteristic layer after the edge refinement.

In one embodiment, the obtaining, according to the sixth feature layer, a seventh feature layer through a decoding operation includes: and obtaining the seventh characteristic layer through an up-sampling operation according to the sixth characteristic layer.

In one embodiment, the obtaining a segmentation result of the image according to the second feature layer includes: obtaining an integrated characteristic layer through splicing operation according to the seventh characteristic layer and the second characteristic layer; and performing convolution operation on the integrated characteristic layer to obtain a segmentation result of the image.

In one embodiment, the obtaining, according to the sixth feature layer, a seventh feature layer through a decoding operation includes: according to the sixth characteristic layer and the second characteristic layer, obtaining a fused characteristic layer through characteristic fusion; according to the sixth characteristic layer, obtaining an eighth characteristic layer through an up-sampling operation; determining the seventh feature layer based on the fused feature layer and the eighth feature layer.

In one embodiment, the obtaining a segmentation result of the image according to the second feature layer includes: obtaining an integrated feature layer through splicing operation according to the eighth feature layer and the fused feature layer; and performing convolution operation on the integrated characteristic layer to obtain a segmentation result of the image.

In one embodiment, the image is a medical image of the brain, and the segmentation result of the image is a multi-class segmentation result of the background, the left ventricle and the right ventricle.

According to a second aspect of embodiments of the present application, there is provided an apparatus for image segmentation, including: an acquisition module configured to acquire a first feature layer based on the image; the information fusion module is configured to obtain a second characteristic layer through a neural network based on a residual error structure according to the first characteristic layer; and the segmentation module is configured to acquire a segmentation result of the image according to the second feature layer.

In one embodiment, the apparatus for image segmentation further comprises: and a module for performing each step in the method for image segmentation mentioned in the above embodiments.

According to a third aspect of embodiments of the present application, there is provided an electronic apparatus, including: a processor for performing the method of image segmentation mentioned in the above embodiments; a memory for storing the processor-executable instructions.

According to a fourth aspect of embodiments of the present application, there is provided a computer-readable storage medium storing a computer program for executing the method of image segmentation mentioned in the above embodiments.

According to the image segmentation method provided by the embodiment of the application, the first characteristic layer is obtained based on the image, the second characteristic layer is obtained through the neural network based on the residual error structure according to the first characteristic layer, and the segmentation result of the image is obtained according to the second characteristic layer, so that the accuracy of the segmentation result of the image can be improved even under the influence of the image quality.

Drawings

The above and other objects, features and advantages of the present application will become more apparent by describing in more detail embodiments of the present application with reference to the attached drawings. The accompanying drawings are included to provide a further understanding of the embodiments of the application and are incorporated in and constitute a part of this specification, illustrate embodiments of the application and together with the description serve to explain the principles of the application. In the drawings, like reference numbers generally represent like parts or steps.

FIG. 1 is a schematic diagram of an implementation environment provided by an embodiment of the present application.

FIG. 2 is a block diagram of a system for image segmentation provided by an embodiment of the present application.

Fig. 3 is a flowchart illustrating a method for image segmentation according to an embodiment of the present application.

Fig. 4 is a schematic diagram of an image processing procedure of a neural network of a residual structure according to an embodiment of the present application.

Fig. 5 is a schematic diagram of an image processing procedure of a first neural network according to an embodiment of the present application.

FIG. 6 is a schematic diagram of an image processing procedure of a second neural network provided by an embodiment of the present application.

Fig. 7 is a schematic diagram of an image processing process of a dense block in a dense convolutional network according to an embodiment of the present application.

Fig. 8 is a schematic diagram of an image processing procedure of a third neural network according to an embodiment of the present application.

Fig. 9 is a flowchart illustrating a method for image segmentation according to another embodiment of the present application.

Fig. 10 is a flowchart illustrating an image segmentation method according to another embodiment of the present application.

Fig. 11 is a flowchart illustrating a method for image segmentation according to still another embodiment of the present application.

Fig. 12a to 12c are schematic diagrams of a segmentation process of a method for image segmentation according to an embodiment of the present application.

Fig. 13 is a block diagram illustrating an apparatus for image segmentation according to an embodiment of the present application.

Fig. 14 is a block diagram illustrating an electronic device according to an embodiment of the present application.

Detailed Description

The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are only a part of the embodiments of the present application, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.

Summary of the application

Medical images are images that reflect the internal structure or internal function of an anatomical region and are composed of a set of image elements, either pixels (2D) or voxels (3D). Medical images are discrete image representations produced by sampling or reconstruction that can map values to different spatial locations. Medical images are mostly radiographic, functional, magnetic resonance, ultrasound imaging. The medical image is mostly a single-channel grayscale image, and although a large number of medical images are 3D, there is no concept of depth of field in the medical image. The current medical digital image devices include CT, MTI, CR, DR, etc., which are generally in DICOM3.0 as the standard file format. Medical image display systems, which are usually configured as high-performance PCs and are equipped with high-resolution displays, provide physicians with information that assists diagnosis and treatment in an intuitive manner, and in particular, enable experienced radiologists and clinicians to derive a great deal of useful information from these images. So for medical images, which are critical for the details in the image, a single image contains a large amount of data. However, in the image acquisition process, the image quality is often reduced due to the influence of various factors such as the acquisition method, the equipment and random interference, so that the segmentation result of the image is inaccurate.

Deep learning implements artificial intelligence in a computing system by building artificial neural networks with hierarchical structures. Because the artificial neural network of the hierarchical structure can extract and screen the input information layer by layer, the deep learning has the characteristic learning capability and can realize end-to-end supervised learning and unsupervised learning. The artificial neural network of the hierarchical structure used for deep learning has various forms, the complexity of the hierarchy is generally called 'depth', and the forms of deep learning comprise a multilayer perceptron, a convolutional neural network, a cyclic neural network, a deep belief network and other mixed structures according to the types of structures. The deep learning uses data to update parameters in the construction of the data to achieve a training target, the process is generally called 'learning', the deep learning provides a method for enabling a computer to automatically learn mode characteristics, and the characteristic learning is integrated into the process of establishing a model, so that the incompleteness caused by artificial design characteristics is reduced.

A neural network is an operational model, which is formed by a large number of nodes (or neurons) connected to each other, each node corresponding to a policy function, and the connection between each two nodes representing a weighted value, called weight, for a signal passing through the connection. The neural network generally comprises a plurality of neural network layers, the upper network layer and the lower network layer are mutually cascaded, the output of the ith neural network layer is connected with the input of the (i + 1) th neural network layer, the output of the (i + 1) th neural network layer is connected with the input of the (i + 2) th neural network layer, and the like. After the training samples are input into the cascaded neural network layers, an output result is output through each neural network layer and is used as the input of the next neural network layer, therefore, the output is obtained through calculation of a plurality of neural network layers, the prediction result of the output layer is compared with a real target value, the weight matrix and the strategy function of each layer are adjusted according to the difference condition between the prediction result and the target value, the neural network continuously passes through the adjusting process by using the training samples, so that the parameters such as the weight of the neural network and the like are adjusted until the prediction result of the output of the neural network is consistent with the real target result, and the process is called the training process of the neural network. After the neural network is trained, a neural network model can be obtained.

In view of the foregoing technical problems, the present application provides an image segmentation method, which mainly includes obtaining a first feature layer based on an image, obtaining a second feature layer based on the first feature layer through a neural network based on a residual error structure, and obtaining a segmentation result of the image according to the second feature layer, so that accuracy of the segmentation result of the image can be improved even under the influence of image quality.

Having described the general principles of the present application, various non-limiting embodiments of the present application will now be described with reference to the accompanying drawings.

Exemplary System

FIG. 1 is a schematic diagram of an implementation environment provided by an embodiment of the present application. Taking a medical image of the brain as an example, the implementation environment comprises a CT scanner 130, a server 120 and a computer device 110. The computer device 110 may acquire medical images of the brain from the CT scanner 130, and the computer device 110 may be connected to the server 120 via a communication network. Optionally, the communication network is a wired network or a wireless network.

The CT scanner 130 is used for performing X-ray scanning on the human tissue to obtain a CT image of the human tissue. In one embodiment, the brain medical image may be obtained by scanning the brain with the CT scanner 130.

The computer device 110 may be a general-purpose computer or a computer device composed of an application-specific integrated circuit, and the like, which is not limited in this embodiment. For example, the Computer device 110 may be a mobile terminal device such as a tablet Computer, or may be a Personal Computer (PC), such as a laptop portable Computer and a desktop Computer. One skilled in the art will appreciate that the number of computer devices 110 described above may be one or more, and that the types may be the same or different. For example, the number of the computer devices 110 may be one, or the number of the computer devices 110 may be several tens or hundreds, or more. The number and the type of the computer devices 110 are not limited in the embodiments of the present application.

A trained image segmentation model for segmenting the brain medical image may be deployed in the computer device 110, and the image segmentation model is composed of a multi-level network (e.g., a neural network based on a residual structure, a dense convolutional network, a semantic segmentation network, or a global convolutional network, etc.), and the image segmentation model with the multi-level network can improve the accuracy of the segmentation result of the image even under the influence of the image quality. In some alternative embodiments, the computer device 110 may segment its medical image of the brain acquired from the CT scanner 130 using the image segmentation model deployed thereon to segment out the multi-class segmentation results of the background, left ventricle, and right ventricle.

The server 120 is a server, or consists of several servers, or is a virtualization platform, or a cloud computing service center. In some optional embodiments, the server 120 acquires a sample image of the brain according to the artificially labeled brain medical image to train an image segmentation model, and the trained image segmentation model is composed of a multi-level network (e.g., a neural network based on a residual structure, a dense convolutional network, a semantic segmentation network, or a global convolutional network, etc.), and the accuracy of the segmentation result of the image can be improved by the image segmentation model with the multi-level network even under the influence of the image quality. The computer device 110 may transmit the medical brain image acquired from the CT scanner 130 to the server 120, and the server 120 segments the medical brain image by using the image segmentation model trained thereon to segment the multi-class segmentation results of the background, the left ventricle and the right ventricle.

FIG. 2 is a block diagram of a system for image segmentation provided by an embodiment of the present application. As shown in fig. 2, the system includes:

the preprocessing module 21 is configured to perform normalization, drying removal and/or image enhancement on the original brain medical image a to obtain a brain medical image B;

and the image segmentation model 22 is used for obtaining a segmentation result C of the image according to the brain medical image B.

The method of image segmentation in the present application is implemented in this way with reference to the data flow shown by the solid arrow line in fig. 2.

Exemplary method

Fig. 3 is a flowchart illustrating a method for image segmentation according to an embodiment of the present application. The method described in fig. 3 is performed by a computing device (e.g., a server), but the embodiments of the present application are not limited thereto. The server may be one server, or may be composed of a plurality of servers, or may be a virtualization platform, or a cloud computing service center, which is not limited in this embodiment of the present application. As shown in fig. 3, the method includes:

s301: a first feature layer is acquired based on the image.

In an embodiment, the image may be a medical image, in particular a brain medical image. Of course, the image may also be a medical image acquired by other medical equipment, or an image in other fields such as a driving field, and the type of the image is not particularly limited in the embodiment of the present invention, that is, the method for segmenting an image in the embodiment of the present application may be used for segmenting various types of images.

For convenience of description, the method for image segmentation provided by the embodiment of the present application is described in detail below by taking a medical image of a brain as an example.

In another embodiment, the brain medical image may be a brain medical image obtained by normalizing, de-drying, and/or image enhancing a raw brain medical image, which may be an image directly obtained by Computed Tomography (CT), Computed Radiography (CR), Digital Radiography (DR), nuclear magnetic resonance, or ultrasound, among other techniques. However, in the process of capturing the original brain medical image, noise may be introduced to affect clear and accurate display of the image, so the original brain medical image may be preprocessed, for example, the noise in the original brain medical image may be removed by using a gaussian filter or a median filter. The image enhancement processing may include resizing, cropping, rotation, normalization, and normalization, etc., to improve the signal-to-noise ratio of the segmented region of the medical image of the brain. During the preprocessing, one or more of them can be used to enhance the original brain medical image for the subsequent image segmentation process. The image enhancement processing may be performed before or after the denoising processing. After the original brain medical image is subjected to some processing or attack, such as image enhancement and/or denoising, a plurality of duplicate images can be obtained, and after the duplicate images are subjected to image normalization processing with the same parameters, the same form of standard image, namely the brain medical image, can be obtained.

In another embodiment, the medical brain image may be subjected to feature extraction through a neural network to obtain the first feature layer, but the embodiment of the present application does not limit the specific type of the neural network, and may be resnet, resnext, or densent.

S302: and obtaining a second characteristic layer through a neural network based on a residual error structure according to the first characteristic layer.

In one embodiment, the first feature layer is input into a neural network based on a residual structure, and a second feature layer can be obtained. When the neural network based on the residual error structure is trained, the neural network based on the residual error structure has better convergence, the learning complexity of the neural network based on the residual error structure is reduced, and meanwhile, the segmentation accuracy can be improved through the trained neural network based on the residual error structure.

However, the embodiment of the present application is not limited to a specific type of the neural network based on the residual structure, and may be configured by at least one network structure of a convolutional neural network, a cyclic neural network, a deep neural network, and the like.

S303: and acquiring a segmentation result of the image according to the second characteristic layer.

In an embodiment, the second feature layer may be input into a convolutional neural network of a convolution kernel of 3 × 3 to obtain a segmentation result of the image, but the embodiment of the present application is not limited to the specific implementation of obtaining the segmentation result of the image, and the segmentation result of the image may also be obtained in other manners, for example, the second feature layer may be input into a convolutional neural network of another convolution kernel size, the second feature layer may also be input into another type of neural network, the second feature layer may also be subjected to post-processing (for example, an edge refinement operation, etc.) first, and then the post-processed second feature layer is input into a corresponding neural network.

In another embodiment, the segmentation result of the image may be a segmentation map, or a binary image obtained by further adjusting the segmentation map, where 0 represents a background region and 1 represents a target region (i.e., a target region such as a left ventricle and a right ventricle), which is not specifically limited in this embodiment of the present application.

Therefore, the first characteristic layer is firstly obtained based on the image, then the first characteristic layer is input into the neural network based on the residual error structure to obtain the second characteristic layer, and finally the segmentation result of the image can be obtained according to the second characteristic layer, so that the accuracy of the segmentation result of the image can be improved even under the influence of the image quality.

In another embodiment of the present application, obtaining, according to the first feature layer, a second feature layer through a neural network based on a residual structure includes: obtaining a third feature layer through a first neural network according to the first feature layer; obtaining a fourth feature layer through a second neural network according to the first feature layer; and acquiring the second characteristic layer according to the first characteristic layer, the third characteristic layer and the fourth characteristic layer.

In an embodiment, as shown in fig. 4, the neural network based on the residual structure is composed of a first neural network and a second neural network, the first feature layer is respectively input into the first neural network and the second neural network, the third feature layer is output after passing through the first neural network, the fourth feature layer is output after passing through the second neural network, and finally the second feature layer can be obtained through the first feature layer, the third feature layer and the fourth feature layer.

In another embodiment, the first neural network may be a global convolutional network GCN (global convolutional network) that increases the kernel size to the spatial size of the first feature layer, so that global information (global convolutional) may be obtained, where the global convolutional network does not directly use a particularly large convolutional kernel, but rather splits the large convolutional kernel into convolutional combinations based on the principle of GoogleNet, as shown in fig. 5, the image processing process of the first neural network is to output two feature layers to be merged (the shape of each feature layer to be merged is kept consistent) after the convolution of the first feature layer (the image size is W × H × C) with 1 × k + k × and the convolution of k × +1 583 k, and then to Sum (Sum) the two feature layers to be merged, so that a third feature layer (the image size is W × H × t) may be obtained after the convolution of the feature layers to be merged with 1 k + 461 k + 9 k) and the convolution of the two feature layers to be merged is operated (Sum) (the shape of the feature layers to be merged is kept consistent), and then the result in that the result in a reduction of the convolution of the result in a smaller number of GCN, which is easier to be compared to be increased when the convolution kernel is larger than when the convolution of the first feature layer is larger region p 6323 k × k + 7, and the convolution of the region is larger.

In another embodiment, the second neural network can be a semantic segmentation network, and the semantic segmentation network can be formed by any one of network structures such as a Full Convolution Network (FCN), a SegNet and a Deeplab, as shown in FIG. 6, the image processing process of the second neural network includes performing 1 × 1 convolution operation on the first feature layer to obtain a shallow feature layer, performing coding operation on the first feature layer (that is, performing hole convolution and downsampling operations with different sampling rates on the first feature layer to output a downsampled feature layer), performing decoding operation on the downsampled feature layer (that is, performing global pooling upsampling operation on the downsampled feature layer to output an upsampled feature layer), and performing concat operation on the shallow feature layer and the upsampled feature layer to obtain a fourth feature layer.

Therefore, the first neural network is combined with the second neural network, the second neural network can prevent overfitting with less parameter quantity, the first neural network can obtain global information, the first neural network and the second neural network achieve complementary effects, and the first neural network and the second neural network are a residual structure and have better convergence and precision.

In another embodiment of the present application, the obtaining the second feature layer according to the first feature layer, the third feature layer, and the fourth feature layer includes: and performing feature fusion on the first feature layer, the third feature layer and the fourth feature layer to obtain the second feature layer.

Specifically, the second feature layer can be obtained by performing feature fusion on the first feature layer, the third feature layer and the fourth feature layer. The feature fusion can comprise add operation and concat operation, through the add operation, the information amount under each feature of the second feature layer is increased, but the dimension (feature) of the second feature layer is not increased; through the concat operation, the number of channels of the second feature layer is the combination of the number of channels of the first feature layer, the third feature layer and the fourth feature layer, that is, the features of the second feature layer are increased, and the information under each feature is not increased. However, the embodiment of the present application does not limit the specific implementation manner of feature fusion.

In another embodiment of the present application, the acquiring a first feature layer based on the image includes: and obtaining the first characteristic layer through a dense convolution network according to the image.

In one embodiment, the image is input into a dense convolution network for feature extraction, and a first feature layer can be obtained. The dense convolutional network may comprise at least one dense block whose image processing procedure is shown in fig. 7, where each layer gets additional input from all the previous layers, e.g., feature layer 3 passes its own feature map to all the subsequent layers (feature layer 4 and feature layer 5) by using a "concat" merging mode at the channel layer (i.e., feature multiplexing by stacking up channel layers). Thus, by using the cascade mode, each layer receives ' Collective Knowledge ' (Collective Knowledge ') from the previous layers, that is, the learned features of each layer can be directly used by all the following layers, so that the features can be reused in the whole dense convolutional network, and the anti-overfitting effect can be achieved. Because a plurality of characteristic layers are separated between the shallow characteristic layer and the loss function in other networks, the gradient is gradually reduced in backward propagation, the gradient tends to 0 when the number of layers is too deep, and the gradient disappears when the gradient is transmitted back to the shallow characteristic layer, so that the network can not be converged for learning, and each layer in the dense convolutional network can be directly supervised by loss in the original network, thereby relieving the disappearance of the gradient and saving parameters and calculated amount.

In another embodiment of the present application, the obtaining a segmentation result of the image according to the second feature layer includes: and performing convolution operation on the second characteristic layer to obtain a segmentation result of the image.

In an embodiment, the second feature layer may be input into a convolutional neural network of a convolution kernel of 3 × 3 to obtain a segmentation result of the image, but the embodiment of the present application is not limited to the specific implementation of obtaining the segmentation result of the image, and the segmentation result of the image may also be obtained in other manners, for example, the second feature layer may be input into a convolutional neural network of other convolution kernel sizes, or into another type of neural network.

In another embodiment of the present application, the obtaining a segmentation result of the image according to the second feature layer includes: obtaining an edge refining feature layer through a third neural network according to the second feature layer; and carrying out convolution operation on the edge thinning characteristic layer to obtain a segmentation result of the image.

In an embodiment, the second feature layer may be input into a third neural network to obtain an edge refined feature layer, and then the edge refined feature layer is input into a convolutional neural network of a 3 × 3 convolutional kernel to obtain a segmentation result of the image.

In another embodiment, the third neural network may be a network similar to a residual structure, and is used for learning boundary information to perform edge refinement on the second feature layer to obtain an edge refined feature layer, as shown in fig. 8, after the second feature layer is subjected to a convolution operation of 3 × 3 and an activation function, one feature layer is output, after the feature layer is subjected to a convolution operation of 3 × 3, another feature layer is output, and then the another feature layer is subjected to a summation (Sum) operation with the second feature layer, an edge refined feature layer may be output.

In another embodiment of the present application, the method further comprises: obtaining a fifth characteristic layer through coding operation according to the first characteristic layer; according to the fifth characteristic layer, a sixth characteristic layer is obtained through the neural network based on the residual error structure; and obtaining a seventh characteristic layer through decoding operation according to the sixth characteristic layer.

In an embodiment, the fifth feature layer may be output after the first feature layer is subjected to an encoding operation, and a specific implementation of the encoding operation is not specifically limited in this application embodiment. After the first feature layer is subjected to encoding operation, at least one feature layer may be output, that is, the number of the fifth feature layers may be one or multiple.

In another embodiment, the fifth feature layer is input into the neural network based on the residual structure mentioned in the foregoing embodiment, so that the sixth feature layer can be obtained, and a specific network structure of the neural network based on the residual structure is not described herein again in this embodiment of the present application, and a specific process of how to obtain the sixth feature layer by the neural network based on the residual structure is also not described herein again in this embodiment of the present application, for details, see the description of the foregoing embodiment. When the number of the fifth feature layers is multiple, one fifth feature layer corresponds to one neural network based on the residual error structure, that is, each fifth feature layer can be input into the corresponding neural network based on the residual error structure, so that multiple sixth feature layers can be obtained.

In another embodiment, the sixth feature layer may output the seventh feature layer after being subjected to a decoding operation, and the embodiment of the present application does not specifically limit the specific implementation of the decoding operation, and the decoding operation and the encoding operation correspond to each other. The sixth feature layer may output a seventh feature layer after being decoded, but in the embodiment of the present application, the specific number of the seventh feature layers is not limited, and may be one or multiple, the number of the seventh feature layers may be determined according to specific application requirements, and the greater the number of the seventh feature layers, the greater the fusion degree between the semantic information and the spatial position information can be improved, so as to improve the accuracy of the segmentation result of the image.

In another embodiment of the present application, obtaining a fifth feature layer through an encoding operation according to the first feature layer includes: obtaining a ninth feature layer through a dense convolution network according to the first feature layer; and performing convolution operation and downsampling operation on the ninth feature layer to obtain the fifth feature layer.

In an embodiment, the first feature layer is input into the dense convolutional network, so that a ninth feature layer can be obtained, which is not described in detail herein in the embodiment of the present application, and a specific process of how to obtain the ninth feature layer by using the dense convolutional network is also not described herein in the embodiment of the present application, for details, see the description of the above embodiment.

In another embodiment, the fifth feature layer may be obtained by performing a convolution operation and a downsampling operation on the ninth feature layer. For example, the downsampling operation may be a pooling layer, that is, the ninth feature layer after the convolution operation is input into the pooling layer, so that the ninth feature layer maintains the invariance of Translation (Translation), Rotation (Rotation), and Scale (Scale), while the main features are retained, the parameters (i.e., dimensionality reduction, similar to PCA) and the calculation amount are reduced to prevent overfitting, thereby improving the model generalization capability, but the pooling layer may cause the loss of picture information, such as spatial position information. In the embodiment of the present application, the number of times of downsampling is not limited, and may be 4 times, 2 times, 8 times, or the like.

In another embodiment, the above two steps may be performed iteratively: 1) obtaining a ninth feature layer through a dense convolution network according to the first feature layer; 2) performing convolution operation and downsampling operation on the ninth feature layer to obtain a fifth feature layer; after the first fifth feature layer is obtained through the two steps, the first fifth feature layer is input into the dense convolution network to obtain a second ninth feature layer, and then the convolution operation and the downsampling operation are performed on the second ninth feature layer to obtain a second fifth feature layer, and so on, the fifth feature layers can be obtained.

It should be noted that, every time a ninth feature layer is obtained, at least one dense block is used, and cross-layer connection of at least one dense block is used in combination, so that semantic information is effectively extracted through the pooling layer (which is helpful for improving classification performance), and meanwhile, spatial position information (which is used for type alignment corresponding to a pixel classification label) is also retained, and in addition, the convergence capability of the model is also improved.

In another embodiment of the present application, before obtaining a seventh feature layer through a decoding operation according to the sixth feature layer, the method further includes: and obtaining a sixth characteristic layer after edge refinement through a third neural network according to the sixth characteristic layer.

In an embodiment, after the sixth feature layer is obtained by the neural network based on the residual structure, the sixth feature layer may be directly input into the third neural network to obtain the sixth feature layer after edge refinement. When the number of the sixth feature layers is multiple, one sixth feature layer corresponds to one third neural network, that is, each sixth feature layer may be input into the corresponding third neural network, so that multiple sixth feature layers with edges being refined may be obtained. The third neural network is not described herein again in the embodiments of the present application, and a specific process of how to obtain the sixth feature layer after edge refinement through the third neural network is also not described herein in the embodiments of the present application, for details, see the description of the above embodiments.

In another embodiment of the present application, obtaining a seventh feature layer through a decoding operation according to the sixth feature layer includes: and obtaining the seventh characteristic layer through the decoding operation according to the sixth characteristic layer after the edge refinement.

In an embodiment, after the sixth feature layer after edge refinement is obtained through the third neural network, a decoding operation is performed on the sixth feature layer after edge refinement to obtain a seventh feature layer. The embodiment of the present application does not describe the decoding operation in detail here, and the embodiment of the present application also does not describe in detail how to obtain the seventh feature layer by the decoding operation here, which is described in detail in the following embodiments.

In another embodiment of the present application, obtaining a seventh feature layer through a decoding operation according to the sixth feature layer includes: and obtaining the seventh characteristic layer through an up-sampling operation according to the sixth characteristic layer.

In an embodiment, the sixth feature layer is upsampled to obtain a seventh feature layer. However, the embodiment of the present application does not limit the specific number of the seventh feature layers, one seventh feature layer may be obtained through the upsampling operation, and multiple seventh feature layers may also be obtained, where the number of the seventh feature layers may be determined according to specific application requirements, and the greater the number of the seventh feature layers obtained through the upsampling operation, the more the fusion degree between the semantic information and the spatial position information can be improved, so as to improve the accuracy of the segmentation result of the image. It should be noted that the present embodiment does not limit the multiple of upsampling, and may be 4 times, 2 times, 8 times, and so on.

For example, when the number of the sixth feature layers is one, and the fifth feature layer is obtained after 2 times of downsampling, 2 times of upsampling may be performed on the sixth feature layer to obtain a seventh feature layer. When the number of the sixth feature layers is plural, specifically, the sixth feature layers include a sixth feature layer 1, a sixth feature layer 2, and a sixth feature layer 3, and the plural fifth feature layers are obtained by down-sampling 2 times, 4 times, and 8 times, respectively. The sixth feature layer 3 may be upsampled by 8 times to obtain a first seventh feature layer, the sixth feature layer 2 may be upsampled by 4 times to obtain a second seventh feature layer, and the sixth feature layer 1 may be upsampled by 2 times to obtain a third seventh feature layer.

In another embodiment, an upsampling operation may be used to compensate for this deficiency, since the pooling layer may result in the loss of spatial location information during the encoding operation. In particular, the upsampling operation may recover the true spatial location of the maximum activation value lost in the pooling layer and place it in the correct location during the upsampling operation. And the up-sampling operation is very meaningful for segmenting relatively small subcutaneous tissue. There are two common ways of upsampling: deconvolution and interpolation, wherein interpolation includes bilinear interpolation (bilinear).

In another embodiment of the present application, the obtaining a segmentation result of the image according to the second feature layer includes: obtaining an integrated characteristic layer through splicing operation according to the seventh characteristic layer and the second characteristic layer; and performing convolution operation on the integrated characteristic layer to obtain a segmentation result of the image.

In an embodiment, when the fifth feature layer is obtained by 2 times of downsampling, 2 times of upsampling may be performed on the sixth feature layer to obtain a seventh feature layer having a size consistent with that of the second feature layer, then the seventh feature layer and the second feature layer are subjected to a splicing operation (i.e., a concat operation, which is not described herein again, for details, see the above embodiment), the feature layers after integration may be obtained, and then the feature layers after integration are subjected to a convolution operation, so that a segmentation result of the image may be obtained.

In another embodiment, after the integrated feature layer is obtained, the integrated feature layer may be further input to a third neural network to perform edge refinement on the integrated feature layer, and then perform convolution operation on the integrated feature layer with edge refinement, so as to obtain a segmentation result of the image, where the obtained segmentation result of the image is more accurate. The third neural network and the specific process of performing edge refinement by the third neural network are not repeated herein in the embodiments of the present application, and for details, refer to the description of the above embodiments.

It should be noted that the embodiment of the present application is not limited to the specific implementation of the convolution operation, and reference may be made to the description of the above embodiment.

In another embodiment of the present application, obtaining a seventh feature layer through a decoding operation according to the sixth feature layer includes: according to the sixth characteristic layer and the second characteristic layer, obtaining a fused characteristic layer through characteristic fusion; according to the sixth characteristic layer, obtaining an eighth characteristic layer through an up-sampling operation; determining the seventh feature layer based on the fused feature layer and the eighth feature layer.

In an embodiment, an upsampling operation may be performed on the sixth feature layer first, so that a scale of the upsampled sixth feature layer is consistent with a scale of the second feature layer, so as to facilitate feature fusion of the upsampled sixth feature layer and the second feature layer. It should be noted that the present embodiment does not limit the multiple of upsampling, and may be 4 times, 2 times, 8 times, and so on.

For example, when the number of the sixth feature layers is one, and the fifth feature layer is obtained through 2 times of downsampling, 2 times of upsampling may be performed on the sixth feature layer to obtain an upsampled sixth feature layer with a size consistent with that of the second feature layer, and then the second feature layer and the upsampled sixth feature layer are feature-fused, so as to obtain a fused feature layer. When the number of the sixth feature layers is plural, specifically, the sixth feature layers include a sixth feature layer 1, a sixth feature layer 2, and a sixth feature layer 3, and the plural fifth feature layers are a fifth feature layer 1, a fifth feature layer 2, and a fifth feature layer 3, respectively. The fifth feature layer 1 is obtained by performing 2-fold down-sampling, the fifth feature layer 2 is obtained by performing 4-fold down-sampling, and the fifth feature layer 3 is obtained by performing 8-fold down-sampling. The sixth feature layer 3 may be up-sampled by 2 times and then feature-fused with the sixth feature layer 2 to obtain the feature-fused feature layer 1. And then performing 2 times of upsampling on the feature fusion feature layer 1, and performing feature fusion on the feature fusion feature layer 1 and the sixth feature layer 1 to obtain a feature fusion feature layer 2. And finally, performing 2 times of upsampling on the feature fusion feature layer 2 and performing feature fusion on the feature fusion feature layer and the second feature layer to obtain a fusion feature layer.

In another embodiment, the eighth feature layer may be obtained by performing an upsampling operation on the sixth feature layer. However, the embodiment of the application does not limit the specific number of the eighth feature layers, one eighth feature layer or multiple eighth feature layers can be obtained through the upsampling operation, the number of the eighth feature layers can be determined according to specific application requirements, and the greater the number of the eighth feature layers obtained through the upsampling operation, the greater the fusion degree between the semantic information and the spatial position information can be, so as to improve the accuracy of the segmentation result of the image. It should be noted that the present embodiment does not limit the multiple of upsampling, and may be 4 times, 2 times, 8 times, and so on.

For example, when the number of the sixth feature layer is one, and the fifth feature layer is obtained through 2 times of downsampling, 2 times of upsampling may be performed on the sixth feature layer to obtain an eighth feature layer consistent with the scale of the second feature layer. When the number of the sixth feature layers is plural, specifically, the sixth feature layers include a sixth feature layer 1, a sixth feature layer 2, and a sixth feature layer 3, and the plural fifth feature layers are a fifth feature layer 1, a fifth feature layer 2, and a fifth feature layer 3, respectively. The fifth feature layer 1 is obtained by performing 2-time down-sampling, the fifth feature layer 2 is obtained by performing 4-time down-sampling, the fifth feature layer 3 is obtained by performing 8-time down-sampling, the sixth feature layer 3 may be up-sampled by 8 times to obtain a first eighth feature layer, the sixth feature layer 2 may be up-sampled by 4 times to obtain a second eighth feature layer, and the sixth feature layer 1 may be up-sampled by 2 times to obtain a third eighth feature layer.

In another embodiment, the combination of the fused feature layer and the eighth feature layer together form the seventh feature layer, that is, the number of the seventh feature layers in this embodiment is at least two.

In another embodiment of the present application, the obtaining a segmentation result of the image according to the second feature layer includes: obtaining an integrated feature layer through splicing operation according to the eighth feature layer and the fused feature layer; and performing convolution operation on the integrated characteristic layer to obtain a segmentation result of the image.

In an embodiment, since the scales of the eighth feature layer and the fused feature layer obtained by the foregoing embodiment are consistent, a splicing operation (that is, a concat operation, which is not described herein again in this embodiment of the present application, for specific details, refer to the foregoing embodiment) may be directly performed on the eighth feature layer and the fused feature layer, an integrated feature layer may be obtained, and then a convolution operation is performed on the integrated feature layer, so that a segmentation result of the image may be obtained.

In another embodiment, after the integrated feature layer is obtained, the integrated feature layer may be further input to a third neural network to perform edge refinement on the integrated feature layer, and then perform convolution operation on the integrated feature layer with edge refinement, so as to obtain a segmentation result of the image, where the obtained segmentation result of the image is more accurate. The third neural network and the specific process of performing edge refinement by the third neural network are not repeated herein in the embodiments of the present application, and for details, refer to the description of the above embodiments.

It should be noted that the embodiment of the present application is not limited to the specific implementation of the convolution operation, and reference may be made to the description of the above embodiment.

In another embodiment of the present application, the image is a medical brain image, and the segmentation result of the image is a multi-class segmentation result of a background, a left ventricle and a right ventricle.

It should be noted that, in the embodiment of the present application, the specific type of the segmentation result is not limited, and the segmentation result may also be a multi-class segmentation result of other target areas.

Fig. 9 is a flowchart illustrating a method for image segmentation according to another embodiment of the present application. The embodiment shown in fig. 9 is a preferred embodiment of the present application, and as shown in fig. 9, the method includes:

s901: acquiring a first feature layer based on the image;

s902: obtaining a second characteristic layer through a neural network based on a residual error structure according to the first characteristic layer;

s903: obtaining an edge refining feature layer through a third neural network according to the second feature layer;

s904: and carrying out convolution operation on the edge thinning characteristic layer to obtain a segmentation result of the image.

S901 to S904 in the embodiments of the present application are specifically explained in the embodiments above, and for details not disclosed in the method shown in fig. 9 of the present application, refer to the embodiments above of the present application.

Fig. 10 is a flowchart illustrating an image segmentation method according to another embodiment of the present application. The embodiment shown in fig. 10 is a preferred embodiment of the present application, and as shown in fig. 10, the method includes:

s1001: acquiring a first feature layer based on the image;

s1002: obtaining a second characteristic layer through a neural network based on a residual error structure according to the first characteristic layer;

s1003: obtaining a fifth characteristic layer through coding operation according to the first characteristic layer;

s1004: according to the fifth characteristic layer, a sixth characteristic layer is obtained through the neural network based on the residual error structure;

s1005: obtaining a sixth characteristic layer after edge refinement through a third neural network according to the sixth characteristic layer;

s1006: obtaining a seventh characteristic layer through decoding operation according to the sixth characteristic layer after edge thinning;

s1007: obtaining an integrated characteristic layer through splicing operation according to the seventh characteristic layer and the second characteristic layer;

s1008: obtaining an edge-refined integrated feature layer through a third neural network according to the integrated feature layer;

s1009: and performing convolution operation on the integrated feature layer with the edge refined to obtain a segmentation result of the image.

S1001 to S1009 in the embodiment of the present application are specifically explained in the above embodiments, and for details that are not disclosed in the method shown in fig. 10 of the present application, refer to the above embodiments of the present application.

Fig. 11 is a flowchart illustrating a method for image segmentation according to still another embodiment of the present application. The embodiment shown in fig. 11 is a preferred embodiment of the present application, and as shown in fig. 11, the method includes:

s1101: acquiring a first feature layer based on the image;

s1102: obtaining a second characteristic layer through a neural network based on a residual error structure according to the first characteristic layer;

s1103: obtaining a fifth characteristic layer through coding operation according to the first characteristic layer;

s1104: according to the fifth characteristic layer, a sixth characteristic layer is obtained through the neural network based on the residual error structure;

s1105: obtaining a sixth characteristic layer after edge refinement through a third neural network according to the sixth characteristic layer;

s1106: according to the sixth characteristic layer after edge thinning and the second characteristic layer, obtaining a fused characteristic layer through characteristic fusion;

s1107: according to the sixth characteristic layer after edge thinning, obtaining an eighth characteristic layer through an up-sampling operation;

s1108: obtaining an integrated feature layer through splicing operation according to the eighth feature layer and the fused feature layer;

s1109: obtaining an edge-refined integrated feature layer through a third neural network according to the integrated feature layer;

s1110: and performing convolution operation on the integrated feature layer with the edge refined to obtain a segmentation result of the image.

S1101 to S1110 in the embodiments of the present application are specifically explained in the above embodiments, and for details not disclosed in the method shown in fig. 11 of the present application, refer to the above embodiments of the present application.

Fig. 12a to 12c are schematic diagrams of a segmentation process of a method for image segmentation according to an embodiment of the present application. As shown in fig. 12a, a is a first feature layer, B is a second feature layer, C is a ninth feature layer, D is a fifth feature layer, E is a sixth feature layer, F is a sixth feature layer after edge refinement, G is a fused feature layer, H is an eighth feature layer, J is a seventh feature layer, and I is an integrated feature layer.

In an embodiment, after the ninth feature layer C is subjected to 2-fold down-sampling, the fifth feature layer D is output; the fifth characteristic layer D passes through a dense convolution network and then outputs a ninth characteristic layer C; the ninth characteristic layer C outputs a fifth characteristic layer D after 4 times of down sampling; the fifth characteristic layer D passes through a dense convolution network and then outputs a ninth characteristic layer C; and the ninth characteristic layer C is subjected to 8-time down-sampling, and then the fifth characteristic layer D is output.

In another embodiment, after each fifth feature layer D passes through the neural network based on the residual structure, a sixth feature layer E is output; and after each sixth feature layer E passes through a third neural network for edge refinement, outputting a sixth feature layer F after edge refinement.

In another embodiment, after the sixth feature layer F after edge refinement is up-sampled by 2 times, feature fusion is performed with the sixth feature layer F after another edge refinement, and a fusion feature layer G is output; after 2 times of upsampling, the fusion feature layer G is subjected to feature fusion with the sixth feature layer F with the thinned edge, and the other fusion feature layer G is output; and after 2 times of upsampling, the fusion characteristic layer G is subjected to characteristic fusion with the second characteristic layer B, and the final fusion characteristic layer G is output.

In another embodiment, after 8 times of upsampling is performed on the sixth feature layer F after edge refinement, an eighth feature layer H is output; outputting another eighth characteristic layer H after one fused characteristic layer G is subjected to up-sampling by 4 times; and outputting a second eighth feature layer H after the other fused feature layer G is subjected to 2 times of upsampling.

In another embodiment, the final fused feature layer G and the three eighth feature layers H combine to form a seventh feature layer J.

In another embodiment, the eighth feature layer H and the fusion feature layer G are subjected to a splicing operation to obtain an integrated feature layer I, and the integrated feature layer I is subjected to a convolution operation to obtain a segmentation result of the image.

As shown in fig. 12B, a is the first feature layer, B is the second feature layer, C is the ninth feature layer, D is the fifth feature layer, E is the sixth feature layer, F is the sixth feature layer after edge refinement, I is the feature layer after integration, and J is the seventh feature layer. Only the differences from the embodiment shown in fig. 12a will be described below, and the same parts will not be described again.

In one embodiment, after 8 times of upsampling, a sixth feature layer F with thinned edges outputs a seventh feature layer J; after the sixth characteristic layer F with the thinned other edge is subjected to up-sampling by 4 times, outputting another seventh characteristic layer J; and outputting a seventh feature layer J after 2 times of upsampling of the sixth feature layer F after the edge refinement.

In another embodiment, the plurality of seventh feature layers J and the second feature layer B are subjected to a splicing operation to obtain an integrated feature layer I, and the integrated feature layer I is subjected to a convolution operation to obtain a segmentation result of the image.

As shown in fig. 12c, a is the first feature layer, B is the second feature layer, and L is the edge refining feature layer.

In an embodiment, the first feature layer A outputs a second feature layer B after passing through a neural network based on a residual structure, the second feature layer B outputs an edge refinement feature layer L after passing through a third neural network for edge refinement, and the edge refinement feature layer L is subjected to convolution operation to obtain a segmentation result of the image.

Exemplary devices

The embodiment of the device can be used for executing the embodiment of the method. For details which are not disclosed in the embodiments of the apparatus of the present application, reference is made to the embodiments of the method of the present application.

Fig. 13 is a block diagram illustrating an apparatus for image segmentation according to an embodiment of the present application. As shown in fig. 13, the image segmentation apparatus 1300 includes:

an acquisition module 1310 configured to acquire a first feature layer based on the image;

an information fusion module 1320, configured to obtain a second feature layer through a neural network based on a residual structure according to the first feature layer;

a segmentation module 1330 configured to obtain a segmentation result of the image according to the second feature layer.

In one embodiment, the apparatus 1300 for image segmentation further comprises: and a module for performing each step in the method for image segmentation mentioned in the above embodiments.

Exemplary electronic device

Next, an electronic apparatus according to an embodiment of the present application is described with reference to fig. 14. FIG. 14 illustrates a block diagram of an electronic device in accordance with an embodiment of the present application.

As shown in fig. 14, the electronic device 1400 includes one or more processors 1410 and memory 1420.

The processor 1410 may be a Central Processing Unit (CPU) or other form of processing unit having data processing capabilities and/or instruction execution capabilities, and may control other components in the electronic device 1400 to perform desired functions.

Memory 1420 may include one or more computer program products that may include various forms of computer-readable storage media, such as volatile memory and/or non-volatile memory. The volatile memory may include, for example, Random Access Memory (RAM), cache memory (cache), and/or the like. The non-volatile memory may include, for example, Read Only Memory (ROM), hard disk, flash memory, etc. One or more computer program instructions may be stored on the computer-readable storage medium and executed by processor 1410 to implement the methods of image segmentation of the various embodiments of the present application described above and/or other desired functions. Various contents such as an input signal, a signal component, a noise component, etc. may also be stored in the computer-readable storage medium.

In one example, the electronic device 1400 may further include: an input device 1430 and an output device 1440, which are interconnected by a bus system and/or other form of connection mechanism (not shown).

For example, the input 1430 may be a microphone or microphone array as described above for capturing an input signal of a sound source. The input 1430 may be a communication network connector when the electronic device is a stand-alone device.

The input devices 1430 may also include, for example, a keyboard, a mouse, and the like.

The output device 1440 may output various information including the identified symptom category information to the outside. The output devices 1440 may include, for example, a display, speakers, a printer, and a communication network and remote output devices connected thereto, among others.

Of course, for simplicity, only some of the components of the electronic device 1400 relevant to the present application are shown in fig. 14, omitting components such as buses, input/output interfaces, and the like. In addition, electronic device 1400 may include any other suitable components, depending on the particular application.

Exemplary computer program product and computer-readable storage Medium

In addition to the above-described methods and apparatus, embodiments of the present application may also be a computer program product comprising computer program instructions that, when executed by a processor, cause the processor to perform the steps in the method of image segmentation according to various embodiments of the present application described in the "exemplary methods" section of this specification, supra.

The computer program product may be written with program code for performing the operations of embodiments of the present application in any combination of one or more programming languages, including an object oriented programming language such as Java, C + + or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code may execute entirely on the user's computing device, partly on the user's device, as a stand-alone software package, partly on the user's computing device and partly on a remote computing device, or entirely on the remote computing device or server.

Furthermore, embodiments of the present application may also be a computer-readable storage medium having stored thereon computer program instructions that, when executed by a processor, cause the processor to perform the steps in the method of image segmentation according to various embodiments of the present application described in the "exemplary methods" section above in this specification.

The computer-readable storage medium may take any combination of one or more readable media. The readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may include, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or a combination of any of the foregoing. More specific examples (a non-exhaustive list) of the readable storage medium include: an electrical connection having one or more wires, a portable disk, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

The foregoing describes the general principles of the present application in conjunction with specific embodiments, however, it is noted that the advantages, effects, etc. mentioned in the present application are merely examples and are not limiting, and they should not be considered essential to the various embodiments of the present application.

The foregoing description has been presented for purposes of illustration and description. Furthermore, the description is not intended to limit embodiments of the application to the form disclosed herein. While a number of example aspects and embodiments have been discussed above, those of skill in the art will recognize certain variations, modifications, alterations, additions and sub-combinations thereof.

Claims (15)

1. A method of image segmentation, comprising:

acquiring a first feature layer based on the image;

obtaining a second characteristic layer through a neural network based on a residual error structure according to the first characteristic layer;

and acquiring a segmentation result of the image according to the second characteristic layer.

2. The method of claim 1, wherein obtaining a second feature layer from the first feature layer through a neural network based on a residual structure comprises:

obtaining a third feature layer through a first neural network according to the first feature layer;

obtaining a fourth feature layer through a second neural network according to the first feature layer;

and acquiring the second characteristic layer according to the first characteristic layer, the third characteristic layer and the fourth characteristic layer.

3. The method of claim 2, wherein obtaining the second feature layer from the first feature layer, the third feature layer, and the fourth feature layer comprises:

and performing feature fusion on the first feature layer, the third feature layer and the fourth feature layer to obtain the second feature layer.

4. The method of claim 1, wherein said obtaining a first feature layer based on the image comprises:

and obtaining the first characteristic layer through a dense convolution network according to the image.

5. The method according to any one of claims 1 to 4, wherein the obtaining a segmentation result of the image according to the second feature layer comprises:

and performing convolution operation on the second characteristic layer to obtain a segmentation result of the image.

6. The method according to any one of claims 1 to 4, wherein the obtaining a segmentation result of the image according to the second feature layer comprises:

obtaining an edge refining feature layer through a third neural network according to the second feature layer;

and carrying out convolution operation on the edge thinning characteristic layer to obtain a segmentation result of the image.

7. The method of any of claims 1 to 4, further comprising:

obtaining a fifth characteristic layer through coding operation according to the first characteristic layer;

according to the fifth characteristic layer, a sixth characteristic layer is obtained through the neural network based on the residual error structure;

and obtaining a seventh characteristic layer through decoding operation according to the sixth characteristic layer.

8. The method of claim 7, wherein the obtaining a fifth feature layer from the first feature layer through an encoding operation comprises:

obtaining a ninth feature layer through a dense convolution network according to the first feature layer;

and performing convolution operation and downsampling operation on the ninth feature layer to obtain the fifth feature layer.

9. The method of claim 7, wherein before obtaining a seventh feature layer from the sixth feature layer via a decoding operation, the method further comprises:

obtaining a sixth characteristic layer after edge refinement through a third neural network according to the sixth characteristic layer,

obtaining a seventh feature layer through a decoding operation according to the sixth feature layer, including:

and obtaining the seventh characteristic layer through the decoding operation according to the sixth characteristic layer after the edge refinement.

10. The method of claim 7, wherein obtaining a seventh feature layer from the sixth feature layer through a decoding operation comprises:

obtaining the seventh characteristic layer through an up-sampling operation according to the sixth characteristic layer,

wherein, the obtaining the segmentation result of the image according to the second feature layer includes:

obtaining an integrated characteristic layer through splicing operation according to the seventh characteristic layer and the second characteristic layer;

and performing convolution operation on the integrated characteristic layer to obtain a segmentation result of the image.

11. The method of claim 7, wherein obtaining a seventh feature layer from the sixth feature layer through a decoding operation comprises:

according to the sixth characteristic layer and the second characteristic layer, obtaining a fused characteristic layer through characteristic fusion;

according to the sixth characteristic layer, obtaining an eighth characteristic layer through an up-sampling operation;

determining the seventh feature layer based on the fused feature layer and the eighth feature layer,

wherein, the obtaining the segmentation result of the image according to the second feature layer includes:

obtaining an integrated feature layer through splicing operation according to the eighth feature layer and the fused feature layer;

and performing convolution operation on the integrated characteristic layer to obtain a segmentation result of the image.

12. The method according to any one of claims 1 to 4, wherein the image is a medical image of the brain, and the segmentation result of the image is a multi-class segmentation result of the background, the left ventricle, and the right ventricle.

13. An apparatus for image segmentation, comprising:

an acquisition module configured to acquire a first feature layer based on the image;

the information fusion module is configured to obtain a second characteristic layer through a neural network based on a residual error structure according to the first characteristic layer;

and the segmentation module is configured to acquire a segmentation result of the image according to the second feature layer.

14. An electronic device, comprising:

a processor for performing the method of image segmentation of any one of the preceding claims 1 to 12;

a memory for storing the processor-executable instructions.

15. A computer-readable storage medium, storing a computer program for performing the method of image segmentation of any of the preceding claims 1 to 12.

Images