CN109544560B

CN109544560B - Image processing method and device, electronic equipment and storage medium

Info

Publication number: CN109544560B
Application number: CN201811287035.2A
Authority: CN
Inventors: 宋涛; 刘蓬博
Original assignee: Shanghai Sensetime Intelligent Technology Co Ltd
Current assignee: Shanghai Sensetime Intelligent Technology Co Ltd
Priority date: 2018-10-31
Filing date: 2018-10-31
Publication date: 2021-04-27
Anticipated expiration: 2038-10-31
Also published as: CN109544560A

Abstract

The present disclosure relates to an image processing method and apparatus, an electronic device, and a storage medium, the method including: preprocessing a plurality of pixel points of an image to be processed to obtain a first image; inputting the first image into a segmentation network for processing to obtain a first probability that each pixel point of the first image belongs to a first target region, a second probability that each pixel point belongs to a second target region and a third probability that each pixel point belongs to a background region; and determining a first target area, a second target area and a background area according to the first probability, the second probability and the third probability. According to the image processing method, the first target area, the second target area and the background area in the first image can be determined according to the first probability, the second probability and the third probability, the types of recognizable areas are increased, different types of areas can be accurately distinguished, and the distinguishing capability of the target areas is improved.

Description

Image processing method and device, electronic equipment and storage medium

Technical Field

The present disclosure relates to the field of computer technologies, and in particular, to an image processing method and apparatus, an electronic device, and a storage medium.

Background

And acquiring a target area in the image, and further analyzing the information in the target area. For example, in medical imaging such as Diffusion Weighted Imaging (DWI), it is necessary to determine a lesion region such as stroke or cerebral infarction, and analyze the lesion region to determine the degree of lesion, so as to provide a basis for treatment. For example, the lesion type of a lesion region may be divided into a core region of the lesion and a penumbra region due to ischemia, and the determination of the core region and the penumbra region has an important role in diagnosis and treatment. In the related art, it is difficult to accurately distinguish different types of regions.

Disclosure of Invention

The disclosure provides an image processing method and device, an electronic device and a storage medium.

According to an aspect of the present disclosure, there is provided an image processing method including:

preprocessing a plurality of pixel points of an image to be processed to obtain a first image;

inputting the first image into a segmentation network for processing to obtain a first probability that each pixel point of the first image belongs to a first target region, a second probability that each pixel point belongs to a second target region and a third probability that each pixel point belongs to a background region;

and determining a first target area, a second target area and a background area in the first image according to the first probability, the second probability and the third probability of each pixel point of the first image.

According to the image processing method of the embodiment of the disclosure, the first probability that each pixel point of the first image obtained after preprocessing belongs to the first target area, the second probability that each pixel point belongs to the second target area and the third probability that each pixel point belongs to the background area can be obtained, the first target area, the second target area and the background area in the first image are determined according to the first probability, the second probability and the third probability, the category of the recognizable area is increased, the areas of different types can be accurately distinguished, and the distinguishing capability of the target areas is improved.

In one possible implementation, the segmentation network includes a probability-determining subnetwork that includes a hole convolution layer.

In a possible implementation manner, the void convolution layer is configured to determine a first probability, a second probability, and a third probability of each pixel point of the first image.

By the method, the accuracy loss caused by loss can be avoided under the condition that the receptive field of the input image is enlarged by the void convolution layer, and the accuracy of probability determination is improved.

In one possible implementation, the segmentation network includes a down-sampling sub-network including a self-adaptive normalization layer for performing self-adaptive normalization processing on an image input to the self-adaptive normalization layer.

By the mode, the self-adaptive normalization layer can perform self-adaptive normalization processing on the input image, the generalization capability of the segmentation network can be improved, and the false recognition rate of the segmentation network is reduced.

In one possible implementation, the segmentation network includes a feature enhancement sub-network, and the feature enhancement sub-network is configured to perform feature enhancement processing on an image input to the feature enhancement sub-network according to a plurality of pixel points of the image input to the feature enhancement sub-network.

By the method, the characteristic of each pixel point of the down-sampling image can be enhanced, the discrimination between the pixel points in different areas is improved, the relation between the pixel points in the same area is increased, and the identification capability and the extraction capability of a segmentation network to a target area are improved.

In one possible implementation, the segmentation network comprises a down-sampling sub-network, a feature enhancement sub-network and a probability determination sub-network,

inputting the first image into a segmentation network for processing to obtain a first probability that each pixel point of the first image belongs to a first target region, a second probability that each pixel point belongs to a second target region, and a third probability that each pixel point belongs to a background region, wherein the method comprises the following steps:

inputting the first image into a down-sampling sub-network for down-sampling processing to obtain a down-sampled image;

inputting the downsampled image into a feature enhancement sub-network for feature enhancement processing to obtain a second image;

and inputting the second image into the probability determination sub-network for processing, and determining the first probability, the second probability and the third probability of each pixel point of the first image.

In one possible implementation manner, inputting the downsampled image into a feature enhancement sub-network for feature enhancement processing to obtain a second image, includes:

according to a plurality of pixel points of a down-sampling image, performing feature enhancement processing on a first pixel point of the down-sampling image to obtain a second pixel point with enhanced features, wherein the first pixel point is any pixel point in the down-sampling image;

and obtaining the second image according to the plurality of second pixel points.

In one possible implementation, the sub-sampling sub-network comprises a first down-sampling layer comprising at least a convolutional layer, a self-adaptive normalization layer, and a second down-sampling layer comprising at least an active layer.

In one possible implementation, inputting the first image into a down-sampling sub-network for down-sampling processing to obtain a down-sampled image, includes:

inputting the first image into a first down-sampling layer for processing to obtain a third image;

inputting the third image into a self-adaptive normalization layer for processing to obtain a fourth image;

and inputting the fourth image into a second down-sampling layer for processing to obtain the down-sampling image.

In a possible implementation manner, inputting the second image into the probability determination sub-network for processing, and determining a first probability, a second probability, and a third probability of each pixel of the first image includes:

inputting the second image into the probability determination sub-network, and determining a fourth probability that each pixel point of the second image belongs to the first target area, a fifth probability that each pixel point belongs to the second target area and a sixth probability that each pixel point belongs to the background area;

and determining the first probability, the second probability and the third probability of each pixel point of the first image according to the fourth probability, the fifth probability and the sixth probability of each pixel point of the second image and the down-sampling multiple of down-sampling processing.

In one possible implementation, the method further includes:

and determining a network for processing the first image input position to obtain the position relation among all pixel points of the first image.

In a possible implementation manner, determining a first target region, a second target region, and a background region in a first image according to a first probability, a second probability, and a third probability of each pixel point of the first image includes:

determining the category of a third pixel point according to the first probability, the second probability and the third probability of the third pixel point, wherein the third pixel point is any pixel point of the first image;

and respectively determining a first target area, a second target area and a background area according to the category of each pixel point of the first image and the position relation among the pixel points.

By the method, the positions of the first target area, the second target area and the background area in the first image can be determined according to the position relation, so that the precision loss in the processes of up-sampling or interpolation and the like of each pixel point is reduced, and the precision of determining each area is improved.

In a possible implementation manner, preprocessing a plurality of pixel points of an image to be processed to obtain a first image includes:

and carrying out batch normalization processing on a plurality of pixel points of the image to be processed to obtain the first image.

By the method, the complexity of the image to be processed can be reduced, and the processing efficiency is improved.

In one possible implementation, the first target region is a lesion core region, and the second target region is a penumbra region.

In one possible implementation, the method further includes:

the segmentation network is trained by an image set comprising a plurality of preprocessed sample images.

In one possible implementation, training the segmentation network by an image set including a plurality of preprocessed sample images includes:

inputting a sample image to the segmentation network, and obtaining a first probability that each pixel point of the sample image belongs to a first sample target area, a second probability that each pixel point of the sample image belongs to a second sample target area and a third probability that each pixel point of the sample image belongs to a sample background area;

determining a first sample target area, a second sample target area and a sample background area in the sample image according to a first probability that each pixel point belongs to a first sample target area, a second probability that each pixel point belongs to a second sample target area and a third probability that each pixel point belongs to a sample background area;

determining the network loss of the segmentation network according to a first sample target area, a second sample target area and a sample background area in the sample image, and the first target area, the second target area and the background area in the sample image;

adjusting network parameters of the segmented network according to the network loss;

and when the segmentation network meets the training condition, obtaining the trained segmentation network.

In one possible implementation, determining a network loss of the segmented network according to the first sample target area, the second sample target area, and the sample background area in the sample image, and the first target area, the second target area, and the background area in the sample image includes:

determining cross entropy loss of each pixel point in the sample image according to a first sample target area, a second sample target area and a sample background area in the sample image, and the first target area, the second target area and the background area in the sample image;

determining a weight coefficient matrix according to the symbol distance functions of the sample image target area, the second sample target area and the sample background area;

determining the coincidence degree of the first sample target area and the second sample target area according to the first probability that each pixel point of the sample image belongs to the target area and the second probability that each pixel point of the sample image belongs to the second sample target area;

and determining the network loss of the segmented network according to the cross entropy loss, the weight coefficient matrix and the coincidence degree.

According to another aspect of the present disclosure, there is provided an image processing apparatus including:

the preprocessing module is used for preprocessing a plurality of pixel points of an image to be processed to obtain a first image;

a probability obtaining module, configured to input the first image into a segmentation network for processing, so as to obtain a first probability that each pixel of the first image belongs to a first target region, a second probability that each pixel belongs to a second target region, and a third probability that each pixel belongs to a background region;

and the region determining module is used for determining a first target region, a second target region and a background region in the first image according to the first probability, the second probability and the third probability of each pixel point of the first image.

wherein the probability obtaining module is further configured to:

In one possible implementation, the probability obtaining module is further configured to:

In one possible implementation, the apparatus further includes:

and the position relation obtaining module is used for determining the position of the first image in the position determining network to process so as to obtain the position relation among all the pixel points of the first image.

In one possible implementation, the region determination module is further configured to:

In one possible implementation, the preprocessing module is further configured to:

In one possible implementation, the apparatus further includes:

a training module to train the segmentation network through an image set comprising a plurality of preprocessed sample images.

In one possible implementation, the training module is further configured to:

According to another aspect of the present disclosure, there is provided an electronic device including:

a processor;

a memory for storing processor-executable instructions;

wherein the processor is configured to: the above-described image processing method is performed.

According to another aspect of the present disclosure, there is provided a computer readable storage medium having stored thereon computer program instructions which, when executed by a processor, implement the above-described image processing method.

According to another aspect of the present disclosure, there is provided a computer program product comprising computer program instructions which, when executed by a processor, implement the above-described image processing method.

It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the disclosure.

Other features and aspects of the present disclosure will become apparent from the following detailed description of exemplary embodiments, which proceeds with reference to the accompanying drawings.

Drawings

The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the principles of the disclosure.

FIG. 1 shows a flow diagram of an image processing method according to an embodiment of the present disclosure;

FIG. 2 shows a flow diagram of an image processing method according to an embodiment of the present disclosure;

FIG. 3 shows a flow diagram of an image processing method according to an embodiment of the present disclosure;

FIG. 4 shows a schematic diagram of an application of an image processing method according to an embodiment of the present disclosure;

fig. 5 shows a block diagram of an image processing apparatus according to an embodiment of the present disclosure;

fig. 6 shows a block diagram of an image processing apparatus according to an embodiment of the present disclosure;

FIG. 7 shows a block diagram of an electronic device in accordance with an embodiment of the disclosure;

fig. 8 illustrates a block diagram of an electronic device in accordance with an embodiment of the disclosure.

Detailed Description

Various exemplary embodiments, features and aspects of the present disclosure will be described in detail below with reference to the accompanying drawings. In the drawings, like reference numbers can indicate functionally identical or similar elements. While the various aspects of the embodiments are presented in drawings, the drawings are not necessarily drawn to scale unless specifically indicated.

The word "exemplary" is used exclusively herein to mean "serving as an example, embodiment, or illustration. Any embodiment described herein as "exemplary" is not necessarily to be construed as preferred or advantageous over other embodiments.

The term "and/or" herein is merely an association describing an associated object, meaning that three relationships may exist, e.g., a and/or B, may mean: a exists alone, A and B exist simultaneously, and B exists alone. In addition, the term "at least one" herein means any one of a plurality or any combination of at least two of a plurality, for example, including at least one of A, B, C, and may mean including any one or more elements selected from the group consisting of A, B and C.

Furthermore, in the following detailed description, numerous specific details are set forth in order to provide a better understanding of the present disclosure. It will be understood by those skilled in the art that the present disclosure may be practiced without some of these specific details. In some instances, methods, means, elements and circuits that are well known to those skilled in the art have not been described in detail so as not to obscure the present disclosure.

Fig. 1 shows a flowchart of an image processing method according to an embodiment of the present disclosure, as shown in fig. 1, the method comprising:

in step S11, preprocessing a plurality of pixel points of the image to be processed to obtain a first image;

in step S12, the first image is input into a segmentation network for processing, and a first probability that each pixel of the first image belongs to a first target region, a second probability that each pixel belongs to a second target region, and a third probability that each pixel belongs to a background region are obtained;

in step S13, a first target region, a second target region, and a background region in the first image are determined according to the first probability, the second probability, and the third probability of each pixel point of the first image.

In one possible implementation, the image processing method may be performed by a terminal device, which may be a User Equipment (UE), a mobile device, a User terminal, a cellular phone, a cordless phone, a Personal Digital Assistant (PDA), a handheld device, a computing device, a vehicle-mounted device, a wearable device, or the like, and the method may be implemented by a processor calling computer-readable instructions stored in a memory. Alternatively, the method may be performed by a server by acquiring an image to be processed by a terminal device or an image capture device (e.g., a camera, etc.) and transmitting the image to be processed to the server.

In a possible implementation manner, in step S11, the image to be processed may include medical imaging such as diffusion weighted imaging of the brain, the first target region is a lesion core region, and the second target region is a penumbra region. The image to be processed may also include other images, and the category of the image to be processed is not limited by the present disclosure. The image to be processed may be preprocessed, and the preprocessing may include flipping, symmetry, clipping, and warping the image to be processed. The present disclosure does not limit the category of pre-processing.

In a possible implementation manner, the preprocessing may be performed on a plurality of pixel points of the image to be processed to obtain the first image, and the preprocessing may include: and carrying out Batch Normalization (Batch Normalization) processing on a plurality of pixel points of the image to be processed to obtain the first image.

In an example, the image to be processed may be input into a batch normalization network for batch normalization processing, where the batch normalization network may include a neural network (i.e., a neural network without affine transformation) such as a BP neural network, a recurrent neural network, or a convolutional neural network without parameters, and the batch normalization network may determine parameters such as standard deviation and mean of a plurality of pixels of the image to be processed, for example, parameters such as standard deviation and mean of at least one of gray values, brightness values, chromatic values, or RGB values of pixel points of the image to be processed are obtained, so as to perform batch normalization processing. When the batch normalization network is used for carrying out batch normalization on the image to be processed, affine parameters can not be preset, so that the batch normalization network iterates at least one of gray values, brightness values, chromatic values or RGB values of a plurality of pixels, the parameters such as the standard deviation and the mean value are continuously updated, and after the iteration processing is finished, the parameters such as the standard deviation and the mean value can be obtained, so that the batch normalization processing is carried out on the image to be processed, and the first image is obtained. In an example, the batch normalization network may be a sub-network of the segmentation network.

In one possible implementation manner, in step S12, the preprocessed first image may be input to a segmentation network for processing, where the segmentation network may be a neural network such as a BP neural network, a recurrent neural network, or a convolutional neural network.

In a possible implementation manner, the segmentation network may be a convolutional neural network, and the segmentation network may include a downsampling subnetwork, a feature enhancement subnetwork, and a probability determination subnetwork, where the first image is input to the segmentation network and processed to obtain a first probability that each pixel of the first image belongs to a first target region, a second probability that each pixel of the first image belongs to a second target region, and a third probability that each pixel of the first image belongs to a background region, and the method includes: inputting the first image into a down-sampling sub-network for down-sampling processing to obtain a down-sampled image; inputting the downsampled image into a feature enhancement sub-network for feature enhancement processing to obtain a second image; and inputting the second image into the probability determination sub-network for processing, and determining the first probability, the second probability and the third probability of each pixel point of the first image.

In an example, the segmentation network includes a down-sampling sub-network including a self-adaptive Normalization (Switchable Normalization) layer for performing self-adaptive Normalization processing on an image input to the self-adaptive Normalization layer.

In one possible implementation, the downsampling subnetwork of the segmentation network comprises a first downsampling layer comprising at least a convolution layer, a normalization layer and a second downsampling layer comprising at least an activation layer. The method for inputting the first image into a down-sampling sub-network to perform down-sampling processing to obtain a down-sampled image comprises the following steps: inputting the first image into a first down-sampling layer for processing to obtain a third image; inputting the third image into a self-adaptive normalization layer for processing to obtain a fourth image; and inputting the fourth image into a second down-sampling layer for processing to obtain the down-sampling image.

In one possible implementation, the first downsampling layer at least includes a convolution layer, and the convolution processing may be performed on each pixel point of the first image, for example, the first image may be downsampled by using a shared weight of a convolution kernel. In an example, the first downsampling layer may further include one or more of network hierarchies such as an activation layer and a pooling layer, and the present disclosure does not limit the kinds and the number of the network hierarchies included in the first downsampling layer. After the first image is processed through all network levels of the first downsampling layer, one or more third images may be obtained.

In an example, the first image is an image having a resolution of 256 × 256, and a plurality of third images having a resolution of 128 × 128 are obtained by performing downsampling processing such as convolution processing on the first image by the first downsampling layer. And the third image can be input into the image of the self-adaptive normalization layer for self-adaptive normalization processing.

In one possible implementation, the normalization layer may perform adaptive normalization on the third image. In an example, the third image (i.e. the image input into the self-adaptive normalization layer) is subjected to self-adaptive normalization processing by the self-adaptive normalization layer to obtain a fourth image, and the self-adaptive normalization processing is performed by using the self-adaptive normalization layer, so that the self-adaptive normalization processing process is not influenced by Batch size (Batch size), and the generalization capability of the segmentation network is remarkably improved. And after the self-adaptive normalization processing is carried out on the first image, a fourth image can be obtained.

In a possible implementation manner, the second downsampling layer at least includes an activation layer, and the fourth image may be activated, for example, each pixel point of the fourth image is activated by an activation function (e.g., Sigmoid function, Tanh function, ReLU function, etc.) of the activation layer. In an example, the second down-sampling layer may further include one or more of a convolutional layer and a pooling layer, and the present disclosure does not limit the kind and number of network layers included in the second down-sampling layer. After the fourth image is processed through all network levels of the second downsampling layer, the downsampled image may be obtained. In an example, the self-adaptive normalization layer may be located between a convolutional layer (i.e., a convolutional layer of a first downsampling subnetwork) and an active layer (i.e., an active layer of a second downsampling subnetwork).

In this way, the down-sampling sub-network can comprise a self-adaptive normalization layer, and the self-adaptive normalization processing can be performed on the input third image, so that the generalization capability of the segmentation network can be improved, and the false identification rate of the segmentation network can be reduced.

In one possible implementation, the segmentation network includes a feature enhancement sub-network configured to perform feature enhancement processing on an image (e.g., a downsampled image) input to the feature enhancement sub-network based on a plurality of pixel points of the image input to the feature enhancement sub-network. Inputting the downsampled image into a feature enhancement sub-network for feature enhancement processing to obtain a second image, wherein the feature enhancement sub-network comprises: according to a plurality of pixel points of a down-sampling image, performing feature enhancement processing on a first pixel point of the down-sampling image to obtain a second pixel point with enhanced features, wherein the first pixel point is any pixel point in the down-sampling image; and obtaining the second image according to the plurality of second pixel points.

In a possible implementation manner, the first pixel point is any pixel point in the down-sampled image, and feature enhancement processing may be performed on the first pixel point by using all pixel points of the down-sampled image to obtain a second pixel point.

In an example, in the split network, a feature space dimension attention mechanism (self-attention) may be included after the sub-network is downsampled, e.g., a feature enhancement sub-network is included after the sub-network is downsampled. In an example, the resolution of the first image is 256 × 256, the first image may be re-calibrated by using a channel attention mechanism of a down-sampling sub-network, that is, the first image is observed through different observation dimensions during the down-sampling process, and observation images (i.e., down-sampling images) of multiple dimensions are obtained, for example, the first image may be convolved by using convolution kernels of different parameters (that is, different observation dimensions are found through convolution kernels of different parameters), and after the processing of network levels such as an adaptive normalization layer, an activation layer, and a pooling layer, a plurality of down-sampling images (i.e., observation images of multiple dimensions) may be obtained. In an example, the downsampling subnetwork may downsample the first image four times to obtain a plurality of downsampled images at a resolution of 64 x 64, each downsampled image may represent a different observation dimension, for each downsampled image, the attention mechanism (i.e., feature enhancement sub-network) of the feature space dimension may be used to perform feature enhancement processing on each pixel point of itself, e.g., for a target down-sampling image (i.e., an image of an input feature enhancement sub-network), the feature enhancement sub-network may perform feature enhancement processing on a first pixel of the target down-sampling image through all pixels of the target down-sampling image, obtain a second pixel of the target down-sampling image, the target down-sampling image is any down-sampling image, and the first pixel point of the target down-sampling image is any pixel point of the target down-sampling image.

In an example, a target downsampled image may be input into the feature enhancement sub-network, the feature enhancement sub-network may copy the target downsampled image, and map a copy of the copied target downsampled image to a preset feature space, and each pixel point mapped to the copy in the feature space may be used to perform a feature enhancement process on a first pixel point, for example, the feature enhancement process may be performed by the following formula (1), and obtain the second pixel point:

wherein x is_i(i is more than or equal to 1 and less than or equal to N) as a first pixel of the target downsampled imagePoint, y_iIs a second pixel point, x'_j(j is more than or equal to 1 and less than or equal to N) is any pixel point of the copy in the feature space, N is the number of pixel points of the target down-sampling image, C (x) is a normalization factor, and g (x'_j) Is a predetermined unary function, f (x)_i，x’_j) As a predetermined binary function, e.g.

Etc. this disclosure deals with g (x'_j) And f (x)_i，x’_j) The specific form of (a) is not limiting.

In a possible implementation manner, feature enhancement processing may be performed on each pixel point of the target downsampled image to obtain a second pixel point corresponding to each pixel point, that is, the second image may be obtained, for example, the feature enhancement processing is performed on each pixel point of the target downsampled image according to formula (1), that is, the second image may be obtained.

In one possible implementation, the segmentation network includes a probability-determining subnetwork that includes a hole convolution layer. The void convolutional layer is used for determining a first probability, a second probability and a third probability of each pixel point of the first image. Inputting the second image into the probability determination sub-network for processing, and determining a first probability, a second probability and a third probability of each pixel point of the first image, including: inputting the second image into the probability determination sub-network, and determining a fourth probability that each pixel point of the second image belongs to the first target area, a fifth probability that each pixel point belongs to the second target area and a sixth probability that each pixel point belongs to the background area; and determining the first probability, the second probability and the third probability of each pixel point of the first image according to the fourth probability, the fifth probability and the sixth probability of each pixel point of the second image and the down-sampling multiple of down-sampling processing.

In one possible implementation, the second image may be input into the probability determining subnetwork. In an example, the probability determination subnetwork includes a hole convolution layer, the probability determination can be performed on the pixels of the plurality of downsampled images one by one, the fourth probability that each pixel of the second image belongs to the first target region, the fifth probability that each pixel of the second image belongs to the second target region, and the sixth probability that each pixel of the second image belongs to the background region can be determined, by using the hole convolution layer, the receptive field of the segmentation network on the second image can be enlarged, the fourth probability, the fifth probability, and the sixth probability that each pixel of the plurality of second images is recognized are identified, the recognition capability of the segmentation network on the pixels of the second image with multiple resolutions is improved, and no precision loss is caused under the condition that the receptive field is enlarged. In an example, the sixth probability belonging to the background region may include a plurality of output values, for example, 3 output values (i.e., the number of output channels is 5, where 3 output values are the sixth probability belonging to the background region, 1 output value is the fourth probability belonging to the first target region, and 1 output value is the fifth probability belonging to the second target region), and the sixth probability may be determined by taking a maximum value (max out), i.e., taking a maximum value of the output values of the 3 sixth probabilities as the sixth probability, thereby determining the fourth, fifth, and sixth probabilities. In an example, the fourth probability, the fifth probability, and the sixth probability may be further normalized by a softmax function such that a sum of the fourth probability, the fifth probability, and the sixth probability is 1.

In a possible implementation manner, the first probability, the second probability, and the third probability of each pixel point of the first image may be determined according to the fourth probability, the fifth probability, and the sixth probability of each pixel point of the second image and a down-sampling multiple of down-sampling processing. In an example, according to the down-sampling multiple, the corresponding relationship between each pixel point in the second image and each pixel point in the first image can be determined, for example, if the down-sampling multiple is 8, then the 1 st pixel point in the second image is the 1 st pixel point in the first image, the 2 nd pixel point in the second image is the 9 th pixel point … in the first image, the 1 st pixel point in another second image is the 2 nd pixel point in the first image, and the 2 nd pixel point in another second image is the 10 th pixel point … in the first image, so that the first probability, the second probability and the third probability of each pixel point in the first image can be determined by the plurality of the fourth probability, the fifth probability and the sixth probability of each pixel point in the second image and the corresponding relationship between each pixel point in the first image, for example, the fourth probability, the fifth probability, and the sixth probability of the 1 st pixel point in the second image are respectively determined as the first probability, the second probability, the third probability, and the like of the 1 st pixel point in the first image. The present disclosure does not limit the correspondence between each pixel point of the second image and each pixel point in the first image. In an example, the void convolutional layer may determine a corresponding relationship between each pixel point of the second image and each pixel point in the first image according to the downsampling multiple, and further determine a first probability, a second probability, and a third probability of each pixel point in the first image according to a fourth probability, a fifth probability, a sixth probability, and the corresponding relationship of each pixel point in the second image, that is, the void convolutional layer may determine the first probability, the second probability, and the third probability of each pixel point of the first image.

In an example, the first image is an image with a resolution of 256 × 256, the second image includes a plurality of images with a resolution of 64 × 64, the hole convolution layer may include four mixed hole convolution modules, each of the mixed hole convolution modules may perform a pixel-by-pixel probability determination according to a fourth probability, a fifth probability, and a sixth probability of each pixel of the second image and the correspondence relationship, that is, each pixel of the plurality of second images may be individually corresponded to a pixel in the first image to determine a first probability, a second probability, and a third probability of each pixel of the first image, for example, the mixed hole convolution module may perform a hole convolution operation (i.e., a deconvolution operation) with holes of 1, 2, and 3 respectively on each pixel of the second image, that is, a probability value of 3 × 64 × 64 (where 64 × 64 is the resolution of the second image, that is, the number of the pixel points included in the second image, 3 is three probability values of each pixel point, that is, the fourth probability, the fifth probability, and the sixth probability), and 3 × 256 × 256 probability values are determined, which are the first probability, the second probability, and the third probability of 256 × 256 pixel points, respectively.

By the method, the fourth probability, the fifth probability and the sixth probability of each pixel point of the second image can be determined through the void convolution layer, the first probability, the second probability and the third probability of each pixel point in the first image are determined, precision loss caused by loss can be avoided under the condition that the receptive field of the second image is enlarged, and accuracy of probability determination is improved.

In a possible implementation manner, in step S13, a first target region, a second target region, and a background region in the first image may be determined according to the first probability, the second probability, and the third probability of each pixel point of the first image.

Fig. 2 shows a flowchart of an image processing method according to an embodiment of the present disclosure, as shown in fig. 2, the method further comprising:

in step S14, the first image input position determination network is processed to obtain the positional relationship between the pixels of the first image.

Determining a first target area, a second target area and a background area in the first image according to the first probability, the second probability and the third probability of each pixel point of the first image, and comprising the following steps: determining the category of a third pixel point according to the first probability, the second probability and the third probability of the third pixel point, wherein the third pixel point is any pixel point of the first image; and respectively determining a first target area, a second target area and a background area according to the category of each pixel point of the first image and the position relation among the pixel points.

In a possible implementation manner, the position determination network may be a BP neural network, a convolutional neural network, or a recurrent neural network, and the position determination network may determine a position relationship between pixels of the first image, for example, which pixels are respectively adjacent to a certain pixel, or number the pixels, and record a position of each numbered pixel, and the like. In an example, the location determination network may be a sub-network of the split network. In an example, the position relationship of each pixel point of the first image can also be recorded by recording the coordinate of each pixel point of the first image.

In one possible implementation, the category of each pixel point may be determined according to a first probability, a second probability, and a third probability of each pixel point in the first image, in this example, the third pixel point is any pixel point of the first image, if a first probability that the third pixel point belongs to the first target region is greater than a second probability that the third pixel point belongs to the second target region and a third probability that the third pixel point belongs to the background region, the category of the third pixel point may be determined as a pixel point in the first target region, if a second probability that the third pixel point belongs to the second target region is greater than the first probability that the third pixel point belongs to the first target region and the third probability that the third pixel point belongs to the background region, the category of the third pixel point may be determined as a pixel point in the second target region, and if the third probability that the third pixel point belongs to the background region is greater than the first probability that the third pixel point belongs to the first target region and the second target probability that the third pixel point belongs to the background region And determining the category of the third pixel point as the pixel point in the background region according to the second probability of the region. In this way, the classification of each pixel of the first image can be determined.

In a possible implementation manner, according to the category of each pixel point of the first image, a pixel point belonging to the first target region, a pixel point belonging to the second target region, and a pixel point belonging to the background region may be determined. Furthermore, the position of the pixel point belonging to the first target area in the first image can be determined through the position relationship among the pixel points, and then the position of the first target area is determined.

In an example, the category of 256 × 256 pixel points may be determined according to 3 × 256 × 256 probability values, and when the position of each region in the first image is determined, because the 256 × 256 pixel points are pixel points of a plurality of second images obtained by performing processing such as downsampling and feature enhancement on each pixel point of the first image, the 256 × 256 pixel points may be subjected to processing such as upsampling or interpolation to restore the pixel points to an image with the same resolution as the first image, but the processing such as upsampling or interpolation may cause accuracy loss, and therefore, pixel recombination (pixel shuffle) processing may be performed through a positional relationship between the 256 × 256 pixel points, and the positions of each pixel point in the first image may be determined, so as to determine the positions of the first target region, the second target region, and the background region in the first image.

In one possible implementation, the segmentation network may be trained prior to processing the first image using the segmentation network.

Fig. 3 shows a flow chart of an image processing method according to an embodiment of the present disclosure, as shown in fig. 3, the method further comprising:

in step S15, the segmentation network is trained by an image set comprising a plurality of pre-processed sample images.

In one possible implementation manner, the plurality of sample images are all images subjected to batch normalization processing, for example, images subjected to batch normalization processing.

In one possible implementation, training the segmentation network by an image set including a plurality of preprocessed sample images may include: inputting a sample image to the segmentation network, and obtaining a first probability that each pixel point of the sample image belongs to a first sample target area, a second probability that each pixel point of the sample image belongs to a second sample target area and a third probability that each pixel point of the sample image belongs to a sample background area; determining a first sample target area, a second sample target area and a sample background area in the sample image according to a first probability that each pixel point belongs to a first sample target area, a second probability that each pixel point belongs to a second sample target area and a third probability that each pixel point belongs to a sample background area; determining the network loss of the segmentation network according to a first sample target area, a second sample target area and a sample background area in the sample image, and the first target area, the second target area and the background area in the sample image; adjusting network parameters of the segmented network according to the network loss; and when the segmentation network meets the training condition, obtaining the trained segmentation network.

In a possible implementation manner, any sample image may be input into the segmentation network, and a first probability that each pixel point of a plurality of pixel points of the sample image belongs to a first sample target region, a second probability that each pixel point belongs to a second sample target region, and a third probability that each pixel point belongs to a sample background region are obtained, where the first probability that each pixel point belongs to the first sample target region, the second probability that each pixel point belongs to the second sample target region, and the third probability that each pixel point belongs to the sample background region are output results of the segmentation network, and there may be errors.

In a possible implementation manner, a first sample target region, a second sample target region and a sample background region in a sample image may be determined according to the first probability, the second probability and the third probability of each pixel point in the sample image. For example, the category of each pixel point and the position relationship between each pixel point can be determined by the first probability, the second probability and the third probability of each pixel point, and the first sample target area, the second sample target area and the sample background area are determined.

In one possible implementation manner, the network loss of the segmented network may be determined according to a first sample target area, a second sample target area and a sample background area in a sample image, and a real first target area, a second target area and a background area of the sample image, where the first target area, the second target area and the background area are real areas in the sample image, and the real areas may be artificially labeled real areas. In an example, a network loss may be determined by an error between the output result and the real area.

In one possible implementation, determining a network loss of the segmented network according to the first sample target area, the second sample target area, and the sample background area in the sample image, and the first target area, the second target area, and the background area in the sample image includes: determining cross entropy loss of each pixel point in the sample image according to a first sample target area, a second sample target area and a sample background area in the sample image, and the first target area, the second target area and the background area in the sample image; determining a weight coefficient matrix according to the symbol distance functions of the sample image target area, the second sample target area and the sample background area; determining the coincidence degree of the first sample target area and the second sample target area according to the first probability that each pixel point of the sample image belongs to the target area and the second probability that each pixel point of the sample image belongs to the second sample target area; and determining the network loss of the segmented network according to the cross entropy loss, the weight coefficient matrix and the coincidence degree.

In an example, the cross entropy loss of each pixel point may be determined according to an error between the output result and the real region and the first, second, and third probabilities of each pixel point. And determining a weight coefficient matrix (i.e., a weight of a pixel point belonging to a first probability of the first sample target region, a second probability of the second sample target region, and a third probability of the sample background region) according to a Sign Distance Function (SDF) of the first sample target region, the second sample target region, and the sample background region in the sample image. And determining the coincidence degree of the first sample target area and the second sample target area according to the first probability, the second probability and the third probability, where the coincidence degree may be determined by the number or the ratio of the coincident pixel points, for example, a first probability that a certain pixel point belongs to the first sample target area and a second probability that a certain pixel point belongs to the second sample target area are close (for example, the difference between the first probability and the second probability is less than a threshold), and the first sample target area and the second sample target area may be considered to coincide at the pixel point.

In an example, a network loss is determined from the cross-entropy loss, a matrix of weight coefficients, and the degree of coincidence. In an example, the network loss may be determined by the following equation (2):

L＝W×CE-log(GD) (2)

according to the network loss, the coincidence degree can be reduced in the process of adjusting network parameters of the segmentation network, the discrimination of the first sample target area and the second sample target area is increased, the gradient ratio among the first sample target area, the second sample target area and the sample background area is balanced, and the performance of distinguishing the first sample target area from the second sample target area by the segmentation network is improved. W is a weight coefficient matrix, in an example, the weight coefficient matrix determined according to the symbol distance function of the first sample target region, the second sample target region and the sample background region can increase the weights of the first probability and the second probability, effectively reduce cross entropy loss, and improve the capability of the segmentation network to identify the first sample target region and the second sample target region.

In a possible implementation manner, the segmentation network may be adjusted according to the network loss, and when the segmentation network satisfies a training condition, a trained segmentation network is obtained. In an example, network parameters of the split network may be adjusted in a direction in which network loss is minimized. The training condition may be an adjustment number of times, and a predetermined number of sample images may be input, that is, a network parameter value of the segmentation network is adjusted a predetermined number of times. In an example, the training condition may be a size or a convergence of the network loss, and the adjustment may be stopped when the network loss decreases to a certain degree or converges within a certain threshold, so as to obtain an adjusted segmented network. In the adjusting process, the network parameters of the segmented network can be adjusted by reversely gradiently propagating the network loss by using a set learning rate, for example, the learning rate can be set to 0.0002, and the learning rate can be attenuated by 10 times when the adjusting times reach at least one of 180 times, 300 times and 500 times. After the adjustment is completed, the adjusted segmentation network may be used in the step of determining the first probability, the second probability, and the third probability of each pixel point of the first image.

According to the image processing method disclosed by the embodiment of the disclosure, the complexity of the image to be processed is reduced and the processing efficiency is improved by performing batch normalization processing on the image to be processed. And the first images after batch normalization processing can be input into a segmentation network to obtain a first probability, a second probability and a third probability, wherein a down-sampling sub-network of the segmentation network can comprise a self-adaptive normalization layer, and self-adaptive normalization processing can be performed on the input third images, so that the generalization capability of the segmentation network can be improved, and the false recognition rate of the segmentation network can be reduced. The segmentation network can perform feature enhancement on each pixel point of the down-sampling image, improve the discrimination between the pixel points in different areas, increase the relation between the pixel points in the same area and improve the identification capability of the segmentation network on a target area. The positions of the first target area, the second target area and the background area in the first image can be determined according to the position relation, so that the precision loss in the processing processes of up-sampling or interpolation and the like of each pixel point is reduced, and the precision of determining each area is improved. Furthermore, the segmentation network can determine the first target area, the second target area and the background area in the first image according to the first probability, the second probability and the third probability, so that the types of recognizable areas are increased, different types of areas can be accurately distinguished, and the resolution capability of the target areas is improved.

Fig. 4 is a schematic application diagram of an image processing method according to an embodiment of the present disclosure, in which the image to be processed is diffusion-weighted imaging of a brain, the first target region is a core region of a lesion such as a stroke or an infarction, and the second target region is a penumbra region caused by ischemia. The image to be processed may be subjected to batch normalization processing, for example, at least one of a gray value, a brightness value, a chromatic value, or an RGB value of the image to be processed may be subjected to batch normalization processing to obtain the first image.

In one possible implementation, as shown in FIG. 4, the first image may be input to a segmentation network for processing, which may include a downsampling subnetwork, a feature enhancement subnetwork, and a probability determination subnetwork. The down-sampling sub-network can comprise a first down-sampling layer, a self-adaptive normalization layer and a second down-sampling layer, the first down-sampling layer comprises one or more of a convolution layer, an activation layer, a pooling layer and other network layers, and each pixel point of the first image can be subjected to convolution processing to obtain a third image. The self-adaptive normalization layer can perform self-adaptive normalization processing on the third image so as to improve the generalization capability of the segmentation network and obtain a fourth image. The second downsampling layer comprises one or more of an activation layer and a network layer such as a convolution layer and a pooling layer, and the fourth image can be downsampled to obtain the downsampled image.

In a possible implementation manner, the feature enhancement network may copy the down-sampled image, and map the copy of the down-sampled image obtained by copying to a preset feature space, and may perform feature enhancement processing on each pixel point of the sampled image by using each pixel point mapped to the copy in the feature space, for example, may perform feature enhancement processing by using formula (1), so as to obtain the second image.

In a possible implementation manner, the probability determining sub-network may process the second image through the void convolution layer to obtain a first probability, a second probability, and a third probability of each pixel of the first image, and may perform pixel reconstruction processing according to a positional relationship between each pixel of the first image to determine the core region and the penumbra region.

In a possible implementation manner, RGB values may be further added to the core region and the semi-dark band region, for example, random RGB values may be added or preset RGB values may be added, so that the core region, the semi-dark band region and the background region have different colors, so as to improve the distinction between different regions.

In addition, the present disclosure also provides an image processing apparatus, an electronic device, a computer-readable storage medium, and a program, which can be used to implement any one of the image processing methods provided by the present disclosure, and the descriptions and corresponding descriptions of the corresponding technical solutions and the corresponding descriptions in the methods section are omitted for brevity.

It will be understood by those skilled in the art that in the method of the present invention, the order of writing the steps does not imply a strict order of execution and any limitations on the implementation, and the specific order of execution of the steps should be determined by their function and possible inherent logic.

Fig. 5 shows a block diagram of an image processing apparatus according to an embodiment of the present disclosure, which, as shown in fig. 5, includes:

the preprocessing module 11 is configured to preprocess a plurality of pixel points of an image to be processed to obtain a first image;

a probability obtaining module 12, configured to input the first image into a segmentation network for processing, so as to obtain a first probability that each pixel of the first image belongs to a first target region, a second probability that each pixel belongs to a second target region, and a third probability that each pixel belongs to a background region;

and the region determining module 13 is configured to determine a first target region, a second target region, and a background region in the first image according to the first probability, the second probability, and the third probability of each pixel point of the first image.

wherein the probability obtaining module is further configured to:

Fig. 6 shows a block diagram of an image processing apparatus according to an embodiment of the present disclosure, which, as shown in fig. 6, further includes:

and the position relation obtaining module 14 is configured to determine a position of the first image in the position determination network and process the first image to obtain a position relation between each pixel of the first image.

In one possible implementation, the apparatus further includes:

a training module 15 for training the segmentation network by an image set comprising a plurality of preprocessed sample images.

In one possible implementation, the training module is further configured to:

In some embodiments, functions of or modules included in the apparatus provided in the embodiments of the present disclosure may be used to execute the method described in the above method embodiments, and specific implementation thereof may refer to the description of the above method embodiments, and for brevity, will not be described again here.

Embodiments of the present disclosure also provide a computer-readable storage medium having stored thereon computer program instructions, which when executed by a processor, implement the above-mentioned method. The computer readable storage medium may be a non-volatile computer readable storage medium.

An embodiment of the present disclosure further provides an electronic device, including: a processor; a memory for storing processor-executable instructions; wherein the processor is configured as the above method.

The electronic device may be provided as a terminal, server, or other form of device.

Fig. 7 is a block diagram illustrating an electronic device 800 according to an example embodiment. For example, the electronic device 800 may be a mobile phone, a computer, a digital broadcast terminal, a messaging device, a game console, a tablet device, a medical device, a fitness device, a personal digital assistant, or the like terminal.

Referring to fig. 7, electronic device 800 may include one or more of the following components: processing component 802, memory 804, power component 806, multimedia component 808, audio component 810, input/output (I/O) interface 812, sensor component 814, and communication component 816.

The processing component 802 generally controls overall operation of the electronic device 800, such as operations associated with display, telephone calls, data communications, camera operations, and recording operations. The processing components 802 may include one or more processors 820 to execute instructions to perform all or a portion of the steps of the methods described above. Further, the processing component 802 can include one or more modules that facilitate interaction between the processing component 802 and other components. For example, the processing component 802 can include a multimedia module to facilitate interaction between the multimedia component 808 and the processing component 802.

The memory 804 is configured to store various types of data to support operations at the electronic device 800. Examples of such data include instructions for any application or method operating on the electronic device 800, contact data, phonebook data, messages, pictures, videos, and so forth. The memory 804 may be implemented by any type or combination of volatile or non-volatile memory devices such as Static Random Access Memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic or optical disks.

The power supply component 806 provides power to the various components of the electronic device 800. The power components 806 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power for the electronic device 800.

The multimedia component 808 includes a screen that provides an output interface between the electronic device 800 and a user. In some embodiments, the screen may include a Liquid Crystal Display (LCD) and a Touch Panel (TP). If the screen includes a touch panel, the screen may be implemented as a touch screen to receive an input signal from a user. The touch panel includes one or more touch sensors to sense touch, slide, and gestures on the touch panel. The touch sensor may not only sense the boundary of a touch or slide action, but also detect the duration and pressure associated with the touch or slide operation. In some embodiments, the multimedia component 808 includes a front facing camera and/or a rear facing camera. The front camera and/or the rear camera may receive external multimedia data when the electronic device 800 is in an operation mode, such as a shooting mode or a video mode. Each front camera and rear camera may be a fixed optical lens system or have a focal length and optical zoom capability.

The audio component 810 is configured to output and/or input audio signals. For example, the audio component 810 includes a Microphone (MIC) configured to receive external audio signals when the electronic device 800 is in an operational mode, such as a call mode, a recording mode, and a voice recognition mode. The received audio signals may further be stored in the memory 804 or transmitted via the communication component 816. In some embodiments, audio component 810 also includes a speaker for outputting audio signals.

The I/O interface 812 provides an interface between the processing component 802 and peripheral interface modules, which may be keyboards, click wheels, buttons, etc. These buttons may include, but are not limited to: a home button, a volume button, a start button, and a lock button.

The sensor assembly 814 includes one or more sensors for providing various aspects of state assessment for the electronic device 800. For example, the sensor assembly 814 may detect an open/closed state of the electronic device 800, the relative positioning of components, such as a display and keypad of the electronic device 800, the sensor assembly 814 may also detect a change in the position of the electronic device 800 or a component of the electronic device 800, the presence or absence of user contact with the electronic device 800, orientation or acceleration/deceleration of the electronic device 800, and a change in the temperature of the electronic device 800. Sensor assembly 814 may include a proximity sensor configured to detect the presence of a nearby object without any physical contact. The sensor assembly 814 may also include a light sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, the sensor assembly 814 may also include an acceleration sensor, a gyroscope sensor, a magnetic sensor, a pressure sensor, or a temperature sensor.

The communication component 816 is configured to facilitate wired or wireless communication between the electronic device 800 and other devices. The electronic device 800 may access a wireless network based on a communication standard, such as WiFi, 2G or 3G, or a combination thereof. In an exemplary embodiment, the communication component 816 receives a broadcast signal or broadcast related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component 816 further includes a Near Field Communication (NFC) module to facilitate short-range communications. For example, the NFC module may be implemented based on Radio Frequency Identification (RFID) technology, infrared data association (IrDA) technology, Ultra Wideband (UWB) technology, Bluetooth (BT) technology, and other technologies.

In an exemplary embodiment, the electronic device 800 may be implemented by one or more Application Specific Integrated Circuits (ASICs), Digital Signal Processors (DSPs), Digital Signal Processing Devices (DSPDs), Programmable Logic Devices (PLDs), Field Programmable Gate Arrays (FPGAs), controllers, micro-controllers, microprocessors or other electronic components for performing the above-described methods.

In an exemplary embodiment, a non-transitory computer-readable storage medium, such as the memory 804, is also provided that includes computer program instructions executable by the processor 820 of the electronic device 800 to perform the above-described methods.

Fig. 8 is a block diagram illustrating an electronic device 1900 according to an example embodiment. For example, the electronic device 1900 may be provided as a server. Referring to fig. 8, electronic device 1900 includes a processing component 1922 further including one or more processors and memory resources, represented by memory 1932, for storing instructions, e.g., applications, executable by processing component 1922. The application programs stored in memory 1932 may include one or more modules that each correspond to a set of instructions. Further, the processing component 1922 is configured to execute instructions to perform the above-described method.

The electronic device 1900 may also include a power component 1926 configured to perform power management of the electronic device 1900, a wired or wireless network interface 1950 configured to connect the electronic device 1900 to a network, and an input/output (I/O) interface 1958. The electronic device 1900 may operate based on an operating system stored in memory 1932, such as Windows Server, Mac OS XTM, UnixTM, LinuxTM, FreeBSDTM, or the like.

In an exemplary embodiment, a non-transitory computer readable storage medium, such as the memory 1932, is also provided that includes computer program instructions executable by the processing component 1922 of the electronic device 1900 to perform the above-described methods.

The present disclosure may be systems, methods, and/or computer program products. The computer program product may include a computer-readable storage medium having computer-readable program instructions embodied thereon for causing a processor to implement various aspects of the present disclosure.

The computer readable storage medium may be a tangible device that can hold and store the instructions for use by the instruction execution device. The computer readable storage medium may be, for example, but not limited to, an electronic memory device, a magnetic memory device, an optical memory device, an electromagnetic memory device, a semiconductor memory device, or any suitable combination of the foregoing. More specific examples (a non-exhaustive list) of the computer readable storage medium would include the following: a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a Static Random Access Memory (SRAM), a portable compact disc read-only memory (CD-ROM), a Digital Versatile Disc (DVD), a memory stick, a floppy disk, a mechanical coding device, such as punch cards or in-groove projection structures having instructions stored thereon, and any suitable combination of the foregoing. Computer-readable storage media as used herein is not to be construed as transitory signals per se, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through a waveguide or other transmission medium (e.g., optical pulses through a fiber optic cable), or electrical signals transmitted through electrical wires.

The computer-readable program instructions described herein may be downloaded from a computer-readable storage medium to a respective computing/processing device, or to an external computer or external storage device via a network, such as the internet, a local area network, a wide area network, and/or a wireless network. The network may include copper transmission cables, fiber optic transmission, wireless transmission, routers, firewalls, switches, gateway computers and/or edge servers. The network adapter card or network interface in each computing/processing device receives computer-readable program instructions from the network and forwards the computer-readable program instructions for storage in a computer-readable storage medium in the respective computing/processing device.

The computer program instructions for carrying out operations of the present disclosure may be assembler instructions, Instruction Set Architecture (ISA) instructions, machine-related instructions, microcode, firmware instructions, state setting data, or source or object code written in any combination of one or more programming languages, including an object oriented programming language such as Smalltalk, C + + or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The computer-readable program instructions may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or the connection may be made to an external computer (for example, through the Internet using an Internet service provider). In some embodiments, the electronic circuitry that can execute the computer-readable program instructions implements aspects of the present disclosure by utilizing the state information of the computer-readable program instructions to personalize the electronic circuitry, such as a programmable logic circuit, a Field Programmable Gate Array (FPGA), or a Programmable Logic Array (PLA).

Various aspects of the present disclosure are described herein with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the disclosure. It will be understood that each block of the flowchart illustrations and/or block diagrams, and combinations of blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer-readable program instructions.

These computer-readable program instructions may be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions/acts specified in the flowchart and/or block diagram block or blocks. These computer-readable program instructions may also be stored in a computer-readable storage medium that can direct a computer, programmable data processing apparatus, and/or other devices to function in a particular manner, such that the computer-readable medium storing the instructions comprises an article of manufacture including instructions which implement the function/act specified in the flowchart and/or block diagram block or blocks.

The computer readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other devices to cause a series of operational steps to be performed on the computer, other programmable apparatus or other devices to produce a computer implemented process such that the instructions which execute on the computer, other programmable apparatus or other devices implement the functions/acts specified in the flowchart and/or block diagram block or blocks.

The flowchart and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of instructions, which comprises one or more executable instructions for implementing the specified logical function(s). In some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.

Having described embodiments of the present disclosure, the foregoing description is intended to be exemplary, not exhaustive, and not limited to the disclosed embodiments. Many modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. The terms used herein were chosen in order to best explain the principles of the embodiments, the practical application, or technical improvements to the techniques in the marketplace, or to enable others of ordinary skill in the art to understand the embodiments disclosed herein.

Claims

1. An image processing method, characterized in that the method comprises:

determining a first target area, a second target area and a background area in the first image according to a first probability, a second probability and a third probability of each pixel point of the first image, wherein the first target area and the second target area are different types of target areas;

the segmentation network comprises a down-sampling sub-network, a feature enhancement sub-network and a probability determination sub-network,

inputting the second image into the probability determination sub-network for processing, and determining a first probability, a second probability and a third probability of each pixel point of the first image;

inputting the second image into the probability determination sub-network for processing, and determining a first probability, a second probability and a third probability of each pixel point of the first image, including:

2. The method of claim 1, wherein the segmentation network comprises a probability determination sub-network comprising hole convolutional layers.

3. The method of claim 2, wherein the hole convolution layer is used to determine a first probability, a second probability, and a third probability for each pixel of the first image.

4. The method of any of claims 1-3, wherein the segmentation network comprises a downsampling subnetwork comprising an adaptive normalization layer for performing adaptive normalization processing on an image input to the adaptive normalization layer.

5. The method of claim 1, wherein the segmentation network comprises a feature enhancement sub-network configured to perform feature enhancement processing on the image input to the feature enhancement sub-network based on a plurality of pixel points of the image input to the feature enhancement sub-network.

6. The method of claim 1, wherein inputting the downsampled image into a feature enhancement sub-network for feature enhancement processing to obtain a second image comprises:

7. The method of claim 1 or 6, wherein the sub-sampling sub-network comprises a first down-sampling layer comprising at least a convolutional layer, a self-adaptive normalization layer, and a second down-sampling layer comprising at least an active layer.

8. The method of claim 7, wherein inputting the first image into a downsampling subnetwork for downsampling to obtain a downsampled image comprises:

9. The method of claim 1, further comprising:

10. The method of claim 9, wherein determining the first target region, the second target region, and the background region in the first image according to the first probability, the second probability, and the third probability of each pixel of the first image comprises:

11. The method of claim 1, wherein preprocessing a plurality of pixel points of an image to be processed to obtain a first image comprises:

12. The method of claim 1, wherein the first target region is a lesion core region and the second target region is a penumbra region.

13. The method of claim 1, further comprising:

14. The method of claim 13, wherein training the segmentation network with an image set comprising a plurality of preprocessed sample images comprises:

15. The method of claim 14, wherein determining the network loss of the segmented network from the first sample target region, the second sample target region, and the sample background region in the sample image, and the first target region, the second target region, and the background region in the sample image comprises:

16. An image processing apparatus characterized by comprising:

the region determining module is used for determining a first target region, a second target region and a background region in the first image according to the first probability, the second probability and the third probability of each pixel point of the first image, wherein the first target region and the second target region are different types of target regions;

wherein the probability obtaining module is further configured to:

the probability obtaining module is further configured to:

17. The apparatus of claim 16, wherein the segmentation network comprises a probability determination subnetwork comprising hole convolutional layers.

18. The apparatus of claim 17, wherein the hole convolution layer is configured to determine a first probability, a second probability, and a third probability for each pixel of the first image.

19. The apparatus of any of claims 15-18, wherein the segmentation network comprises a downsampling subnetwork comprising an adaptive normalization layer for performing adaptive normalization processing on an image input to the adaptive normalization layer.

20. The apparatus of claim 16, wherein the segmentation network comprises a feature enhancement sub-network configured to perform feature enhancement processing on the image input to the feature enhancement sub-network based on a plurality of pixel points of the image input to the feature enhancement sub-network.

21. The apparatus of claim 16, wherein the probability obtaining module is further configured to:

22. The apparatus of claim 16 or 21, wherein the sub-sampling sub-network comprises a first down-sampling layer comprising at least a convolutional layer, a self-adaptive normalization layer, and a second down-sampling layer comprising at least an active layer.

23. The apparatus of claim 22, wherein the probability obtaining module is further configured to:

24. The apparatus of claim 16, further comprising:

25. The apparatus of claim 24, wherein the region determination module is further configured to:

26. The apparatus of claim 16, wherein the pre-processing module is further configured to:

27. The device of claim 16, wherein the first target region is a lesion core region and the second target region is a penumbra region.

28. The apparatus of claim 16, further comprising:

29. The apparatus of claim 28, wherein the training module is further configured to:

30. The apparatus of claim 29, wherein the training module is further configured to:

31. An electronic device, comprising:

a processor;

a memory for storing processor-executable instructions;

wherein the processor is configured to: performing the method of any one of claims 1 to 15.

32. A computer readable storage medium having computer program instructions stored thereon, which when executed by a processor implement the method of any one of claims 1 to 15.