TW202125408A - Image semantic segmentation method, device and storage medium thereof - Google Patents

Abstract

The embodiment of the present disclosure discloses an image semantic segmentation method, a device and a storage medium, wherein the method includes: performing feature extraction on the acquired image to be processed to obtain a first feature image; and Simultaneously extracting multiple context features with different ranges on the first feature image to obtain multiple second feature images; at least according to the multiple second feature images, determining a target image, wherein the target image as the new first feature image is synchronously extracted again context features with different ranges; In response to the number of simultaneous extraction of multiple context features with different ranges for the first feature image reaching the target number of times, a semantic image corresponding to the image to be processed is generated based on the target image obtained at the last time.

Description

Image semantic segmentation method, device and storage medium

The present disclosure relates to the field of deep learning, in particular to image semantic segmentation methods and devices, and storage media.

For movable machinery and equipment, semantic segmentation can be performed on the images collected by the cameras mounted on it to obtain a semantic understanding of the scene, so as to realize functions such as obstacle avoidance and navigation.

At present, on the one hand, due to cost and mobility considerations, the computing resources of movable machines are often limited. On the other hand, mobile machines and equipment need to interact with the real environment in real time. Therefore, how to perform real-time semantic segmentation under limited computing resources is a challenging technical problem.

The embodiments of the present disclosure provide an image semantic segmentation method, device, and storage medium.

According to a first aspect of the embodiments of the present disclosure, there is provided an image semantic segmentation method, the method includes: performing feature extraction on an acquired image to be processed to obtain a first feature image; Synchronously extract multiple contextual features with different ranges to obtain multiple second feature images; at least according to the multiple second feature images, determine a target image, and use the target image as the new first The feature image is again synchronized to extract a plurality of context features with different ranges; in response to the number of times that the first feature image is synchronized to extract a plurality of context features with different ranges reaches the target number of times, based on the target image obtained last time, A semantic image corresponding to the image to be processed is generated.

In some optional embodiments, the synchronously extracting a plurality of contextual features with different ranges from the first characteristic image to obtain a plurality of second characteristic images includes: dividing the first characteristic image into a plurality of Channels perform dimensionality reduction processing synchronously to obtain multiple third feature images; extract contextual features with different ranges from at least two third feature images in the multiple third feature images to obtain multiple second feature images picture.

In some optional embodiments, the extracting context features with different ranges from at least two third feature images of the plurality of third feature images to obtain multiple second feature images includes: using depth The convolution can be separated and the convolution kernel corresponds to the hole convolution with different hole coefficients, and the context features with different ranges are extracted from at least two third feature images in the plurality of third feature images to obtain the plurality of first feature images. Two feature images.

In some optional embodiments, the determining the target image at least according to the plurality of second characteristic images includes: fusing at least the plurality of second characteristic images to obtain a fourth characteristic image; The target image is determined based on at least the fourth characteristic image.

In some optional embodiments, the fusing at least the plurality of second characteristic images to obtain a fourth characteristic image includes: superimposing the plurality of second characteristic images to obtain the first characteristic image. Four characteristic images; or superimposing at least one third characteristic image of the plurality of second characteristic images and the plurality of third characteristic images to obtain the fourth characteristic image.

In some optional embodiments, the determining the target image at least according to the fourth characteristic image includes: up-sampling the fourth characteristic image to obtain the target image; or, Performing sub-pixel convolution on the fourth characteristic image to obtain the target image.

In some optional embodiments, the method further includes: performing feature extraction and dimensionality reduction processing on the image to be processed to obtain a fifth feature image; wherein, feature extraction corresponding to the fifth feature image The number of layers is less than the number of layers of feature extraction corresponding to the first feature image; the determining the target image at least according to the fourth feature image includes: when the number of times is less than the target number of times In case, the fourth characteristic image and the fifth characteristic image are superimposed and then up-sampled to obtain the target image; or, in the case that the number of times is less than the target number of times, the The image obtained by sub-pixel convolution of the fourth characteristic image is superimposed on the fifth characteristic image to obtain the target image.

In some optional embodiments, the dimension corresponding to the target image obtained last time is a target dimension; wherein, the target dimension is based on a preset total number of object categories included in the semantic image The project is determined.

In some optional embodiments, after the semantic image corresponding to the image to be processed is generated, the method further includes: performing machine device navigation according to the semantic image.

According to a second aspect of the embodiments of the present disclosure, there is provided an image semantic segmentation device, the device comprising: a feature extraction module configured to perform feature extraction on the acquired image to be processed to obtain a first feature image; context The feature extraction module is configured to simultaneously extract a plurality of contextual features with different ranges from the first feature image to obtain a plurality of second feature images; the determining module is configured to at least according to the plurality of second feature maps Image, determine the target image, and use the target image as the new first feature image to synchronously extract a plurality of contextual features with different ranges; the semantic image generation module is configured to respond to the first feature image The number of times that a feature image synchronously extracts a plurality of context features with different ranges reaches a target number of times, and a semantic image corresponding to the image to be processed is generated based on the target image obtained last time.

In some optional embodiments, the context feature extraction module includes: a first processing sub-module configured to simultaneously perform dimensionality reduction processing on the first feature image in multiple channels to obtain multiple third features Image; a second processing sub-module configured to extract context features with different ranges from at least two third feature images in the plurality of third feature images to obtain a plurality of second feature images.

In some optional embodiments, the second processing sub-module is configured to adopt a hole convolution with a depth separable convolution and a convolution kernel corresponding to different hole coefficients, and perform the At least two third feature images extract context features with different ranges to obtain the multiple second feature images.

In some optional embodiments, the determining module includes: a first determining sub-module configured to fuse at least the plurality of second characteristic images to obtain a fourth characteristic image; and a second determining sub-module The group is configured to determine the target image at least according to the fourth characteristic image.

In some optional embodiments, the first determining sub-module is configured to superimpose the plurality of second characteristic images to obtain the fourth characteristic image; or to compare the plurality of second characteristic images The image and at least one third characteristic image of the plurality of third characteristic images are superimposed to obtain the fourth characteristic image.

In some optional embodiments, the second determining sub-module is configured to perform up-sampling on the fourth characteristic image to obtain the target image; or, perform sub-modules on the fourth characteristic image. Pixel convolution to obtain the target image.

In some optional embodiments, the device further includes: a processing module configured to perform feature extraction and dimensionality reduction processing on the image to be processed to obtain a fifth feature image; wherein, the fifth feature The number of feature extraction layers corresponding to the image is less than the number of feature extraction layers corresponding to the first feature image; the second determining sub-module is configured to: if the number of times is less than the target number of times, The fourth characteristic image and the fifth characteristic image are superimposed and then up-sampling is performed to obtain the target image; or, when the number of times is less than the target number of times, the fourth characteristic image is The image obtained after sub-pixel convolution is superimposed on the fifth characteristic image to obtain the target image.

In some optional embodiments, the dimension corresponding to the target image obtained last time is a target dimension; wherein, the target dimension is based on a preset total number of object categories included in the semantic image The project is determined.

In some optional embodiments, the device further includes a navigation module configured to perform machine equipment navigation according to the semantic image.

According to a third-party aspect of the embodiments of the present disclosure, a computer-readable storage medium is provided, the storage medium stores a computer program, and the computer program is used to execute the image semantic segmentation method of any one of the above-mentioned first aspects.

According to a fourth aspect of the embodiments of the present disclosure, there is provided an image semantic segmentation device, including: a processor; a memory for storing executable instructions of the processor; wherein the processor is configured to call the memory The executable instructions stored in the body implement the image semantic segmentation method described in any one of the first aspect.

According to a fifth aspect of the embodiments of the present disclosure, there is provided a computer program that enables a computer to execute the image semantic segmentation method according to any one of the first aspects of the embodiments of the present disclosure.

The technical solutions provided by the embodiments of the present disclosure may include the following beneficial effects:

In the embodiment of the present disclosure, feature extraction can be performed on the acquired image to be processed to obtain a first feature image, and then multiple context features with different ranges are simultaneously extracted from the first feature image to obtain multiple second features image. At least according to the multiple second feature images, the target image is determined, and the target image is used as the new first feature image, and multiple context features with different ranges are synchronously extracted again. When the number of times of synchronously extracting a plurality of context features with different ranges from the first feature image reaches the target number of times, semantic segmentation may be performed based on the target image obtained last time to generate a semantic image corresponding to the image to be processed. In the embodiments of the present disclosure, multiple contextual features with different ranges are synchronously extracted by feature images corresponding to images to be processed multiple times, and context information of different scales is fully integrated, thereby improving the accuracy of semantic segmentation.

In the disclosed embodiment, the first feature image may be divided into multiple channels to perform dimensionality reduction processing to obtain multiple third feature images, and then at least two of the multiple third feature images with different extraction ranges Context feature, and obtain multiple corresponding second feature images. The purpose of synchronously extracting a plurality of contextual features with different ranges from the first feature image is realized, which is beneficial to improve the accuracy of semantic segmentation and reduces the amount of calculation in the semantic segmentation process.

In the embodiment of the disclosure, a hole convolution with a depth separable convolution and a convolution kernel corresponding to different hole coefficients can be used to extract at least two context features with different ranges in a plurality of third feature images. The purpose of synchronously extracting multiple contextual features with different ranges in the first feature image is beneficial to improve the accuracy of semantic segmentation.

In the embodiment of the present disclosure, multiple second feature images can be directly superimposed to obtain a fourth feature image, or at least one of multiple second feature images and multiple third feature images can also be superimposed, Obtaining the fourth feature image has high usability, can fuse information of more scales, and improve the accuracy of semantic segmentation.

In the embodiment of the present disclosure, in order to maintain the dimensionality of the target image, the fourth characteristic image may be up-sampled to obtain the target image. Or, sub-pixel convolution can be performed on the fourth feature image to improve the effect of semantic segmentation and make the result of semantic segmentation more accurate.

In the embodiment of the present disclosure, the fifth characteristic image may be acquired before the target image is determined. Among them, the fifth feature image is an image obtained by extracting low-dimensional image features from the image to be processed. The number of feature extraction layers corresponding to the fifth feature image is smaller than the number of feature extraction layers corresponding to the first feature image. The fourth feature image and the fifth feature image are superimposed and then up-sampled to obtain the target image. In the case that the number of times of synchronously extracting a plurality of context features with different ranges from the first feature image reaches the target number of times, Only the fourth feature image can be up-sampled to obtain the target image, which reduces the possibility of losing some important features in the image to be processed after dimensionality reduction processing, and improves the accuracy of semantic segmentation.

In the disclosed embodiment, the dimension of the target image obtained last time is the target dimension, where the target dimension is determined according to the preset total number of object categories included in the semantic image. Ensure that the dimension of the final semantic image is consistent with the dimension of the image to be processed.

In the embodiments of the present disclosure, machine and equipment navigation can be performed according to the semantic image corresponding to the generated image to be processed, and the usability is high.

It should be understood that the above general description and the following detailed description are only exemplary and explanatory, and cannot limit the embodiments of the present disclosure. [Schematic simple invention]

The drawings here are incorporated into the specification and constitute a part of the specification, show embodiments that comply with the disclosure, and are used together with the specification to explain the principle of the disclosure. FIG. 1A is a color image according to an exemplary embodiment of the present disclosure; Figure 1B is a semantic image according to an exemplary embodiment of the present disclosure; Figure 2 is a flow chart of a method for image semantic segmentation according to an exemplary embodiment of the present disclosure; FIG. 3 is a flowchart of another image semantic segmentation method according to an exemplary embodiment of the present disclosure; FIG. 4 is a schematic diagram of a scene for extracting context features in different ranges according to an exemplary embodiment of the present disclosure; FIG. 5 is a flowchart of another image semantic segmentation method according to an exemplary embodiment of the present disclosure; Fig. 6 is a flowchart of another image semantic segmentation method according to an exemplary embodiment of the present disclosure; FIG. 7 is a schematic diagram of a neural network architecture for obtaining semantic images according to an exemplary embodiment of the present disclosure; FIG. 8A is a schematic structural diagram of a rear terminal network according to an exemplary embodiment of the present disclosure; FIG. 8B is a schematic structural diagram of another rear terminal network according to an exemplary embodiment of the present disclosure; FIG. 8C is a schematic structural diagram of another rear terminal network according to an exemplary embodiment of the present disclosure; FIG. 8D is a schematic structural diagram of another rear terminal network according to an exemplary embodiment of the present disclosure; FIG. 9 is a flowchart of still another image semantic segmentation method according to an exemplary embodiment of the present disclosure; Fig. 10 is a block diagram of an image semantic segmentation device according to an exemplary embodiment of the present disclosure; FIG. 11 is a schematic structural diagram of a device for image semantic segmentation according to an exemplary embodiment of the present disclosure.

Detailed ways

The exemplary embodiments will be described in detail here, and examples thereof are shown in the accompanying drawings. When the following description refers to the accompanying drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The implementation manners described in the following exemplary embodiments do not represent all implementation manners consistent with the present disclosure. On the contrary, they are only examples of devices and methods consistent with some aspects of the present disclosure as detailed in the scope of the appended invention application.

The terminology used in the present disclosure is only for the purpose of describing specific embodiments, and is not intended to limit the present disclosure. The singular forms of "a", "the" and "the" used in the scope of the present disclosure and the appended invention applications are also intended to include plural forms, unless the context clearly indicates other meanings. It should also be understood that the term "and/or" as used herein refers to and includes any or all possible combinations of one or more associated listed items.

It should be understood that although the terms first, second, third, etc. may be used in this disclosure to describe various information, the information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other. For example, without departing from the scope of this disclosure, the first information can also be referred to as second information, and similarly, the second information can also be referred to as first information. Depending on the context, the word "if" as used here can be interpreted as "when" or "when" or "responding to certainty".

The embodiment of the present disclosure provides a method for image semantic segmentation, which can be used in machine equipment, such as mobile machines and equipment such as robots, unmanned vehicles, and drones. Alternatively, the method provided in the embodiments of the present disclosure may be implemented by running a computer executable code through a processor.

Image semantic segmentation refers to the estimation of the type of object to which each pixel in the input red, green and blue (RGB) image belongs. This type of object can include but is not limited to various objects, such as grass, people, For cars, buildings, sky, etc., a semantic map with the same object type label with the same size and dimension corresponding to the RGB image is obtained. For example, Figure 1A is an RGB image, and Figure 1B is a corresponding semantic image.

The embodiment of the disclosure obtains the first characteristic image by extracting the features of the image to be processed obtained by the machine and equipment; and then simultaneously extracting multiple context features with different ranges from the first characteristic image multiple times to obtain multiple The second feature image, so that the target image is determined at least according to the multiple second feature images; finally, a semantic image can be generated based on the target image obtained last time. The disclosed embodiment can fully integrate context information of different scales through multiple context feature extraction and fusion, and improve the accuracy of semantic segmentation. The machine equipment can avoid obstacles in front of the machine equipment according to the semantic image corresponding to the image to be processed, reasonably plan the driving route, and have high availability.

The above are only exemplary application scenarios of the present disclosure, and other scenarios where the image semantic segmentation method of the present disclosure can be used belong to the protection scope of the present disclosure.

As shown in Figure 2, Figure 2 is an image semantic segmentation method according to an exemplary embodiment, which includes the following steps:

In step 101, feature extraction is performed on the acquired image to be processed to obtain a first feature image.

In the disclosed embodiment, the image to be processed may be a real-time image. The real-time image can be collected through a camera preset on the machine and equipment, and the collected images can include various moving routes in front of the machine and equipment. object. The image to be processed may also be an image that has been collected by the machine equipment (for example, an image stored in the machine equipment), or an image that needs to be semantically segmented and sent to the machine equipment by other equipment.

Convert the original image information included in the image to be processed into a set of features with obvious physical or statistical significance, so that the first feature image can be obtained; or it can be through a convolutional network, such as a residual network (Residual Networks, ResNet), Visual Geometry Group (VGG) network and other methods extract high-dimensional image features from the image to be processed to obtain the first feature image.

Among them, in some embodiments, when feature extraction is performed on the image to be processed, such as Haar-like features (Haar), Local Binary Pattern (LBP), and LBP can be extracted from the image to be processed. Histogram of Oriented Gradient (HOG) and other features. The Haar feature describes the light-dark transformation information of the pixel value in the local area of the image, LBP describes the texture information corresponding to the image in the local area, and HOG describes the edge gradient information of the image in the local area. . Or, in other embodiments, when performing feature extraction on the image to be processed, high-dimensional visual features of the image to be processed can be extracted.

In step 102, multiple contextual features with different ranges are synchronously extracted from the first feature image to obtain multiple second feature images.

In this embodiment of the disclosure, the context feature extraction is to perform statistics on the distribution of other pixels in the neighborhood of the pixel in the first feature image.

Contextual feature extraction with different ranges refers to contextual feature extraction with different pixel numbers. For example, when contextual feature extraction is performed on the first feature image, the multiple pixel intervals included in the first feature image can be synchronized ( For example, 3 pixels, 7 pixels, 12 pixels intervals) perform context feature extraction to obtain multiple second feature images respectively.

In step 103, a target image is determined according to at least the plurality of second characteristic images, and the target image is used as the new first characteristic image to synchronously extract a plurality of contextual features with different ranges.

In the embodiment of the present disclosure, the target image is an image obtained at least according to a plurality of second characteristic images each time. After the target image is determined, the target image can be used as a new first characteristic image, and step 102 is executed again.

In step 104, in response to the number of times of synchronously extracting a plurality of contextual features in different ranges from the first feature image reaching a target number of times, based on the target image obtained last time, a corresponding image corresponding to the to-be-processed image is generated. Semantic image.

In the embodiment of the present disclosure, the target number of times may be a positive integer greater than or equal to 2.

In the above embodiment, feature extraction can be performed on the acquired image to be processed to obtain a first feature image, and then multiple context features with different ranges can be simultaneously extracted from the first feature image to obtain multiple second feature maps picture. At least according to the multiple second feature images, the target image is determined, and the target image is used as the new first feature image, and multiple context features with different ranges are synchronously extracted again. When the number of times of synchronously extracting a plurality of context features with different ranges from the first feature image reaches the target number of times, semantic segmentation may be performed based on the target image obtained last time to generate a semantic image corresponding to the image to be processed. In the disclosed embodiment, multiple contextual features with different ranges are simultaneously extracted through feature images corresponding to images to be processed multiple times, so that context information of different scales can be fully integrated, and the accuracy of semantic segmentation is improved.

In some optional embodiments, for step 101, a feature extraction network may be used to input the collected image to be processed into the feature extraction network, so that the feature extraction network outputs the first feature image. Among them, the feature extraction network can be a neural network that can perform feature extraction, such as Resnet and VGG.

In some optional embodiments, such as shown in FIG. 3, step 102 may include:

In step 102-1, the first feature image is divided into multiple channels to simultaneously perform dimensionality reduction processing to obtain multiple third feature images.

In the embodiment of the disclosure, the dimensionality reduction processing on the first feature image is for better context feature extraction in the subsequent process, which is beneficial to reduce the calculation amount of subsequent processing. The first feature image is divided into multiple channels to perform dimensionality reduction processing, and then the dimensionality reduction processed images corresponding to multiple channels can be used to extract contextual features with different ranges, which is conducive to improving the accuracy of semantic segmentation and reducing The amount of calculation in the semantic segmentation process.

In the disclosed embodiment, the first feature image can be divided into multiple channels to simultaneously perform dimensionality reduction processing of the same dimension. For example, as shown in Figure 4, the convolutional layer of the 1×1 convolution kernel is used for multi-channel dimensionality reduction processing. The dimensions of the plurality of third feature images obtained later may be 1×1×256 dimensions.

In step 102-2, extracting context features with different ranges from at least two third feature images in the plurality of third feature images, to obtain a plurality of second feature images.

In the disclosed embodiment, a hole convolution with a depth separable convolution and a convolution kernel corresponding to different hole coefficients can be used to extract contexts with different ranges for at least two third characteristic images among the plurality of third characteristic images. Features, obtaining the multiple second feature images, achieves the purpose of extracting multiple context features with different ranges from the first feature images simultaneously, which is beneficial to improving the accuracy of semantic segmentation. Among them, the hole convolution can choose a convolution kernel with a size of 3×3, or a convolution kernel with a size of 5×5 or 7×7. The size of the convolution kernel of the hole convolution is not limited in the embodiment of this disclosure. . Among them, the hole coefficient r of the hole convolution can be set to different values according to the semantic segmentation scene, for example, r can be set to 6, 12, 18, 32, etc., according to the value of r, different numbers of pixels can be separated for contextual features extract.

For example, as shown in Figure 4, after performing dimensionality reduction processing on the first feature image with 4 channels, 4 third feature images are obtained, which are respectively denoted as third feature image 1 to third feature image 4; The third feature image 1 may not be subjected to context feature extraction, and the values of the hole coefficient r corresponding to the third feature images 2, 3, and 4 are 6, 12, and 18 respectively, that is, for the third feature images 2, 3, and 4. 4 At intervals of 6 pixels, 12 pixels, and 18 pixels, respectively, extract the context feature to obtain three second feature images.

In the foregoing embodiment, the first feature image may be divided into multiple channels to perform dimensionality reduction processing to obtain multiple third feature images, and then at least two third feature images of the multiple third feature images Extracting contextual features with different ranges to obtain at least two corresponding second feature images. The purpose of synchronously extracting a plurality of contextual features with different ranges from the first feature image is realized, which is beneficial to improve the accuracy of semantic segmentation and reduces the amount of calculation in the semantic segmentation process.

In some optional embodiments, for example, as shown in Figure 5, the process of determining the target image in step 103 at least based on the plurality of second characteristic images may include:

In step 103-1, at least the plurality of second characteristic images are merged to obtain a fourth characteristic image.

In this embodiment of the disclosure, at least multiple second characteristic images obtained in the above steps may be superimposed to obtain a fourth characteristic image.

For example, a plurality of second feature images are stacked together, and then a convolution operation is used to realize the fusion of multi-scale context features to obtain a fourth feature image. It is also possible to stitch a plurality of second characteristic images to obtain a fourth characteristic image.

In step 103-2, the target image is determined based on at least the fourth characteristic image.

In a possible implementation manner, the fourth characteristic image can be directly used as the target image. In another possible implementation manner, processing that can improve the semantic segmentation effect may be performed on the fourth feature image, so as to obtain the target image. In another possible implementation manner, the target image may also be determined according to the fourth characteristic image and other characteristic images associated with the image to be processed.

In the foregoing embodiment, at least the target image can be determined based on a plurality of second characteristic images, which is highly usable.

In some optional embodiments, for step 103-1, in a possible implementation manner, multiple second characteristic images may be superimposed to obtain a fourth characteristic image. In order to better retain the feature information corresponding to the image to be processed and improve the accuracy of semantic segmentation, in another possible implementation manner, multiple second feature images and multiple third feature images can also be combined The at least one third characteristic image of is superimposed, and the superimposed image is used as the fourth characteristic image.

The multiple third feature images are the images obtained after the first feature image is divided into multiple channels to perform dimensionality reduction processing. In the disclosed embodiment, multiple second feature images and multiple third feature images can be combined. At least one third feature image in the image is superimposed, and then a convolution operation is used to realize the fusion of multi-scale context features to obtain a fourth feature image. It is also possible to stitch together at least one third characteristic image among the plurality of second characteristic images and the plurality of third characteristic images to obtain a fourth characteristic image.

In the foregoing embodiment, multiple second feature images can be directly superimposed to obtain a fourth feature image, or multiple second feature images and multiple third feature images that have not been subjected to context feature extraction can also be combined. At least one of the third feature images is superimposed to obtain the fourth feature image, which has high usability, can fuse information of more scales, and improves the accuracy of semantic segmentation.

In some optional embodiments, for step 103-2, any one of the following methods may be used to determine the target image.

In a possible implementation manner, the determining the target image at least according to the fourth characteristic image includes: up-sampling the fourth characteristic image to obtain the target image.

In the embodiment of the present disclosure, since the target image needs to be subjected to dimensionality reduction processing or to generate a semantic image subsequently, in order to maintain the dimensionality of the target image, the fourth characteristic image needs to be up-sampled. After the fourth feature image is determined, the up-sampling process (for example, linear interpolation) is directly performed on the fourth feature image, so as to obtain the target image. Then, the target image is taken as the new first characteristic image, and step 102 is returned to.

When the fourth feature image is up-sampling, the corresponding up-sampling factor t can be 2, 4, 8, etc. Each time the fourth feature image is up-sampled, the same or different up-sampling can be used. Sampling factor. Among them, the up-sampling factor is the number of new pixels inserted between the pixels using a suitable interpolation algorithm when the original image is enlarged. For example, when the up-sampling factor t is 2, it can be set between two adjacent pixels. A linear interpolation algorithm is used to insert 2 new pixels.

In another possible implementation manner, the determining the target image at least according to the fourth characteristic image includes: performing sub-pixel convolution on the fourth characteristic image to obtain the target image .

Sub-pixel convolution tiling the pixels in the depth direction of the output feature map makes the depth of the feature map smaller and the spatial scale of the two-dimensional plane larger, thereby improving the spatial resolution of the feature map.

By performing sub-pixel convolution on the fourth feature image, the effect of semantic segmentation can be improved, and the result of semantic segmentation can be more accurate. After the sub-pixel convolution processing, an up-sampling process can also be performed, the target image is obtained through the up-sampling process, and then the target image is used as a new first characteristic image, and step 102 is returned to.

In another possible implementation manner, considering that the first feature image was previously processed for dimensionality reduction, the subsequent images are all based on multiple third feature images after the dimensionality reduction process, but the final generation The semantic image is the same high-dimensional image as the image to be processed. In order to reduce the possibility of losing some important features in the image to be processed after dimensionality reduction processing, the accuracy of semantic segmentation can be improved. Before determining the target image, a fifth characteristic image is acquired.

Among them, the fifth feature image is an image obtained by extracting low-dimensional image features from the image to be processed. The number of feature extraction layers corresponding to the fifth feature image is smaller than the number of feature extraction layers corresponding to the first feature image. For example, if 10 layers of feature extraction are performed on the image to be processed to obtain the first feature image, then the image obtained after feature extraction of the first 4 layers can be used as the fifth feature image.

Correspondingly, for example, as shown in Figure 6, the above method may further include:

In step 105, after performing feature extraction and dimensionality reduction processing on the image to be processed, a fifth feature image is obtained.

In the embodiment of the present disclosure, the fourth characteristic image and the fifth characteristic image may be superimposed and then up-sampling processing may be performed to obtain the target image.

Wherein, in the case that the number of times of synchronously extracting a plurality of context features with different ranges from the first feature image is less than the target number of times, step 103-2 may superimpose the fourth feature image and the fifth feature image and then perform up-sampling. To obtain the target image, when the number of times of synchronously extracting a plurality of context features with different ranges from the first characteristic image reaches the target number of times, only the fourth characteristic image may be up-sampled to obtain the target image. The target image is taken as the new first characteristic image, and step 102 is returned to. The corresponding upsampling factor t during each upsampling process can be the same or different.

In another possible implementation manner, the target image can also be determined based on the fourth characteristic image and the fifth characteristic image.

In the case that the number of times of synchronously extracting a plurality of context features with different ranges from the first feature image is less than the target number of times, the image obtained after sub-pixel convolution on the fourth feature image is the same as the fifth feature image Image superimposition to obtain the target image. When the number of times of synchronously extracting a plurality of context features with different ranges from the first feature image reaches the target number of times, sub-pixel convolution is directly performed on the fourth feature image to obtain the target image.

In the embodiment of the present disclosure, in order to ensure the effect of semantic segmentation, it is also possible to perform subtraction on the fourth feature image when the number of times of synchronously extracting multiple context features in different ranges from the first feature image is less than the target number of times. Pixel convolution, superimpose the obtained image with the fifth characteristic image to obtain the target image. If the number of times reaches the target number of times, the fourth feature image can be directly subjected to sub-pixel convolution to obtain the target image. Wherein, after sub-pixel convolution is performed on the fourth feature image, up-sampling may also be performed. Then, the target image is taken as the new first characteristic image, and step 102 is returned to.

It should be noted that each time the target image is determined, the target image is used as a new first feature image to resynchronize and extract multiple contextual features with different ranges, and the new first feature image is divided into multiple channels The dimensions of the new multiple third feature images obtained when the dimensionality reduction processing is performed synchronously may be the same or different from the dimensions of the multiple third feature images obtained after the previous dimensionality reduction processing is performed. For example, the last time the first feature image was divided into multiple channels and the dimensionality reduction process was performed to obtain multiple third feature images of 1×1×256 dimensions, and the new first feature image was divided into multiple channels for synchronization. After the dimensionality reduction process, a plurality of new third feature images with 1×1×128 dimensions can be obtained.

In addition, the hole coefficients when the hole convolution is performed on the plurality of third feature images each time may also be the same or different. For example, when the hole convolution was performed on at least two of the multiple third feature images last time, the corresponding hole coefficients may be 6, 12, and 18, respectively. For at least two of the new multiple third feature images, When a hole convolution is performed, the corresponding hole coefficients can be 6 and 12 respectively.

In the above-mentioned embodiment, at least one target image can be determined based on the fourth characteristic image, so as to ensure the precision and accuracy of semantic segmentation and high usability.

In some optional embodiments, in order to ensure that the dimension of the finally obtained semantic image is consistent with the dimension of the image to be processed, dimensionality reduction and/or dimensionality can be performed before outputting the target image, so as to ensure that the target image corresponds to the The dimension is the target dimension. Wherein, the target dimension is determined according to the preset total number of object categories included in the semantic image.

For example, the target dimension may be 1×1×16N, where N is the preset total number of object categories included in the semantic image. If 4 types of object categories need to be analyzed in the semantic image, the target dimension can be 1×1×64.

In the above-mentioned embodiment, the dimension corresponding to the target image obtained last time may be dimensionality reduction and/or dimension increase processing (for example, a convolutional layer with a preset number of channels is used for convolution operation) before the target image is output. Ensure that the dimensions of the target image are associated with the target dimensions, which improves the accuracy and precision of semantic segmentation.

In some optional embodiments, for step 104, after obtaining the target image for the last time, an interpolation algorithm may be used to generate the semantic image, and the interpolation algorithm may include, but is not limited to, a bilinear interpolation algorithm.

The above embodiment is further illustrated as follows. For example, as shown in Figure 7, the collected image to be processed (such as the real-time image shown in the figure) can be input into a fully convolutional neural network, and the fully convolution The neural network outputs the corresponding semantic image.

The fully convolutional neural network may include a front terminal network and a rear terminal network.

The front terminal network can be a feature extraction network, and neural networks such as Resnet and VGG can be used.

In the process of training the front terminal network, manually labeled image classification sample data sets, such as ImageNet, can be used. The ImageNet collection includes images and corresponding image feature tags. By adjusting the network parameters of the front terminal network, the output result of the front terminal network can match the label content in the ImageNet sample collection or be within the tolerance range.

The first characteristic image corresponding to the image to be processed can be obtained through the front terminal network, and the first characteristic image is further input into the rear terminal network to obtain the semantic image output by the rear terminal network.

When training the back-terminal network, you can use artificially annotated image semantic segmentation sample collections, such as CityScapes, and train the network parameters of the entire neural network through the backward propagation algorithm, including the front-terminal network and the back-terminal network The network parameters of the road, so that the output result of the back terminal network matches the label content in the CityScapes sample set or is within the fault tolerance range.

In order to facilitate the introduction of the network architecture adopted by the rear terminal network, in the embodiment of this disclosure, only the target number of times is 2 for illustration. It should be noted that any positive integer value greater than 2 belongs to the disclosure of the target number of times. protected range.

In a possible implementation, the network architecture of the back terminal network can be as shown in Figure 8A.

Through the subnet 1, the first feature image is divided into multiple channels to perform dimensionality reduction processing simultaneously to obtain multiple third feature images. Then, extracting context features with different ranges for at least two third feature images among the plurality of third feature images can be obtained through hole convolution with depth separable convolution and the convolution kernel corresponding to different hole coefficients to obtain multiple first feature images. Two feature images.

Further, it is possible to superimpose multiple second feature images and then perform upsampling processing (the upsampling process is not shown in Figure 8A) to obtain the target image, or combine multiple first images with no context feature The at least one extracted third characteristic image is superimposed and then subjected to up-sampling processing to obtain the target image.

The target image is directly used as the new first feature image, and through the subnet 2, the new first feature image is divided into multiple channels to perform dimensionality reduction processing simultaneously, and multiple third feature images are obtained again. Then extract the context features with different ranges for at least two third feature images in the plurality of third feature images, for example, the depth separable convolution can be performed and the convolution kernel corresponds to the hole convolution with different hole coefficients to obtain multiple The second feature image. The multiple second feature images are superimposed again and then up-sampling is performed to obtain the target image. Alternatively, multiple first images and at least one third feature image that has not been subjected to context feature extraction can be superimposed and then up-sampling is performed. Get the target image.

A bilinear interpolation algorithm is used for the target image output by the subnet 2 to generate the semantic image.

In the foregoing embodiment, multiple contextual feature extraction and fusion of different ranges can be synchronously extracted from the first feature image in multiple times, so that context information of different scales is fully integrated, and the accuracy of semantic segmentation is improved. And because the depth of the separable hole convolution is used, the amount of calculation in the semantic segmentation process is reduced.

In another possible implementation, the network architecture of the back terminal network can be as shown in Figure 8B.

Through the subnet 1, the first feature image is divided into multiple channels to perform dimensionality reduction processing simultaneously to obtain multiple third feature images. Then extract the context features with different ranges for at least two third feature images in the plurality of third feature images, for example, a depth-separable hole convolution operation can be performed, and the hole coefficients are different from each other to obtain multiple second features image.

In order to improve the effect of semantic segmentation, multiple second feature images can be superimposed and then subjected to sub-pixel convolution and up-sampling processing to obtain the target image. Alternatively, multiple second feature images and at least one third feature image that has not been subjected to context feature extraction can be superimposed to perform sub-pixel convolution and up-sampling processing (the up-sampling process is not shown in Figure 8B) to obtain the target image.

The target image is directly used as the new first feature image, and through the subnet 2, the new first feature image is divided into multiple channels to perform dimensionality reduction processing simultaneously, and multiple third feature images are obtained again. Then extract the context features with different ranges for at least two third feature images in the plurality of third feature images, for example, a depth-separable hole convolution operation can be performed, and the hole coefficients are different from each other to obtain multiple second features image. Superimpose multiple second feature images again to perform sub-pixel convolution and up-sampling processing to obtain the target image, or superimpose multiple first images with at least one third feature image that has not been subjected to context feature extraction Then, sub-pixel convolution and up-sampling are performed to obtain the target image.

A bilinear interpolation algorithm is used for the target image output by the subnet 2 to generate the semantic image.

In the foregoing embodiment, multiple contextual feature extraction and fusion of different ranges can be synchronously extracted from the first feature image in multiple times, so that context information of different scales is fully integrated, and the accuracy of semantic segmentation is improved. And because the depth of the separable hole convolution is used, the amount of calculation in the semantic segmentation process is reduced. In addition, the effect of semantic segmentation can also be improved through sub-pixel convolution.

In another possible implementation, the network architecture of the back terminal network can be as shown in Figure 8C.

Through the subnet 1, the first feature image is divided into multiple channels to perform dimensionality reduction processing simultaneously to obtain multiple third feature images. Then, for at least two contextual features with different extraction ranges in the plurality of third feature images, for example, a depth-separable hole convolution operation may be performed, and the hole coefficients are different from each other to obtain a plurality of second feature images.

Further, multiple second feature images and at least one third feature image that has not been subjected to context feature extraction can be superimposed, and then superimposed with the fifth feature image, and the superimposed image can be upsampled ( The up-sampling process is not shown in Figure 8C) to obtain the target image. Wherein, the number of feature extraction layers corresponding to the fifth feature image is smaller than the number of feature extraction layers corresponding to the first feature image.

The target image is directly used as the new first feature image, and through the subnet 2, the new first feature image is divided into multiple channels to perform dimensionality reduction processing simultaneously, and multiple third feature images are obtained again. Then extract the context features with different ranges for at least two third feature images in the plurality of third feature images, for example, a depth-separable hole convolution operation can be performed, and the hole coefficients are different from each other to obtain multiple second features image. After superimposing a plurality of second feature images and at least one third feature image that has not been subjected to context feature extraction, up-sampling processing is performed on the superimposed images to obtain a target image.

A bilinear interpolation algorithm is used for the target image output by the subnet 2 to generate the semantic image.

In another possible implementation manner, the network architecture of the back terminal network can be as shown in Figure 8D.

Through the subnet 1, the first feature image is divided into multiple channels to perform dimensionality reduction processing simultaneously to obtain multiple third feature images. Then extract the context features with different ranges for at least two third feature images in the plurality of third feature images, for example, a depth-separable hole convolution operation can be performed, and the hole coefficients are different from each other to obtain multiple second features image.

Further, after superimposing multiple second feature images and at least one third feature image that has not been subjected to context feature extraction, sub-pixel convolution and up-sampling processing can be performed (the up-sampling process is not shown in Figure 8D), Then it is superimposed with the fifth characteristic image to obtain the target image. Wherein, the number of feature extraction layers corresponding to the fifth feature image is smaller than the number of feature extraction layers corresponding to the first feature image.

The target image is directly used as the new first feature image, and through the subnet 2, the new first feature image is divided into multiple channels to perform dimensionality reduction processing simultaneously, and multiple third feature images are obtained again. Then, for at least two contextual features with different extraction ranges in the plurality of third feature images, for example, a depth-separable hole convolution operation may be performed, and the hole coefficients are different from each other to obtain a plurality of second feature images. After superimposing a plurality of second feature images and at least one third feature image that has not been subjected to context feature extraction, sub-pixel convolution and up-sampling processing are performed on the superimposed images to obtain a target image.

In addition, in order to ensure that the dimension of the target image is the target dimension, after superimposing a plurality of second feature images and at least one third feature image that has not been subjected to context feature extraction, the superimposed image may be subjected to dimensionality reduction processing. And the dimension upscaling process, then the sub-pixel convolution and up-sampling process are performed to obtain the target image.

A bilinear interpolation algorithm is used for the target image output by the subnet 2 to generate the semantic image.

In the foregoing embodiment, multiple contextual feature extraction and fusion of different ranges can be synchronously extracted from the first feature image in multiple times, so that context information of different scales is fully integrated, and the accuracy of semantic segmentation is improved. Due to the use of depth-separable hollow convolution, the amount of calculation in the semantic segmentation process is reduced. In addition, using the fifth feature image to determine the target image can ensure that important information in the image to be detected will not be lost, and the effect of semantic segmentation is also improved.

In some optional embodiments, such as shown in Figure 9, after step 104 is completed, the method may further include: in step 106, machine device navigation is performed according to the semantic image.

In the embodiment of the present disclosure, the machine equipment can be navigated according to the generated semantic image. For example, if obstacles are included in the semantic image, navigation to avoid obstacles can be performed. If the semantic image includes a fork in the road, it can be determined whether it is necessary to go straight or turn according to the designated route.

In the foregoing embodiment, machine device navigation can be performed according to the generated semantic image corresponding to the image to be processed, which has high usability.

Corresponding to the foregoing method embodiments, the present disclosure also provides device embodiments.

As shown in Figure 10, Figure 10 is a block diagram of an image semantic segmentation device according to an exemplary embodiment of the present disclosure. The device includes: a feature extraction module 210 configured to perform processing on the acquired image to be processed Feature extraction to obtain a first feature image; a context feature extraction module 220 configured to simultaneously extract a plurality of context features with different ranges from the first feature image to obtain a plurality of second feature images; a determination module 230 , Configured to determine a target image at least according to the plurality of second characteristic images, and use the target image as the new first characteristic image to synchronously extract a plurality of contextual features with different ranges; semantic map The image generation module 240 is configured to generate the to-be-processed image based on the target image obtained the last time in response to the number of times of synchronously extracting a plurality of contextual features in different ranges from the first characteristic image reaching a target number of times Like the corresponding semantic image.

In some optional embodiments, the context feature extraction module 220 includes: a first processing sub-module configured to simultaneously perform dimensionality reduction processing on the first feature image in multiple channels to obtain multiple third A feature image; a second processing sub-module configured to extract context features with different ranges from at least two third feature images in the plurality of third feature images to obtain a plurality of second feature images.

In some optional embodiments, the second processing sub-module is configured to adopt a hole convolution with a depth separable convolution and a convolution kernel corresponding to different hole coefficients, and perform the At least two third feature images extract context features with different ranges to obtain the multiple second feature images.

In some optional embodiments, the determining module includes: a first determining sub-module configured to fuse at least the plurality of second characteristic images to obtain a fourth characteristic image; and a second determining sub-module The group is configured to determine the target image at least according to the fourth characteristic image.

In some optional embodiments, the first determining sub-module is configured to superimpose the plurality of second characteristic images to obtain the fourth characteristic image; or, for the plurality of second characteristic images The characteristic image and at least one third characteristic image of the plurality of third characteristic images are superimposed to obtain the fourth characteristic image.

In some optional embodiments, the second determining sub-module is configured to perform up-sampling on the fourth characteristic image to obtain the target image; or, perform sub-modules on the fourth characteristic image. Pixel convolution to obtain the target image.

In some optional embodiments, the device further includes: a processing module configured to perform feature extraction and dimensionality reduction processing on the image to be processed to obtain a fifth feature image; wherein, the fifth feature The number of feature extraction layers corresponding to the image is less than the number of feature extraction layers corresponding to the first feature image; the second determining sub-module is configured to: if the number of times is less than the target number of times, The fourth characteristic image and the fifth characteristic image are superimposed and then up-sampling is performed to obtain the target image; or, when the number of times is less than the target number of times, the fourth characteristic image is The image obtained after sub-pixel convolution is superimposed on the fifth characteristic image to obtain the target image.

In some optional embodiments, the dimension corresponding to the target image obtained last time is a target dimension; wherein, the target dimension is based on a preset total number of object categories included in the semantic image The project is determined.

In some optional embodiments, the device further includes a navigation module configured to perform machine equipment navigation according to the semantic image.

For the device embodiment, since it basically corresponds to the method embodiment, the relevant part can refer to the part of the description of the method embodiment. The device embodiments described above are merely illustrative, where the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place. , Or it can be distributed to multiple network units. Some or all of the modules can be selected according to actual needs to achieve the purpose of the disclosed solution. Those of ordinary skill in the art can understand and implement it without making progressive labor.

The embodiment of the present disclosure also provides a computer-readable storage medium, and the storage medium stores a computer program, and the computer program is used to execute any one of the aforementioned image semantic segmentation methods.

The embodiment of the present disclosure also provides a computer program that enables a computer to execute the image semantic segmentation method described in any one of the above.

In some optional embodiments, the embodiments of the present disclosure provide a computer program product, including computer-readable code. When the computer-readable code runs on the device, the processor in the device executes to implement any of the above embodiments. Provide instructions for image semantic segmentation methods.

In some optional embodiments, the disclosed embodiments also provide another computer program product for storing computer-readable instructions, which when executed, cause the computer to perform the operations of the image semantic segmentation method provided by any of the above-mentioned embodiments .

The computer program product can be implemented through hardware, software, or a combination thereof. In an optional embodiment, the computer program product is specifically embodied as a computer storage medium. In another optional embodiment, the computer program product is specifically embodied as a software product, such as a software development kit (SDK), etc. Wait.

The embodiment of the present disclosure also provides an image semantic segmentation device, including: a processor; a memory for storing executable instructions of the processor; wherein the processor is configured to call the executable instructions stored in the memory, Realize any one of the image semantic segmentation methods described above.

Figure 11 is a schematic diagram of the hardware structure of an image semantic segmentation device provided by an embodiment of the application. The image semantic segmentation device 310 includes a processor 311, and may also include an input device 312, an output device 313, and a memory 314. The input device 312, the output device 313, the memory 314 and the processor 311 are connected to each other through a bus.

Memory includes but is not limited to Random Access Memory (RAM), Read-Only Memory (ROM), Erasable Programmable Read Only Memory (EPROM) ), or portable CD-ROM (Compact Disc Read-Only Memory, CD-ROM), which is used for related commands and data.

The input device is used to input data and/or signals, and the output device is used to output data and/or signals. The output device and the input device can be independent devices or a whole device.

The processor may include one or more processors, for example, including one or more central processing units (CPU). In the case that the processor is a CPU, the CPU may be a single-core CPU or Multi-core CPU.

The memory is used to store the code and data of the network equipment.

The processor is used to call the program code and data in the memory to execute the steps in the above method embodiment. For details, please refer to the description in the method embodiment, which will not be repeated here.

It is understandable that Figure 11 only shows a simplified design of an image semantic segmentation device. In practical applications, the image semantic segmentation device may also include other necessary components, including but not limited to any number of input/output devices, processors, controllers, memory, etc., and all the devices that can implement the embodiments of the present application The image semantic segmentation devices are all within the protection scope of the embodiment of the disclosure.

In some embodiments, the functions or modules contained in the device provided in the embodiments of the present disclosure can be used to execute the methods described in the above method embodiments. For specific implementation, please refer to the description of the above method embodiments. For brevity, I won't repeat it here.

After considering the specification and practicing the invention disclosed herein, those skilled in the art will easily think of other implementation schemes of the embodiments of the present disclosure. The embodiments of the present disclosure are intended to cover any modifications, uses, or adaptive changes of the present disclosure. These modifications, uses, or adaptive changes follow the general principles of the embodiments of the present disclosure and include those in the technical field that are not disclosed in the embodiments of the present disclosure. Common knowledge or conventional technical means. The description and the embodiments are only regarded as exemplary, and the true scope and spirit of the embodiments of the present disclosure are pointed out by the following invention patent application scope.

The foregoing descriptions are only optional embodiments of the present disclosure, and are not intended to limit the embodiments of the present disclosure. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the embodiments of the present disclosure shall be applicable. It is included in the protection scope of the embodiments of the present disclosure.

101,102,102-1,102-2,103,103-1,103-2,104,105,106: steps 210: Feature Extraction Module 220: The following feature extraction module 230: Determine the module 240: Semantic Image Generation Module 310: Image Semantic Segmentation Device 311: processor 312: input device 313: output device 314: Memory

Claims (11)

A method for image semantic segmentation, including: Perform feature extraction on the acquired image to be processed to obtain a first feature image; Synchronously extracting a plurality of contextual features with different ranges from the first feature image to obtain a plurality of second feature images; Determining a target image at least according to the plurality of second characteristic images, and using the target image as the new first characteristic image to synchronously extract a plurality of contextual features with different ranges; In response to the number of times of synchronously extracting a plurality of contextual features with different ranges from the first feature image reaching a target number of times, a semantic image corresponding to the image to be processed is generated based on the target image obtained last time. The method according to claim 1, wherein the synchronously extracting a plurality of contextual features with different ranges from the first characteristic image to obtain a plurality of second characteristic images includes: Perform dimensionality reduction processing on the first feature image in multiple channels simultaneously to obtain multiple third feature images; and A plurality of second feature images are obtained by extracting context features with different ranges from at least two third feature images in the plurality of third feature images. The method according to claim 2, wherein the extracting context features with different ranges from at least two third feature images of the plurality of third feature images to obtain multiple second feature images includes : The depth separable convolution is adopted and the convolution kernel corresponds to the hole convolution with different hole coefficients, and context features with different ranges are extracted from at least two third feature images in the plurality of third feature images to obtain the multiple A second feature image. The method according to any one of claim items 1-3, wherein the determining a target image at least according to the plurality of second characteristic images includes: Fusing at least the plurality of second characteristic images to obtain a fourth characteristic image; and The target image is determined based on at least the fourth characteristic image. The method according to claim 4, wherein the fusing at least the plurality of second characteristic images to obtain a fourth characteristic image includes: Superimposing the plurality of second characteristic images to obtain the fourth characteristic image; or, At least one third characteristic image of the plurality of second characteristic images and the plurality of third characteristic images is superimposed to obtain the fourth characteristic image. The method according to claim 4, wherein the determining the target image according to at least the fourth characteristic image includes: Up-sampling the fourth characteristic image to obtain the target image; or, Performing sub-pixel convolution on the fourth characteristic image to obtain the target image. The method according to claim 4, wherein the method further includes: After performing feature extraction and dimensionality reduction processing on the image to be processed, a fifth feature image is obtained; wherein the number of feature extraction layers corresponding to the fifth feature image is smaller than the number of features corresponding to the first feature image The number of layers extracted; The determining the target image at least according to the fourth characteristic image includes: In the case that the number of times is less than the target number of times, the fourth characteristic image and the fifth characteristic image are superimposed and then up-sampling is performed to obtain the target image; or, In the case that the number of times is less than the target number of times, an image obtained after sub-pixel convolution of the fourth characteristic image is superimposed on the fifth characteristic image to obtain the target image. The method according to any one of claim items 1-3, wherein the dimension corresponding to the target image obtained last time is a target dimension; wherein the target dimension is a preset semantic image The total number of object categories included in it is determined. The method according to any one of claim items 1-3, wherein after the generating the semantic image corresponding to the image to be processed, the method further includes: Navigate the machine and equipment according to the semantic image. A computer-readable storage medium, the storage medium stores a computer program, and the computer program is used to execute the image semantic segmentation method described in any one of the above-mentioned request items 1-9. An image semantic segmentation device, including: processor; A memory for storing executable instructions of the processor; Wherein, the processor is configured to call executable instructions stored in the memory to implement the image semantic segmentation method described in any one of request items 1-9.

Images