CN115908459A - Image segmentation method and device, computer equipment and readable storage medium - Google Patents
Image segmentation method and device, computer equipment and readable storage medium Download PDFInfo
- Publication number
- CN115908459A CN115908459A CN202310229248.4A CN202310229248A CN115908459A CN 115908459 A CN115908459 A CN 115908459A CN 202310229248 A CN202310229248 A CN 202310229248A CN 115908459 A CN115908459 A CN 115908459A
- Authority
- CN
- China
- Prior art keywords
- feature
- feature map
- resolution
- image
- target feature
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Landscapes
- Image Analysis (AREA)
Abstract
The application provides an image segmentation method, an image segmentation device, computer equipment and a readable storage medium, and relates to the field of industrial visual inspection. The method comprises the following steps: acquiring an image to be segmented; carrying out neural network convolution processing on an image to be segmented to obtain N characteristic graphs with different resolutions, wherein the characteristic depths of the N characteristic graphs are sequentially increased; acquiring a first target characteristic diagram corresponding to a central area of the N characteristic diagrams, and acquiring a second target characteristic diagram according to the last M characteristic diagrams in the N characteristic diagrams; adjusting the resolutions of the first target feature map and the second target feature map to a first preset resolution; performing fusion convolution processing on the first target characteristic diagram and the second target characteristic diagram to obtain a segmentation result; n and M are positive integers larger than 1, and N is larger than M. According to the image segmentation method and device, the problem of missing detection of the edge area of the image is solved, and meanwhile the image segmentation precision is improved.
Description
Technical Field
The present application relates to the field of industrial vision inspection, and in particular, to an image segmentation method, an image segmentation apparatus, a computer device, and a readable storage medium.
Background
At present, the appearance quality detection of industrial products is an indispensable link in the production process of the products, is used for positioning and removing products with flaws in appearance, and is very important for the control and upgrade of the product quality and the maintenance of brand images. In order to realize an efficient and stable appearance quality inspection link, the detection of the surface defects of the automatic products is realized through industrial machine vision. Image segmentation is a commonly used defect region detection model, which outputs a segmentation result of the same size by analyzing the attribute characteristics of an input image, indicating the position where a defect may exist. In order to ensure a high-quality defect area detection effect, an original image is generally divided into a series of image blocks at equal intervals, and then the image blocks are analyzed, and then a detection result at an original resolution is obtained by image stitching. According to the difference of the resolution of the segmentation result in the detection process, the image segmentation model for defect detection in the related art can be roughly divided into the following two schemes:
(1) A resolution preserving segmentation model, i.e. the resolution of the output segmentation result is the same as the input image, is one of the most common segmentation models. In the detection process of such a model, the original image is generally cut and processed separately in a manner with a certain overlap ratio, as shown in fig. 1, wherein the arrow represents the interval between two cuts.
(2) The reduced resolution segmentation model, i.e. for the input image, results in a smaller resolution segmentation. The segmentation of such a model is shown in fig. 2, where the arrows represent the interval between two cropping, the thin solid lines represent the image area of each cropping, and the thick solid lines represent the corresponding segmentation result.
The detection model according to the above-mentioned means (1) focuses more on the central region of the image, but neglects the image characteristics of the edge position, and thus the detection capability of the model for the object falling at the edge position is insufficient, and there is a problem that the detection quality of the edge region is poor. The model corresponding to the scheme (2) has a complex structure adjusting process and is difficult to flexibly apply, more segmentation operations are adopted in the forward transmission process of the characteristic diagram, a large amount of information in the characteristic diagram is removed, and the detection precision of the model is limited to a certain extent.
Disclosure of Invention
In view of this, the present application provides an image segmentation method, an image segmentation apparatus, a computer device, and a readable storage medium, which improve the image segmentation accuracy while solving the problem of missing detection in the image edge region.
In a first aspect, an embodiment of the present application provides an image segmentation method, including:
acquiring an image to be segmented, and performing neural network convolution processing on the image to be segmented to obtain N feature maps with different resolutions, wherein the feature depths of the N feature maps are sequentially increased;
acquiring a first target feature map corresponding to the central area of the N feature maps, and acquiring a second target feature map according to the last M feature maps in the N feature maps;
adjusting the resolutions of the first target feature map and the second target feature map to a first preset resolution;
performing fusion convolution processing on the first target characteristic diagram and the second target characteristic diagram to obtain a segmentation result;
wherein N and M are positive integers greater than 1, and N is greater than M.
According to the above method of the embodiment of the present application, the following additional technical features may also be provided:
in the above technical solution, optionally, before acquiring the image to be segmented, the method further includes: and acquiring a first preset resolution of a segmentation result set by a user and a second preset resolution of an image to be segmented.
In any of the above technical solutions, optionally, the feature map includes features of the defect region and features of a surrounding image of the defect region.
In any of the above technical solutions, optionally, the obtaining a first target feature map corresponding to a central area of the N feature maps includes: respectively carrying out feature clipping processing on the N feature graphs to obtain the central region feature of each feature graph; and performing feature fusion processing on the N central region features to obtain a first target feature map.
In any of the above technical solutions, optionally, the first target feature map has image bottom layer texture feature information; wherein the image bottom layer texture feature information comprises at least one of the following items: edge information, direction information.
In any of the above technical solutions, optionally, the performing feature clipping processing on the N feature maps respectively to obtain the central region feature of each feature map includes: calculating a first characteristic resolution of each characteristic image according to a second preset resolution of the image to be segmented, and calculating a second characteristic resolution according to the first preset resolution of the segmentation result, wherein the second preset resolution and the first preset resolution are both set by a user; and performing feature clipping processing on the feature map of each first feature resolution to obtain the feature of the central area of the second feature resolution in the central area of each feature map.
In any of the above technical solutions, optionally, the width and the height corresponding to the second preset resolution are equal, the width and the height corresponding to the first preset resolution are equal, the width and the height corresponding to the second preset resolution are equal, and the width and the height corresponding to the first preset resolution are equal;
the formula for calculating the width or height corresponding to the resolution of the first feature is as follows:
S Fi =S in /2 i+1
wherein S is Fi Width or height corresponding to resolution of the first feature, S in I =1,2,3,. N for a width or height corresponding to a second preset resolution;
the formula for calculating the width or height corresponding to the resolution of the second feature is as follows:
C Fi =S out /2 i+1
wherein, C Fi Width or height, S, corresponding to the resolution of the second feature out I =1,2, 3.. N for the width or height corresponding to the first preset resolution.
In any of the above technical solutions, optionally, performing feature fusion processing on the N central region features to obtain a first target feature map, including: scaling the N central region features; and performing feature fusion processing on the N central region features after the scaling processing to obtain a first target feature map.
In any of the above technical solutions, optionally, obtaining the second target feature map according to the last M feature maps of the N feature maps includes: performing global average pooling on the latter M feature maps in the N feature maps to obtain M pooled features, and performing feature fusion processing on the M pooled features to obtain a third target feature map; carrying out feature extraction processing on the third target feature map to obtain feature vectors with the same number as channels of the third target feature map; and multiplying the feature vector by the third target feature map to obtain a second target feature map.
In any of the above technical solutions, optionally, adjusting the resolutions of the first target feature map and the second target feature map to a first preset resolution includes: and performing up-sampling processing on the first target feature map and the second target feature map to enable the first target feature map and the second target feature map to respectively reach a first preset resolution.
In any of the above technical solutions, optionally, performing fusion convolution processing on the first target feature map and the second target feature map to obtain a segmentation result, including: performing feature fusion processing on the first target feature map and the second target feature map after the resolution is adjusted to generate a fourth target feature map; and sequentially carrying out multilayer convolution processing and maximum independent variable point set function processing on the fourth target characteristic graph to obtain a segmentation result.
In any of the above technical solutions, optionally, the acquiring an image to be segmented includes: acquiring an image of a product to be detected; and performing image extraction on the image of the product to be detected by using a preset extraction method to obtain an image to be segmented.
In a second aspect, an embodiment of the present application provides an image segmentation apparatus, including:
the first acquisition module is used for acquiring an image to be segmented;
the first processing module is used for carrying out neural network convolution processing on an image to be segmented to obtain N characteristic graphs with different resolutions, and the characteristic depths of the N characteristic graphs are sequentially increased;
the second processing module is used for acquiring a first target feature map corresponding to the central area of the N feature maps and acquiring a second target feature map according to the last M feature maps in the N feature maps;
the third processing module is used for adjusting the resolutions of the first target feature map and the second target feature map to a first preset resolution, and performing fusion convolution processing on the first target feature map and the second target feature map to obtain a segmentation result;
wherein N and M are positive integers greater than 1, and N is greater than M.
The above device according to the embodiment of the present application may further have the following additional technical features:
in the above technical solution, optionally, the apparatus further includes: and the second acquisition module is used for acquiring a first preset resolution of a segmentation result set by a user and a second preset resolution of the image to be segmented.
In any of the above technical solutions, optionally, the feature map includes features of the defect region and features of a surrounding image of the defect region.
In any of the above technical solutions, optionally, the second processing module is specifically configured to: respectively carrying out feature clipping processing on the N feature graphs to obtain the central region feature of each feature graph; and performing feature fusion processing on the N central region features to obtain a first target feature map.
In any of the above technical solutions, optionally, the first target feature map has image bottom layer texture feature information; wherein the image bottom layer texture feature information comprises at least one of the following items: edge information, direction information.
In any of the above technical solutions, optionally, the second processing module is specifically configured to: calculating a first characteristic resolution of each characteristic image according to a second preset resolution of the image to be segmented, and calculating a second characteristic resolution according to the first preset resolution of the segmentation result, wherein the second preset resolution and the first preset resolution are both set by a user; and performing feature clipping processing on the feature map of each first feature resolution to obtain the feature of the central area of the second feature resolution in the central area of each feature map.
In any of the above technical solutions, optionally, the width and the height corresponding to the second preset resolution are equal, the width and the height corresponding to the first preset resolution are equal, the width and the height corresponding to the second preset resolution are equal, and the width and the height corresponding to the first preset resolution are equal;
the formula for calculating the width or height corresponding to the resolution of the first feature is as follows:
S Fi =S in /2 i+1
wherein S is Fi Width or height, S, corresponding to the resolution of the first feature in I =1,2,3,. N for a width or height corresponding to a second preset resolution;
the formula for calculating the width or height corresponding to the resolution of the second feature is as follows:
C Fi =S out /2 i+1
wherein, C Fi Width or height, S, corresponding to the resolution of the second feature out I =1,2, 3.. N for the width or height corresponding to the first preset resolution.
In any of the above technical solutions, optionally, the second processing module is specifically configured to: scaling the N central region features; and performing feature fusion processing on the N central region features after the scaling processing to obtain a first target feature map.
In any of the above technical solutions, optionally, the second processing module is specifically configured to: performing global average pooling processing on the last M characteristic graphs in the N characteristic graphs to obtain M pooled characteristics, and performing characteristic fusion processing on the M pooled characteristics to obtain a third target characteristic graph; performing feature extraction processing on the third target feature map to obtain feature vectors with the same number as channels of the third target feature map; and multiplying the feature vector by the third target feature map to obtain a second target feature map.
In any of the above technical solutions, optionally, the third processing module is specifically configured to: and performing up-sampling processing on the first target feature map and the second target feature map to enable the first target feature map and the second target feature map to respectively reach a first preset resolution.
In any of the above technical solutions, optionally, the third processing module is specifically configured to: performing feature fusion processing on the first target feature map and the second target feature map after the resolution is adjusted to generate a fourth target feature map; and sequentially carrying out multilayer convolution processing and maximum independent variable point set function processing on the fourth target characteristic graph to obtain a segmentation result.
In any of the above technical solutions, optionally, the first obtaining module is specifically configured to: acquiring an image of a product to be detected; and performing image extraction on the image of the product to be detected by using a preset extraction method to obtain an image to be segmented.
In a third aspect, embodiments of the present application provide a computer device comprising a processor and a memory, the memory storing a program or instructions executable on the processor, the program or instructions, when executed by the processor, implementing the steps of the method as in the first aspect.
In a fourth aspect, embodiments of the present application provide a readable storage medium on which a program or instructions are stored, which when executed by a processor, implement the steps of the method according to the first aspect.
In a fifth aspect, embodiments of the present application provide a chip, where the chip includes a processor and a communication interface, where the communication interface is coupled to the processor, and the processor is configured to execute a program or instructions to implement the method according to the first aspect.
In a sixth aspect, embodiments of the present application provide a computer program product, stored on a storage medium, for execution by at least one processor to implement a method as in the first aspect.
In the embodiment of the application, an image to be segmented is obtained, and multi-layer convolution feature extraction processing is performed on the image to be segmented to obtain N layers of feature maps. The central area of the feature map obtained by fully extracting the features is segmented and fused to obtain a first target feature map with image bottom layer texture feature information, the last M feature maps in the N feature maps are determined, namely M deep feature maps in the N feature maps are determined, and then a second target feature map is obtained according to the M deep feature maps. And adjusting the resolutions of the first target feature map and the second target feature map to a first preset resolution, and performing fusion convolution processing on the first target feature map and the second target feature map to obtain a segmentation result.
According to the image segmentation method and device, segmentation of the defect area can be assisted through spatial context information, the problem of missing detection in the edge area is reduced, the defect area is accurately located, the information utilization rate is improved through the double-branch structure design of the first target feature graph and the second target feature graph, and the image segmentation precision is improved.
The foregoing description is only an overview of the technical solutions of the present application, and the present application can be implemented according to the content of the description in order to make the technical means of the present application more clearly understood, and the following detailed description of the present application is given in order to make the above and other objects, features, and advantages of the present application more clearly understandable.
Drawings
The accompanying drawings, which are included to provide a further understanding of the application and are incorporated in and constitute a part of this application, illustrate embodiment(s) of the application and together with the description serve to explain the application and not to limit the application. In the drawings:
FIG. 1 is a diagram illustrating image segmentation of a resolution-preserving segmentation model in the related art;
FIG. 2 is a diagram illustrating image segmentation of a reduced resolution segmentation model according to the related art;
FIG. 3 is a flow chart of an image segmentation method according to an embodiment of the present application;
FIG. 4 shows a schematic image segmentation diagram of an embodiment of the present application;
fig. 5 is a block diagram showing a configuration of an image segmentation apparatus according to an embodiment of the present application;
fig. 6 shows a block diagram of a computer device according to an embodiment of the present application.
Detailed Description
The technical solutions in the embodiments of the present application will be described clearly below with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are some, but not all, embodiments of the present application. All other embodiments that can be derived by one of ordinary skill in the art from the embodiments given herein are intended to be within the scope of the present disclosure.
The terms first, second and the like in the description and in the claims of the present application are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order. It will be appreciated that the data so used may be interchanged under appropriate circumstances such that embodiments of the application may be practiced in sequences other than those illustrated or described herein, and that the terms "first," "second," and the like are generally used herein in a generic sense and do not limit the number of terms, e.g., the first term can be one or more than one. In addition, "and/or" in the specification and claims means at least one of connected objects, a character "/", and generally means that the former and latter related objects are in an "or" relationship.
The image segmentation method, the image segmentation apparatus, the computer device and the readable storage medium provided by the embodiments of the present application are described in detail below with reference to the accompanying drawings through specific embodiments and application scenarios thereof.
The embodiment of the application provides an image segmentation method, which is realized by an image segmentation model based on deep learning, and is used for segmenting various target regions (namely, defect regions) from a background region in an input image, namely, providing a category for each pixel point in the input image so as to distinguish the pixel point as a background pixel or a defect pixel.
As shown in fig. 3, the image segmentation method includes:
In this step, an image to be segmented is acquired. Specifically, a high-definition camera is used for shooting a product to be detected to obtain an image of the product to be detected, and then a preset extraction method is used for extracting the image of the product to be detected to obtain an image to be segmented.
The preset extraction method is any one Of gradient calculation, numerical value smoothing, region Of Interest (ROI) selection, binarization, contour extraction, and the like, and the specific mode Of the preset extraction method is not limited in the embodiment Of the present application.
In one embodiment of the present application, the resolution of the image to be segmented and the segmentation result obtained by image segmentation is set by the user, that is, the resolution of the input image and the output result of the image segmentation model is set by the user. Specifically, before the image to be segmented is obtained, a second preset resolution of the image to be segmented set by a user and a first preset resolution of a segmentation result obtained by image segmentation are obtained. For example, the user sets the size of the image to be segmented to S in ×S in (width x height), and setting the size of the division result obtained by dividing the image to S out ×S out (width x height).
The output size of the segmentation model in the related art is strictly related to the input size, and the segmentation model is difficult to be applied to industrial visual inspection scenes needing flexible adjustment of the segmentation size. In the embodiment of the application, the user can arbitrarily specify the resolution of the output segmentation result, the model can automatically adapt to the resolution setting and output the segmentation image with the expected size, and the user does not need to manually change the specific structure of the model. That is to say, when the user needs the output with different resolutions, the user only needs to change the parameters outside the model, and does not need to manually change the model structure, thereby expanding the application of the segmentation model in the industrial appearance quality inspection.
And 302, performing neural network convolution processing on the image to be segmented to obtain N characteristic graphs with different resolutions, wherein N is a positive integer greater than 1, and the characteristic depths of the N characteristic graphs are sequentially increased.
In this step, a feature extraction model, such as a convolutional neural network, is used to perform multi-layer convolutional feature extraction on the image to be segmented, so as to obtain N layers of feature maps. Illustratively, as shown in FIG. 4, a feature map F with 4 layers of different resolutions is obtained 1 、F 2 、F 3 、F 4 Wherein, the characteristic diagram F 1 、F 2 、F 3 、F 4 Are sequentially increased.
It should be noted that the convolution operation in the convolutional neural network already considers, to some extent, spatial context information around a target region to be segmented (i.e., a defect region), where the spatial context information refers to information between the defect region and its surrounding image features, so that the feature map includes, in addition to the defect region, a part of the features of the surrounding image of the defect region, and therefore, accurate positioning of the defect region can be assisted by introducing the spatial context information subsequently.
According to the embodiment of the application, the segmentation of the defect area can be assisted by utilizing the spatial context information, the missing detection problem existing in the edge area of the image is reduced, and the image segmentation quality is improved.
In the step, feature clipping processing is respectively carried out on the N feature maps, the central region feature of each feature map is extracted, and feature fusion processing is carried out on the N central region features to obtain a first target feature map. That is, the central region of the feature map obtained by fully extracting the features is divided and fused to obtain the first target feature map with the image bottom layer texture feature information, wherein the image bottom layer texture feature information comprises edge information, direction information and the like.
In an embodiment of the present application, the specific step of obtaining the first target feature map includes:
(1) According to a second preset resolution S of the image to be segmented in And a first preset resolution S of the segmentation result out Calculating each feature mapFirst characteristic resolution S of Fi And a second feature resolution C of the central region features to be cropped Fi The specific calculation formula is as follows:
S Fi =S in /2 i+1
C Fi =S out /2 i+1
wherein S is Fi Width or height corresponding to resolution of the first feature, S in Width or height, C, for second predetermined resolution Fi Width or height, S, corresponding to the resolution of the second feature out I =1,2, 3.. N for the width or height corresponding to the first preset resolution.
It should be noted that, because the image usually selected for the segmentation task is a square, the width and height of the feature map involved in the embodiment of the present application are equal, and therefore, the resolution variables are all represented by one variable to simplify the description process.
(2) According to the calculated first characteristic resolution S Fi And a second feature resolution C Fi Cutting out C in the middle position of each layer of feature map Fi ×C Fi And respectively obtaining corresponding central area characteristics in the central areas with different sizes. Illustratively, as shown in FIG. 4, the feature maps F are cut out respectively 1 、F 2 、F 3 、F 4 Central region feature F of 1 ’、F 2 ’、F 3 ’、F 4 ’。
(3) And zooming the N cut central region features and splicing the N cut central region features in channel dimensions to obtain a first target feature map with image bottom layer texture feature information. Illustratively, as shown in FIG. 4, the center region is characterized by F 1 ’、F 2 ’、F 3 ’、F 4 After zooming, feature fusion processing is carried out to obtain a first target feature map F L 。
The embodiment of the application adopts a central cutting mode, most of information contained in the characteristics of the central area obtained by cutting is from a target area (namely, a defect area) to be cut in an image to be cut, namely, the characteristics of the defect area of the image to be cut can be extracted more. In addition, when the defect area is determined, as described above, the information of the image features of the peripheral area is also introduced as a reference based on the spatial context information, so that the segmentation result of the central area is ensured to be more reliable, and the defect location effect is improved.
And step 304, acquiring a second target feature map according to the last M feature maps in the N feature maps.
In this step, the last M feature maps of the N feature maps, that is, the feature maps of M deep layers of the N feature maps are determined, where M is a positive integer greater than 1 and M is smaller than N. And further acquiring a second target characteristic diagram according to the M deep characteristic diagrams.
In an embodiment of the present application, acquiring the second target feature map according to the last M feature maps of the N feature maps specifically includes: carrying out global average pooling on the M feature maps to obtain M pooled features, and carrying out feature fusion processing on the M pooled features to obtain a third target feature map; carrying out feature extraction processing on the third target feature map to obtain feature vectors with the same number as channels of the third target feature map; and multiplying the feature vector by the third target feature map to obtain a second target feature map.
Exemplarily, as shown in fig. 4, a feature map F for two deep layers 3 And F 4 Obtaining the resolution ratio S by utilizing the self-adaptive global average pooling out Two pooling characteristics F of/32 H3 And F H4 To pooling characteristic F H3 And F H4 Splicing to obtain a third target characteristic diagram F H . Combining the multi-layer convolution layer with the third target feature map F H Extracting the characteristics to obtain a length and a third target characteristic diagram F H The feature vectors V with the same number of channels are compared with a third target feature map F H Multiplying to obtain a second target characteristic diagram F H '. The feature vector V is an index indicating the importance of the feature map, and functions as an adaptive weighting operation.
According to the embodiment of the application, the deep feature map is subjected to feature fusion, and the self-adaptive weighting operation is combined, so that the spatial semantic information of the deep feature is enhanced, and the accuracy of subsequent image detection is improved.
In the step, the first target feature map and the second target feature map are up-sampled to make the first target feature map and the second target feature map reach a first preset resolution S respectively out 。
And step 306, performing fusion convolution processing on the first target characteristic diagram and the second target characteristic diagram to obtain a segmentation result.
In this step, the first target feature map and the second target feature map after the resolution adjustment are fused to obtain a fourth target feature map, and then the fourth target feature map is subjected to multilayer convolution processing and maximum value argument point set function (i.e., argmax function) processing to obtain a final segmentation result R.
The resolution of the segmentation result R is the same as a first preset resolution set by a user, the pixel value of the background region of the segmentation result R is 0, and the pixel values of the defect regions of different classes respectively have a corresponding class ID.
According to the embodiment of the application, the defect area can be partitioned by the aid of the spatial context information, the missing detection problem of the edge area is reduced, the defect area is accurately positioned, the information utilization rate is improved through the double-branch structure design of the first target feature diagram and the second target feature diagram, and the image partitioning precision is improved.
Further, as a specific implementation of the image segmentation method, an embodiment of the present application provides an image segmentation apparatus. As shown in fig. 5, the image segmentation apparatus 500 includes: a first obtaining module 501, a first processing module 502, a second processing module 503, and a third processing module 504.
The first obtaining module 501 is configured to obtain an image to be segmented;
the first processing module 502 is configured to perform neural network convolution processing on an image to be segmented to obtain N feature maps with different resolutions;
a second processing module 503, configured to obtain a first target feature map corresponding to a central area of the N feature maps, and obtain a second target feature map according to M feature maps in the N feature maps;
a third processing module 504, configured to adjust the resolutions of the first target feature map and the second target feature map to a first preset resolution, and perform fusion convolution processing on the first target feature map and the second target feature map to obtain a segmentation result;
wherein N and M are positive integers greater than 1, and N is greater than M.
In the embodiment, an image to be segmented is obtained, and multi-layer convolution feature extraction processing is performed on the image to be segmented to obtain N layers of feature maps. The central area of the feature map obtained by fully extracting the features is segmented and fused to obtain a first target feature map with image bottom layer texture feature information, the last M feature maps in the N feature maps are determined, namely M deep feature maps in the N feature maps are determined, and then a second target feature map is obtained according to the M deep feature maps. And adjusting the resolutions of the first target feature map and the second target feature map to a first preset resolution, and performing fusion convolution processing on the first target feature map and the second target feature map to obtain a segmentation result.
According to the image segmentation method and device, segmentation of the defect area can be assisted through spatial context information, the problem of missing detection in the edge area is reduced, the defect area is accurately located, the information utilization rate is improved through the double-branch structure design of the first target feature graph and the second target feature graph, and the image segmentation precision is improved.
Further, the apparatus further comprises: and the second acquisition module is used for acquiring a first preset resolution of a segmentation result set by a user and a second preset resolution of the image to be segmented.
Further, the feature map includes features of the defective region and features of a surrounding image of the defective region.
Further, the second processing module 503 is specifically configured to: respectively carrying out feature cutting processing on the N feature graphs to obtain the central region feature of each feature graph; and performing feature fusion processing on the N central region features to obtain a first target feature map.
Further, the first target feature map has image bottom layer texture feature information; wherein the image bottom layer texture feature information comprises at least one of the following items: edge information, direction information.
Further, the second processing module 503 is specifically configured to: calculating a first characteristic resolution of each characteristic image according to a second preset resolution of the image to be segmented, and calculating a second characteristic resolution according to the first preset resolution of the segmentation result, wherein the second preset resolution and the first preset resolution are both set by a user; and performing feature clipping processing on the feature map of each first feature resolution to obtain the feature of the central area of the second feature resolution in the central area of each feature map.
Further, the width and the height corresponding to the second preset resolution are equal, the width and the height corresponding to the first preset resolution are equal, the width and the height corresponding to the second preset resolution are equal, and the width and the height corresponding to the first preset resolution are equal;
the formula for calculating the width or height corresponding to the resolution of the first feature is as follows:
S Fi =S in /2 i+1
wherein S is Fi Width or height corresponding to resolution of the first feature, S in I =1,2,3,. N for a width or height corresponding to a second preset resolution;
the formula for calculating the width or height corresponding to the resolution of the second feature is as follows:
C Fi =S out /2 i+1
wherein, C Fi Width or height, S, corresponding to the resolution of the second feature out I =1,2, 3.. N for the width or height corresponding to the first preset resolution.
Further, the second processing module 503 is specifically configured to: scaling the N central region features; and performing feature fusion processing on the N central region features after the scaling processing to obtain a first target feature map.
Further, the second processing module 503 is specifically configured to: performing global average pooling on M feature maps in the N feature maps to obtain M pooled features, and performing feature fusion processing on the M pooled features to obtain a third target feature map; carrying out feature extraction processing on the third target feature map to obtain feature vectors with the same number as channels of the third target feature map; and multiplying the feature vector by the third target feature map to obtain a second target feature map.
Further, the third processing module 504 is specifically configured to: and performing up-sampling processing on the first target characteristic diagram and the second target characteristic diagram to enable the first target characteristic diagram and the second target characteristic diagram to respectively reach a first preset resolution.
Further, the third processing module 504 is specifically configured to: performing feature fusion processing on the first target feature map and the second target feature map after the resolution is adjusted to generate a fourth target feature map; and sequentially carrying out multilayer convolution processing and maximum independent variable point set function processing on the fourth target characteristic graph to obtain a segmentation result.
Further, the first obtaining module 501 is specifically configured to: acquiring an image of a product to be detected; and performing image extraction on the image of the product to be detected by using a preset extraction method to obtain an image to be segmented.
The image segmentation apparatus 500 in the embodiment of the present application may be a computer device, or may be a component in a computer device, such as a chip.
The image segmentation apparatus 500 provided in this embodiment of the application can implement each process implemented in the embodiment of the image segmentation method in fig. 3, and is not described herein again to avoid repetition.
As shown in fig. 6, the computer device 600 includes a processor 601 and a memory 602, where the memory 602 stores a program or an instruction that can be executed on the processor 601, and when the program or the instruction is executed by the processor 601, the steps of the image segmentation method embodiment can be implemented, and the same technical effects can be achieved.
It should be noted that the computer devices in the embodiments of the present application include the mobile computer device and the non-mobile computer device described above.
The memory 602 may be used to store software programs as well as various data. The memory 602 may mainly include a first storage area storing programs or instructions and a second storage area storing data, wherein the first storage area may store an operating system, application programs or instructions required for at least one function (such as a sound playing function, an image playing function, and the like), and the like. Further, the memory 602 may include volatile memory or nonvolatile memory, or the memory 602 may include both volatile and nonvolatile memory. The non-volatile memory may be a Read-only memory (ROM), a Programmable ROM (PROM), an Erasable PROM (EPROM), an Electrically Erasable PROM (EEPROM), or a flash memory. Volatile memories may be Random Access Memories (RAMs), static random access memories (Static RAMs, SRAMs), dynamic random access memories (Dynamic RAMs, DRAMs), synchronous Dynamic random access memories (Synchronous DRAMs, SDRAMs), double data Rate Synchronous Dynamic random access memories (doubldatarate SDRAMs, DDRSDRAMs), enhanced Synchronous SDRAMs (enhanced SDRAMs, ESDRAMs), synchronous link Dynamic random access memories (Synchlink DRAMs, SLDRAMs) and direct Memory bus random access memories (direcrambus RAMs, DRRAMs). The memory 602 in the embodiments of the subject application includes, but is not limited to, these and any other suitable types of memory.
The embodiment of the present application further provides a readable storage medium, where a program or an instruction is stored on the readable storage medium, and when the program or the instruction is executed by a processor, the program or the instruction implements each process of the embodiment of the image segmentation method, and can achieve the same technical effect, and in order to avoid repetition, details are not repeated here.
The embodiment of the present application further provides a chip, where the chip includes a processor and a communication interface, the communication interface is coupled to the processor, and the processor is configured to execute a program or an instruction to implement each process of the embodiment of the image segmentation method, and can achieve the same technical effect, and in order to avoid repetition, the description is omitted here.
It should be understood that the chips mentioned in the embodiments of the present application may also be referred to as a system-on-chip, or a system-on-chip.
The embodiments of the present application further provide a computer program product, where the program product is stored in a storage medium, and the program product is executed by at least one processor to implement the processes of the foregoing embodiment of the image segmentation method, and can achieve the same technical effect, and in order to avoid repetition, details are not repeated here.
It should be noted that, in this document, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrases "comprising a component of' 8230; \8230;" does not exclude the presence of another like element in a process, method, article, or apparatus that comprises the element. Further, it should be noted that the scope of the methods and apparatus of the embodiments of the present application is not limited to performing the functions in the order illustrated or discussed, but may include performing the functions in a substantially simultaneous manner or in a reverse order based on the functions involved, e.g., the methods described may be performed in an order different than that described, and various steps may be added, omitted, or combined. In addition, features described with reference to certain examples may be combined in other examples.
While the present embodiments have been described with reference to the accompanying drawings, it is to be understood that the invention is not limited to the precise embodiments described above, which are meant to be illustrative and not restrictive, and that various changes may be made therein by those skilled in the art without departing from the spirit and scope of the invention as defined by the appended claims.
Claims (26)
1. An image segmentation method, comprising:
acquiring an image to be segmented;
carrying out neural network convolution processing on the image to be segmented to obtain N characteristic graphs with different resolutions, wherein the characteristic depths of the N characteristic graphs are sequentially increased;
acquiring first target feature maps corresponding to central areas of the N feature maps, and acquiring second target feature maps according to the last M feature maps in the N feature maps;
adjusting the resolutions of the first target feature map and the second target feature map to a first preset resolution;
performing fusion convolution processing on the first target feature map and the second target feature map to obtain a segmentation result;
wherein N and M are positive integers greater than 1, and N is greater than M.
2. The method of claim 1, further comprising, prior to said acquiring an image to be segmented:
and acquiring the first preset resolution of the segmentation result set by a user and a second preset resolution of the image to be segmented.
3. The method of claim 1,
the feature map includes features of the defective region and features of a surrounding image of the defective region.
4. The method according to claim 1, wherein the obtaining a first target feature map corresponding to a central region of the N feature maps comprises:
respectively carrying out feature clipping processing on the N feature graphs to obtain the central region feature of each feature graph;
and performing feature fusion processing on the N central region features to obtain a first target feature map.
5. The method of claim 4,
the first target feature map has image bottom layer texture feature information;
wherein the image bottom layer texture feature information comprises at least one of: edge information, direction information.
6. The method according to claim 4, wherein the performing feature clipping processing on the N feature maps respectively to obtain the feature of the central area of each feature map comprises:
calculating a first feature resolution of each feature map according to a second preset resolution of an image to be segmented, and calculating a second feature resolution according to the first preset resolution of a segmentation result, wherein the second preset resolution and the first preset resolution are both set by a user;
and performing feature clipping processing on the feature map of each first feature resolution to obtain the feature of the central area of the second feature resolution in the central area of each feature map.
7. The method according to claim 6, wherein the width and height corresponding to the second predetermined resolution are equal, the width and height corresponding to the first predetermined resolution are equal, the width and height corresponding to the second predetermined resolution are equal, and the width and height corresponding to the first predetermined resolution are equal;
and calculating the width or height corresponding to the first characteristic resolution by the following formula:
S Fi =S in /2 i+1
wherein S is Fi For width or height, S, corresponding to said first feature resolution in I =1,2,3,. N for the width or height corresponding to the second preset resolution;
the formula for calculating the width or height corresponding to the second feature resolution is as follows:
C Fi =S out /2 i+1
wherein, C Fi Width or height corresponding to the second feature resolution, S out I =1,2,3,. N for a width or height corresponding to the first preset resolution.
8. The method according to claim 4, wherein the performing feature fusion processing on the N central region features to obtain a first target feature map comprises:
scaling the N central region features;
and performing feature fusion processing on the N zoomed central region features to obtain a first target feature map.
9. The method according to claim 1, wherein the obtaining a second target feature map according to the last M feature maps of the N feature maps comprises:
performing global average pooling on the last M feature maps in the N feature maps to obtain M pooled features, and performing feature fusion processing on the M pooled features to obtain a third target feature map;
performing feature extraction processing on the third target feature map to obtain feature vectors with the same number as that of channels of the third target feature map;
and multiplying the feature vector by the third target feature map to obtain a second target feature map.
10. The method of claim 1, wherein the adjusting the resolution of each of the first target feature map and the second target feature map to a first preset resolution comprises:
and performing up-sampling processing on the first target feature map and the second target feature map to enable the first target feature map and the second target feature map to respectively reach a first preset resolution.
11. The method according to any one of claims 1 to 10, wherein the performing a fusion convolution process on the first target feature map and the second target feature map to obtain a segmentation result comprises:
performing feature fusion processing on the first target feature map and the second target feature map after the resolution is adjusted to generate a fourth target feature map;
and sequentially carrying out multilayer convolution processing and maximum independent variable point set function processing on the fourth target feature graph to obtain the segmentation result.
12. The method according to any one of claims 1 to 10, wherein the acquiring an image to be segmented comprises:
acquiring an image of a product to be detected;
and carrying out image extraction on the image of the product to be detected by using a preset extraction method to obtain an image to be segmented.
13. An image segmentation apparatus, comprising:
the first acquisition module is used for acquiring an image to be segmented;
the first processing module is used for carrying out neural network convolution processing on the image to be segmented to obtain N characteristic graphs with different resolutions, and the characteristic depths of the N characteristic graphs are sequentially increased;
the second processing module is used for acquiring a first target feature map corresponding to a central area of the N feature maps and acquiring a second target feature map according to the last M feature maps in the N feature maps;
the third processing module is used for adjusting the resolutions of the first target feature map and the second target feature map to a first preset resolution, and performing fusion convolution processing on the first target feature map and the second target feature map to obtain a segmentation result;
wherein N and M are positive integers greater than 1, and N is greater than M.
14. The apparatus of claim 13, further comprising:
and the second acquisition module is used for acquiring the first preset resolution of the segmentation result set by the user and the second preset resolution of the image to be segmented.
15. The apparatus of claim 13,
the feature map includes features of the defective region and features of a surrounding image of the defective region.
16. The apparatus of claim 13, wherein the second processing module is specifically configured to:
respectively carrying out feature clipping processing on the N feature graphs to obtain the central region feature of each feature graph;
and performing feature fusion processing on the N central region features to obtain a first target feature map.
17. The apparatus of claim 16,
the first target feature map has image bottom layer texture feature information;
wherein the image bottom layer texture feature information comprises at least one of: edge information, direction information.
18. The apparatus of claim 16, wherein the second processing module is specifically configured to:
calculating a first feature resolution of each feature map according to a second preset resolution of an image to be segmented, and calculating a second feature resolution according to the first preset resolution of a segmentation result, wherein the second preset resolution and the first preset resolution are both set by a user;
and performing feature clipping processing on the feature map of each first feature resolution to obtain the feature of the central area of the second feature resolution in the central area of each feature map.
19. The apparatus of claim 18, wherein the width and height of the second predetermined resolution are equal, the width and height of the first predetermined resolution are equal, the width and height of the second predetermined resolution are equal, and the width and height of the first predetermined resolution are equal;
and calculating the width or height corresponding to the first characteristic resolution by the following formula:
S Fi =S in /2 i+1
wherein S is Fi For width or height, S, corresponding to said first feature resolution in I =1,2,3,. N for the width or height corresponding to the second preset resolution;
and calculating the width or height corresponding to the second feature resolution by the following formula:
C Fi =S out /2 i+1
wherein, C Fi For width or height, S, corresponding to said second feature resolution out I =1,2,3,. N for a width or height corresponding to the first preset resolution.
20. The apparatus according to claim 16, wherein the second processing module is specifically configured to:
scaling the N central region features;
and performing feature fusion processing on the N zoomed central region features to obtain a first target feature map.
21. The apparatus of claim 13, wherein the second processing module is specifically configured to:
performing global average pooling on the last M feature maps in the N feature maps to obtain M pooled features, and performing feature fusion processing on the M pooled features to obtain a third target feature map;
performing feature extraction processing on the third target feature map to obtain feature vectors with the same number as that of channels of the third target feature map;
and multiplying the feature vector by the third target feature map to obtain a second target feature map.
22. The apparatus according to claim 13, wherein the third processing module is specifically configured to:
and performing upsampling processing on the first target feature map and the second target feature map to enable the first target feature map and the second target feature map to respectively reach a first preset resolution.
23. The apparatus according to any one of claims 13 to 22, wherein the third processing module is specifically configured to:
performing feature fusion processing on the first target feature map and the second target feature map after the resolution is adjusted to generate a fourth target feature map;
and sequentially carrying out multilayer convolution processing and maximum independent variable point set function processing on the fourth target feature graph to obtain the segmentation result.
24. The apparatus according to any one of claims 13 to 22, wherein the first obtaining module is specifically configured to:
acquiring an image of a product to be detected;
and performing image extraction on the image of the product to be detected by using a preset extraction method to obtain an image to be segmented.
25. A computer device comprising a processor and a memory, the memory storing a program or instructions running on the processor, which program or instructions, when executed by the processor, carry out the steps of the image segmentation method according to any one of claims 1 to 12.
26. A readable storage medium on which a program or instructions are stored, which program or instructions, when executed by a processor, carry out the steps of the image segmentation method according to any one of claims 1 to 12.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310229248.4A CN115908459B (en) | 2023-03-10 | 2023-03-10 | Image segmentation method, device, computer equipment and readable storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310229248.4A CN115908459B (en) | 2023-03-10 | 2023-03-10 | Image segmentation method, device, computer equipment and readable storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN115908459A true CN115908459A (en) | 2023-04-04 |
| CN115908459B CN115908459B (en) | 2023-05-05 |
Family
ID=85771877
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310229248.4A Active CN115908459B (en) | 2023-03-10 | 2023-03-10 | Image segmentation method, device, computer equipment and readable storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115908459B (en) |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111681273A (en) * | 2020-06-10 | 2020-09-18 | 创新奇智(青岛)科技有限公司 | Image segmentation method and device, electronic equipment and readable storage medium |
| CN111709929A (en) * | 2020-06-15 | 2020-09-25 | 北京航空航天大学 | Lung canceration region segmentation and classification detection system |
| CN112884782A (en) * | 2021-03-02 | 2021-06-01 | 深圳市瑞图生物技术有限公司 | Biological object segmentation method, apparatus, computer device and storage medium |
| CN113538313A (en) * | 2021-07-22 | 2021-10-22 | 深圳大学 | Polyp segmentation method and device, computer equipment and storage medium |
| CN113762396A (en) * | 2021-09-10 | 2021-12-07 | 西南科技大学 | Two-dimensional image semantic segmentation method |
| WO2022105622A1 (en) * | 2020-11-18 | 2022-05-27 | 北京有竹居网络技术有限公司 | Image segmentation method and apparatus, readable medium, and electronic device |
| CN114565048A (en) * | 2022-03-02 | 2022-05-31 | 安徽大学 | Three-stage pest image identification method based on adaptive feature fusion pyramid network |
-
2023
- 2023-03-10 CN CN202310229248.4A patent/CN115908459B/en active Active
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111681273A (en) * | 2020-06-10 | 2020-09-18 | 创新奇智(青岛)科技有限公司 | Image segmentation method and device, electronic equipment and readable storage medium |
| CN111709929A (en) * | 2020-06-15 | 2020-09-25 | 北京航空航天大学 | Lung canceration region segmentation and classification detection system |
| WO2022105622A1 (en) * | 2020-11-18 | 2022-05-27 | 北京有竹居网络技术有限公司 | Image segmentation method and apparatus, readable medium, and electronic device |
| CN112884782A (en) * | 2021-03-02 | 2021-06-01 | 深圳市瑞图生物技术有限公司 | Biological object segmentation method, apparatus, computer device and storage medium |
| CN113538313A (en) * | 2021-07-22 | 2021-10-22 | 深圳大学 | Polyp segmentation method and device, computer equipment and storage medium |
| CN113762396A (en) * | 2021-09-10 | 2021-12-07 | 西南科技大学 | Two-dimensional image semantic segmentation method |
| CN114565048A (en) * | 2022-03-02 | 2022-05-31 | 安徽大学 | Three-stage pest image identification method based on adaptive feature fusion pyramid network |
Also Published As
| Publication number | Publication date |
|---|---|
| CN115908459B (en) | 2023-05-05 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111027547B (en) | Automatic detection method for multi-scale polymorphic target in two-dimensional image | |
| CN109377445B (en) | Model training method, method and device for replacing image background and electronic system | |
| CN110084817B (en) | Digital elevation model production method based on deep learning | |
| US20220366544A1 (en) | Method for high resolution image inpainting, processing system and associated computer program product | |
| CN109919145B (en) | Mine card detection method and system based on 3D point cloud deep learning | |
| CN110188495B (en) | Method for generating three-dimensional house type graph based on two-dimensional house type graph of deep learning | |
| CN110992318A (en) | Special metal flaw detection system based on deep learning | |
| JP2019207535A (en) | Information processing apparatus, information processing method, and program | |
| CN113962959A (en) | Three-dimensional image processing method, three-dimensional image processing device, computer equipment and storage medium | |
| CN110782385A (en) | Image watermark removing method based on deep learning | |
| WO2014198029A1 (en) | Image completion based on patch offset statistics | |
| CN111860208B (en) | Super-pixel-based remote sensing image ground object classification method, system, device and medium | |
| CN113077477B (en) | Image vectorization method and device and terminal equipment | |
| CN113763415B (en) | Target tracking method, device, electronic equipment and storage medium | |
| CN112669300A (en) | Defect detection method and device, computer equipment and storage medium | |
| CN113744142A (en) | Image restoration method, electronic device and storage medium | |
| CN116664590B (en) | Automatic segmentation method and device based on dynamic contrast enhancement magnetic resonance image | |
| CN115908459B (en) | Image segmentation method, device, computer equipment and readable storage medium | |
| CN110991408B (en) | Method and device for segmenting white matter high signal based on deep learning method | |
| CN116596895A (en) | Substation equipment image defect identification method and system | |
| CN111241891A (en) | Face image cutting method and device and computer readable storage medium | |
| CN112348021A (en) | Text detection method, device, equipment and storage medium | |
| CN113034366A (en) | Splicing seam-free SAR image segmentation parallel acceleration processing method | |
| CN114581448B (en) | Image detection method, device, terminal equipment and storage medium | |
| CN112669299B (en) | Flaw detection method and device, computer equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |