CN115908459B - Image segmentation method, device, computer equipment and readable storage medium - Google Patents
Image segmentation method, device, computer equipment and readable storage medium Download PDFInfo
- Publication number
- CN115908459B CN115908459B CN202310229248.4A CN202310229248A CN115908459B CN 115908459 B CN115908459 B CN 115908459B CN 202310229248 A CN202310229248 A CN 202310229248A CN 115908459 B CN115908459 B CN 115908459B
- Authority
- CN
- China
- Prior art keywords
- feature
- resolution
- feature map
- image
- target feature
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Abstract
The application provides an image segmentation method, an image segmentation device, computer equipment and a readable storage medium, and relates to the field of industrial vision detection. The method comprises the following steps: acquiring an image to be segmented; performing neural network convolution processing on an image to be segmented to obtain N feature images with different resolutions, wherein the feature depths of the N feature images are sequentially increased; acquiring first target feature images corresponding to central areas of the N feature images, and acquiring second target feature images according to the last M feature images in the N feature images; the resolutions of the first target feature map and the second target feature map are adjusted to a first preset resolution; performing fusion convolution processing on the first target feature map and the second target feature map to obtain a segmentation result; n, M is a positive integer greater than 1, and N is greater than M. The method and the device solve the problem of missing detection of the image edge area and improve the image segmentation precision.
Description
Technical Field
The present application relates to the field of industrial visual inspection, and in particular, to an image segmentation method, an image segmentation apparatus, a computer device, and a readable storage medium.
Background
At present, the appearance quality detection of industrial products is an indispensable link in the production process of the products, is used for positioning and removing products with defects in appearance, and is very important for upgrading the control of the product quality and maintaining the brand image. In order to realize an efficient and stable appearance quality inspection link, the detection of surface defects of an automatic product is realized through industrial machine vision. Image segmentation is a commonly used defect region detection model, which outputs a segmentation result of the same size by analyzing the attribute characteristics of an input image, and indicates the position where a defect may exist. In order to ensure a high-quality defect area detection effect, an original image is generally cut into a series of image blocks at equal intervals, and then analyzed respectively, and then a detection result under the original resolution is obtained by an image stitching mode. According to the different resolutions of the segmentation results in the detection process, the image segmentation model for defect detection in the related art can be roughly divided into the following two schemes:
(1) The segmentation model that maintains resolution, i.e. the resolution of the output segmentation result is the same as the input image, is the most common segmentation model. In the detection process of the model, original pictures are generally cut and processed respectively in a mode with a certain overlapping proportion, the process is shown in fig. 1, and the arrow represents the interval between two cuts.
(2) The reduced resolution segmentation model, i.e. for the input image, results in a segmentation result of smaller resolution. The segmentation of such a model is shown in fig. 2, where the arrow represents the interval between two cuts, the thin solid line represents the image area of each cut, and the thick solid line represents the corresponding segmentation result.
The detection model corresponding to the scheme (1) focuses more on the central area of the image, but ignores the image characteristics of the edge position, so that the detection capability of the model on the target falling on the edge position is insufficient, and the problem of poor detection quality of the edge area exists. The model corresponding to the scheme (2) is complex in structure adjustment process and difficult to flexibly apply, and more segmentation operations are adopted in the forward transmission process of the feature map, so that a large amount of information in the feature map is removed, and the detection accuracy of the model is limited to a certain extent.
Disclosure of Invention
In view of this, the present application provides an image segmentation method, an image segmentation apparatus, a computer device, and a readable storage medium, which solve the problem of missing detection of an image edge region and improve image segmentation accuracy.
In a first aspect, an embodiment of the present application provides an image segmentation method, including:
Acquiring an image to be segmented, and performing neural network convolution processing on the image to be segmented to obtain N feature images with different resolutions, wherein the feature depths of the N feature images are sequentially increased;
acquiring a first target feature map corresponding to a central area of the N feature maps, and acquiring a second target feature map according to the last M feature maps in the N feature maps;
the resolutions of the first target feature map and the second target feature map are adjusted to a first preset resolution;
performing fusion convolution processing on the first target feature map and the second target feature map to obtain a segmentation result;
wherein N, M is a positive integer greater than 1, and N is greater than M.
The method according to the embodiment of the application can also have the following additional technical characteristics:
in the above technical solution, optionally, before acquiring the image to be segmented, the method further includes: and acquiring a first preset resolution of a segmentation result set by a user and a second preset resolution of an image to be segmented.
In any of the above embodiments, optionally, the feature map includes features of the defect area and features of a surrounding image of the defect area.
In any of the above solutions, optionally, obtaining a first target feature map corresponding to a central area of the N feature maps includes: respectively carrying out feature clipping treatment on the N feature images to obtain the central region feature of each feature image; and carrying out feature fusion processing on the N central region features to obtain a first target feature map.
In any of the above solutions, optionally, the first target feature map has image bottom texture feature information; wherein the image underlying texture feature information comprises at least one of: edge information and direction information.
In any of the above technical solutions, optionally, performing feature clipping processing on the N feature maps to obtain a central region feature of each feature map, where the feature clipping processing includes: according to a second preset resolution of the image to be segmented, calculating a first feature resolution of each feature map, and according to the first preset resolution of the segmentation result, calculating a second feature resolution, wherein the second preset resolution and the first preset resolution are set by a user; and carrying out feature clipping processing on the feature graphs of each first feature resolution to obtain the features of the central region of the second feature resolution in the central region of each feature graph.
In any of the above technical solutions, optionally, the widths and heights corresponding to the second preset resolution are equal, the widths and heights corresponding to the first preset resolution are equal, the widths and heights corresponding to the second preset resolution are equal, and the widths and heights corresponding to the first preset resolution are equal;
the formula for calculating the width or height corresponding to the first feature resolution is:
S Fi =S in /2 i+1
Wherein S is Fi For the width or height corresponding to the first feature resolution, S in I=1, 2,3 for the width or height corresponding to the second preset resolution.
The equation for calculating the width or height corresponding to the second feature resolution is:
C Fi =S out /2 i+1
wherein C is Fi For the width or height corresponding to the second feature resolution, S out I=1, 2,3 for the width or height corresponding to the first preset resolution.
In any of the above technical solutions, optionally, performing feature fusion processing on the N central region features to obtain a first target feature map, including: scaling the N central region features; and carrying out feature fusion processing on the N central region features after the scaling processing to obtain a first target feature map.
In any of the above solutions, optionally, obtaining the second target feature map according to the last M feature maps in the N feature maps includes: carrying out global average pooling treatment on the last M feature graphs in the N feature graphs to obtain M pooled features, and carrying out feature fusion treatment on the M pooled features to obtain a third target feature graph; performing feature extraction processing on the third target feature map to obtain feature vectors with the same channel number as the third target feature map; and multiplying the feature vector with the third target feature map to obtain a second target feature map.
In any of the above solutions, optionally, adjusting the resolutions of the first target feature map and the second target feature map to a first preset resolution includes: and carrying out up-sampling processing on the first target feature map and the second target feature map to enable the first target feature map and the second target feature map to respectively reach a first preset resolution.
In any of the above technical solutions, optionally, performing fusion convolution processing on the first target feature map and the second target feature map to obtain a segmentation result, including: performing feature fusion processing on the first target feature map and the second target feature map after resolution adjustment to generate a fourth target feature map; and sequentially carrying out multi-layer convolution processing and maximum value independent variable point set function processing on the fourth target feature map to obtain a segmentation result.
In any of the above technical solutions, optionally, acquiring an image to be segmented includes: acquiring an image of a product to be detected; and carrying out image extraction on the image of the product to be detected by using a preset extraction method to obtain an image to be segmented.
In a second aspect, an embodiment of the present application provides an image segmentation apparatus, including:
the first acquisition module is used for acquiring an image to be segmented;
The first processing module is used for carrying out neural network convolution processing on the image to be segmented to obtain N feature images with different resolutions, and the feature depths of the N feature images are sequentially increased;
the second processing module is used for acquiring first target feature images corresponding to the central areas of the N feature images and acquiring second target feature images according to the last M feature images in the N feature images;
the third processing module is used for adjusting the resolutions of the first target feature map and the second target feature map to a first preset resolution, and carrying out fusion convolution processing on the first target feature map and the second target feature map to obtain a segmentation result;
wherein N, M is a positive integer greater than 1, and N is greater than M.
The device according to the embodiment of the application may further have the following additional technical features:
in the above technical solution, optionally, the apparatus further includes: the second acquisition module is used for acquiring a first preset resolution of the segmentation result set by the user and a second preset resolution of the image to be segmented.
In any of the above embodiments, optionally, the feature map includes features of the defect area and features of a surrounding image of the defect area.
In any of the above solutions, optionally, the second processing module is specifically configured to: respectively carrying out feature clipping treatment on the N feature images to obtain the central region feature of each feature image; and carrying out feature fusion processing on the N central region features to obtain a first target feature map.
In any of the above solutions, optionally, the first target feature map has image bottom texture feature information; wherein the image underlying texture feature information comprises at least one of: edge information and direction information.
In any of the above solutions, optionally, the second processing module is specifically configured to: according to a second preset resolution of the image to be segmented, calculating a first feature resolution of each feature map, and according to the first preset resolution of the segmentation result, calculating a second feature resolution, wherein the second preset resolution and the first preset resolution are set by a user; and carrying out feature clipping processing on the feature graphs of each first feature resolution to obtain the features of the central region of the second feature resolution in the central region of each feature graph.
In any of the above technical solutions, optionally, the widths and heights corresponding to the second preset resolution are equal, the widths and heights corresponding to the first preset resolution are equal, the widths and heights corresponding to the second preset resolution are equal, and the widths and heights corresponding to the first preset resolution are equal;
the formula for calculating the width or height corresponding to the first feature resolution is:
S Fi =S in /2 i+1
wherein S is Fi For the width or height corresponding to the first feature resolution, S in I=1, 2,3 for the width or height corresponding to the second preset resolution.
The equation for calculating the width or height corresponding to the second feature resolution is:
C Fi =S out /2 i+1
wherein C is Fi For the width or height corresponding to the second feature resolution, S out I=1, 2,3 for the width or height corresponding to the first preset resolution.
In any of the above solutions, optionally, the second processing module is specifically configured to: scaling the N central region features; and carrying out feature fusion processing on the N central region features after the scaling processing to obtain a first target feature map.
In any of the above solutions, optionally, the second processing module is specifically configured to: carrying out global average pooling treatment on the last M feature graphs in the N feature graphs to obtain M pooled features, and carrying out feature fusion treatment on the M pooled features to obtain a third target feature graph; performing feature extraction processing on the third target feature map to obtain feature vectors with the same channel number as the third target feature map; and multiplying the feature vector with the third target feature map to obtain a second target feature map.
In any of the above solutions, optionally, a third processing module is specifically configured to: and carrying out up-sampling processing on the first target feature map and the second target feature map to enable the first target feature map and the second target feature map to respectively reach a first preset resolution.
In any of the above solutions, optionally, a third processing module is specifically configured to: performing feature fusion processing on the first target feature map and the second target feature map after resolution adjustment to generate a fourth target feature map; and sequentially carrying out multi-layer convolution processing and maximum value independent variable point set function processing on the fourth target feature map to obtain a segmentation result.
In any of the foregoing solutions, optionally, the first obtaining module is specifically configured to: acquiring an image of a product to be detected; and carrying out image extraction on the image of the product to be detected by using a preset extraction method to obtain an image to be segmented.
In a third aspect, embodiments of the present application provide a computer device comprising a processor and a memory storing a program or instructions executable on the processor, the program or instructions implementing the steps of the method as in the first aspect when executed by the processor.
In a fourth aspect, embodiments of the present application provide a readable storage medium having stored thereon a program or instructions which when executed by a processor perform the steps of the method as in the first aspect.
In a fifth aspect, embodiments of the present application provide a chip comprising a processor and a communication interface, the communication interface being coupled to the processor, the processor being configured to execute programs or instructions to implement a method as in the first aspect.
In a sixth aspect, embodiments of the present application provide a computer program product stored in a storage medium, the program product being executable by at least one processor to implement a method as in the first aspect.
In the embodiment of the application, an image to be segmented is obtained, and multi-layer convolution feature extraction processing is carried out on the image to be segmented to obtain an N-layer feature map. And dividing and fusing the central area of the feature map obtained by fully extracting the features to obtain a first target feature map with the image bottom texture feature information, determining the last M feature maps in the N feature maps, namely determining the M deep feature maps in the N feature maps, and further obtaining a second target feature map according to the M deep feature maps. And adjusting the resolutions of the first target feature map and the second target feature map to a first preset resolution, and then carrying out fusion convolution processing on the first target feature map and the second target feature map to obtain a segmentation result.
The method and the device can help the segmentation of the defect area through the spatial context information, reduce the problem of missing detection in the edge area, and therefore accurately position the defect area, improve the information utilization rate through the double-branch structure design of the first target feature map and the second target feature map, and improve the image segmentation precision.
The foregoing description is only an overview of the technical solutions of the present application, and may be implemented according to the content of the specification in order to make the technical means of the present application more clearly understood, and in order to make the above-mentioned and other objects, features and advantages of the present application more clearly understood, the following detailed description of the present application will be given.
Drawings
The accompanying drawings, which are included to provide a further understanding of the application and are incorporated in and constitute a part of this application, illustrate embodiments of the application and together with the description serve to explain the application and do not constitute an undue limitation to the application. In the drawings:
FIG. 1 shows an image segmentation schematic of a segmentation model with maintained resolution in the related art;
FIG. 2 shows an image segmentation schematic of a reduced resolution segmentation model in the related art;
FIG. 3 shows a flow diagram of an image segmentation method according to an embodiment of the present application;
FIG. 4 shows an image segmentation schematic diagram of an embodiment of the present application;
fig. 5 shows a block diagram of the image dividing apparatus of the embodiment of the present application;
fig. 6 shows a block diagram of a computer device according to an embodiment of the present application.
Detailed Description
Technical solutions in the embodiments of the present application will be clearly described below with reference to the drawings in the embodiments of the present application, and it is apparent that the described embodiments are some embodiments of the present application, but not all embodiments. All other embodiments obtained by a person of ordinary skill in the art based on the embodiments in the present application are within the scope of the protection of the present application.
The terms first, second and the like in the description and in the claims, are used for distinguishing between similar objects and not necessarily for describing a particular sequential or chronological order. It is to be understood that the data so used may be interchanged, as appropriate, such that embodiments of the present application may be implemented in sequences other than those illustrated or described herein, and that the objects identified by "first," "second," etc. are generally of a type and not limited to the number of objects, e.g., the first object may be one or more. Furthermore, in the description and claims, "and/or" means at least one of the connected objects, and the character "/", generally means that the associated object is an "or" relationship.
The image segmentation method, the image segmentation apparatus, the computer device and the readable storage medium provided in the embodiments of the present application are described in detail below with reference to the accompanying drawings by means of specific embodiments and application scenarios thereof.
The embodiment of the application provides an image segmentation method, which is realized by an image segmentation model based on deep learning, and is used for segmenting various target areas (namely, defect areas) and background areas in an input image, namely, a category is given to each pixel point in the input image, so that the pixel point is divided into background pixels or defect pixels.
As shown in fig. 3, the image segmentation method includes:
In this step, an image to be segmented is acquired. Specifically, shooting a product to be detected by using a high-definition camera to obtain an image of the product to be detected, and extracting the image of the product to be detected by using a preset extraction method to obtain an image to be segmented.
The preset extraction method is any one of gradient calculation, numerical smoothing, ROI (Region Of Interest ) selection, binarization, contour extraction and the like, and the specific mode of the preset extraction method is not limited in the embodiment of the present application.
In one embodiment of the present application, the resolution of the image to be segmented and the segmentation result resulting from the segmentation of the image, i.e. the resolution of the input image and the output result of the image segmentation model, is set by the user. Specifically, before the image to be segmented is acquired, a second preset resolution of the image to be segmented set by a user and a first preset resolution of a segmentation result obtained by performing image segmentation are acquired. For example, the user sets the size of the image to be segmented to S in ×S in (width×height), and setting the size of the division result obtained by the image division to S out ×S out (width. Times. Height).
The output size and the input size of the segmentation model in the related art are strictly related, and the segmentation model is difficult to apply to industrial vision detection scenes which need flexible adjustment of the segmentation size. In the embodiment of the application, the user can arbitrarily specify the resolution of the output segmentation result, the model can automatically adapt to the resolution setting and output the segmentation image with the expected size, and the user is not required to manually change the specific structure of the model. That is, when the user needs the output with different resolutions, the user only needs to change the parameters outside the model, and the model structure is not manually changed, so that the application of the segmentation model in the industrial appearance quality inspection is enlarged.
In the step, a feature extraction model, such as a convolutional neural network, is used to perform multi-layer convolutional feature extraction processing on the image to be segmented to obtain an N-layer feature map. Illustratively, as shown in FIG. 4, a 4-layer resolution-differentiated feature map F is obtained 1 、F 2 、F 3 、F 4 Wherein, the characteristic diagram F 1 、F 2 、F 3 、F 4 Sequentially increasing the depth of (c).
It should be noted that the convolution operation in the convolutional neural network already considers, to a certain extent, the spatial context information around the target region to be segmented (i.e. the defect region), where the spatial context information refers to the information between the defect region and its surrounding image features, so that the feature map includes, in addition to the defect region, the features of the surrounding image of a part of the defect region, and therefore, the accurate positioning of the defect region can be assisted by introducing the spatial context information later.
According to the method and the device for dividing the defect area, the space context information can be used for assisting in dividing the defect area, the problem of missing detection in the image edge area is reduced, and the image dividing quality is improved.
In the step, feature clipping processing is respectively carried out on the N feature images, the central region feature of each feature image is extracted, and feature fusion processing is carried out on the N central region features, so that a first target feature image is obtained. That is, the central region of the feature map obtained by sufficiently extracting the features is segmented and fused to obtain the first target feature map having the image-based texture feature information including the edge information, the direction information, and the like.
In one embodiment of the present application, the specific step of obtaining the first target feature map includes:
(1) According to a second preset resolution S of the image to be segmented in First preset resolution S of segmentation result out Calculating a first feature resolution S of each feature map Fi Second feature resolution C of center region features to be cropped Fi The specific calculation formula is as follows:
S Fi =S in /2 i+1
C Fi =S out /2 i+1
wherein S is Fi For the width or height corresponding to the first feature resolution, S in C is the width or the height corresponding to the second preset resolution Fi For the width or height corresponding to the second feature resolution, S out I=1, 2,3 for the width or height corresponding to the first preset resolution.
It should be noted that, because the image generally selected in the segmentation task is square, the width and height of the feature map in the embodiment of the present application are equal, and thus the variable of resolution is represented by one variable to simplify the description process.
(2) According to the calculated first feature resolution S Fi And a second feature resolution C Fi Cutting out C at the middle position of each layer of characteristic diagram Fi ×C Fi And the central area of the size is respectively obtained to obtain the corresponding central area characteristics. Exemplary, as shown in FIG. 4, feature map F is cut out 1 、F 2 、F 3 、F 4 Central region feature F of (2) 1 ’、F 2 ’、F 3 ’、F 4 ’。
(3) And scaling the N central region features obtained by cutting, and then splicing in the channel dimension to obtain a first target feature map with the image bottom texture feature information. Illustratively, as shown in FIG. 4, the central region feature F 1 ’、F 2 ’、F 3 ’、F 4 After' scaling treatment, carrying out feature fusion treatment to obtain a first target feature map F L 。
In the embodiment of the present application, a central clipping manner is adopted, and most of information included in central region features obtained by clipping is from a target region to be segmented (i.e., a defect region) in an image to be segmented, that is, features of the defect region of the image to be segmented can be extracted more. In addition, when the decision is made on the defect area, as described above, the information of the image characteristics of the surrounding area is also introduced as a reference based on the spatial context information, so that the segmentation result of the central area is ensured to be more reliable, and the defect positioning effect is improved.
And step 304, acquiring a second target feature map according to the last M feature maps in the N feature maps.
In this step, the last M feature maps of the N feature maps, that is, the feature maps of M deep layers of the N feature maps, are determined, where M is a positive integer greater than 1 and M is less than N. And further acquiring a second target feature map according to the M deep feature maps.
In one embodiment of the present application, the obtaining a second target feature map according to the last M feature maps in the N feature maps specifically includes: performing global average pooling treatment on the M feature images to obtain M pooled features, and performing feature fusion treatment on the M pooled features to obtain a third target feature image; performing feature extraction processing on the third target feature map to obtain feature vectors with the same channel number as the third target feature map; and multiplying the feature vector with the third target feature map to obtain a second target feature map.
Exemplary, as shown in FIG. 4, feature maps F for two deep layers 3 And F 4 Obtaining the resolution of S by using self-adaptive global average pooling out Two pooling features F of/32 H3 And F H4 To pool feature F H3 And F H4 Splicing to obtain a third target feature map F H . Combining multiple convolution layers with a third target feature map F H Extracting features to obtain a length and third target feature map F H The feature vector V with the same channel number is then combined with the third target feature map F H Multiplying to obtain a second target feature map F H '. The feature vector V is an index representing the importance of the feature map, and functions as an adaptive weighting operation.
According to the embodiment of the application, feature fusion is carried out on the deep feature map, and the self-adaptive weighting operation is combined, so that the space semantic information of the deep features is enhanced, and the accuracy of subsequent image detection is improved.
In step 305, the resolutions of the first target feature map and the second target feature map are adjusted to a first preset resolution.
In the step, the up-sampling process is performed on the first target feature map and the second target feature map to respectively reach the first preset resolution S out 。
And 306, performing fusion convolution processing on the first target feature map and the second target feature map to obtain a segmentation result.
In the step, the first target feature map and the second target feature map after the resolution adjustment are fused to obtain a fourth target feature map, and then the fourth target feature map is subjected to multi-layer convolution processing and maximum value independent variable point set function (namely, argmax function) processing to obtain a final segmentation result R.
The resolution of the segmentation result R is the same as the first preset resolution set by the user, the pixel value of the background area of the segmentation result R is 0, and the pixel values of the defect areas of different categories have respective corresponding category IDs.
According to the method and the device, the segmentation of the defect area can be assisted by the spatial context information, the missing detection problem in the edge area is reduced, so that the defect area is accurately positioned, the information utilization rate is improved through the double-branch structure design of the first target feature map and the second target feature map, and the image segmentation precision is improved.
Further, as a specific implementation of the image segmentation method, an embodiment of the present application provides an image segmentation apparatus. As shown in fig. 5, the image segmentation apparatus 500 includes: a first acquisition module 501, a first processing module 502, a second processing module 503, and a third processing module 504.
The first acquiring module 501 is configured to acquire an image to be segmented;
the first processing module 502 is configured to perform neural network convolution processing on an image to be segmented to obtain N feature maps with different resolutions;
a second processing module 503, configured to obtain a first target feature map corresponding to a central area of the N feature maps, and obtain a second target feature map according to M feature maps in the N feature maps;
the third processing module 504 is configured to adjust the resolutions of the first target feature map and the second target feature map to a first preset resolution, and perform fusion convolution processing on the first target feature map and the second target feature map to obtain a segmentation result;
Wherein N, M is a positive integer greater than 1, and N is greater than M.
In this embodiment, an image to be segmented is acquired, and feature extraction processing of multi-layer convolution is performed on the image to be segmented, so as to obtain an N-layer feature map. And dividing and fusing the central area of the feature map obtained by fully extracting the features to obtain a first target feature map with the image bottom texture feature information, determining the last M feature maps in the N feature maps, namely determining the M deep feature maps in the N feature maps, and further obtaining a second target feature map according to the M deep feature maps. And adjusting the resolutions of the first target feature map and the second target feature map to a first preset resolution, and then carrying out fusion convolution processing on the first target feature map and the second target feature map to obtain a segmentation result.
The method and the device can help the segmentation of the defect area through the spatial context information, reduce the problem of missing detection in the edge area, and therefore accurately position the defect area, improve the information utilization rate through the double-branch structure design of the first target feature map and the second target feature map, and improve the image segmentation precision.
Further, the apparatus further comprises: the second acquisition module is used for acquiring a first preset resolution of the segmentation result set by the user and a second preset resolution of the image to be segmented.
Further, the feature map includes features of the defective region and features of the surrounding image of the defective region.
Further, the second processing module 503 is specifically configured to: respectively carrying out feature clipping treatment on the N feature images to obtain the central region feature of each feature image; and carrying out feature fusion processing on the N central region features to obtain a first target feature map.
Further, the first target feature map has image bottom texture feature information; wherein the image underlying texture feature information comprises at least one of: edge information and direction information.
Further, the second processing module 503 is specifically configured to: according to a second preset resolution of the image to be segmented, calculating a first feature resolution of each feature map, and according to the first preset resolution of the segmentation result, calculating a second feature resolution, wherein the second preset resolution and the first preset resolution are set by a user; and carrying out feature clipping processing on the feature graphs of each first feature resolution to obtain the features of the central region of the second feature resolution in the central region of each feature graph.
Further, the width and the height corresponding to the second preset resolution are equal, the width and the height corresponding to the first preset resolution are equal, the width and the height corresponding to the second preset resolution are equal, and the width and the height corresponding to the first preset resolution are equal;
The formula for calculating the width or height corresponding to the first feature resolution is:
S Fi =S in /2 i+1
wherein S is Fi For the width or height corresponding to the first feature resolution, S in I=1, 2,3 for the width or height corresponding to the second preset resolution.
The equation for calculating the width or height corresponding to the second feature resolution is:
C Fi =S out /2 i+1
wherein C is Fi For the width or height corresponding to the second feature resolution, S out I=1, 2,3 for the width or height corresponding to the first preset resolution.
Further, the second processing module 503 is specifically configured to: scaling the N central region features; and carrying out feature fusion processing on the N central region features after the scaling processing to obtain a first target feature map.
Further, the second processing module 503 is specifically configured to: carrying out global average pooling treatment on M feature graphs in the N feature graphs to obtain M pooled features, and carrying out feature fusion treatment on the M pooled features to obtain a third target feature graph; performing feature extraction processing on the third target feature map to obtain feature vectors with the same channel number as the third target feature map; and multiplying the feature vector with the third target feature map to obtain a second target feature map.
Further, the third processing module 504 is specifically configured to: and carrying out up-sampling processing on the first target feature map and the second target feature map to enable the first target feature map and the second target feature map to respectively reach a first preset resolution.
Further, the third processing module 504 is specifically configured to: performing feature fusion processing on the first target feature map and the second target feature map after resolution adjustment to generate a fourth target feature map; and sequentially carrying out multi-layer convolution processing and maximum value independent variable point set function processing on the fourth target feature map to obtain a segmentation result.
Further, the first obtaining module 501 is specifically configured to: acquiring an image of a product to be detected; and carrying out image extraction on the image of the product to be detected by using a preset extraction method to obtain an image to be segmented.
The image segmentation apparatus 500 in the embodiment of the present application may be a computer device, or may be a component in a computer device, such as a chip.
The image segmentation apparatus 500 provided in the embodiment of the present application can implement each process implemented by the embodiment of the image segmentation method in fig. 3, and in order to avoid repetition, a detailed description is omitted here.
The embodiment of the present application further provides a computer device, as shown in fig. 6, where the computer device 600 includes a processor 601 and a memory 602, and a program or an instruction that can be executed on the processor 601 is stored in the memory 602, and when the program or the instruction is executed by the processor 601, the steps of the above-mentioned embodiment of the image segmentation method are implemented, and the same technical effects can be achieved, so that repetition is avoided, and no further description is given here.
It should be noted that, the computer device in the embodiment of the present application includes the mobile computer device and the non-mobile computer device described above.
The memory 602 may be used to store software programs as well as various data. The memory 602 may mainly include a first memory area storing programs or instructions and a second memory area storing data, wherein the first memory area may store an operating system, application programs or instructions (such as a sound playing function, an image playing function, etc.) required for at least one function, and the like. Further, the memory 602 may include volatile memory or nonvolatile memory, or the memory 602 may include both volatile and nonvolatile memory. The nonvolatile memory may be a Read-only memory (ROM), a Programmable ROM (PROM), an Erasable Programmable ROM (EPROM), an electrically Erasable Programmable ROM (ElectricallyEPROM, EEPROM), or a flash memory, among others. The volatile memory may be random access memory (RandomAccess Memory, RAM), static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double Data Rate Synchronous DRAM (DDRSDRAM), enhanced Synchronous DRAM (EnhancedSDRAM, ESDRAM), synchronous Link DRAM (SLDRAM), and direct memory bus RAM (DRRAM). Memory 602 in embodiments of the present application includes, but is not limited to, these and any other suitable types of memory.
The processor 601 may include one or more processing units; optionally, the processor 601 integrates an application processor that primarily handles operations involving an operating system, user interface, application programs, etc., and a modem processor that primarily handles wireless communication signals, such as a baseband processor. It will be appreciated that the modem processor described above may not be integrated into the processor 601.
The embodiment of the present application further provides a readable storage medium, where a program or an instruction is stored, where the program or the instruction realizes each process of the above embodiment of the image segmentation method when executed by a processor, and the same technical effects can be achieved, so that repetition is avoided, and no further description is given here.
The embodiment of the application also provides a chip, the chip comprises a processor and a communication interface, the communication interface is coupled with the processor, the processor is used for running programs or instructions, the processes of the embodiment of the image segmentation method can be realized, the same technical effects can be achieved, and the repetition is avoided, and the description is omitted here.
It should be understood that the chips referred to in the embodiments of the present application may also be referred to as system-on-chip chips, chip systems, or system-on-chip chips, etc.
The embodiments of the present application further provide a computer program product, which is stored in a storage medium, and the program product is executed by at least one processor to implement the respective processes of the embodiments of the image segmentation method, and achieve the same technical effects, so that repetition is avoided, and a detailed description is omitted here.
It should be noted that, in this document, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one … …" does not exclude the presence of other like elements in a process, method, article, or apparatus that comprises the element. Furthermore, it should be noted that the scope of the methods and apparatus in the embodiments of the present application is not limited to performing the functions in the order shown or discussed, but may also include performing the functions in a substantially simultaneous manner or in an opposite order depending on the functions involved, e.g., the described methods may be performed in an order different from that described, and various steps may also be added, omitted, or combined. Additionally, features described with reference to certain examples may be combined in other examples.
The embodiments of the present application have been described above with reference to the accompanying drawings, but the present application is not limited to the above-described embodiments, which are merely illustrative and not restrictive, and many forms may be made by those of ordinary skill in the art without departing from the spirit of the present application and the scope of the claims, which are also within the protection of the present application.
Claims (22)
1. An image segmentation method, comprising:
acquiring an image to be segmented;
performing neural network convolution processing on the image to be segmented to obtain N feature images with different resolutions, wherein the feature depths of the N feature images are sequentially increased;
acquiring first target feature images corresponding to central areas of the N feature images, and acquiring second target feature images according to the last M feature images in the N feature images;
the resolutions of the first target feature map and the second target feature map are adjusted to a first preset resolution;
performing fusion convolution processing on the first target feature map and the second target feature map to obtain a segmentation result;
wherein N, M is a positive integer greater than 1, N is greater than M;
The obtaining the first target feature images corresponding to the central areas of the N feature images includes:
respectively carrying out feature cutting processing on the N feature images to obtain the central region feature of each feature image;
performing feature fusion processing on the N central region features to obtain a first target feature map, wherein the first target feature map has image bottom texture feature information, and the image bottom texture feature information comprises at least one of the following items: edge information and direction information.
2. The method of claim 1, further comprising, prior to the acquiring the image to be segmented:
and acquiring the first preset resolution of the segmentation result set by a user and the second preset resolution of the image to be segmented.
3. The method of claim 1, wherein the step of determining the position of the substrate comprises,
the feature map includes features of a defective region and features of a surrounding image of the defective region.
4. The method according to claim 1, wherein the performing feature clipping processing on the N feature maps to obtain a central region feature of each feature map includes:
calculating a first feature resolution of each feature map according to a second preset resolution of an image to be segmented, and calculating a second feature resolution according to the first preset resolution of a segmentation result, wherein the second preset resolution and the first preset resolution are set by a user;
And carrying out feature clipping processing on the feature graphs of each first feature resolution to obtain the central region features of the second feature resolution in the central region of each feature graph.
5. The method of claim 4, wherein the second preset resolution corresponds to equal width and height, the first preset resolution corresponds to equal width and height, the second feature resolution corresponds to equal width and height, and the first feature resolution corresponds to equal width and height;
the formula for calculating the width or height corresponding to the first feature resolution is as follows:
S Fi =S in /2 i+1
wherein S is Fi For the width or height corresponding to the first feature resolution, S in I=1, 2,3 for the width or height corresponding to the second preset resolution;
the formula for calculating the width or height corresponding to the second feature resolution is as follows:
C Fi =S out /2 i+1
wherein C is Fi For the width or height corresponding to the second feature resolution, S out I=1, 2,3 for the width or height corresponding to the first preset resolution.
6. The method according to claim 1, wherein the performing feature fusion processing on the N central region features to obtain a first target feature map includes:
scaling N central region features;
And carrying out feature fusion processing on the N central region features subjected to scaling processing to obtain a first target feature map.
7. The method according to claim 1, wherein the obtaining a second target feature map according to the last M feature maps in the N feature maps includes:
carrying out global average pooling treatment on the last M feature graphs in the N feature graphs to obtain M pooled features, and carrying out feature fusion treatment on the M pooled features to obtain a third target feature graph;
performing feature extraction processing on the third target feature map to obtain feature vectors with the same channel number as the third target feature map;
and multiplying the feature vector with the third target feature map to obtain a second target feature map.
8. The method of claim 1, wherein adjusting the resolution of both the first target feature map and the second target feature map to a first preset resolution comprises:
and carrying out up-sampling processing on the first target feature map and the second target feature map so that the first target feature map and the second target feature map respectively reach a first preset resolution.
9. The method according to any one of claims 1 to 8, wherein the performing a fusion convolution process on the first target feature map and the second target feature map to obtain a segmentation result includes:
performing feature fusion processing on the first target feature map and the second target feature map after the resolution is adjusted to generate a fourth target feature map;
and sequentially carrying out multi-layer convolution processing and maximum value independent variable point set function processing on the fourth target feature map to obtain the segmentation result.
10. The method according to any one of claims 1 to 8, wherein the acquiring an image to be segmented comprises:
acquiring an image of a product to be detected;
and extracting the image of the product to be detected by using a preset extraction method to obtain an image to be segmented.
11. An image dividing apparatus, comprising:
the first acquisition module is used for acquiring an image to be segmented;
the first processing module is used for carrying out neural network convolution processing on the image to be segmented to obtain N feature images with different resolutions, and the feature depths of the N feature images are sequentially increased;
the second processing module is used for acquiring first target feature images corresponding to the central areas of the N feature images and acquiring second target feature images according to the last M feature images in the N feature images;
The third processing module is used for adjusting the resolutions of the first target feature map and the second target feature map to a first preset resolution, and performing fusion convolution processing on the first target feature map and the second target feature map to obtain a segmentation result;
wherein N, M is a positive integer greater than 1, N is greater than M;
the second processing module is specifically configured to:
respectively carrying out feature cutting processing on the N feature images to obtain the central region feature of each feature image;
performing feature fusion processing on the N central region features to obtain a first target feature map, wherein the first target feature map has image bottom texture feature information, and the image bottom texture feature information comprises at least one of the following items: edge information and direction information.
12. The apparatus as recited in claim 11, further comprising:
the second acquisition module is used for acquiring the first preset resolution of the segmentation result set by a user and the second preset resolution of the image to be segmented.
13. The apparatus of claim 11, wherein the device comprises a plurality of sensors,
the feature map includes features of a defective region and features of a surrounding image of the defective region.
14. The apparatus according to claim 11, wherein the second processing module is specifically configured to:
calculating a first feature resolution of each feature map according to a second preset resolution of an image to be segmented, and calculating a second feature resolution according to the first preset resolution of a segmentation result, wherein the second preset resolution and the first preset resolution are set by a user;
and carrying out feature clipping processing on the feature graphs of each first feature resolution to obtain the central region features of the second feature resolution in the central region of each feature graph.
15. The apparatus of claim 14, wherein the second preset resolution corresponds to equal width and height, the first preset resolution corresponds to equal width and height, the second feature resolution corresponds to equal width and height, and the first feature resolution corresponds to equal width and height;
the formula for calculating the width or height corresponding to the first feature resolution is as follows:
S Fi =S in /2 i+1
wherein S is Fi For the width or height corresponding to the first feature resolution, S in I=1, 2,3 for the width or height corresponding to the second preset resolution;
The formula for calculating the width or height corresponding to the second feature resolution is as follows:
C Fi =S out /2 i+1
wherein C is Fi For the width or height corresponding to the second feature resolution, S out I=1, 2,3 for the width or height corresponding to the first preset resolution.
16. The apparatus according to claim 11, wherein the second processing module is specifically configured to:
scaling N central region features;
and carrying out feature fusion processing on the N central region features subjected to scaling processing to obtain a first target feature map.
17. The apparatus according to claim 11, wherein the second processing module is specifically configured to:
carrying out global average pooling treatment on the last M feature graphs in the N feature graphs to obtain M pooled features, and carrying out feature fusion treatment on the M pooled features to obtain a third target feature graph;
performing feature extraction processing on the third target feature map to obtain feature vectors with the same channel number as the third target feature map;
and multiplying the feature vector with the third target feature map to obtain a second target feature map.
18. The apparatus according to claim 11, wherein the third processing module is specifically configured to:
And carrying out up-sampling processing on the first target feature map and the second target feature map so that the first target feature map and the second target feature map respectively reach a first preset resolution.
19. The apparatus according to any one of claims 11 to 18, wherein the third processing module is specifically configured to:
performing feature fusion processing on the first target feature map and the second target feature map after the resolution is adjusted to generate a fourth target feature map;
and sequentially carrying out multi-layer convolution processing and maximum value independent variable point set function processing on the fourth target feature map to obtain the segmentation result.
20. The apparatus according to any one of claims 11 to 18, wherein the first acquisition module is specifically configured to:
acquiring an image of a product to be detected;
and extracting the image of the product to be detected by using a preset extraction method to obtain an image to be segmented.
21. A computer device comprising a processor and a memory storing a program or instructions that when executed by the processor implement the steps of the image segmentation method as claimed in any one of claims 1 to 10.
22. A readable storage medium having stored thereon a program or instructions which when executed by a processor realizes the steps of the image segmentation method according to any one of claims 1 to 10.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310229248.4A CN115908459B (en) | 2023-03-10 | 2023-03-10 | Image segmentation method, device, computer equipment and readable storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310229248.4A CN115908459B (en) | 2023-03-10 | 2023-03-10 | Image segmentation method, device, computer equipment and readable storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN115908459A CN115908459A (en) | 2023-04-04 |
| CN115908459B true CN115908459B (en) | 2023-05-05 |
Family
ID=85771877
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310229248.4A Active CN115908459B (en) | 2023-03-10 | 2023-03-10 | Image segmentation method, device, computer equipment and readable storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115908459B (en) |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113762396A (en) * | 2021-09-10 | 2021-12-07 | 西南科技大学 | Two-dimensional image semantic segmentation method |
| CN114565048A (en) * | 2022-03-02 | 2022-05-31 | 安徽大学 | Three-stage pest image identification method based on adaptive feature fusion pyramid network |
Family Cites Families (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111681273B (en) * | 2020-06-10 | 2023-02-03 | 创新奇智(青岛)科技有限公司 | Image segmentation method and device, electronic equipment and readable storage medium |
| CN111709929B (en) * | 2020-06-15 | 2023-01-20 | 北京航空航天大学 | Lung canceration region segmentation and classification detection system |
| CN112418232A (en) * | 2020-11-18 | 2021-02-26 | 北京有竹居网络技术有限公司 | Image segmentation method and device, readable medium and electronic equipment |
| CN112884782B (en) * | 2021-03-02 | 2024-01-05 | 深圳市瑞图生物技术有限公司 | Biological object segmentation method, apparatus, computer device, and storage medium |
| CN113538313B (en) * | 2021-07-22 | 2022-03-25 | 深圳大学 | Polyp segmentation method and device, computer equipment and storage medium |
-
2023
- 2023-03-10 CN CN202310229248.4A patent/CN115908459B/en active Active
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113762396A (en) * | 2021-09-10 | 2021-12-07 | 西南科技大学 | Two-dimensional image semantic segmentation method |
| CN114565048A (en) * | 2022-03-02 | 2022-05-31 | 安徽大学 | Three-stage pest image identification method based on adaptive feature fusion pyramid network |
Also Published As
| Publication number | Publication date |
|---|---|
| CN115908459A (en) | 2023-04-04 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111027547B (en) | Automatic detection method for multi-scale polymorphic target in two-dimensional image | |
| CN111881720B (en) | Automatic enhancement and expansion method, recognition method and system for data for deep learning | |
| CN110991560B (en) | Target detection method and system combining context information | |
| CN112634209A (en) | Product defect detection method and device | |
| CN114529459B (en) | Method, system and medium for enhancing image edge | |
| CN107871319B (en) | Method and device for detecting beam limiter area, X-ray system and storage medium | |
| US20220366544A1 (en) | Method for high resolution image inpainting, processing system and associated computer program product | |
| CN112712546A (en) | Target tracking method based on twin neural network | |
| CN113657528B (en) | Image feature point extraction method and device, computer terminal and storage medium | |
| CN113362331A (en) | Image segmentation method and device, electronic equipment and computer storage medium | |
| CN112380926A (en) | Weeding path planning system of field weeding robot | |
| US20130236099A1 (en) | Apparatus and method for extracting foreground layer in image sequence | |
| CN110782385A (en) | Image watermark removing method based on deep learning | |
| WO2014198029A1 (en) | Image completion based on patch offset statistics | |
| CN112669300A (en) | Defect detection method and device, computer equipment and storage medium | |
| CN115908459B (en) | Image segmentation method, device, computer equipment and readable storage medium | |
| CN113592714A (en) | Image amplification method, module and system | |
| CN111241891B (en) | Face image cutting method and device and computer readable storage medium | |
| CN107945196B (en) | Image edge detection method for aerial photography correction image splicing | |
| CN113688753B (en) | Static face dynamic method, system, computer equipment and readable storage medium | |
| CN115631198A (en) | Crack detection method and device for glass display screen and computer equipment | |
| CN111523533B (en) | Method and device for determining area of object from image | |
| CN112348021A (en) | Text detection method, device, equipment and storage medium | |
| CN108470326B (en) | Image completion method and device | |
| CN108364273B (en) | Method for multi-focus image fusion in spatial domain |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |