CN115731397A - Method and device for repairing uncertain edge points in significance detection - Google Patents
Method and device for repairing uncertain edge points in significance detection Download PDFInfo
- Publication number
- CN115731397A CN115731397A CN202211173894.5A CN202211173894A CN115731397A CN 115731397 A CN115731397 A CN 115731397A CN 202211173894 A CN202211173894 A CN 202211173894A CN 115731397 A CN115731397 A CN 115731397A
- Authority
- CN
- China
- Prior art keywords
- uncertain
- edge
- boundary
- significance
- points
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Landscapes
- Image Analysis (AREA)
Abstract
The invention discloses a method and a device for repairing an uncertain edge point by significance detection, wherein the method comprises the following steps: A. integrating a boundary renderer into an encoding and decoding style significance detection network, and performing model training to obtain optimal model parameters; B. performing threshold segmentation processing on the output result of the edge detection branch by using a trained model to obtain a boundary region of the target class object; C. acquiring a predetermined number of salient uncertain points at a boundary region of a target object by using a position generator of the uncertain boundary points; D. fusing the coarse features and the band features of each saliency uncertainty point based on the position of the saliency uncertainty point; E. and using a multilayer perceptron to reclassify by using the fused rough features and the strip features. The repairing method and the repairing device utilize the multilayer perceptron to reclassify and replace the reclassification result into the initial result.
Description
Technical Field
The invention relates to an intelligent image processing technology, in particular to an edge thinning method for saliency detection.
Background
In the saliency detection task, saliency detection predicts each pixel. Convolutional neural networks can easily provide coding of segmentation information for various methods. Full Convolution Networks (FCNs) use full convolution structures for saliency detection and build jump structures to connect saliency information at different depths in the convolution layers. In FCN, the mask result is predicted by transposed convolution using features 8x, 16x, 32x smaller than the input. UNet is also a fully convolutional network with a symmetric architecture in the encoding and decoding of feature maps. UNet connects feature maps using the same resolution in both the encoder and decoder and recovers these features using transposed convolution, outputting the mask result at a higher resolution.
Although these methods utilize the excellent feature extraction capability of the convolution operator, the feature mapping, which is 8 or 16 times smaller than the input, is too rough to be segmented. When these coarse masks are upsampled or resized with the result that the input image size is the same, there is a blurring problem on the target class boundary of the mask result, which affects the accuracy of saliency detection. To avoid the down-sampling effect in the mask-based model, the scholars have proposed some contour-based segmentation models that differentiate objects by contours formed by the vertices of the contours. These models can obtain sharp contours of the same resolution as the input image by determining the coordinates of the vertices of the contours. The Polygon-RNN model and the Polygon-RNN + + model use RNN to find the silhouette vertices one by one. The Curve-GCN implements Graph Convolution Network (GCN), and the coordinates of the contour vertex are obtained through regression. The Curve-GCN can adjust the coordinates of a fixed number of vertices from the initial contour to the target at the same time.
Although the above contour-based segmentation methods avoid the influence by down-sampling and directly restore the resolution, they cannot provide complicated edges due to the limitation that the number of contour vertices is fixed, and thus the segmentation effect on uncertain boundaries is not good.
Disclosure of Invention
In order to solve the technical problem that the segmentation effect on an uncertain boundary is poor due to the fact that complex edges cannot be provided due to the fact that the number of contour vertexes is limited in the prior art, the invention provides a repairing method and a repairing device for significance detection of uncertain edge points. The rendering of segmentation results was learned in the work of PointRend, which uses mask scores to select uncertain points around the contour. Different from PointRend, the boundary renderer of the uncertain points of the method acquires a certain number of uncertain points by focusing on the edge area of the target object, reclassifies the uncertain points by using a multi-layer perceptron in combination with the rough characteristic and the strip characteristic of each point, and replaces the reclassification result into an initial result.
In order to achieve the aim, the invention adopts the following technical scheme.
A repair method for saliency detection of uncertain edge points, said method comprising the steps of:
A. integrating a boundary renderer into a coding and decoding style significance detection network, outputting a high-dimensional characteristic diagram obtained by a decoding module to a significance detection branch and an edge detection branch, and performing model training to obtain optimal model parameters;
B. performing threshold segmentation processing on the output result of the edge detection branch by using a trained model to obtain a boundary region of the target class object;
C. acquiring a predetermined number of salient uncertainty points at a boundary region of a target object using a location generator of uncertainty boundary points;
D. fusing the coarse features and the band features of each saliency uncertainty point based on the position of the saliency uncertainty point;
E. and using a multilayer perceptron to reclassify by using the fused rough features and the strip features.
In the method for repairing uncertain edge points for significance detection, the boundary renderer is integrated into a coding and decoding style significance detection network, the high-dimensional feature map obtained by the decoding module is output to a significance detection branch and an edge detection branch, model training is performed, and the optimal model parameters are obtained by:
a1, obtaining high-dimensional characteristic diagram output of the last stage of a decoding module;
a2, taking the high-dimensional characteristic diagram as an input of a significance detection branch, outputting a significance detection mask, and calculating a loss value to update model parameters;
a3, taking the high-dimensional feature map as the input of an edge detection branch, outputting an edge detection mask, and calculating a loss value to update model parameters;
a4, classifying the points acquired in the training stage by using a multilayer perceptron, outputting the significance score of each point, and calculating a loss value to update the model parameters;
and A5, when the loss values in A2, A3 and A4 are converged, obtaining the optimal model parameters.
In the method for repairing the uncertain edge points for significance detection, loss values in the steps A2, A3 and A4 are binary cross entropies, and the calculation method comprises the following steps:
wherein y is the real value of the parameter in the training sample, and y ^ is the predicted value.
In the method for repairing the uncertain edge points in the significance detection, the threshold segmentation processing is carried out on the output result of the edge detection branch by using a trained model, and the obtaining of the boundary area of the target class object comprises the following steps:
b1, mapping the high-dimensional feature graph into a single-channel feature graph through an edge detection branch output result by utilizing 1x1 convolution;
b2, compressing the value range of the characteristic diagram of the single channel to 0-1 to obtain the edge strength;
and B3, performing threshold segmentation on the edge strength according to a preset threshold to obtain a boundary region of the target class object.
In the method for repairing an uncertain edge point for significance detection according to the present invention, obtaining a certain number of uncertain points of significance in a boundary region of a target object by using a position generator of an uncertain boundary point comprises:
c1, filtering out non-boundary regions based on the boundary regions of the target class objects;
c2, mapping the high-dimensional feature map into a single-channel feature map through a significance detection branch output result by utilizing 1x1 convolution;
c3, compressing the value range of the feature map of the single channel to 0-1 to obtain the significance score of each point in the boundary area;
and C4, selecting a preset number of points with the most uncertain significance by utilizing the significance score condition of each point.
In the method for repairing an uncertain edge point by saliency detection, fusing the rough feature and the strip feature of each uncertain point based on the position of the uncertain point comprises the following steps:
d1, mapping the high-dimensional characteristic graph to obtain rough characteristics through a significance detection branch output result by utilizing 1 multiplied by 1 convolution;
d2, mapping the high-dimensional characteristic graph through an edge detection branch output result by using a strip convolution module to obtain high-dimensional strip characteristics;
and D3, fusing the rough characteristic and the strip characteristic of each saliency uncertain point.
In the method for repairing the uncertain edge points in the significance detection, the method for mapping the high-dimensional characteristic graph by the strip convolution module through the edge detection branch output result to obtain the high-dimensional strip characteristics comprises the following steps:
the method comprises the steps of capturing context information from four different directions of a horizontal direction, a vertical direction, a left diagonal line and a right diagonal line by utilizing a strip convolution module, inputting an input tensor of the strip convolution module into four paths after 1x1 convolution, connecting output mappings of the four strip convolutions, and performing up-sampling operation and convolution to obtain the output of the strip convolution module.
The invention also comprises a repairing device for prominence detection of uncertain edge points, which comprises a boundary renderer training module, an edge segmentation module, a position generator of uncertain edge points, a feature fusion module and a reclassification module, wherein,
the boundary renderer training module is integrated into the coding and decoding style significance detection network and used for outputting the high-dimensional feature map obtained by the decoding module to the significance detection branch and the edge detection branch to perform model training to obtain optimal model parameters;
the edge segmentation module performs threshold segmentation processing on an edge detection branch output result by using a trained model to obtain a target class object boundary region;
the position generator of the uncertain boundary points obtains a certain number of significant uncertain points at the boundary area of the target object by utilizing the position generator of the uncertain boundary points;
the feature fusion module is used for fusing the rough feature and the strip feature of each saliency uncertainty point based on the position of the saliency uncertainty point;
and the reclassification module is used for reclassifying by using the fused rough features and the strip features by using the multilayer perceptron.
In the restoration device for detecting uncertain edge points with significance, the boundary renderer training module comprises: a high-dimensional characteristic diagram obtaining unit, a significance detection branch, an edge detection branch, a multilayer perceptron and an optimal model parameter obtaining unit, wherein,
the high-dimensional characteristic diagram acquisition unit is used for acquiring the high-dimensional characteristic diagram output of the last stage of the decoding module;
the significance detection branch is used for taking the high-dimensional characteristic diagram as input, outputting a significance detection mask, and calculating a loss value to update model parameters;
the edge detection branch is used for taking the high-dimensional characteristic graph as input, outputting an edge detection mask, and calculating a loss value to update the model parameters;
classifying the points acquired in the training stage by the multilayer perceptron, outputting the significance score of each point, and calculating a loss value to update the model parameters;
the optimal model parameter obtaining unit is configured to obtain optimal model parameters when the loss values in A2, A3, and A4 converge.
Drawings
Fig. 1 is a flowchart illustrating a repairing method for saliency detection of an uncertain edge point according to an embodiment of the present invention.
Fig. 2 is a processing diagram of a repairing method for saliency detection of uncertain edge points according to an embodiment of the present invention.
Fig. 3 is a schematic diagram of a strip convolution process of a repairing method for saliency detection of uncertain edge points according to an embodiment of the present invention.
Fig. 4 is a schematic diagram of a point selection process of an uncertain boundary point of a repairing method for saliency detection of an uncertain edge point according to an embodiment of the present invention.
Fig. 5 is a schematic diagram illustrating an application result of the repairing method for saliency detection of an uncertain edge point according to the embodiment of the present invention.
Detailed Description
The present invention will be described in detail below with reference to the accompanying drawings.
Detailed exemplary embodiments are disclosed below. However, specific structural and functional details disclosed herein are merely for purposes of describing example embodiments.
It should be understood, however, that the intention is not to limit the invention to the particular exemplary embodiments disclosed, but to cover all modifications, equivalents, and alternatives falling within the scope of the disclosure. Like reference numerals refer to like elements throughout the description of the figures.
Referring to the drawings, the structures, the proportions, the sizes, and the like shown in the drawings are only used for matching the disclosure of the present invention, so as to be understood and read by those skilled in the art, and are not used for limiting the limit conditions of the present invention, so that the present invention has no technical significance, and any structural modification, proportion relationship change, or size adjustment shall still fall within the scope of the present invention without affecting the efficacy and the achievable purpose of the present invention. Meanwhile, the positional limitation terms used in the present specification are for clarity of description only, and are not intended to limit the scope of the present invention, and changes or modifications of the relative relationship therebetween may be regarded as the scope of the present invention without substantial changes in the technical content.
It will also be understood that the term "and/or" as used herein includes any and all combinations of one or more of the associated listed items. It will be further understood that when an element or unit is referred to as being "connected" or "coupled" to another element or unit, it can be directly connected or coupled to the other element or unit or intervening elements or units may also be present. Moreover, other words used to describe the relationship between components or elements should be understood in the same manner (e.g., "between" versus "directly between," "adjacent" versus "directly adjacent," etc.).
The invention aims to comprehensively and systematically reveal the bacterial interaction relationship in the vaginal micro-ecology through a data processing method, and show the significance of the change of a bacterial interaction network on statistical data under the influence of different individuals and various factors. Provides scientific basis and basic support for data analysis as personalized accurate analysis of vaginitis and restoration of vaginal microecological balance. In order to achieve the purpose, the following technical scheme is adopted in the specific embodiment of the invention.
In the prior art, when a saliency detection model is used for segmenting a target class, a large number of points with uncertain probability appear on a segmentation object boundary, and the saliency detection precision is influenced.
In view of the above problems, an embodiment of the present invention provides an algorithm module for repairing an uncertain edge point of significance detection, where the method first generates an uncertain point list by using an uncertain boundary point position generator, then fuses a coarse feature of each point and a stripe feature having remote context information, and finally reclassifies the fused features by using a multi-layer perceptron, and replaces the reclassification result with an initial result.
Various embodiments of the present application are described in detail below with reference to specific figures. Fig. 1 is a flowchart illustrating a repairing method for saliency detection of an uncertain edge point according to an embodiment of the present invention. Fig. 2 is a processing diagram of a repairing method for saliency detection of uncertain edge points according to an embodiment of the present invention.
Specifically, in an embodiment of the present invention, a repairing method for detecting an uncertain edge point with significance is provided, as shown in the flowchart included in fig. 1, the method specifically includes the following steps.
S1, integrating a boundary renderer into an encoding and decoding style significance detection network, and performing model training to obtain optimal model parameters.
Further, in the embodiment of the present invention, as shown in fig. 2, the obtaining of the optimal parameters of the integrated model mainly includes the following steps:
and S11, acquiring the high-dimensional characteristic diagram output of the last stage of the decoding module.
And S12, taking the high-dimensional characteristic diagram obtained in the S11 as the input of the significance detection branch, outputting a significance detection mask, and calculating a loss value to update the model parameters.
And when the similarity is higher, namely target inheritance between the characteristic graphs of different scales at the front and the back is better, the significance coefficient of the corresponding area is larger, otherwise, the significance coefficient is smaller. The process of multiplying the significant coefficient map by the feature map can be regarded as the process of reassigning the weights of the components of the image, and can highlight the important areas and suppress the flat background.
And on the basis of the acquired significance coefficient, carrying out binarization and local area mean value judgment processing on the significance coefficient, thereby obtaining a significance detection mask capable of removing a flat background.
Therefore, in the specific embodiment of the invention, the local features of the image are enhanced by utilizing the significant feature extraction method, which is beneficial to improving the target detection performance; and the feature map is screened by using a detection mask generated by the significance coefficient, a flat background area is excluded, a prediction target is generated only in an effective area of the image, and the execution efficiency of target detection can be improved.
In the image processing according to the embodiment of the present invention, the convolution kernel refers to a weighting matrix having a predetermined specification, and convolution filtering is performed using the weighting matrix: and rotating the convolution kernel matrix, sequentially sliding each corresponding area on the high-dimensional feature map detection result, and taking the sum of the products of the corresponding sliding area and the corresponding element of the convolution kernel as a new value on the corresponding area of the high-dimensional feature map detection result.
In a specific embodiment, a convolution kernel of 3*3 specification and a sigmoid activation function are adopted to obtain the significance mask Ms of the target object:
M s ＝Sigmoid(conv(SODB(F d )))，
after a significance detection block (SODB) is performed on the high-dimensional feature map Fd of the last stage of the decoding module, conv is a convolution function, and the expression of the Sigmoid function is f (x) = 1/(1+e) -x ) The function has the following characteristics: as x approaches negative infinity, y approaches 0; as x approaches positive infinity, y approaches 1; when x =0, y =1/2.
S13, taking the high-dimensional feature map obtained in the S11 as the input of an edge detection branch, outputting an edge mask, and calculating a loss value to update model parameters;
an edge refers to the portion of the image where the local intensity variation is most significant. The method mainly exists between objects, objects and backgrounds, and regions (including different colors), and is an important basis for image analysis such as image segmentation, texture features, shape features and the like. And performing edge detection on the Gaussian feature map, wherein the obtained result image has the same wave band name in the Gaussian feature map, and a non-zero region is indicated as an edge region.
Similarly, in the embodiment of the present invention, a convolution kernel of 3*3 specification and a sigmoid activation function are used to obtain the edge mask Mb of the target object:
M b ＝Sigmoid(conv(BDB(F d )))，
after carrying out edge detection branching (BDB) on the high-dimensional feature map Fd of the last stage of the decoding module, carrying out convolution and Sigmoid function processing to obtain an edge mask.
And S14, classifying the points acquired in the training stage by the multilayer perceptron, outputting the significance score of each point, and calculating a loss value to update the model parameters.
Specifically, the loss function related to S12, S13, and S14 is binary cross entropy, and the calculation formula is:
wherein, y is a true value,
is a predicted value.
And S15, obtaining the optimal model parameters when the loss values are converged.
And S2, performing threshold segmentation processing on the output result of the edge detection branch to obtain a boundary region of the target object. The method comprises the following specific steps:
s21, mapping the high-dimensional edge detection branch output feature graph into a single-channel feature graph by utilizing 1x1 convolution;
s22, compressing the value range of the characteristic diagram to be between [0,1] by using a sigmoid function, and obtaining the edge strength.
S23, threshold segmentation is performed by setting a threshold of the hyper-parameter according to the predetermined target, for example, in one embodiment of the present invention, the threshold is set to 0.5.
And S3, acquiring a certain number of significant uncertain points at the boundary area of the target object by using the position generator of the uncertain boundary points. The method comprises the following specific steps:
s31, filtering out non-boundary regions based on the boundary regions obtained in the S2;
s32, mapping the high-dimensional saliency detection branch output feature map into a single-channel feature map by utilizing 1x1 convolution;
s33, compressing the value range of the feature map to be between [0,1] by using a sigmoid function to obtain a significance score;
s34, selecting N most uncertain points in the significance detection result according to the scoring condition
The formula adopted is as follows:
wherein, S (n) i ) Representing point n i In the significance mask M s Probability value of (1), B (n) i ) Representing point n i Masking the film M at the edges b The probability value of (c).
Thus, the uncertain boundary point processing provides K uncertain boundary points and a set of high-dimensional bar feature maps with remote context information resulting from the edge detection processing for boundary rendering.
And S4, fusing the rough characteristic and the strip characteristic of each point based on the position of the uncertain point. The method comprises the following specific steps:
s41, obtaining a high-dimensional characteristic diagram of the significance detection branch, and mapping the high-dimensional characteristic diagram by using 1x1 convolution to obtain a rough characteristic F c 。
S42, obtaining Gao Weitiao band feature F output by band convolution module in edge detection branch s 。
Fig. 3 is a schematic diagram of a strip convolution process of a repairing method for saliency detection of uncertain edge points according to an embodiment of the present invention. In steps S41 and S42, the high-dimensional stripe feature map is obtained by the stripe convolution module capturing remote context information from four different directions, namely, four horizontal, vertical, left diagonal and right diagonal directions. Let x be an element of R H×W×C Represents the input tensor of the stripe convolution module, where H, W and C represent the height, width and number of channels. In the strip coil block, X is input into four parallel paths after being subjected to 1 × 1 convolution, each path containing a strip convolution of a shape. The output feature maps of the four strip convolutions are then concatenated and then upsampled and convolved by 1x1 to obtain the output of the strip convolution block.
The definition of the stripe convolution employed therein is as follows:
wherein w ∈ R 2K+1 Represents the convolution kernel with size of 2K +1, D = (D) h ,D w ) Indicates the direction of w, where (1,1), (-1,1), (0,1), and (1,0) indicate the left diagonal, right diagonal, horizontal, and vertical, respectively. Z D ∈R H×W×C' Indicating the direction of the strip convolution and X w the result of the convolution operation. For filter w, k =4 is set in one embodiment of the invention, with 9 parameters per strip convolution, as in the case of the 3*3 convolution filter.
In the above-described strip convolution module, each position in the output feature map is allowed to be associated with a plurality of positions in four directions in the input feature map. The four directions selected coincide with the distribution of most of the borders in the input image and are relatively easy to implement.
Sample points of the edge renderer extract feature information of the points according to position information of the points on the basis of edge points provided by an uncertain point generator, then the foreground probability of the sampling points is predicted through a multi-layer perceptron formed by convolution of 1*1, and finally new scores of the sample points are replaced into an initial significance mask, so that a refined result is obtained.
Fig. 4 is a schematic diagram illustrating a point selection process of an uncertain boundary point of a repairing method for significantly detecting an uncertain edge point according to an embodiment of the present invention. When the sampling points are processed in the training and testing phases, the specific implementation mode of the invention can respectively adopt the rules of two sampling points. In the training phase, in order to increase the diversity of the sampling points, the sampling points consist of three parts: (1) Random sampling, the embodiment of the invention randomly samples KN (K > 1) uncertain candidate points from the uniform distribution; (2) uncertain boundary sampling: generating beta N (N > 1) sampling points by using an uncertain boundary point generator; (3) non-boundary region sampling: (1- β) N uncertainty points are sampled from the non-boundary region.
To balance accuracy and training complexity, a lightly biased sampling strategy is used in one embodiment of the invention, e.g., setting N =18 2 K =3, β =0.75. In the testing phase, since no gradient needs to be calculated, more points are used to obtain a denser prediction, which all originate from the uncertain boundary point generator.
S43, fusing the characteristics obtained in S41 and S42, wherein the fusion formula is as follows:
F fusion ＝Concatenate(F s ,F c )，
and S5, reclassifying the fused features by using a multilayer perceptron and replacing the fused features into an initial result, wherein a final comparison result is shown in FIG. 5.
The boundary renderer optimizes the segmentation result based on the sample point reclassification. Specifically, in the specific embodiment of the present invention, the feature fusion is performed on the band feature map Fs generated by the band convolution module and the rough feature map Fc generated by the saliency detection branch, and then the obtained feature fusion is input into the multilayer perceptron, and the multilayer perceptron predicts the scores of corresponding points and replaces the points generated by the boundary renderer into the initial mask, so that the details of the boundary of the complex object are ensured while the high resolution is restored, and the multilayer perceptron aims to refine the features of the uncertain boundary points.
S point ＝Sigmoid(MLP(F fusion ))，
MLP is a multi-layer perceptron that introduces one to many hidden layers on the basis of a single-layer neural network, the hidden layers being located between an input layer and an output layer.
Corresponding to the method for repairing the uncertain edge points through the significance detection in the embodiment of the invention, the embodiment of the invention also comprises a repairing device for the uncertain edge points through the significance detection, which comprises a boundary renderer training module, an edge segmentation module, a position generator of the uncertain edge points, a feature fusion module and a reclassification module, wherein,
the boundary renderer training module is integrated into the coding and decoding style significance detection network and used for outputting the high-dimensional feature map obtained by the decoding module to the significance detection branch and the edge detection branch to perform model training to obtain optimal model parameters;
the edge segmentation module performs threshold segmentation processing on the output result of the edge detection branch by using the trained model to obtain a boundary region of the target object;
the position generator of the uncertain boundary points obtains a certain number of significant uncertain points at the boundary area of the target object by utilizing the position generator of the uncertain boundary points;
the feature fusion module is used for fusing the rough feature and the strip feature of each saliency uncertainty point based on the position of the saliency uncertainty point;
and the reclassification module is used for reclassifying by using the fused rough features and the strip features by using the multilayer perceptron.
In the restoration device for detecting uncertain edge points in significance, the boundary renderer training module comprises: a high-dimensional characteristic diagram obtaining unit, a significance detection branch, an edge detection branch, a multilayer perceptron and an optimal model parameter obtaining unit, wherein,
the high-dimensional characteristic diagram acquisition unit is used for acquiring the high-dimensional characteristic diagram output of the last stage of the decoding module;
the significance detection branch is used for taking the high-dimensional characteristic diagram as input, outputting a significance detection mask, and calculating a loss value to update model parameters;
the edge detection branch is used for taking the high-dimensional characteristic graph as input, outputting an edge detection mask, and calculating a loss value to update model parameters;
classifying the points acquired in the training stage by the multilayer perceptron, outputting the significance score of each point, and calculating a loss value to update the model parameters;
the optimal model parameter obtaining unit is configured to obtain an optimal model parameter when the loss values in A2, A3, and A4 converge.
The above description shows and describes several preferred embodiments of the present invention, and the embodiments of the present invention are only exemplary examples of the technical process of the present invention under the current technical conditions, and there is a great optimization and promotion space without departing from the technical principles, steps, functions, applications and implementation framework of the present invention, and these improvements, optimizations, etc. are also considered as the protection scope of the present patent. Therefore, as previously stated, it is to be understood that the invention is not limited to the forms disclosed herein, but is not to be construed as excluding other embodiments and may be used in various other combinations, modifications, and environments and is capable of changes within the scope of the inventive concept as expressed herein, commensurate with the above teachings, or the skill or knowledge of the relevant art. And that modifications and variations may be effected by those skilled in the art without departing from the spirit and scope of the invention as defined by the appended claims.
Claims (9)
1. A method for repairing an uncertain edge point by saliency detection, the method comprising the steps of:
A. integrating a boundary renderer into a coding and decoding style significance detection network, outputting a high-dimensional feature map obtained by a decoding module to a significance detection branch and an edge detection branch, and performing model training to obtain optimal model parameters;
B. performing threshold segmentation processing on the output result of the edge detection branch by using a trained model to obtain a boundary region of the target class object;
C. acquiring a certain number of significant uncertain points at the boundary area of the target object by utilizing a position generator of the uncertain boundary points;
D. fusing the coarse features and the band features of each saliency uncertainty point based on the position of the saliency uncertainty point;
E. and using a multilayer perceptron to reclassify by using the fused rough features and the strip features.
2. The method for repairing an uncertain edge point through saliency detection according to claim 1, wherein the step of integrating the boundary renderer into the coding and decoding style saliency detection network outputs the high-dimensional feature map obtained by the decoding module to the saliency detection branch and the edge detection branch, and the step of performing model training to obtain the optimal model parameters comprises the steps of:
a1, obtaining high-dimensional characteristic diagram output of the last stage of a decoding module;
a2, taking the high-dimensional characteristic diagram as an input of a significance detection branch, outputting a significance detection mask, and calculating a loss value to update model parameters;
a3, taking the high-dimensional feature map as the input of an edge detection branch, outputting an edge detection mask, and calculating a loss value to update model parameters;
a4, classifying the points acquired in the training stage by using a multilayer perceptron, outputting the significance score of each point, and calculating a loss value to update the model parameters;
and A5, when the loss values in A2, A3 and A4 are converged, obtaining the optimal model parameters.
3. The restoration method for saliency detection of uncertain edge points according to claim 2, characterized in that the loss values in steps A2, A3 and A4 are binary cross entropy calculated by:
wherein y is the real value of the parameter in the training sample,
is a predicted value.
4. The method for repairing an uncertain edge point in significance detection according to claim 1, wherein the threshold segmentation processing is performed on the output result of the edge detection branch by using a trained model to obtain the boundary region of the target object, and the method comprises the following steps:
b1, mapping the high-dimensional feature graph into a single-channel feature graph through an edge detection branch output result by utilizing 1x1 convolution;
b2, compressing the value range of the characteristic diagram of the single channel to 0-1 to obtain the edge strength;
and B3, performing threshold segmentation on the edge strength according to a preset threshold to obtain a boundary region of the target class object.
5. The method for repairing an uncertain edge point for saliency detection as claimed in claim 1, wherein the obtaining a certain number of uncertain points of saliency at the boundary region of the target object using the position generator of uncertain boundary points comprises:
c1, filtering out non-boundary regions based on the boundary regions of the target class objects;
c2, mapping the high-dimensional feature map into a single-channel feature map through a significance detection branch output result by utilizing 1x1 convolution;
c3, compressing the value range of the feature map of the single channel to 0-1 to obtain the significance score of each point in the boundary area;
and C4, selecting a certain number of most uncertain points by utilizing the significance scoring condition of each point.
6. A restoration method for saliency detection uncertain edge points as claimed in claim 1, characterized in that fusing the coarse and band features of each saliency uncertainty point based on its position comprises:
d1, mapping the high-dimensional characteristic graph through a significance detection branch output result by utilizing 1x1 convolution to obtain a rough characteristic;
d2, mapping the high-dimensional characteristic graph through an edge detection branch output result by using a strip convolution module to obtain high-dimensional strip characteristics;
and D3, fusing the rough characteristic and the strip characteristic of each saliency uncertain point.
7. The method for repairing uncertain edge points of significance detection as claimed in claim 6, wherein the obtaining of high-dimensional band features by mapping the high-dimensional feature graph through the edge detection branch output result by using the band convolution module comprises:
the method comprises the steps of capturing context information from four different directions of a horizontal direction, a vertical direction, a left diagonal line and a right diagonal line by utilizing a strip convolution module, inputting an input tensor of the strip convolution module into four paths after 1x1 convolution, connecting output mappings of the four strip convolutions, and performing up-sampling operation and convolution to obtain the output of the strip convolution module.
8. A restoration device for detecting uncertain edge points in a significance mode comprises a boundary renderer training module, an edge segmentation module, a position generator of the uncertain edge points, a feature fusion module and a reclassification module, wherein,
the boundary renderer training module is integrated into the coding and decoding style significance detection network and used for outputting the high-dimensional feature map obtained by the decoding module to the significance detection branch and the edge detection branch to perform model training to obtain optimal model parameters;
the edge segmentation module performs threshold segmentation processing on an edge detection branch output result by using a trained model to obtain a target class object boundary region;
the position generator of the uncertain boundary points obtains a certain number of significant uncertain points at the boundary area of the target object by utilizing the position generator of the uncertain boundary points;
the feature fusion module is used for fusing the rough feature and the strip feature of each saliency uncertainty point based on the position of the saliency uncertainty point;
and the reclassification module is used for reclassifying by using the fused rough features and the strip features by using the multilayer perceptron.
9. The restoration apparatus for detecting an uncertain edge point according to claim 8, wherein the boundary renderer training module comprises: a high-dimensional characteristic diagram obtaining unit, a significance detection branch, an edge detection branch, a multilayer perceptron and an optimal model parameter obtaining unit, wherein,
the high-dimensional characteristic diagram acquisition unit is used for acquiring the high-dimensional characteristic diagram output of the last stage of the decoding module;
the significance detection branch is used for taking the high-dimensional characteristic diagram as input, outputting a significance detection mask, and calculating a loss value to update model parameters;
the edge detection branch is used for taking the high-dimensional characteristic graph as input, outputting an edge detection mask, and calculating a loss value to update the model parameters;
the multi-layer perceptron classifies the points acquired in the training stage, outputs the significance score of each point, and calculates the loss value to update the model parameters;
the optimal model parameter obtaining unit is configured to obtain an optimal model parameter when the loss values in A2, A3, and A4 converge.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211173894.5A CN115731397A (en) | 2022-09-26 | 2022-09-26 | Method and device for repairing uncertain edge points in significance detection |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211173894.5A CN115731397A (en) | 2022-09-26 | 2022-09-26 | Method and device for repairing uncertain edge points in significance detection |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN115731397A true CN115731397A (en) | 2023-03-03 |
Family
ID=85293295
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202211173894.5A Pending CN115731397A (en) | 2022-09-26 | 2022-09-26 | Method and device for repairing uncertain edge points in significance detection |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115731397A (en) |
-
2022
- 2022-09-26 CN CN202211173894.5A patent/CN115731397A/en active Pending
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109446992B (en) | Remote sensing image building extraction method and system based on deep learning, storage medium and electronic equipment | |
| CN109829893B (en) | Defect target detection method based on attention mechanism | |
| CN109584248B (en) | Infrared target instance segmentation method based on feature fusion and dense connection network | |
| CN108921799B (en) | Remote sensing image thin cloud removing method based on multi-scale collaborative learning convolutional neural network | |
| CN107564025B (en) | Electric power equipment infrared image semantic segmentation method based on deep neural network | |
| CN109840556B (en) | Image classification and identification method based on twin network | |
| CN111145209B (en) | Medical image segmentation method, device, equipment and storage medium | |
| CN111160407B (en) | Deep learning target detection method and system | |
| CN108961180B (en) | Infrared image enhancement method and system | |
| CN110570440A (en) | Image automatic segmentation method and device based on deep learning edge detection | |
| CN113901972A (en) | Method, device and equipment for detecting remote sensing image building and storage medium | |
| CN115830004A (en) | Surface defect detection method, device, computer equipment and storage medium | |
| Xu et al. | HA U-Net: Improved model for building extraction from high resolution remote sensing imagery | |
| CN116012291A (en) | Industrial part image defect detection method and system, electronic equipment and storage medium | |
| CN115082675A (en) | Transparent object image segmentation method and system | |
| CN113673556A (en) | Hyperspectral image classification method based on multi-scale dense convolution network | |
| Manaf et al. | Hybridization of SLIC and Extra Tree for Object Based Image Analysis in Extracting Shoreline from Medium Resolution Satellite Images. | |
| CN116205876A (en) | Unsupervised notebook appearance defect detection method based on multi-scale standardized flow | |
| CN114445620A (en) | Target segmentation method for improving Mask R-CNN | |
| CN113627481A (en) | Multi-model combined unmanned aerial vehicle garbage classification method for smart gardens | |
| CN113658206A (en) | Plant leaf segmentation method | |
| Bressan et al. | Semantic segmentation with labeling uncertainty and class imbalance | |
| Jiang et al. | Semantic segmentation network combined with edge detection for building extraction in remote sensing images | |
| Ahmadian et al. | Single image super-resolution with self-organization neural networks and image laplace gradient operator | |
| Avots et al. | A new kernel development algorithm for edge detection using singular value ratios |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |