CN114332489A - Image salient target detection method and system based on uncertainty perception - Google Patents

Image salient target detection method and system based on uncertainty perception Download PDF

Info

Publication number
CN114332489A
CN114332489A CN202210249631.1A CN202210249631A CN114332489A CN 114332489 A CN114332489 A CN 114332489A CN 202210249631 A CN202210249631 A CN 202210249631A CN 114332489 A CN114332489 A CN 114332489A
Authority
CN
China
Prior art keywords
uncertainty
features
contour
image
characteristic
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN202210249631.1A
Other languages
Chinese (zh)
Other versions
CN114332489B (en
Inventor
方玉明
张海艳
鄢杰斌
左一帆
姜文晖
刘扬
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Jiangxi University of Finance and Economics
Original Assignee
Jiangxi University of Finance and Economics
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Jiangxi University of Finance and Economics filed Critical Jiangxi University of Finance and Economics
Priority to CN202210249631.1A priority Critical patent/CN114332489B/en
Publication of CN114332489A publication Critical patent/CN114332489A/en
Application granted granted Critical
Publication of CN114332489B publication Critical patent/CN114332489B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Landscapes

  • Image Analysis (AREA)

Abstract

The invention provides an image salient target detection method and system based on uncertainty perception, and the method comprises the following steps: calculating an inner contour uncertainty image and an outer contour uncertainty image through a saliency map in the image; extracting and obtaining an inner contour uncertainty characteristic of the detail, an outer contour uncertainty characteristic of the detail and a significance characteristic through a characteristic encoder; fusing the salient features and the contour uncertainty features of the details to generate complementary features, calculating through a dynamic weighting branch to obtain dynamic weights, distributing, and fusing the salient features through addition operation to obtain refined salient features; inputting the refined significant features and the detailed contour uncertainty features into a decoding branch, and circularly outputting corresponding features to finally obtain a contour uncertainty prediction graph and a significant prediction graph. The method has good effectiveness and superiority in the aspect of detecting the salient object of the image, and has better performance in processing a complex scene.

Description

Image salient target detection method and system based on uncertainty perception
Technical Field
The invention relates to the technical field of computer image processing, in particular to a method and a system for detecting an image salient target based on uncertainty perception.
Background
Today, with the rapid development of informatization, people are exposed to a large amount of image information every day, and people often only care about a small amount of information which is useful for themselves in images. Therefore, it is extremely important and meaningful to extract a portion attracting attention of people from an image efficiently and quickly. The salient object detection is one of the most basic and important research contents in the image processing field, can be used for quickly positioning a region concerned by people and extracting salient information from mass data, and can also be used as a preprocessing step of computer vision tasks such as semantic segmentation, object tracking, image description and the like. Meanwhile, the saliency target detection is widely applied to the artificial intelligence fields of human eye detection, intelligent video monitoring, automobile auxiliary driving systems and the like.
Under the influence of deep learning techniques, Significant Object Detection (SOD) research has made significant progress in the field of computer vision. However, the "uncertainty" problem remains an obstacle to SOD development. In particular, most SOD models are prone to false predictions when facing pixels around (inside or outside) the contours of a salient object, assigning similar saliency probabilities to the pixels of that region. To solve this problem, some existing research methods introduce contour or detail maps as supervisory signals to train the SOD model to make the predicted salient objects visually clearer.
However, the prior art methods only consider the target contour or salient pixels near the target contour and ignore the non-salient pixels near the target contour, which makes them susceptible to the non-salient pixels near the contour. Furthermore, the underutilization of contour information and saliency information also makes prediction of contours difficult.
Disclosure of Invention
In view of the above situation, the main objective of the present invention is to provide a method and a system for detecting an image salient object based on uncertainty perception, so as to solve the above technical problems.
The embodiment of the invention provides an image salient target detection method based on uncertainty perception, wherein the method comprises the following steps:
step one, calculating an inner contour uncertainty image and an outer contour uncertainty image through a saliency image in an image, wherein the saliency image represents a salient target in the image, the inner contour uncertainty image represents salient pixel points near a target contour, the outer contour uncertainty image represents non-salient pixel points near the target contour, and the inner contour uncertainty image and the outer contour uncertainty image are used as binarization labels during iterative calculation;
inputting the image into a feature encoder, and extracting an internal contour uncertainty feature of the detail, an external contour uncertainty feature of the detail and a significant feature through the feature encoder;
fusing the salient features and the detail inner contour uncertainty features through an attention mask to generate first complementary features, fusing the salient features and the detail outer contour uncertainty features through a reverse attention mask to generate second complementary features, respectively carrying out weight distribution on the first complementary features and the second complementary features after dynamic weights are obtained through dynamic weighting branch calculation, and fusing the first complementary features and the second complementary features through addition operation to obtain refined salient features;
inputting the refined significant features and the detailed internal contour uncertainty features into an internal contour decoding branch to output and obtain internal contour uncertainty features; inputting the refined significant features and the outer contour uncertainty features of the details into an outer contour decoding branch to output the outer contour uncertainty features; inputting the refined salient features into a salient decoding branch, and obtaining the salient features of the next stage through two convolution operations;
and step five, circulating the extraction output process corresponding to the internal contour uncertainty characteristic, the external contour uncertainty characteristic and the next-stage saliency characteristic in the step four or more for four times to gradually refine the internal contour uncertainty characteristic, the external contour uncertainty characteristic and the next-stage saliency characteristic respectively to finally obtain an internal contour uncertainty characteristic prediction graph, an external contour uncertainty characteristic prediction graph and a saliency prediction graph.
According to the image salient target detection method based on uncertainty perception, an inner contour uncertainty image and an outer contour uncertainty image are obtained through calculation of a salient image in an image; extracting and obtaining an inner contour uncertainty characteristic of the detail, an outer contour uncertainty characteristic of the detail and a significance characteristic through a characteristic encoder; fusing the salient features and the contour uncertainty features of the details to generate complementary features, calculating through a dynamic weighting branch to obtain dynamic weights, distributing, and fusing the salient features through addition operation to obtain refined salient features; inputting the refined significant features and the detailed contour uncertainty features into a decoding branch, and circularly outputting corresponding features to finally obtain a contour uncertainty prediction graph and a significant prediction graph. The method has good effectiveness and superiority in the aspect of detecting the salient object of the image, and has better performance in processing a complex scene.
In the image salient object detection method based on uncertainty perception, in the first step, the calculation formula of the inner contour uncertainty image and the outer contour uncertainty image obtained through calculation of the saliency map in the image is represented as follows:
Figure 501340DEST_PATH_IMAGE001
wherein the content of the first and second substances,
Figure 267171DEST_PATH_IMAGE002
a saliency map representing the image is shown,
Figure 422209DEST_PATH_IMAGE003
representing the internal contour uncertainty map in a manner that,
Figure 471942DEST_PATH_IMAGE004
representing the outer contour uncertainty map,
Figure 850971DEST_PATH_IMAGE005
the corrosion function is expressed as a function of corrosion,
Figure 307360DEST_PATH_IMAGE006
the function of the expansion is represented by,
Figure 62826DEST_PATH_IMAGE007
which represents the size of the kernel for the etching operation,
Figure 671793DEST_PATH_IMAGE008
representing the kernel size of the dilation operation.
The image salient object detection method based on uncertainty perception comprises the following two specific steps:
inputting the image into a ResNet-50 network to generate features of five scales;
using two 64 channels for the five scale features
Figure 221723DEST_PATH_IMAGE009
The convolution operation and a spatial attention module to obtain an inner contour uncertainty characteristic of the detail and an outer contour uncertainty characteristic of the detail respectively;
based on the characteristics of the five scales, the significant characteristics are obtained by extracting two convolution layers
Figure 165408DEST_PATH_IMAGE010
。
The image salient object detection method based on uncertainty perception is characterized in that the characteristics of the five scales are represented as follows:
Figure 458987DEST_PATH_IMAGE011
；
the internal profile uncertainty characteristics of the detail are expressed as:
Figure 420995DEST_PATH_IMAGE012
；
the outer contour uncertainty characteristics of the detail are represented as:
Figure 204143DEST_PATH_IMAGE013
；
the spatial attention module is calculated in the following way:
Figure 635125DEST_PATH_IMAGE014
wherein the content of the first and second substances,
Figure 483126DEST_PATH_IMAGE015
the spatial attention module is represented as a function of the spatial attention module,
Figure 50374DEST_PATH_IMAGE016
a feature representing the input is presented to the user,
Figure 207686DEST_PATH_IMAGE017
to represent
Figure 125963DEST_PATH_IMAGE018
The amount of the above-mentioned convolution layer is,
Figure 10611DEST_PATH_IMAGE019
a sigmoid function is represented as a function,
Figure 697945DEST_PATH_IMAGE020
representing a multiplication operation at the element level,
Figure 26158DEST_PATH_IMAGE021
an internal contour uncertainty characteristic representing the detail,
Figure 166152DEST_PATH_IMAGE022
showing detailsAn outer contour uncertainty feature.
The image salient object detection method based on uncertainty perception comprises the following steps of fusing a salient feature and an internal contour uncertainty feature of a detail through an attention mask to generate a first complementary feature:
connecting the salient features with the internal contour uncertainty features of the detail to obtain first connected features, using
Figure 621535DEST_PATH_IMAGE023
The convolution operation of (a) compresses the first connected feature to obtain a first compressed feature
Figure 163375DEST_PATH_IMAGE024
；
Applying the first compression characteristic
Figure 662490DEST_PATH_IMAGE024
Inputting into two convolution layers, and generating attention mask by sigmoid function
Figure 289780DEST_PATH_IMAGE025
；
Enhancing complementary information between the salient features and the internal contour uncertainty features of the detail by using multiplication to obtain the first complementary features;
the corresponding calculation formula is expressed as:
Figure 516231DEST_PATH_IMAGE026
Figure 178156DEST_PATH_IMAGE027
Figure 848172DEST_PATH_IMAGE028
wherein the content of the first and second substances,
Figure 962759DEST_PATH_IMAGE029
the uncertainty characteristics of the internal contour are represented,
Figure 494365DEST_PATH_IMAGE030
a first compression characteristic is represented by a first compression characteristic,
Figure 10797DEST_PATH_IMAGE031
to represent
Figure 851714DEST_PATH_IMAGE023
The convolution operation of (a) is performed,
Figure 453597DEST_PATH_IMAGE032
to represent
Figure 241424DEST_PATH_IMAGE033
The convolution operation of (a) is performed,
Figure 658368DEST_PATH_IMAGE034
it is shown that the connection operation is performed,
Figure 670186DEST_PATH_IMAGE035
an attention mask is shown in which the position of the eye,
Figure 493786DEST_PATH_IMAGE036
the salient features are represented by a representation of,
Figure 85304DEST_PATH_IMAGE037
representing the first complementary feature.
The image salient object detection method based on uncertainty perception comprises the following steps of fusing salient features and external contour uncertainty features of details through a reverse attention mask to generate second complementary features:
connecting the salient features with the outer contour uncertainty features of the detail to obtain second connected features, using
Figure 61482DEST_PATH_IMAGE023
The convolution operation of (a) compresses the second connected feature to obtain a second compressed feature
Figure 244201DEST_PATH_IMAGE038
；
Subtracting each pixel value in the generated attention mask with 1 to obtain a reverse attention mask
Figure 820676DEST_PATH_IMAGE039
；
Enhancing complementary information between the salient features and the outer contour uncertainty features of the detail using multiplication operations to obtain the second complementary features;
the corresponding calculation formula is expressed as:
Figure 747044DEST_PATH_IMAGE040
Figure 76263DEST_PATH_IMAGE041
Figure 695463DEST_PATH_IMAGE042
wherein the content of the first and second substances,
Figure 493655DEST_PATH_IMAGE043
the uncertainty characteristics of the outer contour are represented,
Figure 958134DEST_PATH_IMAGE044
the second compression characteristic is represented by a second set of values,
Figure 908904DEST_PATH_IMAGE045
the reverse attention mask is shown with the intent of,
Figure 699005DEST_PATH_IMAGE046
represents the aboveA second complementary feature.
The image salient object detection method based on uncertainty perception is characterized in that a calculation formula of dynamic weight obtained through dynamic weighting branch calculation is represented as follows:
Figure 718914DEST_PATH_IMAGE047
wherein the content of the first and second substances,
Figure 518243DEST_PATH_IMAGE048
the dynamic weight is represented by a weight value representing the dynamic weight,
Figure 556475DEST_PATH_IMAGE049
which represents the average pooling layer, is,
Figure 517477DEST_PATH_IMAGE050
a fully-connected layer is shown,
Figure 24682DEST_PATH_IMAGE051
it is indicated that the softmax function is,
Figure 362123DEST_PATH_IMAGE052
indicating the ReLU activation function.
The image salient object detection method based on uncertainty perception comprises the following steps of:
first output characteristic of previous layer through up-sampling layer
Figure 756326DEST_PATH_IMAGE053
Upsampling to obtain internal profile uncertainty features associated with the detail
Figure 153809DEST_PATH_IMAGE021
Same size sampled first output characteristic
Figure 882731DEST_PATH_IMAGE054
；
Pairing the salient features by an upsampling layer
Figure 23862DEST_PATH_IMAGE036
Upsampling to obtain internal profile uncertainty features associated with the detail
Figure 302266DEST_PATH_IMAGE021
Same size post-sampling first salient feature
Figure 339492DEST_PATH_IMAGE055
；
To the first output characteristic after sampling
Figure 86868DEST_PATH_IMAGE054
And the first significant feature after sampling
Figure 500532DEST_PATH_IMAGE055
Performing multiplication and addition to obtain a decoded first intermediate feature
Figure 134906DEST_PATH_IMAGE056
；
According to the decoded first intermediate feature
Figure 343034DEST_PATH_IMAGE056
Internal profile uncertainty characterization from said detail
Figure 312127DEST_PATH_IMAGE021
Performing a join and convolution operation to obtain the internal contour uncertainty feature prediction graph
Figure 529482DEST_PATH_IMAGE057
；
The corresponding calculation formula is expressed as:
Figure 516898DEST_PATH_IMAGE058
Figure 161506DEST_PATH_IMAGE059
Figure 617895DEST_PATH_IMAGE060
Figure 373362DEST_PATH_IMAGE061
wherein the content of the first and second substances,
Figure 982329DEST_PATH_IMAGE062
it is indicated that the up-sampling operation,
Figure 1100DEST_PATH_IMAGE063
representing an element-level addition operation.
The image salient object detection method based on uncertainty perception comprises the following steps of:
second output characteristic of previous layer by up-sampling layer
Figure 475944DEST_PATH_IMAGE064
Upsampling to obtain an outer contour uncertainty characteristic associated with the detail
Figure 35101DEST_PATH_IMAGE022
Same size of sampled second output characteristic
Figure 997110DEST_PATH_IMAGE065
；
Pairing the salient features by an upsampling layer
Figure 186783DEST_PATH_IMAGE036
Upsampling to obtain an outer contour uncertainty characteristic associated with the detail
Figure 617764DEST_PATH_IMAGE022
Same size post-sampling second salient feature
Figure 511771DEST_PATH_IMAGE066
；
To the second output characteristic after sampling
Figure 829751DEST_PATH_IMAGE065
And second significant feature after sampling
Figure 190325DEST_PATH_IMAGE066
Performing multiplication and addition to obtain a decoded second intermediate feature
Figure 843023DEST_PATH_IMAGE067
；
According to the decoded second intermediate characteristic
Figure 743983DEST_PATH_IMAGE067
External profile uncertainty characterization of said detail
Figure 431316DEST_PATH_IMAGE022
Performing a join and convolution operation to obtain the outer contour uncertainty feature prediction map
Figure 32235DEST_PATH_IMAGE068
；
The corresponding calculation formula is expressed as:
Figure 172229DEST_PATH_IMAGE069
Figure 611300DEST_PATH_IMAGE070
Figure 153140DEST_PATH_IMAGE071
Figure 402987DEST_PATH_IMAGE072
the invention also provides an image salient object detection system based on uncertainty perception, wherein the system comprises:
the image processing device comprises a first calculation module, a second calculation module and a third calculation module, wherein the first calculation module is used for calculating an inner contour uncertainty image and an outer contour uncertainty image through a saliency image in an image, the saliency image represents a salient object in the image, the inner contour uncertainty image represents salient pixel points near a contour of the object, the outer contour uncertainty image represents non-salient pixel points near the contour of the object, and the inner contour uncertainty image and the outer contour uncertainty image are used as binarization labels during iterative calculation;
the characteristic extraction module is used for inputting the image into a characteristic encoder, and extracting and obtaining an inner contour uncertainty characteristic of the detail, an outer contour uncertainty characteristic of the detail and a significance characteristic through the characteristic encoder;
the second calculation module is used for fusing the salient features and the internal contour uncertainty features of the details through an attention mask to generate first complementary features, fusing the salient features and the external contour uncertainty features of the details through a reverse attention mask to generate second complementary features, respectively carrying out weight distribution on the first complementary features and the second complementary features after dynamic weights are obtained through dynamic weighting branch calculation, and fusing the first complementary features and the second complementary features through addition operation to obtain refined salient features;
a feature output module, configured to input the refined significant features and the internal contour uncertainty features of the details into an internal contour decoding branch to output an internal contour uncertainty feature; inputting the refined significant features and the outer contour uncertainty features of the details into an outer contour decoding branch to output the outer contour uncertainty features; inputting the refined salient features into a salient decoding branch, and obtaining the salient features of the next stage through two convolution operations;
and the prediction graph determining module is used for circulating the extraction output process corresponding to the internal contour uncertainty characteristic, the external contour uncertainty characteristic and the next-stage saliency characteristic for more than four times so as to gradually refine the internal contour uncertainty characteristic, the external contour uncertainty characteristic and the next-stage saliency characteristic respectively and finally obtain an internal contour uncertainty characteristic prediction graph, an external contour uncertainty characteristic prediction graph and a saliency prediction graph.
Additional aspects and advantages of the invention will be set forth in part in the description which follows and, in part, will be obvious from the description, or may be learned by practice of the invention.
Drawings
FIG. 1 is a flowchart of a method for detecting a salient object in an image based on uncertainty sensing according to the present invention;
fig. 2 is a schematic structural diagram of an image salient object detection system based on uncertainty perception according to the present invention.
Detailed Description
Reference will now be made in detail to embodiments of the present invention, examples of which are illustrated in the accompanying drawings, wherein like or similar reference numerals refer to the same or similar elements or elements having the same or similar function throughout. The embodiments described below with reference to the accompanying drawings are illustrative only for the purpose of explaining the present invention, and are not to be construed as limiting the present invention.
These and other aspects of embodiments of the invention will be apparent with reference to the following description and attached drawings. In the description and drawings, particular embodiments of the invention have been disclosed in detail as being indicative of some of the ways in which the principles of the embodiments of the invention may be practiced, but it is understood that the scope of the embodiments of the invention is not limited correspondingly. On the contrary, the embodiments of the invention include all changes, modifications and equivalents coming within the spirit and terms of the claims appended hereto.
Referring to fig. 1, the present invention provides a method for detecting an image salient object based on uncertainty perception, wherein the method includes the following steps:
and S101, calculating an inner contour uncertainty image and an outer contour uncertainty image through a saliency map in the image.
The salient map represents a salient target in an image, the inner contour uncertainty map represents salient pixel points near a target contour, the outer contour uncertainty map represents non-salient pixel points near the target contour, and the inner contour uncertainty map and the outer contour uncertainty map are used as binarization labels in iterative computation.
In the invention, in order to relieve the uncertainty problem of pixel significance probability prediction around a significant target contour, three supervision signals of an internal contour uncertainty image, a significant image of an image and an external contour uncertainty image are introduced for joint supervision so as to guide model learning. Wherein the saliency map focuses on salient objects in the image, and the inner contour uncertainty map and the outer contour uncertainty map focus on pixels around the contours of the salient objects.
Specifically, the calculation formula of the inner contour uncertainty map and the outer contour uncertainty map obtained by calculating the saliency map in the image is expressed as follows:
Figure 30278DEST_PATH_IMAGE001
wherein the content of the first and second substances,
Figure 273040DEST_PATH_IMAGE002
a saliency map representing the image is shown,
Figure 934966DEST_PATH_IMAGE003
representing the internal contour uncertainty map in a manner that,
Figure 854249DEST_PATH_IMAGE004
representing the outer contour uncertainty map,
Figure 703256DEST_PATH_IMAGE005
indicating corrosion contentThe number of the first and second groups is,
Figure 15289DEST_PATH_IMAGE006
the function of the expansion is represented by,
Figure 531721DEST_PATH_IMAGE007
which represents the size of the kernel for the etching operation,
Figure 123370DEST_PATH_IMAGE008
representing the kernel size of the dilation operation.
S102, inputting the image into a feature encoder, and extracting the internal contour uncertainty feature, the external contour uncertainty feature and the saliency feature of the detail through the feature encoder.
Specifically, step S102 includes the following substeps:
and S1021, inputting the image into a ResNet-50 network to generate characteristics of five scales.
S1022, for the characteristics of the five scales, the method utilizes two 64 channels
Figure 459674DEST_PATH_IMAGE073
And a spatial attention module to obtain an inner contour uncertainty characteristic of the detail and an outer contour uncertainty characteristic of the detail, respectively.
S1023, extracting the salient features through two convolution layers based on the features of the five scales
Figure 44239DEST_PATH_IMAGE010
。
Wherein, the characteristics of the five scales are represented as:
Figure 415177DEST_PATH_IMAGE011
；
the internal profile uncertainty characteristics of the detail are expressed as:
Figure 676263DEST_PATH_IMAGE012
；
the outer contour uncertainty characteristics of the detail are represented as:
Figure 765442DEST_PATH_IMAGE013
；
the spatial attention module is calculated in the following way:
Figure 888119DEST_PATH_IMAGE014
wherein the content of the first and second substances,
Figure 582405DEST_PATH_IMAGE015
the spatial attention module is represented as a function of the spatial attention module,
Figure 765125DEST_PATH_IMAGE016
a feature representing the input is presented to the user,
Figure 826753DEST_PATH_IMAGE017
to represent
Figure 753121DEST_PATH_IMAGE018
The amount of the above-mentioned convolution layer is,
Figure 98651DEST_PATH_IMAGE019
a sigmoid function is represented as a function,
Figure 452272DEST_PATH_IMAGE020
representing a multiplication operation at the element level,
Figure 499732DEST_PATH_IMAGE021
an internal contour uncertainty characteristic representing the detail,
Figure 229790DEST_PATH_IMAGE022
an outer contour uncertainty characteristic representing the detail.
S103, fusing the salient features and the internal contour uncertainty features of the details through an attention mask to generate first complementary features, fusing the salient features and the external contour uncertainty features of the details through a reverse attention mask to generate second complementary features, respectively carrying out weight distribution on the first complementary features and the second complementary features after dynamic weights are obtained through dynamic weighting branch calculation, and fusing the first complementary features and the second complementary features through addition operation to obtain refined salient features.
Wherein the method of fusing the salient features and the internal contour uncertainty features of the detail through the attention mask to generate first complementary features comprises the steps of:
s1031a, connecting the significant features and the internal contour uncertainty features of the details to obtain first connected features, and using the first connected features
Figure 429827DEST_PATH_IMAGE023
The convolution operation of (a) compresses the first connected feature to obtain a first compressed feature
Figure 954350DEST_PATH_IMAGE074
。
S1031b, compressing the first compression characteristic
Figure 990570DEST_PATH_IMAGE074
Inputting into two convolution layers, and generating attention mask by sigmoid function
Figure 789899DEST_PATH_IMAGE025
。
And S1031c, enhancing complementary information between the significant features and the internal contour uncertainty features of the details by utilizing multiplication operation to obtain the first complementary features.
The corresponding calculation formula is expressed as:
Figure 844442DEST_PATH_IMAGE026
Figure 539866DEST_PATH_IMAGE027
Figure 561917DEST_PATH_IMAGE028
wherein the content of the first and second substances,
Figure 633779DEST_PATH_IMAGE029
the uncertainty characteristics of the internal contour are represented,
Figure 542829DEST_PATH_IMAGE030
a first compression characteristic is represented by a first compression characteristic,
Figure 674733DEST_PATH_IMAGE031
to represent
Figure 419966DEST_PATH_IMAGE023
The convolution operation of (a) is performed,
Figure 498780DEST_PATH_IMAGE032
to represent
Figure 59075DEST_PATH_IMAGE033
The convolution operation of (a) is performed,
Figure 565143DEST_PATH_IMAGE034
it is shown that the connection operation is performed,
Figure 46939DEST_PATH_IMAGE035
an attention mask is shown in which the position of the eye,
Figure 709871DEST_PATH_IMAGE036
the salient features are represented by a representation of,
Figure 593513DEST_PATH_IMAGE037
representing the first complementary feature.
Further, the method of fusing the salient features and the outer contour uncertainty features of the detail through the inverse attention mask to generate second complementary features comprises the steps of:
s1032a, distinguishing the significanceConnecting the feature with the outer contour uncertainty feature of the detail to obtain a second connected feature, using
Figure 67220DEST_PATH_IMAGE023
The convolution operation of (a) compresses the second connected feature to obtain a second compressed feature
Figure 36313DEST_PATH_IMAGE038
。
S1032b, subtracting each pixel value in the generated attention mask by 1 to obtain a reverse attention mask
Figure 4400DEST_PATH_IMAGE039
。
S1032c, enhancing complementary information between the salient feature and the outer contour uncertainty feature of the detail by utilizing multiplication operation to obtain the second complementary feature.
The corresponding calculation formula is expressed as:
Figure 742549DEST_PATH_IMAGE040
Figure 387157DEST_PATH_IMAGE041
Figure 843546DEST_PATH_IMAGE042
wherein the content of the first and second substances,
Figure 848280DEST_PATH_IMAGE043
the uncertainty characteristics of the outer contour are represented,
Figure 706515DEST_PATH_IMAGE044
the second compression characteristic is represented by a second set of values,
Figure 256445DEST_PATH_IMAGE045
the reverse attention mask is shown with the intent of,
Figure 200130DEST_PATH_IMAGE046
representing the second complementary feature.
In this embodiment, the calculation formula of the dynamic weight obtained by the dynamic weighting branch calculation is represented as:
Figure 775599DEST_PATH_IMAGE047
wherein the content of the first and second substances,
Figure 488340DEST_PATH_IMAGE048
the dynamic weight is represented by a weight value representing the dynamic weight,
Figure 474750DEST_PATH_IMAGE049
which represents the average pooling layer, is,
Figure 374573DEST_PATH_IMAGE050
a fully-connected layer is shown,
Figure 721110DEST_PATH_IMAGE051
it is indicated that the softmax function is,
Figure 553937DEST_PATH_IMAGE052
indicating the ReLU activation function.
As supplementary explanation, after the dynamic weight is obtained by calculation, weights are assigned to the salient features, the inner contour uncertainty features and the outer contour uncertainty features to extract complementary features between the contour uncertainty features and the salient features, and the salient features are fused by addition operation to refine and enhance the salient features.
S104, inputting the refined significant features and the detailed internal contour uncertainty features into an internal contour decoding branch to output and obtain internal contour uncertainty features; inputting the refined significant features and the outer contour uncertainty features of the details into an outer contour decoding branch to output the outer contour uncertainty features; inputting the refined salient features into a salient decoding branch, and obtaining the salient features of the next stage through two convolution operations.
In this step, the method for generating the prediction graph of uncertainty characteristics of the internal contour includes the following steps:
s1041a, first output characteristic of previous layer through up-sampling layer
Figure 976828DEST_PATH_IMAGE053
Upsampling to obtain internal profile uncertainty features associated with the detail
Figure 363947DEST_PATH_IMAGE021
Same size sampled first output characteristic
Figure 281218DEST_PATH_IMAGE054
；
S1041b pairing the salient features by an upsampling layer
Figure 968552DEST_PATH_IMAGE036
Upsampling to obtain internal profile uncertainty features associated with the detail
Figure 296765DEST_PATH_IMAGE021
Same size post-sampling first salient feature
Figure 436759DEST_PATH_IMAGE055
；
S1041c, sampling the first output characteristic
Figure 125098DEST_PATH_IMAGE054
And the first significant feature after sampling
Figure 666938DEST_PATH_IMAGE055
Performing multiplication and addition to obtain a decoded first intermediate feature
Figure 166052DEST_PATH_IMAGE056
；
S1041d, according to the decoded first intermediate characteristic
Figure 793343DEST_PATH_IMAGE056
Internal profile uncertainty characterization from said detail
Figure 239368DEST_PATH_IMAGE021
Performing a join and convolution operation to obtain the internal contour uncertainty feature prediction graph
Figure 386446DEST_PATH_IMAGE057
；
The corresponding calculation formula is expressed as:
Figure 56462DEST_PATH_IMAGE058
Figure 171049DEST_PATH_IMAGE059
Figure 217502DEST_PATH_IMAGE060
Figure 717622DEST_PATH_IMAGE061
wherein the content of the first and second substances,
Figure 558540DEST_PATH_IMAGE062
it is indicated that the up-sampling operation,
Figure 426001DEST_PATH_IMAGE063
representing an element-level addition operation.
Further, the method for generating the external contour uncertainty characteristic prediction map comprises the following steps:
s1042a, throughSecond output characteristic of sampling layer to previous layer
Figure 744987DEST_PATH_IMAGE064
Upsampling to obtain an outer contour uncertainty characteristic associated with the detail
Figure 132237DEST_PATH_IMAGE022
Same size of sampled second output characteristic
Figure 144056DEST_PATH_IMAGE065
；
S1042b, comparing the significant features through an upsampling layer
Figure 233234DEST_PATH_IMAGE036
Upsampling to obtain an outer contour uncertainty characteristic associated with the detail
Figure 355911DEST_PATH_IMAGE022
Same size post-sampling second salient feature
Figure 830624DEST_PATH_IMAGE066
；
S1042c, second output characteristic after sampling
Figure 278923DEST_PATH_IMAGE065
And second significant feature after sampling
Figure 589818DEST_PATH_IMAGE066
Performing multiplication and addition to obtain a decoded second intermediate feature
Figure 516186DEST_PATH_IMAGE067
；
S1042d, according to the decoded second intermediate characteristic
Figure 612449DEST_PATH_IMAGE067
External profile uncertainty characterization of said detail
Figure 966070DEST_PATH_IMAGE022
Performing a join and convolution operation to obtain the outer contour uncertainty feature prediction map
Figure 764262DEST_PATH_IMAGE068
；
The corresponding calculation formula is expressed as:
Figure 494320DEST_PATH_IMAGE069
Figure 943625DEST_PATH_IMAGE070
Figure 671410DEST_PATH_IMAGE071
Figure 956898DEST_PATH_IMAGE072
and S105, circulating the extraction output process corresponding to the internal contour uncertainty characteristic, the external contour uncertainty characteristic and the next-stage saliency characteristic for more than four times to respectively refine the internal contour uncertainty characteristic, the external contour uncertainty characteristic and the next-stage saliency characteristic step by step so as to finally obtain an internal contour uncertainty characteristic prediction graph, an external contour uncertainty characteristic prediction graph and a saliency prediction graph.
In this embodiment, for the final output inner contour uncertainty prediction map, the significant prediction map and the outer contour uncertainty prediction map, a loss function corresponding to each is used, and an overall training loss function is combined to optimize the prediction. The significance prediction graph is calculated by using loss binarization cross entropy and an intersection ratio, and a loss function corresponding to the internal contour uncertainty characteristic prediction graph and a loss function corresponding to the external contour uncertainty prediction graph are calculated by using the intersection ratio.
Specifically, the cross entropy of binarization
Figure 225068DEST_PATH_IMAGE075
Cross-over ratio
Figure 279612DEST_PATH_IMAGE076
Is expressed as:
Figure 991347DEST_PATH_IMAGE077
Figure 764131DEST_PATH_IMAGE078
wherein the content of the first and second substances,
Figure 570413DEST_PATH_IMAGE079
a normalized prediction graph is represented that represents the predicted graph,
Figure 745042DEST_PATH_IMAGE080
a label representing the binary value is attached to the label,
Figure 860634DEST_PATH_IMAGE081
and
Figure 120714DEST_PATH_IMAGE082
respectively represent the width and height corresponding to the binary label,
Figure 730687DEST_PATH_IMAGE083
pixel coordinate values representing the binarized label,
Figure 25402DEST_PATH_IMAGE084
pixel coordinate values representing the prediction map,
Figure 78940DEST_PATH_IMAGE085
indicating the coordinate position.
In addition, the overall training loss function is a weighted sum of losses corresponding to the internal contour uncertainty prediction graph, the significant prediction graph, and the external contour uncertainty prediction graph, and is specifically expressed as:
Figure 560737DEST_PATH_IMAGE086
wherein the content of the first and second substances,
Figure 974401DEST_PATH_IMAGE087
the training loss function as a whole is represented,
Figure 858043DEST_PATH_IMAGE088
a function representing the significant loss is represented by,
Figure 581018DEST_PATH_IMAGE089
the internal profile loss function is represented as,
Figure 550111DEST_PATH_IMAGE090
the outer contour loss function is represented as,
Figure 501886DEST_PATH_IMAGE091
and
Figure 505614DEST_PATH_IMAGE092
are all weights used to balance the losses.
According to the image salient target detection method based on uncertainty perception, an inner contour uncertainty image and an outer contour uncertainty image are obtained through calculation of a salient image in an image; extracting and obtaining an inner contour uncertainty characteristic of the detail, an outer contour uncertainty characteristic of the detail and a significance characteristic through a characteristic encoder; fusing the salient features and the contour uncertainty features of the details to generate complementary features, calculating through a dynamic weighting branch to obtain dynamic weights, distributing, and fusing the salient features through addition operation to obtain refined salient features; inputting the refined significant features and the detailed contour uncertainty features into a decoding branch, and circularly outputting corresponding features to finally obtain a contour uncertainty prediction graph and a significant prediction graph. The method has good effectiveness and superiority in the aspect of detecting the salient object of the image, and has better performance in processing a complex scene.
The invention has the advantages that:
(1) guiding the proposed model to pay attention to the detection of the significant target by jointly supervising three labels of the internal contour uncertainty image, the significant image and the external contour uncertainty image, and transferring partial attention to 'uncertainty' pixels near the contour of the significant target;
(2) a new feature interaction module is provided to enhance complementary feature interaction between the inner contour uncertainty feature, the saliency feature and the outer contour uncertainty feature;
(3) the method for detecting the salient object of the research image is not only beneficial to people to quickly find out useful information from the image, but also beneficial to the development of other computer vision tasks taking salient object detection as a preprocessing step. Therefore, an effective and accurate prediction image salient object detection algorithm can promote the development of image processing.
Referring to fig. 2, the present invention further provides a system for detecting an image salient object based on uncertainty perception, wherein the system includes:
the image processing device comprises a first calculation module, a second calculation module and a third calculation module, wherein the first calculation module is used for calculating an inner contour uncertainty image and an outer contour uncertainty image through a saliency image in an image, the saliency image represents a salient object in the image, the inner contour uncertainty image represents salient pixel points near a contour of the object, the outer contour uncertainty image represents non-salient pixel points near the contour of the object, and the inner contour uncertainty image and the outer contour uncertainty image are used as binarization labels during iterative calculation;
the characteristic extraction module is used for inputting the image into a characteristic encoder, and extracting and obtaining an inner contour uncertainty characteristic of the detail, an outer contour uncertainty characteristic of the detail and a significance characteristic through the characteristic encoder;
the second calculation module is used for fusing the salient features and the internal contour uncertainty features of the details through an attention mask to generate first complementary features, fusing the salient features and the external contour uncertainty features of the details through a reverse attention mask to generate second complementary features, respectively carrying out weight distribution on the first complementary features and the second complementary features after dynamic weights are obtained through dynamic weighting branch calculation, and fusing the first complementary features and the second complementary features through addition operation to obtain refined salient features;
a feature output module, configured to input the refined significant features and the internal contour uncertainty features of the details into an internal contour decoding branch to output an internal contour uncertainty feature; inputting the refined significant features and the outer contour uncertainty features of the details into an outer contour decoding branch to output the outer contour uncertainty features; inputting the refined salient features into a salient decoding branch, and obtaining the salient features of the next stage through two convolution operations;
and the prediction graph determining module is used for circulating the extraction output process corresponding to the internal contour uncertainty characteristic, the external contour uncertainty characteristic and the next-stage saliency characteristic for more than four times so as to gradually refine the internal contour uncertainty characteristic, the external contour uncertainty characteristic and the next-stage saliency characteristic respectively and finally obtain an internal contour uncertainty characteristic prediction graph, an external contour uncertainty characteristic prediction graph and a saliency prediction graph.
It should be understood that portions of the present invention may be implemented in hardware, software, firmware, or a combination thereof. In the above embodiments, the various steps or methods may be implemented in software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, any one or combination of the following techniques, which are known in the art, may be used: a discrete logic circuit having a logic gate circuit for implementing a logic function on a data signal, an application specific integrated circuit having an appropriate combinational logic gate circuit, a Programmable Gate Array (PGA), a Field Programmable Gate Array (FPGA), or the like.
In the description herein, references to the description of the term "one embodiment," "some embodiments," "an example," "a specific example," or "some examples," etc., mean that a particular feature, structure, material, or characteristic described in connection with the embodiment or example is included in at least one embodiment or example of the invention. In this specification, the schematic representations of the terms used above do not necessarily refer to the same embodiment or example. Furthermore, the particular features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples.
The above-mentioned embodiments only express several embodiments of the present invention, and the description thereof is more specific and detailed, but not construed as limiting the scope of the present invention. It should be noted that, for a person skilled in the art, several variations and modifications can be made without departing from the inventive concept, which falls within the scope of the present invention. Therefore, the protection scope of the present patent shall be subject to the appended claims.

Claims (10)

1. An image salient object detection method based on uncertainty perception is characterized by comprising the following steps: step one, calculating an inner contour uncertainty image and an outer contour uncertainty image through a saliency image in an image, wherein the saliency image represents a salient target in the image, the inner contour uncertainty image represents salient pixel points near a target contour, the outer contour uncertainty image represents non-salient pixel points near the target contour, and the inner contour uncertainty image and the outer contour uncertainty image are used as binarization labels during iterative calculation;
inputting the image into a feature encoder, and extracting an internal contour uncertainty feature of the detail, an external contour uncertainty feature of the detail and a significant feature through the feature encoder;
fusing the salient features and the detail inner contour uncertainty features through an attention mask to generate first complementary features, fusing the salient features and the detail outer contour uncertainty features through a reverse attention mask to generate second complementary features, respectively carrying out weight distribution on the first complementary features and the second complementary features after dynamic weights are obtained through dynamic weighting branch calculation, and fusing the first complementary features and the second complementary features through addition operation to obtain refined salient features;
inputting the refined significant features and the detailed internal contour uncertainty features into an internal contour decoding branch to output and obtain internal contour uncertainty features; inputting the refined significant features and the outer contour uncertainty features of the details into an outer contour decoding branch to output the outer contour uncertainty features; inputting the refined salient features into a salient decoding branch, and obtaining the salient features of the next stage through two convolution operations;
and step five, circulating the extraction output process corresponding to the internal contour uncertainty characteristic, the external contour uncertainty characteristic and the next-stage saliency characteristic in the step four or more for four times to gradually refine the internal contour uncertainty characteristic, the external contour uncertainty characteristic and the next-stage saliency characteristic respectively to finally obtain an internal contour uncertainty characteristic prediction graph, an external contour uncertainty characteristic prediction graph and a saliency prediction graph.
2. The method for detecting the image salient objects based on the uncertainty perception according to claim 1, wherein in the step one, the calculation formula of the inner contour uncertainty map and the outer contour uncertainty map obtained by the calculation of the saliency map in the image is represented as follows:
Figure 403759DEST_PATH_IMAGE001
wherein the content of the first and second substances,
Figure 366905DEST_PATH_IMAGE002
a saliency map representing the image is shown,
Figure 677800DEST_PATH_IMAGE003
representing the internal contour uncertainty map in a manner that,
Figure 604168DEST_PATH_IMAGE004
representing the outer contour uncertainty map,
Figure 949699DEST_PATH_IMAGE005
the corrosion function is expressed as a function of corrosion,
Figure 54052DEST_PATH_IMAGE006
the function of the expansion is represented by,
Figure 586665DEST_PATH_IMAGE007
which represents the size of the kernel for the etching operation,
Figure 582303DEST_PATH_IMAGE008
representing the kernel size of the dilation operation.
3. The image salient object detection method based on uncertainty perception according to claim 2, wherein the second step specifically comprises:
inputting the image into a ResNet-50 network to generate features of five scales;
using two 64 channels for the five scale features
Figure 782340DEST_PATH_IMAGE009
The convolution operation and a spatial attention module to obtain an inner contour uncertainty characteristic of the detail and an outer contour uncertainty characteristic of the detail respectively;
based on the characteristics of the five scales, the significant characteristics are obtained by extracting two convolution layers
Figure 556130DEST_PATH_IMAGE010
。
4. The image salient object detection method based on uncertainty perception according to claim 3, wherein the five scales of features are represented as:
Figure 576038DEST_PATH_IMAGE011
；
the internal profile uncertainty characteristics of the detail are expressed as:
Figure 109788DEST_PATH_IMAGE012
；
the outer contour uncertainty characteristics of the detail are represented as:
Figure 164331DEST_PATH_IMAGE013
；
the spatial attention module is calculated in the following way:
Figure 876067DEST_PATH_IMAGE014
wherein the content of the first and second substances,
Figure 648851DEST_PATH_IMAGE015
the spatial attention module is represented as a function of the spatial attention module,
Figure 455132DEST_PATH_IMAGE016
a feature representing the input is presented to the user,
Figure 364183DEST_PATH_IMAGE017
to represent
Figure 745354DEST_PATH_IMAGE018
The amount of the above-mentioned convolution layer is,
Figure 5434DEST_PATH_IMAGE019
a sigmoid function is represented as a function,
Figure 615407DEST_PATH_IMAGE020
representing a multiplication operation at the element level,
Figure 378964DEST_PATH_IMAGE021
an internal contour uncertainty characteristic representing the detail,
Figure 432502DEST_PATH_IMAGE022
an outer contour uncertainty characteristic representing the detail.
5. The image salient object detection method based on uncertainty perception according to claim 4, wherein the method for fusing the salient features and the internal contour uncertainty features of the details through the attention mask to generate the first complementary features comprises the following steps:
connecting the salient features with the internal contour uncertainty features of the detail to obtain first connected features, using
Figure 914299DEST_PATH_IMAGE023
The convolution operation of (a) compresses the first connected feature to obtain a first compressed feature
Figure 327962DEST_PATH_IMAGE024
；
Applying the first compression characteristic
Figure 211605DEST_PATH_IMAGE024
Inputting into two convolution layers, and generating attention mask by sigmoid function
Figure 934579DEST_PATH_IMAGE025
；
Enhancing complementary information between the salient features and the internal contour uncertainty features of the detail by using multiplication to obtain the first complementary features;
the corresponding calculation formula is expressed as:
Figure 903672DEST_PATH_IMAGE026
Figure 121027DEST_PATH_IMAGE027
Figure 124755DEST_PATH_IMAGE028
wherein the content of the first and second substances,
Figure 520095DEST_PATH_IMAGE029
the uncertainty characteristics of the internal contour are represented,
Figure 976484DEST_PATH_IMAGE030
a first compression characteristic is represented by a first compression characteristic,
Figure 731951DEST_PATH_IMAGE031
to represent
Figure 324606DEST_PATH_IMAGE023
The convolution operation of (a) is performed,
Figure 389383DEST_PATH_IMAGE032
to represent
Figure 333068DEST_PATH_IMAGE033
The convolution operation of (a) is performed,
Figure 892226DEST_PATH_IMAGE034
it is shown that the connection operation is performed,
Figure 604967DEST_PATH_IMAGE035
an attention mask is shown in which the position of the eye,
Figure 810951DEST_PATH_IMAGE036
the salient features are represented by a representation of,
Figure 241932DEST_PATH_IMAGE037
representing the first complementary feature.
6. The image salient object detection method based on uncertainty perception according to claim 5, wherein the method for fusing the salient features and the external contour uncertainty features of the details through the inverse attention mask to generate second complementary features comprises the following steps:
connecting the salient features with the outer contour uncertainty features of the detail to obtain second connected features, using
Figure 339201DEST_PATH_IMAGE023
The convolution operation of (a) compresses the second connected feature to obtain a second compressed feature
Figure 172028DEST_PATH_IMAGE038
；
Subtracting each pixel value in the generated attention mask with 1 to obtain a reverse attention mask
Figure 313028DEST_PATH_IMAGE039
；
Enhancing complementary information between the salient features and the outer contour uncertainty features of the detail using multiplication operations to obtain the second complementary features;
the corresponding calculation formula is expressed as:
Figure 231306DEST_PATH_IMAGE040
Figure 132266DEST_PATH_IMAGE041
Figure 819599DEST_PATH_IMAGE042
wherein the content of the first and second substances,
Figure 632965DEST_PATH_IMAGE043
the uncertainty characteristics of the outer contour are represented,
Figure 772960DEST_PATH_IMAGE044
the second compression characteristic is represented by a second set of values,
Figure 477611DEST_PATH_IMAGE045
the reverse attention mask is shown with the intent of,
Figure 19450DEST_PATH_IMAGE046
representing the second complementary feature.
7. The image salient object detection method based on uncertainty perception according to claim 6, wherein the calculation formula for obtaining dynamic weight through dynamic weighting branch calculation is represented as:
Figure 767832DEST_PATH_IMAGE047
wherein the content of the first and second substances,
Figure 395123DEST_PATH_IMAGE048
the dynamic weight is represented by a weight value representing the dynamic weight,
Figure 637885DEST_PATH_IMAGE049
which represents the average pooling layer, is,
Figure 768652DEST_PATH_IMAGE050
a fully-connected layer is shown,
Figure 438668DEST_PATH_IMAGE051
it is indicated that the softmax function is,
Figure 303987DEST_PATH_IMAGE052
indicating the ReLU activation function.
8. The image salient object detection method based on uncertainty perception according to claim 7, wherein the generation method of the internal contour uncertainty feature prediction map comprises the following steps:
first output characteristic of previous layer through up-sampling layer
Figure 84861DEST_PATH_IMAGE053
Upsampling to obtain internal profile uncertainty features associated with the detail
Figure 601293DEST_PATH_IMAGE021
Same size sampled first output characteristic
Figure 442210DEST_PATH_IMAGE054
；
Pairing the salient features by an upsampling layer
Figure 293361DEST_PATH_IMAGE036
Upsampling to obtain internal profile uncertainty features associated with the detail
Figure 877926DEST_PATH_IMAGE021
Same size post-sampling first salient feature
Figure 983285DEST_PATH_IMAGE055
；
To the first output characteristic after sampling
Figure 995103DEST_PATH_IMAGE054
And the first significant feature after sampling
Figure 569435DEST_PATH_IMAGE055
Performing multiplication and addition to obtain a decoded first intermediate feature
Figure 223270DEST_PATH_IMAGE056
；
According to the decoded first intermediate feature
Figure 183136DEST_PATH_IMAGE056
Internal profile uncertainty characterization from said detail
Figure 365856DEST_PATH_IMAGE021
Performing a join and convolution operation to obtain the internal contour uncertainty feature prediction graph
Figure 191598DEST_PATH_IMAGE057
；
The corresponding calculation formula is expressed as:
Figure 117966DEST_PATH_IMAGE058
Figure 197917DEST_PATH_IMAGE059
Figure 817118DEST_PATH_IMAGE060
Figure 366042DEST_PATH_IMAGE061
wherein the content of the first and second substances,
Figure 96100DEST_PATH_IMAGE062
it is indicated that the up-sampling operation,
Figure 30558DEST_PATH_IMAGE063
representing an element-level addition operation.
9. The image salient object detection method based on uncertainty perception according to claim 7, wherein the generation method of the outer contour uncertainty feature prediction map comprises the following steps:
second output characteristic of previous layer by up-sampling layer
Figure 820660DEST_PATH_IMAGE064
Upsampling to obtain an outer contour uncertainty characteristic associated with the detail
Figure 355415DEST_PATH_IMAGE022
Same size of sampled second output characteristic
Figure 623585DEST_PATH_IMAGE065
；
Pairing the salient features by an upsampling layer
Figure 412550DEST_PATH_IMAGE036
Upsampling to obtain an outer contour uncertainty characteristic associated with the detail
Figure 373553DEST_PATH_IMAGE022
Same size post-sampling second salient feature
Figure 880757DEST_PATH_IMAGE066
；
To the second output characteristic after sampling
Figure 703351DEST_PATH_IMAGE065
And second significant feature after sampling
Figure 877980DEST_PATH_IMAGE066
Performing multiplication and addition to obtain a decoded second intermediate feature
Figure 744305DEST_PATH_IMAGE067
；
According to the decoded second intermediate characteristic
Figure 4385DEST_PATH_IMAGE067
External profile uncertainty characterization of said detail
Figure 863626DEST_PATH_IMAGE022
Performing a join and convolution operation to obtain the outer contour uncertainty feature prediction map
Figure 892762DEST_PATH_IMAGE068
；
The corresponding calculation formula is expressed as:
Figure 929988DEST_PATH_IMAGE069
Figure 677364DEST_PATH_IMAGE070
Figure 841760DEST_PATH_IMAGE071
Figure 725403DEST_PATH_IMAGE072
。
10. an image salient object detection system based on uncertainty perception, characterized in that the system comprises:
the image processing device comprises a first calculation module, a second calculation module and a third calculation module, wherein the first calculation module is used for calculating an inner contour uncertainty image and an outer contour uncertainty image through a saliency image in an image, the saliency image represents a salient object in the image, the inner contour uncertainty image represents salient pixel points near a contour of the object, the outer contour uncertainty image represents non-salient pixel points near the contour of the object, and the inner contour uncertainty image and the outer contour uncertainty image are used as binarization labels during iterative calculation;
the characteristic extraction module is used for inputting the image into a characteristic encoder, and extracting and obtaining an inner contour uncertainty characteristic of the detail, an outer contour uncertainty characteristic of the detail and a significance characteristic through the characteristic encoder;
the second calculation module is used for fusing the salient features and the internal contour uncertainty features of the details through an attention mask to generate first complementary features, fusing the salient features and the external contour uncertainty features of the details through a reverse attention mask to generate second complementary features, respectively carrying out weight distribution on the first complementary features and the second complementary features after dynamic weights are obtained through dynamic weighting branch calculation, and fusing the first complementary features and the second complementary features through addition operation to obtain refined salient features;
a feature output module, configured to input the refined significant features and the internal contour uncertainty features of the details into an internal contour decoding branch to output an internal contour uncertainty feature; inputting the refined significant features and the outer contour uncertainty features of the details into an outer contour decoding branch to output the outer contour uncertainty features; inputting the refined salient features into a salient decoding branch, and obtaining the salient features of the next stage through two convolution operations;
and the prediction graph determining module is used for circulating the extraction output process corresponding to the internal contour uncertainty characteristic, the external contour uncertainty characteristic and the next-stage saliency characteristic for more than four times so as to gradually refine the internal contour uncertainty characteristic, the external contour uncertainty characteristic and the next-stage saliency characteristic respectively and finally obtain an internal contour uncertainty characteristic prediction graph, an external contour uncertainty characteristic prediction graph and a saliency prediction graph.
CN202210249631.1A 2022-03-15 2022-03-15 Image salient target detection method and system based on uncertainty perception Active CN114332489B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202210249631.1A CN114332489B (en) 2022-03-15 2022-03-15 Image salient target detection method and system based on uncertainty perception

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202210249631.1A CN114332489B (en) 2022-03-15 2022-03-15 Image salient target detection method and system based on uncertainty perception

Publications (2)

Publication Number Publication Date
CN114332489A true CN114332489A (en) 2022-04-12
CN114332489B CN114332489B (en) 2022-06-24

Family

ID=81033136

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202210249631.1A Active CN114332489B (en) 2022-03-15 2022-03-15 Image salient target detection method and system based on uncertainty perception

Country Status (1)

Country Link
CN (1) CN114332489B (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115359258A (en) * 2022-08-26 2022-11-18 中国科学院国家空间科学中心 Weak and small target detection method and system for component uncertainty measurement

Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2015017941A1 (en) * 2013-08-09 2015-02-12 Sweep3D Corporation Systems and methods for generating data indicative of a three-dimensional representation of a scene
US20160140424A1 (en) * 2014-11-13 2016-05-19 Nec Laboratories America, Inc. Object-centric Fine-grained Image Classification
CN110443257A (en) * 2019-07-08 2019-11-12 大连理工大学 A kind of conspicuousness detection method based on Active Learning
US20210146531A1 (en) * 2019-11-20 2021-05-20 Nvidia Corporation Guided uncertainty-aware policy optimization: combining model-free and model-based strategies for sample-efficient learning
CN113326886A (en) * 2021-06-16 2021-08-31 中山大学 Salient object detection method and system based on unsupervised learning
CN114067101A (en) * 2021-11-05 2022-02-18 浙江工业大学 Image significance detection method of double-stream decoder based on information complementation
CN114078138A (en) * 2021-11-23 2022-02-22 中国银行股份有限公司 Image significance detection method and device

Patent Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2015017941A1 (en) * 2013-08-09 2015-02-12 Sweep3D Corporation Systems and methods for generating data indicative of a three-dimensional representation of a scene
US20160140424A1 (en) * 2014-11-13 2016-05-19 Nec Laboratories America, Inc. Object-centric Fine-grained Image Classification
CN110443257A (en) * 2019-07-08 2019-11-12 大连理工大学 A kind of conspicuousness detection method based on Active Learning
US20210146531A1 (en) * 2019-11-20 2021-05-20 Nvidia Corporation Guided uncertainty-aware policy optimization: combining model-free and model-based strategies for sample-efficient learning
CN113326886A (en) * 2021-06-16 2021-08-31 中山大学 Salient object detection method and system based on unsupervised learning
CN114067101A (en) * 2021-11-05 2022-02-18 浙江工业大学 Image significance detection method of double-stream decoder based on information complementation
CN114078138A (en) * 2021-11-23 2022-02-22 中国银行股份有限公司 Image significance detection method and device

Non-Patent Citations (3)

* Cited by examiner, † Cited by third party
Title
A. LI等: ""Uncertainty-aware Joint Salient Object and Camouflaged Object Detection"", 《2021 IEEE/CVF CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION (CVPR)》 *
YUMING FANG等: ""Visual attention prediction for Autism Spectrum Disorder with hierarchical semantic fusion"", 《SIGNAL PROCESSING: IMAGE COMMUNICATION》 *
包晓安等: ""基于背景感知的显著性目标检测算法"", 《计算机系统应用》 *

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115359258A (en) * 2022-08-26 2022-11-18 中国科学院国家空间科学中心 Weak and small target detection method and system for component uncertainty measurement

Also Published As

Publication number Publication date
CN114332489B (en) 2022-06-24

Similar Documents

Publication Publication Date Title
CN112990211B (en) Training method, image processing method and device for neural network
KR20220002067A (en) method, apparatus, computer equipment, computer readable storage medium and computer program for visual qustion answering
CN113592060A (en) Neural network optimization method and device
CN115512103A (en) Multi-scale fusion remote sensing image semantic segmentation method and system
CN113901900A (en) Unsupervised change detection method and system for homologous or heterologous remote sensing image
CN116416416A (en) Training method of virtual fitting model, virtual fitting method and electronic equipment
CN114049356B (en) Method, device and system for detecting structure apparent crack
CN113012177A (en) Three-dimensional point cloud segmentation method based on geometric feature extraction and edge perception coding
CN114359289A (en) Image processing method and related device
CN114332489B (en) Image salient target detection method and system based on uncertainty perception
CN117078930A (en) Medical image segmentation method based on boundary sensing and attention mechanism
CN114821096A (en) Image processing method, neural network training method and related equipment
CN112668608A (en) Image identification method and device, electronic equipment and storage medium
CN111652181A (en) Target tracking method and device and electronic equipment
Zuo et al. A remote sensing image semantic segmentation method by combining deformable convolution with conditional random fields
CN113158970B (en) Action identification method and system based on fast and slow dual-flow graph convolutional neural network
CN114037678A (en) Urine visible component detection method and device based on deep learning
CN112884702A (en) Polyp identification system and method based on endoscope image
CN114565092A (en) Neural network structure determining method and device
CN116824291A (en) Remote sensing image learning method, device and equipment
CN116309050A (en) Image super-resolution method, program product, storage medium and electronic device
CN115984093A (en) Depth estimation method based on infrared image, electronic device and storage medium
CN114399648A (en) Behavior recognition method and apparatus, storage medium, and electronic device
CN113256556A (en) Image selection method and device
CN116824308B (en) Image segmentation model training method and related method, device, medium and equipment

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant