CN110443257B - Significance detection method based on active learning - Google Patents

Significance detection method based on active learning Download PDF

Info

Publication number
CN110443257B
CN110443257B CN201910609780.2A CN201910609780A CN110443257B CN 110443257 B CN110443257 B CN 110443257B CN 201910609780 A CN201910609780 A CN 201910609780A CN 110443257 B CN110443257 B CN 110443257B
Authority
CN
China
Prior art keywords
sample
samples
score
image
training
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN201910609780.2A
Other languages
Chinese (zh)
Other versions
CN110443257A (en
Inventor
张立和
闵一璠
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Dalian University of Technology
Original Assignee
Dalian University of Technology
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Dalian University of Technology filed Critical Dalian University of Technology
Priority to CN201910609780.2A priority Critical patent/CN110443257B/en
Publication of CN110443257A publication Critical patent/CN110443257A/en
Application granted granted Critical
Publication of CN110443257B publication Critical patent/CN110443257B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
    • G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
    • G06F18/2163—Partitioning the feature space
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/24—Classification techniques
    • G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
    • G06F18/2411—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches based on the proximity to a decision surface, e.g. support vector machines
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/20—Image preprocessing
    • G06V10/26—Segmentation of patterns in the image field; Cutting or merging of image elements to establish the pattern region, e.g. clustering-based techniques; Detection of occlusion
    • G06V10/267—Segmentation of patterns in the image field; Cutting or merging of image elements to establish the pattern region, e.g. clustering-based techniques; Detection of occlusion by performing operations on regions, e.g. growing, shrinking or watersheds
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/40—Extraction of image or video features
    • G06V10/46—Descriptors for shape, contour or point-related descriptors, e.g. scale invariant feature transform [SIFT] or bags of words [BoW]; Salient regional features
    • G06V10/462—Salient features, e.g. scale invariant feature transforms [SIFT]

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Data Mining & Analysis (AREA)
  • General Physics & Mathematics (AREA)
  • Physics & Mathematics (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Evolutionary Biology (AREA)
  • Evolutionary Computation (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • General Engineering & Computer Science (AREA)
  • Artificial Intelligence (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Multimedia (AREA)
  • Image Analysis (AREA)

Abstract

The invention belongs to the technical field of artificial intelligence, and provides a significance detection method based on active learning. And then, in order to optimize the target boundary of the saliency map, a super-pixel-level post-processing method is designed to further improve the performance. The invention reduces the marking cost and simultaneously reduces the redundancy of the training set, thereby greatly improving the experimental effect compared with the original KSR model. Meanwhile, comparative experiments show that the performance of the method is superior to that of many classical algorithms.

Description

Significance detection method based on active learning
Technical Field
The invention belongs to the technical field of artificial intelligence, relates to computer vision, and particularly relates to an image saliency detection method.
Background
The economic and technological levels of today's society are rapidly developing and a variety of different fragmented information is received by humans all at once, and image and video information are the most important of these information. How to process image data quickly and effectively becomes a difficult problem to be solved. Usually, one only focuses on the areas of the image that are most attractive to human eyes, i.e. foreground areas or salient objects, while ignoring background areas. Therefore, one uses a computer to simulate the human visual system for saliency detection. At present, the research on significance can be widely applied to various fields of computer vision, including image retrieval, image classification, target recognition, image segmentation and the like.
The target of the saliency detection is to accurately detect the saliency target from the image. The significance detection algorithm based on supervised learning generally has a problem that a large amount of manual marking data is generally needed in the model training process, a large amount of resources are needed for marking a significant region, and redundant information exists in a plurality of training samples, and the redundant information adversely affects the model precision.
Disclosure of Invention
The technical problem to be solved by the invention is as follows: the method makes up the defects of the existing method, provides an image significance detection method based on active learning, and achieves the purpose of obtaining higher model precision by using fewer training samples.
The technical scheme of the invention is as follows:
a significance detection method based on active learning comprises the following steps:
(1) firstly, randomly selecting 500 images from an MSRA database, adding the images into a training set L as an initial training set, respectively generating region candidate segmentation (Propusals) of all the images, and extracting the CNN characteristics of the regions of all the region candidate segmentation;
(2) defining positive and negative samples of the area candidate segmentation, and designing a confidence score (confidence score) to measure the scoring of the samples relative to the foreground and the background of a true value graph, wherein the confidence score is as follows:
Figure GDA0003348160130000021
the score calculates in advance two scores A and C, A being the accuracy score and C being the coverage score, where
Figure GDA0003348160130000022
While
Figure GDA0003348160130000023
Wherein O isiRepresenting the target candidate segmentation of the ith sample, G representing the true value map of the image; where ξ is the weight used to balance the accuracy score and the coverage score; in the method, a sample with a confidence value higher than 0.9 is set as a positive sample, and a sample with a confidence value less than 0.6 is set as a negative sample; because the number of positive samples is found to be much smaller than the number of negative samples when calculating the confidence valueThe number of the samples is that all the positive samples are used, and the negative samples with the same number as the positive samples are randomly selected; for training a sequencing support vector machine, forming positive and negative sample pairs by all positive and negative samples, and defining a positive sample minus negative sample as a positive sample pair, otherwise, a negative sample pair;
using the formula:
Figure GDA0003348160130000024
performing sequencing support vector machine and subspace learning combined training to obtain a sequencer KSR, wherein the sequencer performs significance sequencing on the region candidate segmentation of the sample, and the similarity between the front-ranked region candidate segmentation and the foreground is high; wherein w is the ordering coefficient of the ordering support vector machine; in the formula
Figure GDA0003348160130000025
Is a logic loss function, a is a loss function parameter, e is an exponential function; phi (x)i) Characteristic x representing a sampleiFeatures after mapping by kernel; p is the sample pair xinAnd xjnThe number of constraints of (2); (in, jn) indicates that the sample pair is a subscript to the nth pair constraint; y isnE { +1, -1} indicates whether the sample belongs to the same class or different classes, or whether the sample belongs to the foreground or the background at the same time; l is belonged to Rl×d(l < d) is the learned mapping matrix, l is the initial feature dimension, d is the mapped feature dimension, and μ and λ represent regularization parameters;
selecting training samples by active learning; firstly, the model generated by the initialization is used for carrying out significance ordering on target candidate segmentation of all samples in an unmarked sample pool, and the segmentation is carried out according to si＝wTPkiObtaining a ranking score, and introducing L ═ P phi to simplify the calculation of joint trainingT(X) wherein P ∈ Rl×NN is the number of samples, phiT(X) is a kernel operation, and a kernel function is introduced in the simplification process; using formulas
Figure GDA0003348160130000031
All ranking scores siThe normalization is carried out, and the normalization is carried out,
Figure GDA0003348160130000032
normalized score, s, representing rank scoreminRepresenting the smallest ranking score, s, of the set of ranking scores for the imagemaxRepresenting the largest ranking score in the set of ranking scores for the image; finding images with normalized ranking scores between 0.4 and 0.9 for target candidate segmentation of all images
Figure GDA0003348160130000033
XpRepresenting the selected target candidate segmentation component set, and X represents all target candidate segmentation component sets of the image; by the formula
Figure GDA0003348160130000034
Calculating the ratio beta of the number of the target candidate segmentation scores of the image between 0.4 and 0.9 to all the target candidate segmentations, wherein card (X) represents the number of the target candidate segmentations of the image, and card (X)P) Representative set XpThe target candidate segmentation number of (1); taking the score as an uncertain value of each image; in this way, a set of indeterminate values for the unlabeled samples in all pools is obtained, and its beta ═ beta is selected1,β2,…,βnManually marking samples with medium and high uncertainty, adding a training set, and performing formula
Figure GDA0003348160130000035
Each selection is made where mu0To not determine the mean of the set of values B, δ is the standard deviation of the set, λ0Is a weight parameter, selecting λ01.145; design each selection uncertainty β is greater than μ0+λ0Sample composition set Q of δuc(ii) a For image QucApplying a density clustering algorithm to obtain an optimal parameter epsilon of 0.05, setting the size of MinPts to be 2, and classifying samples into one class when the number of samples in the circle center neighborhood is 2 or more; after clustering, a high density of sample clusters C ═ { C ═ C is obtained1,c2,…cnAnd a cluster of only 1 isolated sample O ═ O1,o2,…om}, final image set QucIs divided into: quc＝{ci,i＝1,2,...n}∪{oiI ═ 1,2,. m }; by the formula
Figure GDA0003348160130000036
From each high-density cluster ctTo select the sample U with the greatest uncertaintytIn addition, all isolated samples are selected and added into the candidate set Q, and the sample points can increase the generalization capability of the training model; the final candidate set is Q ═ Ut,t＝1,...n}∪{OiI ═ 1.. m }; the sample set Q represents a sample selected by a selection model which considers uncertainty and diversity design at the same time, and is added into the training set L after being manually marked;
(3) the sample set Q selected by the work is marked manually, then the marked sample set Q is added into a training set L, a sequencer KSR is trained again by using the updated training set L, then the performance of the model trained at this time is verified on a verification set, the step (2) is repeated continuously until the performance of the model is changed slightly or the performance is reduced, the training set selected by the last iteration is selected as a final training set, the trained model is used as a final training model, the model is used for carrying out significance sorting on the region candidate segmentation of each test image, the region candidate segmentation of 16 bits before ranking is selected for carrying out weighted fusion, and a significance map M of the image is obtainedp；
(4) The saliency map M obtained from step (3)pThe edge detail processing of the target is still insufficient, so the invention provides a processing method on a super-pixel level, and the purpose of optimizing the boundary is achieved. Firstly, the super-pixel segmentation algorithm of SLIC is utilized to set the number of segmented super-pixel blocks to be 100, 150 and 200 respectively, and the segmented super-pixel blocks are used for forming a super-pixel set SP of an image iiSeparately extracting CNN feature x of each superpixel blockj(ii) a The saliency map M obtained from step (3) is usedpBinarization as a prior saliency map Ei(ii) a Determining positive and negative samples of the superpixel, and completely locating the superpixel in the prior saliency map E in order to make the confidence coefficient highestiSuperpixels of foreground region form positive sample set POiCompletely locate the super pixel in the firstSignificance test chart EiThe super pixels of the background region of (2) constitute a negative sample set Ni(ii) a Forming positive and negative sample pairs from the positive and negative sample sets, and using formula
Figure GDA0003348160130000041
To train a model KSR for image ii(ii) a For this model KSRiUsing the formula si＝wTPkiAll superpixels of the image are scored, and all superpixels are sorted into S ═ S1,s2,…snThe higher the score, the closer to the foreground, whereas the lower the score, the closer to the background. Using formulas
Figure GDA0003348160130000042
Obtaining the score of each pixel in the superpixel, normalizing all the scores to be between 0 and 1, and finally obtaining the saliency map M synthesized at the superpixel level through weighting and fusions. The final saliency map is given by the formula M ═ w1×Mp+w2×MSObtaining, wherein M is the final saliency map, MpIs an original significant picture, MsIs a super-pixel level saliency map.
The invention has the beneficial effects that: the significance detection algorithm based on active learning provided by the invention applies the idea of active learning to the significance detection field, selects the sample which is most beneficial to model training from the unlabeled sample set by considering the uncertainty and diversity of the sample, adds the sample into the training set, trains to obtain the final KSR model, and outputs the initial significance map of the test sample by the model. And then, in order to optimize the target boundary of the saliency map, a super-pixel-level post-processing method is designed to further improve the performance. The invention reduces the marking cost and simultaneously reduces the redundancy of the training set, thereby greatly improving the experimental effect compared with the original KSR model. Meanwhile, comparative experiments show that the performance of the method is superior to that of many classical algorithms.
Drawings
FIG. 1 is a basic flow diagram of the present invention.
Fig. 2 is an initial saliency map resulting from the application of active learning to the KSR model.
Fig. 3 is a final saliency map resulting from applying super-pixel level post-processing fusion to an initial saliency map.
Detailed Description
The following further describes a specific embodiment of the present invention with reference to the drawings and technical solutions.
The conception of the invention is as follows: since the supervised learning training process usually requires a large amount of manually labeled data, labeling salient regions requires a large amount of resources, and redundant information exists in many training samples, which adversely affects the model accuracy. The active learning utilizes a selection mechanism to select samples with larger information content for training, and the purpose of obtaining higher model precision by using fewer training samples is achieved. Based on this, the invention combines the idea of Active Learning (AL) with the Kernel Subspace Ranking (KSR) algorithm. The invention designs a pool-based active learning strategy, namely, samples with larger information amount are selected to participate in training by considering the uncertainty and diversity of unmarked samples, so that the aims of reducing the number of training samples and reducing the marking cost are fulfilled.
The method extracts the CNN characteristics of the convolutional neural network of the target-level region candidate segmentation (propusals), utilizes subspace mapping and a sequencing support vector machine to jointly learn a sequencer, performs significance sequencing on the region candidate segmentation of a test image by the sequencer, and performs weighted fusion on the region candidate segmentation in the front of the sequencing to obtain a significance map. Finally, in order to optimize the target boundary of the saliency map, the invention designs a super-pixel-level post-processing method to further improve the performance.
The invention is implemented as follows:
(1) firstly, 500 images are randomly selected from an MSRA database and added into a training set L as an initial training set, region candidate segmentation (propusals) of all the images is respectively generated, and the CNN characteristics of the regions of all the region candidate segmentation are extracted.
(2) Defining positive and negative samples of the region candidate segmentation, and designing a confidence score (confidence score) to measure the sample phase by the algorithmFor the scoring of the truth map foreground and background, the confidence values are:
Figure GDA0003348160130000061
the score calculates in advance two scores A and C, A being the accuracy score and C being the coverage score, where
Figure GDA0003348160130000062
While
Figure GDA0003348160130000063
Wherein O isiRepresenting the target candidate segmentation of the ith sample, G representing the true value map of the image; where ξ is the weight used to balance the accuracy score and the coverage score; in the present algorithm, samples with confidence values higher than 0.9 are set as positive samples, and samples with confidence values less than 0.6 are determined as negative samples; because the number of the positive samples is far less than that of the negative samples when the confidence value is calculated, all the positive samples are used, and the negative samples with the same number as the positive samples are randomly selected; for training a sequencing support vector machine, forming positive and negative sample pairs by all positive and negative samples, and defining a positive sample minus negative sample as a positive sample pair, otherwise, a negative sample pair;
using the formula:
Figure GDA0003348160130000064
performing combined training of a sequencing support vector machine and subspace learning, and performing training to obtain a sequencer KSR, wherein the sequencer performs significance sequencing on the region candidate segmentation of the sample, and the similarity between the front ranking and the foreground is high; wherein w is the ordering coefficient of the ordering support vector machine; in the formula
Figure GDA0003348160130000071
Is a logic loss function, a is a loss function parameter, e is an exponential function; phi (x)i) Characteristic x representing a sampleiFeatures after mapping by kernel; p is the sample pair xinAnd xjnThe number of constraints of (2); (in, jn) indicates that the sample pair is a subscript to the nth pair constraint; y isnE { +1, -1} indicates that the sample belongs to the same classOr whether they are not of the same class, or both, foreground or background; l is belonged to Rl×d(l < d) is the learned mapping matrix, l is the initial feature dimension, d is the mapped feature dimension, and μ and λ represent the regularization parameters.
Selecting training samples by active learning; firstly, the model generated by the initialization is used for carrying out significance ordering on target candidate segmentation of all samples in an unmarked sample pool, and the segmentation is carried out according to si＝wTPkiObtaining ranking scores, and introducing for simplifying calculation of joint training TL＝Pφ(X)Wherein N is the number of samples, Tφ(X)the method is a kernel operation, and a kernel function is introduced in the method in the simplification process. Using formulas
Figure GDA0003348160130000072
All ranking scores siThe normalization is carried out, and the normalization is carried out,
Figure GDA0003348160130000073
normalized score, s, representing rank scoreminRepresenting the smallest ranking score, s, of the set of ranking scores for the imagemaxRepresenting the largest ranking score in the set of ranking scores for the image; finding images with normalized ranking scores between 0.4 and 0.9 for target candidate segmentation of all images
Figure GDA0003348160130000074
XpRepresents the selected set of target candidate segmentation components, and X represents the set of all target candidate segmentation components of the image. By the formula
Figure GDA0003348160130000075
Calculating the ratio beta of the number of the target candidate segmentation scores of the image between 0.4 and 0.9 to all the target candidate segmentations, wherein card (X) represents the number of the target candidate segmentations of the image, and card (X)P) Representative set XpThe target candidate segmentation number in (1). Taking the score as an uncertain value of each image; in this way, an uncertainty value for all unlabeled samples in the sample pool is obtainedAggregate of beta ═ beta1,β2,…,βnSelecting samples with high uncertainty, manually marking the samples, adding the samples into a training set, and performing formula analysis
Figure GDA0003348160130000076
Each selection is made where mu0To not determine the mean of the set of values B, δ is the standard deviation of the set, λ0Is a weight parameter, selecting λ01.145; design each selection uncertainty β is greater than μ0+λ0Sample composition set Q of δuc. For image QucBy applying a density clustering algorithm, the optimal parameter epsilon is obtained through experiments and is 0.05, the size of MinPts is set to be 2, and samples in the neighborhood of the center of a circle are 2 or more, so that the samples can be classified into one class. After clustering, a high density of sample clusters C ═ { C ═ C can be obtained1,c2,…cnAnd some clusters O of only 1 isolated sample O ═ O1,o2,…om}, final image set QucCan be divided into: quc＝{ci,i＝1,2,...n}∪{oiI 1,2,. m }. By the formula
Figure GDA0003348160130000081
From each high-density cluster ctTo select the sample U with the greatest uncertaintytBesides, all isolated samples are selected to be added into the candidate set Q, and such sample points can increase the generalization capability of the training model. The final sum candidate set is Q ═ Ut,t＝1,...n}∪{OiI 1.. m }. The sample set Q represents samples selected by a selection model designed by considering uncertainty and diversity at the same time, and the samples are added into the training set L after being manually marked.
(3) Manually marking the sample set Q selected by the work, adding the sample set Q into a training set L, training a sequencer KSR again by using the updated training set L, verifying the performance of the model trained at this time on a verification set, continuously repeating the step (2) until the performance of the model is changed little or the performance is reduced, selecting the training set selected by the last iteration as a final training set, and training the modelTaking the model as a final training model, carrying out significance ordering on the region candidate segmentation of each test image by the model, selecting the region candidate segmentation with 16 bits at the top of the ranking for weighting and fusing to obtain a significance map M of the imagep。
(4) The saliency map M obtained from step (3)pThe edge detail processing of the target is still insufficient, so the invention provides a processing method on a super-pixel level, and the purpose of optimizing the boundary is achieved. Firstly, the super-pixel segmentation algorithm of SLIC is utilized to set the number of segmented super-pixel blocks to be 100, 150 and 200 respectively, and the segmented super-pixel blocks are used for forming a super-pixel set SP of an image iiSeparately extracting CNN feature x of each superpixel blockj(ii) a The saliency map M obtained from step (3) is usedpBinarization as a prior saliency map Ei(ii) a Determining positive and negative samples of the superpixel, and completely locating the superpixel in the prior saliency map E in order to make the confidence coefficient highestiSuperpixels of foreground region form positive sample set POiCompletely locate the super-pixel in the prior saliency map EiThe super pixels of the background region of (2) constitute a negative sample set Ni(ii) a Forming positive and negative sample pairs from the positive and negative sample sets, and using formula
Figure GDA0003348160130000091
To train a model KSR for image ii(ii) a For this model KSRiUsing the formula si＝wTPkiAll superpixels of the image are scored, and all superpixels are sorted into S ═ S1,s2,…snThe higher the score, the closer to the foreground, whereas the lower the score, the closer to the background. Using formulas
Figure GDA0003348160130000092
Obtaining the score of each pixel in the superpixel, normalizing all the scores to be between 0 and 1, and finally obtaining the saliency map M synthesized at the superpixel level through weighting and fusions. The final saliency map is given by the formula M ═ w1×Mp+w2×MSObtaining, wherein M is the final saliency map, MpIs an original significant picture, MsFor super pixel level saliency maps, w1Is 1, w2Take 0.3.

Claims (1)

1. A significance detection method based on active learning is characterized by comprising the following steps:
(1) firstly, randomly selecting 500 images from an MSRA database, adding the images into a training set L as an initial training set, respectively generating region candidate segmentation of all the images, and extracting the CNN characteristics of the regions of the region candidate segmentation;
(2) defining positive and negative samples of the region candidate segmentation, and designing a confidence value to measure the scoring of the samples relative to the foreground and the background of the truth diagram, wherein the confidence value is as follows:
Figure FDA0003348160120000011
the score calculates in advance two scores A and C, A being the accuracy score and C being the coverage score, where
Figure FDA0003348160120000012
While
Figure FDA0003348160120000013
Wherein O isiRepresenting the target candidate segmentation of the ith sample, G representing the true value map of the image; where ξ is the weight used to balance the accuracy score and the coverage score; in the method, a sample with a confidence value higher than 0.9 is set as a positive sample, and a sample with a confidence value less than 0.6 is set as a negative sample; because the number of the positive samples is far less than that of the negative samples when the confidence value is calculated, all the positive samples are used, and the negative samples with the same number as the positive samples are randomly selected; for training a sequencing support vector machine, forming positive and negative sample pairs by all positive and negative samples, and defining a positive sample minus negative sample as a positive sample pair, otherwise, a negative sample pair;
using the formula:
Figure FDA0003348160120000014
performing rank support vector machine and subspace learningPerforming combined training, wherein a sequencer KSR is obtained through training, the sequencer performs significance sequencing on the region candidate segmentation of the sample, and the similarity between the foreground and the foreground in the front ranking is high; wherein w is the ordering coefficient of the ordering support vector machine; in the formula
Figure FDA0003348160120000015
Is a logic loss function, a is a loss function parameter, e is an exponential function; phi (x)i) Characteristic x representing a sampleiFeatures after mapping by kernel; p is the sample pair xinAnd xjnThe number of constraints of (2); (in, jn) indicates that the sample pair is a subscript to the nth pair constraint; y isnE { +1, -1} indicates whether the samples belong to the same class or different classes; l is belonged to Rl×d(l < d) is the learned mapping matrix, l is the initial feature dimension, d is the mapped feature dimension, and μ and λ represent regularization parameters;
selecting training samples by active learning; firstly, the model generated by the initialization is used for carrying out significance ordering on target candidate segmentation of all samples in an unmarked sample pool, and the segmentation is carried out according to si＝wTPkiObtaining a ranking score, and introducing L ═ P phi to simplify the calculation of joint trainingT(X) wherein P ∈ Rl×NN is the number of samples, phiT(X) is a kernel operation, and a kernel function is introduced in the simplification process; using formulas
Figure FDA0003348160120000021
All ranking scores siThe normalization is carried out, and the normalization is carried out,
Figure FDA0003348160120000022
normalized score, s, representing rank scoreminRepresenting the smallest ranking score, s, of the set of ranking scores for the imagemaxRepresenting the largest ranking score in the set of ranking scores for the image; finding images with normalized ranking scores between 0.4 and 0.9 for target candidate segmentation of all images
Figure FDA0003348160120000023
XpRepresenting the selected target candidate segmentation component set, and X represents all target candidate segmentation component sets of the image; by the formula
Figure FDA0003348160120000024
Calculating the ratio beta of the number of the target candidate segmentation scores of the image between 0.4 and 0.9 to all the target candidate segmentations, wherein card (X) represents the number of the target candidate segmentations of the image, and card (X)P) Representative set XpThe target candidate segmentation number of (1); taking the sorting score as an uncertain value of each image; in this way, a set of indeterminate values beta is obtained for the unlabeled samples in all the cuvettes1,β2,…,βnSelecting beta ═ beta1,β2,…,βnManually marking samples corresponding to medium-high uncertainty and then adding the samples into a training set, namely adding the samples into the training set through a formula
Figure FDA0003348160120000025
Each selection is made where mu0To not determine the mean of the set of values B, δ is the standard deviation of the set B, λ0Is a weight parameter, selecting λ01.145; such that each selection uncertainty β is greater than μ0+λ0Sample composition set Q of δuc(ii) a For image QucApplying a density clustering algorithm to obtain a parameter epsilon of 0.05, setting the size of MinPts to be 2, and classifying samples into a class when the number of samples in the circle center neighborhood is 2 or more; after clustering, a high density of sample clusters C ═ { C ═ C is obtained1,c2,…cnAnd a cluster of only 1 isolated sample O ═ O1,o2,…om}, final image set QucIs divided into: quc＝{ci,i＝1,2,...n}∪{oiI ═ 1,2,. m }; by the formula
Figure FDA0003348160120000026
From each high-density cluster ctTo select the sample U with the greatest uncertaintytIn addition to this, all isolated samples plusIn a candidate set Q, the sample points can increase the generalization capability of the training model; the final candidate set is Q ═ Ut,t＝1,...n}∪{OiI ═ 1.. m }; the sample set Q represents a sample selected by a selection model which considers uncertainty and diversity design at the same time, and is added into the training set L after being manually marked;
(3) the sample set Q selected by the work is marked manually, then the marked sample set Q is added into a training set L, a sequencer KSR is trained again by using the updated training set L, then the performance of the model trained at this time is verified on a verification set, the step (2) is repeated continuously until the performance of the model is changed slightly or the performance is reduced, the training set selected by the last iteration is selected as a final training set, the trained model is used as a final training model, the model is used for carrying out significance sorting on the region candidate segmentation of each test image, the region candidate segmentation of 16 bits before ranking is selected for carrying out weighted fusion, and a significance map M of the image is obtainedp；
(4) Providing a processing method on a super-pixel level to achieve the aim of optimizing a boundary; firstly, the super-pixel segmentation algorithm of SLIC is utilized to set the number of segmented super-pixel blocks to be 100, 150 and 200 respectively, and the segmented super-pixel blocks are used for forming a super-pixel set SP of an image iiSeparately extracting CNN feature x of each superpixel blockj(ii) a The saliency map M obtained from step (3) is usedpBinarization as a prior saliency map Ei(ii) a Determining positive and negative samples of the superpixel, and completely locating the superpixel in the prior saliency map E in order to make the confidence coefficient highestiSuperpixels of foreground region form positive sample set POiCompletely locate the super-pixel in the prior saliency map EiThe super pixels of the background region of (2) constitute a negative sample set Ni(ii) a Forming positive and negative sample pairs from the positive and negative sample sets, and using formula
Figure FDA0003348160120000031
To train a model KSR for image ii(ii) a For this model KSRiUsing the formula si＝wTPkiAll superpixels of the image are scored, and all the superpixels are classifiedSuper-pixel sorting S ═ S1,s2,…snThe higher the score is, the closer the score is to the foreground, and otherwise, the lower the score is, the closer the score is to the background; using formulas
Figure FDA0003348160120000032
Obtaining the score of each pixel in the superpixel, normalizing all the scores to be between 0 and 1, and finally obtaining the saliency map M synthesized at the superpixel level through weighting and fusions(ii) a The final saliency map is given by the formula M ═ w1×Mp+w2×MSObtaining, wherein M is the final saliency map, MpIs an original significant picture, MsIs a super-pixel level saliency map.
CN201910609780.2A 2019-07-08 2019-07-08 Significance detection method based on active learning Active CN110443257B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201910609780.2A CN110443257B (en) 2019-07-08 2019-07-08 Significance detection method based on active learning

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201910609780.2A CN110443257B (en) 2019-07-08 2019-07-08 Significance detection method based on active learning

Publications (2)

Publication Number Publication Date
CN110443257A CN110443257A (en) 2019-11-12
CN110443257B true CN110443257B (en) 2022-04-12

Family

ID=68429598

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201910609780.2A Active CN110443257B (en) 2019-07-08 2019-07-08 Significance detection method based on active learning

Country Status (1)

Country Link
CN (1) CN110443257B (en)

Families Citing this family (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112149688A (en) * 2020-09-24 2020-12-29 北京汽车研究总院有限公司 Image processing method and device, computer readable storage medium, computer device
CN114118413A (en) * 2021-11-30 2022-03-01 上海商汤临港智能科技有限公司 Network training and equipment control method, device, equipment and storage medium
CN114120048B (en) * 2022-01-26 2022-05-13 中兴通讯股份有限公司 Image processing method, electronic device, and computer-readable storage medium
CN114332489B (en) * 2022-03-15 2022-06-24 江西财经大学 Image salient target detection method and system based on uncertainty perception
CN115880268B (en) * 2022-12-28 2024-01-30 南京航空航天大学 Method, system, equipment and medium for detecting inferior goods in plastic hose production

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103927394A (en) * 2014-05-04 2014-07-16 苏州大学 Multi-label active learning classification method and system based on SVM
US9414048B2 (en) * 2011-12-09 2016-08-09 Microsoft Technology Licensing, Llc Automatic 2D-to-stereoscopic video conversion
CN107103608A (en) * 2017-04-17 2017-08-29 大连理工大学 A kind of conspicuousness detection method based on region candidate samples selection
CN107133955A (en) * 2017-04-14 2017-09-05 大连理工大学 A kind of collaboration conspicuousness detection method combined at many levels

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US9414048B2 (en) * 2011-12-09 2016-08-09 Microsoft Technology Licensing, Llc Automatic 2D-to-stereoscopic video conversion
CN103927394A (en) * 2014-05-04 2014-07-16 苏州大学 Multi-label active learning classification method and system based on SVM
CN107133955A (en) * 2017-04-14 2017-09-05 大连理工大学 A kind of collaboration conspicuousness detection method combined at many levels
CN107103608A (en) * 2017-04-17 2017-08-29 大连理工大学 A kind of conspicuousness detection method based on region candidate samples selection

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
Kernelized Subspace Ranking for Saliency Detection;Tiantian Wang etal.;《ECCV 2016》;20160917;全文 *
核子空间样本选择方法的核最近邻凸包分类器;周晓飞等;《计算机工程与应用》;20071231;第43卷(第32期);全文 *

Also Published As

Publication number Publication date
CN110443257A (en) 2019-11-12

Similar Documents

Publication Publication Date Title
CN110443257B (en) Significance detection method based on active learning
CN111814584B (en) Vehicle re-identification method based on multi-center measurement loss under multi-view environment
Endres et al. Category-independent object proposals with diverse ranking
CN111368683B (en) Face image feature extraction method and face recognition method based on modular constraint CenterFace
CN111445488B (en) Method for automatically identifying and dividing salt body by weak supervision learning
CN110837836A (en) Semi-supervised semantic segmentation method based on maximized confidence
CN111553837B (en) Artistic text image generation method based on neural style migration
CN107392919B (en) Adaptive genetic algorithm-based gray threshold acquisition method and image segmentation method
CN110647907B (en) Multi-label image classification algorithm using multi-layer classification and dictionary learning
CN109033978B (en) Error correction strategy-based CNN-SVM hybrid model gesture recognition method
CN111612008A (en) Image segmentation method based on convolution network
CN110008899B (en) Method for extracting and classifying candidate targets of visible light remote sensing image
CN112364791B (en) Pedestrian re-identification method and system based on generation of confrontation network
CN111833322B (en) Garbage multi-target detection method based on improved YOLOv3
CN105574545B (en) The semantic cutting method of street environment image various visual angles and device
CN110633727A (en) Deep neural network ship target fine-grained identification method based on selective search
CN112541491A (en) End-to-end text detection and identification method based on image character region perception
CN112287935B (en) Image semantic segmentation method and system based on significance prior
CN111815582B (en) Two-dimensional code region detection method for improving background priori and foreground priori
CN113705579A (en) Automatic image annotation method driven by visual saliency
CN112364881A (en) Advanced sampling consistency image matching algorithm
CN114550134A (en) Deep learning-based traffic sign detection and identification method
CN108765384B (en) Significance detection method for joint manifold sequencing and improved convex hull
CN113469270B (en) Semi-supervised intuitive clustering method based on decomposition multi-target differential evolution superpixel
CN115294424A (en) Sample data enhancement method based on generation countermeasure network

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant