CN110111340B - Weak supervision example segmentation method based on multi-path segmentation - Google Patents

Weak supervision example segmentation method based on multi-path segmentation Download PDF

Info

Publication number
CN110111340B
CN110111340B CN201910347532.5A CN201910347532A CN110111340B CN 110111340 B CN110111340 B CN 110111340B CN 201910347532 A CN201910347532 A CN 201910347532A CN 110111340 B CN110111340 B CN 110111340B
Authority
CN
China
Prior art keywords
segmentation
object recommendation
category
image
instance
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN201910347532.5A
Other languages
Chinese (zh)
Other versions
CN110111340A (en
Inventor
程明明
刘云
吴宇寰
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Nankai University
Original Assignee
Nankai University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Nankai University filed Critical Nankai University
Priority to CN201910347532.5A priority Critical patent/CN110111340B/en
Publication of CN110111340A publication Critical patent/CN110111340A/en
Application granted granted Critical
Publication of CN110111340B publication Critical patent/CN110111340B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/10—Segmentation; Edge detection
    • G06T7/11—Region-based segmentation
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/10—Segmentation; Edge detection
    • G06T7/136—Segmentation; Edge detection involving thresholding
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20081—Training; Learning
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20084—Artificial neural networks [ANN]

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • General Health & Medical Sciences (AREA)
  • Computing Systems (AREA)
  • Computational Linguistics (AREA)
  • Data Mining & Analysis (AREA)
  • Evolutionary Computation (AREA)
  • Biomedical Technology (AREA)
  • Molecular Biology (AREA)
  • Biophysics (AREA)
  • General Engineering & Computer Science (AREA)
  • Artificial Intelligence (AREA)
  • Mathematical Physics (AREA)
  • Software Systems (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Health & Medical Sciences (AREA)
  • Image Analysis (AREA)

Abstract

A weak supervision example segmentation method based on multi-path segmentation is disclosed. The method trains a convolutional neural network for instance segmentation using only image-level annotation data. Specifically, a training set only with image level labels is given, and a plurality of object recommendation areas irrelevant to the category are calculated for each image by a simulation sampling algorithm; and then, taking the image and the corresponding object recommendation area as input, taking the labeled image category as a learning target, and calculating category probability distribution and semantic features of each object recommendation area through a multi-instance learning framework. Establishing a large-scale graph model by taking object recommendation areas in the whole data set as nodes, regarding the graph model as a multi-path segmentation problem, and giving a category label to each object recommendation area as a result by a segmentation result; or as a training set to train any convolutional neural network for instance segmentation. Experiments show that the method is obviously superior to the existing weak supervision example segmentation method.

Description

Weak supervision example segmentation method based on multi-path segmentation
Technical Field
The invention belongs to the technical field of computer vision, and particularly relates to a method for partitioning a weak supervision instance based on multi-way partitioning.
Background
Example segmentation is directed to segmenting each object in an image separately and identifying the class of the object. Based on the enormous value of business and academia, instance segmentation is an important task in computer vision. Recent example segmentation techniques have advanced primarily from some basic models based on convolutional neural networks, such as Fast R-CNN proposed by Ross Girshick at the ICCV conference 2015, Fast R-CNN proposed by Shaoqing Ren et al at the NIPS conference 2015, and Mask R-CNN proposed by Kaiming He et al at the ICCV conference 2017. However, these deep learning models rely heavily on a large amount of training data, which are all labeled with instances of objects at the pixel level. It is time consuming to label an image from the pixel level and collecting this much data is therefore a very expensive matter.
To reduce the need for pixel-level labeling data, some research efforts have trained the example segmentation model with object labeling boxes as the supervised information. Anna Khoreva et al published in 2017 on the CVPR conference, "Simple dots it: week superior instance and segmentation" paper used a modified version of the GrabCut algorithm to estimate object segmentations in the object marker box, and then used the MCG algorithm to correct these instance segmentations. The paper "weak-and semi-pervised corporate segmentation" published by Qizhu Li et al at the 2018 ECCV conference extends the method of Anna Khoreva et al, which uses an iterative approach to correct the estimated segmentation of the instances. Specifically, they first train the network by obtaining the initial example segmentation similar to Anna Khoreva, and then retrain the network by using the prediction result after the network training as a new segmentation estimate, and so on for several iterations to obtain the final result.
However, marking a large number of object frames is still time and labor consuming, and other tasks requiring object marking frames as supervisory information, such as object detection, have all begun to seek strategies for weakly supervised learning. Therefore, Yanzhao Zhou et al further relaxes the supervision information to image-level labeling in the "weak Supervised Segmentation using Class Peak Response" paper published on the CVPR in 2018, i.e. training the example Segmentation model using only images with Class labels as training data. They propose a new concept, "similar response peak value", that is, when the provided picture is used to train the image classification model, the convolution neural network has a larger response peak value on each object through a certain process, so as to obtain the approximate position of the object, and then the object recommendation area calculated by the analog sampling is combined to obtain the example segmentation result.
Disclosure of Invention
The invention aims to solve the technical problem of the prior example segmentation technology that a large amount of training data labeled at the pixel level is needed, and provides a method for weakly supervised example segmentation based on multi-way segmentation. The method only needs to provide pictures with category labels, and an example segmentation model can be learned.
In order to achieve the above purpose, the present invention firstly designs a multi-instance learning framework, which takes images and corresponding similarity sampling results as input, takes image categories as learning targets, and a trained model can calculate probability distribution and semantic features of each object recommendation region for one input image. Based on the probability distribution and semantic features, a multi-segmentation problem is constructed, and a correct class label is assigned to each object recommendation area.
The invention provides a weak supervision instance segmentation method based on multi-path segmentation, which comprises the following steps:
a. given a data set comprising a training set and a test set, each image in the training set has image-level labels, the method uses a general similarity sampling algorithm to generate object recommendation regions for each image in the data set, wherein the object recommendation regions may include objects of a target class, and the object recommendation regions may or may not include objects of the target class (i.e. background); and, these object recommendation areas are not category labeled, but only indicate that these areas may contain objects of the target category.
b. A multi-instance learning framework based on object recommendation areas is designed, the images and the corresponding object recommendation areas are used as input, the labeled categories of the images are used as learning targets, and the category probability distribution and semantic information can be learned and calculated for each object recommendation area through a loss function of the designed multi-instance learning framework.
The multi-instance learning framework based on the object recommendation regions designs a convolutional neural network model shown in fig. 2, so that the model can predict a probability distribution for each object recommendation region, and according to the probability distribution, the class labels of the images can be used as the supervision target of each object recommendation region. The loss function of the multi-instance learning framework based on the object recommendation region is composed of three parts, namely an attention loss function, a multi-instance learning loss function and a clustering center loss function, wherein the first two loss functions are mainly used for learning category information, and the clustering center loss function is used for learning semantic features of the object recommendation region.
c. And c, establishing a large-scale graph model by using the class probability distribution and the semantic information of the object recommendation areas calculated in the step b and using the object recommendation areas in the whole data set as nodes, regarding the graph model as a large-scale multi-segmentation problem, and giving a class mark to each object recommendation area by using a segmentation result.
Specifically, each object recommendation area is regarded as a node of the graph, each target class is regarded as a vertex of the graph, the distance from one node to the edge of one vertex is the predicted class probability, the distance between two nodes is the cosine value of the included angle between the semantic feature vectors of the two nodes, and the distance between two vertices is infinite. The goal of the multi-cut is to divide the entire graph into several subsets, with one and only one vertex in each subset, and with each node belonging to one and only one subset. It is impractical to solve this large-scale multi-segmentation problem, however, it can be decomposed into several small-scale multi-segmentation problems by limiting the maximum number of edges connected to each node. For each small-scale multi-segmentation problem, the collection of their solutions is the solution of the large graph. And dividing each node representing the object recommendation area into a subset by multi-path division, wherein the category corresponding to the vertex contained in the subset is the category of the object recommendation area.
d. Deleting the object recommendation areas marked as backgrounds in the step c, and taking the rest object recommendation areas and the corresponding category marks as segmentation results; any convolutional neural network for example segmentation can also be trained by using the remaining object recommendation regions as training data, and the network after training can be used for example segmentation of the image.
Advantages and advantageous effects of the invention
The method can simultaneously calculate the probability distribution and the semantic features of an object recommendation area through a multi-instance learning framework, and finally establishes a multi-path segmentation problem by using the probability distribution and the semantic features. This can be done in conjunction with information on the object instance, image, and entire data set to filter out unwanted object recommendation areas, preserve the correct object recommendation areas and assign category labels. This is more robust and accurate than the attention model based image classification networks.
Drawings
FIG. 1 is an overall flow chart of the present invention.
Fig. 2 is a convolutional neural network in the proposed multi-instance learning framework.
FIG. 3 is a comparison of experimental results of the present invention and related methods.
Fig. 4 shows several sets of exemplary results of the present invention. The first and fourth lines are the original input image, the second and fifth lines are the correct segmentations, and the third and sixth lines are the results of the method of the invention output, and a segmentation mask of the results is drawn into the artwork for viewing.
Detailed Description
The following describes in further detail embodiments of the present invention with reference to the accompanying drawings. The following examples are intended to illustrate the invention but are not intended to limit the scope of the invention.
The weak supervision example segmentation method based on multi-path segmentation specifically comprises the following operations:
a. the network model is a Convolutional neural network model with multi-example learning of object recommendation region pooling, wherein the feature extraction part can be a VGG16 framework mentioned in a 'Very Deep conditional Networks for Large-Scale Image registration' article published by Karen Simoyan, or a ResNet framework mentioned in a 'Deep residual learning for Image registration' article published by Kaming He, or other basic network architectures. For the ResNet-50 network, we add a region of interest pooling module to the last module of ResNet (before global averaging pooling), as shown in FIG. 2. The interest region pooling module inputs a plurality of object recommendation region frames obtained by analog sampling, then cuts out region features with the same position and size of the object recommendation region frames from the feature map, and performs maximum pooling sampling on the region features, wherein each module can obtain a feature map of 7x7 with the same channel number as that of the input feature map. Therefore, after the feature map is extracted by ResNet, we input the object recommendation area boxes into the module, and for each recommendation box, obtain 7 × 7 feature maps with the same number of channels (2048 channels) as the input feature map. And finally, after a layer of global average pooling, each recommendation box obtains a feature vector with a corresponding 2048 dimension. Inputting the feature vectors into a full-connection layer of 21 neurons and passing through a softmax layer, wherein each recommendation box corresponds to a 21-dimensional probability feature vector, and the i-th 2048-dimensional feature vector is recorded as fiThe ith 21-dimensional probability feature vector is pi。
b. For f obtained in aiAnd piAnd the multiple instance learning framework in fig. 1, we propose several loss functions as joint supervision. The first is the loss of network attention, we calculated a class attention map using the CAM method proposed by Bolei Zhou et al in "Learning Deep Features for cognitive Localization" published in the 2016 CVPR conference, setting the normalization of the ith image to [0,1 [ ], C]The kth class of attention is sought as
Figure GDA0002956609270000041
The jth recommended frame in the ith image is marked as the attention category of the jth recommended frame
Figure GDA0002956609270000042
Setting:
Figure GDA0002956609270000043
Figure GDA0002956609270000044
the attention score of category k of box j is recommended for the ith picture,
Figure GDA0002956609270000045
can be calculated from the following formula:
Figure GDA0002956609270000046
attention loss function
Figure GDA0002956609270000047
Can be calculated from the following formula:
Figure GDA0002956609270000048
wherein, | Si| is the total number of recommended boxes,
Figure GDA0002956609270000049
and
Figure GDA00029566092700000410
respectively predicting the jth recommendation frame in the ith image as a category
Figure GDA00029566092700000411
And K', K being the total number of target classes. After the attention loss function is given, a multi-instance learning loss function is proposed, a feature map intercepted by using a log-sum-exp function for all recommendation boxes of the ith image is used, and probability vectors of each category under all the recommendation boxes are estimated
Figure GDA00029566092700000412
Maximum value of (1) is
Figure GDA00029566092700000413
Is the probability estimation value of the kth' category of the ith image,
Figure GDA0002956609270000051
can be calculated from the following equation:
Figure GDA0002956609270000052
where r is a parameter of the log-sum-exp function, where r is 5, such that the function is the maximum of the estimated input vector. Probability estimation value of k' th category of estimated ith image
Figure GDA0002956609270000053
Later, multi-instance learning loss function
Figure GDA0002956609270000054
Can be calculated from the following formula:
Figure GDA0002956609270000055
wherein Y isi′Is a positive example of the category of the case,
Figure GDA0002956609270000056
are negative case classes, which are mutually exclusive. After introducing the multiple instance learning penalty function, we present a third penalty function as follows: a cluster center loss function based on multi-instance learning. Cluster center loss function
Figure GDA0002956609270000057
The calculation is obtained by the following two formulas:
Figure GDA0002956609270000058
Figure GDA0002956609270000059
wherein
Figure GDA00029566092700000510
The category corresponding to the maximum probability of the jth recommendation box of the ith image is shown,
Figure GDA00029566092700000511
2048-dimensional feature vectors corresponding to the same recommendation box,
Figure GDA00029566092700000512
is a category
Figure GDA00029566092700000513
The statistical feature vector, | · | | non-conducting phosphor2Represents the 2-norm, | S of the vectoriAnd | is the total number of recommended boxes. Statistical feature vector
Figure GDA00029566092700000514
Will change slowly as training progresses:
Figure GDA00029566092700000515
wherein
Figure GDA00029566092700000516
Obtained for the last iteration
Figure GDA00029566092700000517
Calculated for this iteration
Figure GDA00029566092700000518
θ is a parameter of the update speed, and we use θ as 0.01. After introducing the three loss functions we propose, we use the fusion of these loss functions as the final loss function:
Figure GDA00029566092700000519
where α, β, γ are the weights of the three loss functions, respectively, where we use α ═ 0.5, β ═ 0.5, and γ ═ 0.1. In the above, we input pictures and their corresponding recommendation boxes derived from the similarity sampling method into the multi-instance learning framework and use L(i)Supervised training is performed as a loss function.
c. And c, after the training is finished by the method in the step b, inputting the pictures and the recommendation boxes into the frame, and obtaining the feature vector and the category probability vector corresponding to each recommendation box of each picture. With them, we can build a knowledge undirected graph. And (V, E) is an undirected graph, V represents a node set, and E represents an edge set. Recommendation box
Figure GDA00029566092700000520
And a set of object classes
Figure GDA0002956609270000061
Will act as a node, so:
Figure GDA0002956609270000062
wherein S isiRepresenting a set of bounding boxes in the ith image. Is provided with
Figure GDA0002956609270000063
For terminals (terminals), the capacity E (u, v) ═ E (v, u) of the edge uv ∈ E is:
Figure GDA0002956609270000064
by using the above formula, we have established an undirected graph. It should be noted that we only reserve three edges of maximum capacity for each node except the terminal node when we build the graph.
An undirected graph has been built as above and is subsequently multi-sliced. I.e. solving an optimization problem:
Figure GDA0002956609270000065
wherein the content of the first and second substances,
Figure GDA0002956609270000066
||·||1is the 1-norm of the vector. When the optimization problem is solved, the maximum number of edges connected with each node is limited to be 3, namely three edges with the maximum weight, so that an undirected graph to be solved is converted into sub-graphs of a plurality of connected domains, and the sub-graphs G aret＝(Vt,Et) Are not communicated with each other, and:
Figure GDA0002956609270000067
we solve the above optimization problem independently under each subgraph. For each subgraph, a multiple cut D can be obtainedtAnd, and:
∪tDt＝D，
where D is a multiple cut of G.
d. In the multi-way cutting in the step c, each node representing the object recommendation area is divided into a subset, and the category corresponding to the vertex contained in the subset is the category of the object recommendation area. Deleting the object recommendation areas marked as backgrounds, and taking the remaining object recommendation areas and the corresponding category marks as segmentation results; any convolutional neural network for example segmentation can also be trained by using the remaining object recommendation regions as training sets, and the network after training can perform example segmentation on the image.
FIG. 3 shows a comparison of our method with other methods. mAP0.5 rAnd mAP0.75 rThe average precision of the class-wise averages at thresholds of 0.5 and 0.75, respectively, is indicated, and ABO indicates the average best coverage. The CAM method is a method proposed by Bolei Zhou et al in "Learning Deep Features for cognitive Localization" published by CVPR conference 2016, and SPN is Zhu Yi et al in 2017The method proposed in "Soft porous networks for week super Object localization" published in ICCV conference, MELM is the method proposed in "Min-enhanced tension Model for week super Object Detection" published in 2018 CVPR by Fan et al, PRM is the method proposed in "Weakly super Object localization using Class Peak Response" published in 2018 CVPR by Yanzhao Zhou et al. LIID is the method we propose. Representative uses a block overlay attention map, Ellipse representative uses an attention map, and MCG representative uses the method mentioned in PabloArbelaez et al, "Multiscale composite group," published by CVPR conference 2014 to overlay attention maps. It was found that our method is superior to these methods in all indications.
FIG. 4 is 10 example graphs of example segmentation results obtained using our method. The first and fourth lines are the original input image, the second and fifth lines are the correct segmentations, and the third and sixth lines are the results of the method of the invention output, and a segmentation mask of the results is drawn into the artwork for viewing.
The top picture of each example graph is the original picture, the middle picture is the human labeled reference result, and the bottom is the result generated by our method.

Claims (6)

1. A weak supervision example segmentation method based on multi-path segmentation is characterized by comprising the following steps:
a. given a data set comprising a training set and a test set, wherein each image in the training set has an image-level label, and an object recommendation area which possibly comprises an object of a target class is generated for each image in the data set by using an analog sampling algorithm;
b. designing a multi-instance learning framework based on an object recommendation region, wherein the multi-instance learning framework takes an image and a corresponding object recommendation region as input, takes a mark category of the image as a learning target, and designs a multi-instance learning loss function to learn and calculate category probability distribution and semantic information for each object recommendation region;
c. b, using the class probability distribution and semantic information of the object recommendation areas calculated in the step b, and establishing a large-scale graph model by using the object recommendation areas in the whole data set as nodes, specifically, taking each object recommendation area as a node of the graph, taking each target class as a vertex of the graph, wherein the distance from one node to the edge of one vertex is the predicted class probability, the distance between the edges of two nodes is the cosine value of the included angle between the semantic feature vectors of the two nodes, and the distance between the two vertices is infinite; regarding the graph model as a large-scale multi-path segmentation problem, and giving a category label to each object recommendation area by a segmentation result;
d. deleting the object recommendation areas marked as the background in the step c, and taking the rest object recommendation areas and the corresponding class marks as segmentation results; or training any convolution neural network for example segmentation by using the rest object recommendation area as training data, wherein the network after training is used for example segmentation of the image.
2. The method of multi-way segmentation-based weakly supervised instance segmentation as recited in claim 1, wherein: the multi-instance learning framework based on the object recommendation areas designs a convolutional neural network model, so that the model can predict a probability distribution for each object recommendation area, and class marks of images are used as supervision targets of each object recommendation area.
3. The method of multi-way segmentation-based weakly supervised instance segmentation as recited in claim 1, wherein: the loss function of the multi-instance learning framework based on the object recommendation area is composed of three parts, namely an attention loss function, a multi-instance learning loss function and a clustering center loss function, wherein the first two loss functions are used for learning category information, and the clustering center loss function is used for learning semantic features of the object recommendation area.
4. The multi-way cut-based weakly supervised instance partitioning party of claim 3The method is characterized in that: said attention loss function
Figure FDA0003003006030000011
Calculated from the following formula:
Figure FDA0003003006030000012
wherein, | Si| is the total number of recommended boxes,
Figure FDA0003003006030000021
and
Figure FDA0003003006030000022
respectively predicting the jth recommendation frame in the ith image as a category
Figure FDA0003003006030000023
And K', K being the total number of target classes;
multiple instance learning loss function
Figure FDA0003003006030000024
Calculated from the following formula:
Figure FDA0003003006030000025
wherein Y isi′Is a positive example of the category of the case,
Figure FDA0003003006030000026
is a negative case category, the two categories are mutually exclusive,
Figure FDA0003003006030000027
the probability estimated value of the ith image belonging to the kth' category;
cluster center loss function
Figure FDA0003003006030000028
The calculation is obtained by the following two formulas:
Figure FDA0003003006030000029
Figure FDA00030030060300000210
wherein
Figure FDA00030030060300000211
The category corresponding to the maximum probability of the jth recommendation box of the ith image is shown,
Figure FDA00030030060300000212
2048-dimensional feature vectors corresponding to the same recommendation box,
Figure FDA00030030060300000213
is a category
Figure FDA00030030060300000214
Is a statistical feature vector of | · |2Represents the 2-norm, | S of the vectoriAnd | is the total number of recommended boxes.
5. The method of multi-way segmentation-based weakly supervised instance segmentation as recited in claim 4, wherein: the loss function of the multi-instance learning framework is finally expressed by fusing an attention loss function, a multi-instance learning loss function and a clustering center loss function as follows:
Figure FDA00030030060300000215
where α, β, γ are the weights of the three loss functions, respectively.
6. The method of multi-way segmentation-based weakly supervised instance segmentation as recited in claim 1, wherein: the large-scale multi-segmentation problem decomposes the large-scale multi-segmentation problem into several small-scale multi-segmentation problems by limiting the maximum number of edges connected to each node.
CN201910347532.5A 2019-04-28 2019-04-28 Weak supervision example segmentation method based on multi-path segmentation Active CN110111340B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201910347532.5A CN110111340B (en) 2019-04-28 2019-04-28 Weak supervision example segmentation method based on multi-path segmentation

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201910347532.5A CN110111340B (en) 2019-04-28 2019-04-28 Weak supervision example segmentation method based on multi-path segmentation

Publications (2)

Publication Number Publication Date
CN110111340A CN110111340A (en) 2019-08-09
CN110111340B true CN110111340B (en) 2021-05-14

Family

ID=67487090

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201910347532.5A Active CN110111340B (en) 2019-04-28 2019-04-28 Weak supervision example segmentation method based on multi-path segmentation

Country Status (1)

Country Link
CN (1) CN110111340B (en)

Families Citing this family (23)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2018176000A1 (en) 2017-03-23 2018-09-27 DeepScale, Inc. Data synthesis for autonomous control systems
US11157441B2 (en) 2017-07-24 2021-10-26 Tesla, Inc. Computational array microprocessor system using non-consecutive data formatting
US11409692B2 (en) 2017-07-24 2022-08-09 Tesla, Inc. Vector computational unit
US11893393B2 (en) 2017-07-24 2024-02-06 Tesla, Inc. Computational array microprocessor system with hardware arbiter managing memory requests
US10671349B2 (en) 2017-07-24 2020-06-02 Tesla, Inc. Accelerated mathematical engine
US11561791B2 (en) 2018-02-01 2023-01-24 Tesla, Inc. Vector computational unit receiving data elements in parallel from a last row of a computational array
US11215999B2 (en) 2018-06-20 2022-01-04 Tesla, Inc. Data pipeline and deep learning system for autonomous driving
US11361457B2 (en) 2018-07-20 2022-06-14 Tesla, Inc. Annotation cross-labeling for autonomous control systems
US11636333B2 (en) 2018-07-26 2023-04-25 Tesla, Inc. Optimizing neural network structures for embedded systems
US11562231B2 (en) 2018-09-03 2023-01-24 Tesla, Inc. Neural networks for embedded devices
KR20210072048A (en) 2018-10-11 2021-06-16 테슬라, 인크. Systems and methods for training machine models with augmented data
US11196678B2 (en) 2018-10-25 2021-12-07 Tesla, Inc. QOS manager for system on a chip communications
US11816585B2 (en) 2018-12-03 2023-11-14 Tesla, Inc. Machine learning models operating at different frequencies for autonomous vehicles
US11537811B2 (en) 2018-12-04 2022-12-27 Tesla, Inc. Enhanced object detection for autonomous vehicles based on field view
US11610117B2 (en) 2018-12-27 2023-03-21 Tesla, Inc. System and method for adapting a neural network model on a hardware platform
US10997461B2 (en) 2019-02-01 2021-05-04 Tesla, Inc. Generating ground truth for machine learning from time series elements
US11567514B2 (en) 2019-02-11 2023-01-31 Tesla, Inc. Autonomous and user controlled vehicle summon to a target
US10956755B2 (en) 2019-02-19 2021-03-23 Tesla, Inc. Estimating object properties using visual image data
CN111833356B (en) * 2020-06-15 2023-02-28 五邑大学 Brain glioma image grading method and device and storage medium
CN111914107B (en) * 2020-07-29 2022-06-14 厦门大学 Instance retrieval method based on multi-channel attention area expansion
CN112232355B (en) * 2020-12-11 2021-04-02 腾讯科技（深圳）有限公司 Image segmentation network processing method, image segmentation device and computer equipment
CN113379773B (en) * 2021-05-28 2023-04-28 陕西大智慧医疗科技股份有限公司 Segmentation model establishment and segmentation method and device based on dual-attention mechanism
CN116342627B (en) * 2023-05-23 2023-09-08 山东大学 Intestinal epithelial metaplasia area image segmentation system based on multi-instance learning

Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN107688821A (en) * 2017-07-11 2018-02-13 西安电子科技大学 View-based access control model conspicuousness and across the modality images natural language description methods of semantic attribute
CN107833213A (en) * 2017-11-02 2018-03-23 哈尔滨工业大学 A kind of Weakly supervised object detecting method based on pseudo- true value adaptive method
CN108647684A (en) * 2018-05-02 2018-10-12 深圳市唯特视科技有限公司 A kind of Weakly supervised semantic segmentation method based on guiding attention inference network
CN105138580B (en) * 2015-07-31 2018-11-23 中国科学院信息工程研究所 A kind of network negative information influence minimum method based on the company of blocking side
CN108922599A (en) * 2018-06-27 2018-11-30 西南交通大学 A kind of accurate mask method of medical image lesion point based on MIL
CN109345540A (en) * 2018-09-15 2019-02-15 北京市商汤科技开发有限公司 A kind of image processing method, electronic equipment and storage medium
CN109409371A (en) * 2017-08-18 2019-03-01 三星电子株式会社 The system and method for semantic segmentation for image

Family Cites Families (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN105069774B (en) * 2015-06-30 2017-11-10 长安大学 The Target Segmentation method of optimization is cut based on multi-instance learning and figure
US10049279B2 (en) * 2016-03-11 2018-08-14 Qualcomm Incorporated Recurrent networks with motion-based attention for video understanding
US10424064B2 (en) * 2016-10-18 2019-09-24 Adobe Inc. Instance-level semantic segmentation system
US10049297B1 (en) * 2017-03-20 2018-08-14 Beihang University Data driven method for transferring indoor scene layout and color style
US20180336454A1 (en) * 2017-05-19 2018-11-22 General Electric Company Neural network systems
CN109086811B (en) * 2018-07-19 2021-06-22 南京旷云科技有限公司 Multi-label image classification method and device and electronic equipment
CN109558898B (en) * 2018-11-09 2023-09-05 复旦大学 Multi-choice learning method with high confidence based on deep neural network

Patent Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN105138580B (en) * 2015-07-31 2018-11-23 中国科学院信息工程研究所 A kind of network negative information influence minimum method based on the company of blocking side
CN107688821A (en) * 2017-07-11 2018-02-13 西安电子科技大学 View-based access control model conspicuousness and across the modality images natural language description methods of semantic attribute
CN109409371A (en) * 2017-08-18 2019-03-01 三星电子株式会社 The system and method for semantic segmentation for image
CN107833213A (en) * 2017-11-02 2018-03-23 哈尔滨工业大学 A kind of Weakly supervised object detecting method based on pseudo- true value adaptive method
CN108647684A (en) * 2018-05-02 2018-10-12 深圳市唯特视科技有限公司 A kind of Weakly supervised semantic segmentation method based on guiding attention inference network
CN108922599A (en) * 2018-06-27 2018-11-30 西南交通大学 A kind of accurate mask method of medical image lesion point based on MIL
CN109345540A (en) * 2018-09-15 2019-02-15 北京市商汤科技开发有限公司 A kind of image processing method, electronic equipment and storage medium

Non-Patent Citations (4)

* Cited by examiner, † Cited by third party
Title
Associating inter-image Salient Instances for weakly supervised semantic segmentation;Fan ruochen 等;《European Conference on Computer Vision》;20181005;第371-388页 *
Multiple Structured-Instance Learning for Semantic Segmentation with Uncertain Training Data;Feng-Ju Chang 等;《computer Vision fundation》;20141230;第1-8页 *
STC_ A Simple to Complex Framework for__Weakly-supervised Semantic Segmentation;Yunchao Wei 等;《IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE,》;20161207;第1-8页 *
基于多示例深度学习与损失函数优化的交通标志识别算法;张永雄 等;《现代电子技术》;20180801;第41卷(第15期);第133-140页 *

Also Published As

Publication number Publication date
CN110111340A (en) 2019-08-09

Similar Documents

Publication Publication Date Title
CN110111340B (en) Weak supervision example segmentation method based on multi-path segmentation
CN108319972B (en) End-to-end difference network learning method for image semantic segmentation
CN109685067B (en) Image semantic segmentation method based on region and depth residual error network
CN109886121B (en) Human face key point positioning method for shielding robustness
Dornaika et al. Building detection from orthophotos using a machine learning approach: An empirical study on image segmentation and descriptors
CN108280397B (en) Human body image hair detection method based on deep convolutional neural network
CN104537676B (en) Gradual image segmentation method based on online learning
Li et al. Adaptive deep convolutional neural networks for scene-specific object detection
CN110276264B (en) Crowd density estimation method based on foreground segmentation graph
CN101482923B (en) Human body target detection and sexuality recognition method in video monitoring
CN110910391A (en) Video object segmentation method with dual-module neural network structure
CN112036231B (en) Vehicle-mounted video-based lane line and pavement indication mark detection and identification method
CN105184772A (en) Adaptive color image segmentation method based on super pixels
CN110032952B (en) Road boundary point detection method based on deep learning
CN113362341B (en) Air-ground infrared target tracking data set labeling method based on super-pixel structure constraint
Xu et al. Weakly supervised deep semantic segmentation using CNN and ELM with semantic candidate regions
Qin et al. A robust framework combined saliency detection and image recognition for garbage classification
CN112528845B (en) Physical circuit diagram identification method based on deep learning and application thereof
CN112347930B (en) High-resolution image scene classification method based on self-learning semi-supervised deep neural network
Li et al. Transmission line detection in aerial images: An instance segmentation approach based on multitask neural networks
CN110516512B (en) Training method of pedestrian attribute analysis model, pedestrian attribute identification method and device
CN113592894A (en) Image segmentation method based on bounding box and co-occurrence feature prediction
Maggiolo et al. Improving maps from CNNs trained with sparse, scribbled ground truths using fully connected CRFs
CN113420827A (en) Semantic segmentation network training and image semantic segmentation method, device and equipment
CN110287970B (en) Weak supervision object positioning method based on CAM and covering

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant