CN110297933A - A kind of theme label recommended method and tool based on deep learning - Google Patents
A kind of theme label recommended method and tool based on deep learning Download PDFInfo
- Publication number
- CN110297933A CN110297933A CN201910587052.6A CN201910587052A CN110297933A CN 110297933 A CN110297933 A CN 110297933A CN 201910587052 A CN201910587052 A CN 201910587052A CN 110297933 A CN110297933 A CN 110297933A
- Authority
- CN
- China
- Prior art keywords
- theme label
- image
- label
- model
- video clip
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/50—Information retrieval; Database structures therefor; File system structures therefor of still image data
- G06F16/58—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
- G06F16/5866—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually using information manually generated, e.g. tags, keywords, comments, manually generated location and time information
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/70—Information retrieval; Database structures therefor; File system structures therefor of video data
- G06F16/78—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
- G06F16/7867—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually using information manually generated, e.g. tags, keywords, comments, title and artist information, manually generated time, location and usage information, user ratings
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
- G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
- G06F18/2411—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches based on the proximity to a decision surface, e.g. support vector machines
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- General Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- Life Sciences & Earth Sciences (AREA)
- Artificial Intelligence (AREA)
- Evolutionary Computation (AREA)
- Molecular Biology (AREA)
- Library & Information Science (AREA)
- Databases & Information Systems (AREA)
- Software Systems (AREA)
- Health & Medical Sciences (AREA)
- Biomedical Technology (AREA)
- Biophysics (AREA)
- Computational Linguistics (AREA)
- General Health & Medical Sciences (AREA)
- Mathematical Physics (AREA)
- Computing Systems (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Bioinformatics & Computational Biology (AREA)
- Evolutionary Biology (AREA)
- Multimedia (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Image Analysis (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The present invention discloses a kind of theme label recommended method based on deep learning, it is related to technical field of information processing, this method is based on zero-shot learning thought, feature extraction is carried out to image or video clip using the ability in feature extraction of depth network model, the tagsort of theme label is carried out to the feature of extraction using support vector machines model, and it obtains about image or a prediction theme label of video clip, prediction theme label is extended using bluebeard compound incorporation model word2vec and k nearest neighbor algorithm, and then it obtains and the semantic relevant K theme label of prediction theme label, prediction theme label and K theme label are labeled as the final theme label of input picture or video clip, so that annotation results are more reliable.The theme label recommendation tool based on deep learning that invention additionally discloses a kind of, remaining aforementioned theme label recommended method combine, may be used in major social media network platform.
Description
Technical field
The present invention relates to technical field of information processing, specifically a kind of theme label recommendation side based on deep learning
Method and tool.
Background technique
With the development of internet technology, occur currently on the market miscellaneous from media social application software, example
Such as microblogging, trill, the Little Red Book, watermelon video, all users can upload oneself video or image on App, simultaneously
Suitable theme label is marked for it, when the theme label that we are marked meets the interest of viewer, theme label has newly
When newness and attraction or theme label meet the fashion trend on current network, which will be will receive more
Concern amount.It is appropriate for its mark due to the magnanimity of the diversification of media information content and media information data on network
Theme label is still a urgent problem to be solved.
Traditional method some directly using single machine learning algorithm to the media contents such as image or video directly into
The prediction of row theme label, but this is still unable to satisfy content slightly abundant or complicated image or video subject label recommendations and asks
Topic.
Currently, the theme label classification that method also is recommended is relatively more fixed, current prevalence can not be well adapted for
Trend causes recommended theme label not have attraction.
Summary of the invention
The present invention is directed to the demand and shortcoming of current technology development, provides a kind of theme label based on deep learning
Recommended method and tool.
Firstly, the present invention discloses a kind of theme label recommended method based on deep learning, solves above-mentioned technical problem and adopt
Technical solution is as follows:
A kind of theme label recommended method based on deep learning, this method are based on zero-shot learning thought,
Feature extraction is carried out to image or video clip using the ability in feature extraction of depth network model, utilizes support vector machines
Model carries out the tagsort of theme label to the feature of extraction, and obtains about image or a prediction theme of video clip
Label is extended prediction theme label using bluebeard compound incorporation model word2vec and k nearest neighbor algorithm, so obtain in advance
The semantic relevant K theme label of theme label is surveyed, predicts theme label and K theme label as input picture or piece of video
The final theme label of section is labeled.
Specifically, using depth network model ability in feature extraction to image or video clip carry out feature extraction it
Before, need the ability in feature extraction to depth network model to be trained, concrete operations are as follows:
The image with label or video clip are collected as training set；
Feature extraction is carried out using image or video clip of the depth network model to training set；
The feature that depth network model is extracted inputs support vector machines model as image feature vector；
Support vector machines model carries out the tagsort of theme label to the feature of extraction, and obtain about image or
One prediction theme label of video clip；
Bluebeard compound incorporation model word2vec and k nearest neighbor algorithm obtain multiple themes after being extended to prediction theme label
Label；
Judge whether the multiple theme labels for predicting to obtain after theme label and extension are related；
If related, continue to train next image or video clip；
If uncorrelated, continue to train next image or video clip after correcting.
Specifically, involved training set is divided into three data sets；
There are three depth network models, and image classification model respectively based on CNN convolutional neural networks is followed based on RNN
The image classification model of ring neural network, the image classification model based on DNN deep neural network；
Three image classification models carry out feature extraction to the image or video clip of three data sets respectively；
Three kinds of features that above three different images sorter network extracts are concatenated together to form new multidimensional image spy
Vector is levied, multidimensional image feature vector inputs support vector machines model, is with the original label having of image or video clip
Reference generates a prediction theme label.
Specifically, involved bluebeard compound incorporation model word2vec has modeled the correlative relationship between text, bluebeard compound
Text representation is converted term vector form by incorporation model word2vec, for semantic similar text representation be converted into vector it
After will have a lesser distance, semantic dissimilar text representation will have biggish distance, be based on this feature, and bluebeard compound is embedding
Enter model word2vec by predict theme label project to term vector space, and further using k nearest neighbor algorithm search obtain with
The similar K theme label of the prediction theme label is labeled as the final theme label of input picture or video clip.
Specifically, being extended using bluebeard compound incorporation model word2vec and k nearest neighbor algorithm to prediction theme label, have
Gymnastics is made
Prediction theme label is mapped to text vector space by bluebeard compound incorporation model word2vec；
Text vector is calculated at a distance from other vectors of corpus kind using the method for cosine similarity or Euclidean distance
Relationship；
It is obtained and the most similar K theme label of prediction theme label currently entered by k nearest neighbor method；
Using the prediction theme label of this K theme label and support vector machines model as image or video clip
Final theme label is labeled.
Specifically, involved corpus is periodically updated, meanwhile, corpus is also to bluebeard compound incorporation model word2vec
It is updated.
Secondly, invention additionally discloses a kind of theme label recommendation tool based on deep learning, which includes depth net
Network model, support vector machines model, bluebeard compound incorporation model word2vec, k nearest neighbor algoritic module；
Depth network model carries out feature extraction to image or video clip；
Support vector machines model carries out the tagsort of theme label to the feature of extraction, and obtain about image or
One prediction theme label of video clip；
Bluebeard compound incorporation model word2vec and k nearest neighbor algoritic module are extended prediction theme label, and obtain with
The most similar K theme label of prediction theme label currently entered；
K theme label of acquisition and the prediction theme label of support vector machines model word2vec as image or
The final theme label of video clip is labeled.
Specifically, theme label recommendation tool further includes the instruction being trained to the ability in feature extraction of depth network model
Practice module；
Training module includes collecting submodule and judging submodule；
It collects image or video clip of the submodule collection with label and is stored in training set, training set as training sample
Image or video clip sequentially input depth network model；
Depth network model carries out feature extraction to the image or video clip of training set, extracts feature as characteristics of image
Vector inputs support vector machines model, and support vector machines model carries out the feature point of theme label to the feature of extraction
Class, and obtain and calculated about image or the prediction theme label bluebeard compound incorporation model word2vec and k nearest neighbor of video clip
Method obtains multiple theme labels after being extended to prediction theme label；
Judging submodule judges whether the multiple theme labels for predicting to obtain after theme label and extension are related, and in correlation
When directly export training set in next image or video clip, when uncorrelated, first correct export again it is next in training set
Image or video clip.
Specifically, involved training set is divided into three data sets；
There are three depth network models, is image classification model based on CNN convolutional neural networks respectively, is followed based on RNN
The image classification model of ring neural network, the image classification model based on DNN deep neural network；
Three image classification models carry out feature extraction to the image or video clip of three data sets respectively；
Three kinds of features that above three different images sorter network extracts are concatenated together to form new multidimensional image spy
Vector is levied, multidimensional image feature vector inputs support vector machines model, then, has so that image or video clip are original
Label is reference, generates a prediction theme label.
Specifically, involved bluebeard compound incorporation model word2vec and k nearest neighbor algorithm are extended prediction theme label
Concrete operations include:
Prediction theme label is mapped to text vector space by bluebeard compound incorporation model word2vec；
Text vector is calculated at a distance from other vectors of corpus kind using the method for cosine similarity or Euclidean distance
Relationship；
It is obtained and the most similar K theme label of prediction theme label currently entered by k nearest neighbor method；
Using the prediction theme label of this K theme label and support vector machines model as image or video clip
Final theme label is labeled.
A kind of theme label recommended method and tool based on deep learning of the invention, has compared with prior art
Beneficial effect is:
1) the present invention is based on zero-shot learning thoughts, using the ability in feature extraction of depth network model to figure
Picture or video clip carry out feature extraction, carry out the feature of theme label to the feature of extraction using support vector machines model
Classification, and obtain about image or a prediction theme label of video clip, utilize bluebeard compound incorporation model word2vec and K
Nearest neighbor algorithm is extended prediction theme label, and then obtains K theme label relevant to prediction theme label semanteme, in advance
It surveys theme label and K theme label is labeled as the final theme label of input picture or video clip, so that mark
As a result more reliable；
12) present embodiments can apply to the image or videos that in major social media network platform, uploaded by user are automatic
Suitable theme label is selected, when the theme label of image or video labeling and the matching degree of its content are higher, and is more accorded with
When closing current popular topic, image or video are concerned degree will be higher.
Detailed description of the invention
Attached drawing 1 is the principle flow chart of the embodiment of the present invention one；
Attached drawing 2 is the structural block diagram of the embodiment of the present invention two.
Each label information indicates in attached drawing 2:
1, depth network model, 2, support vector machines model,
3, bluebeard compound incorporation model word2vec, 4, k nearest neighbor algoritic module,
5, the image classification model of CNN convolutional neural networks,
6, based on the image classification model of RNN Recognition with Recurrent Neural Network,
7, based on the image classification model of DNN deep neural network,
8, training module, 9, collection submodule, 10, judging submodule.
Specific embodiment
The technical issues of to make technical solution of the present invention, solving and technical effect are more clearly understood, below in conjunction with tool
Body embodiment carries out clear, complete description to technical solution of the present invention, it is clear that described embodiment is only this hair
Bright a part of the embodiment, instead of all the embodiments.Based on the embodiment of the present invention, those skilled in the art are not doing
All embodiments obtained under the premise of creative work out, all within protection scope of the present invention.
Embodiment one:
In conjunction with attached drawing 1, the present embodiment proposes that a kind of theme label recommended method based on deep learning, this method are based on
Zero-shot learning thought carries out image or video clip using the ability in feature extraction of three depth network models
Feature extraction is carried out the tagsort of theme label to the feature of extraction using support vector machines model, and obtained about figure
One prediction theme label of picture or video clip leads prediction using bluebeard compound incorporation model word2vec and k nearest neighbor algorithm
Topic label is extended, and then obtains K theme label relevant to prediction theme label semanteme, predicts theme label and K
Theme label is labeled as the final theme label of input picture or video clip.
In the present embodiment, feature is carried out to image or video clip using the ability in feature extraction of depth network model to mention
Before taking, the ability in feature extraction to depth network model is needed to be trained, concrete operations are as follows:
The image with label or video clip are collected as training set, training set is divided into three data sets；
Feature extraction, three depth are carried out using image or video clip of three depth network models to three data sets
Network model is respectively the image classification model based on CNN convolutional neural networks, the image classification based on RNN Recognition with Recurrent Neural Network
Model, the image classification model based on DNN deep neural network；
Three kinds of features that above three different images sorter network extracts are concatenated together to form new multidimensional image spy
Vector is levied, multidimensional image feature vector inputs support vector machines model；
Support vector machines model carries out the tagsort of theme label to the feature of extraction, and obtain about image or
One prediction theme label of video clip；
Bluebeard compound incorporation model word2vec and k nearest neighbor algorithm obtain multiple themes after being extended to prediction theme label
Label；
Judge whether the multiple theme labels for predicting to obtain after theme label and extension are related；
If related, continue to train next image or video clip；
If uncorrelated, continue to train next image or video clip after correcting.
In the present embodiment, involved bluebeard compound incorporation model word2vec has modeled the correlative relationship between text,
Text representation is converted term vector form by bluebeard compound incorporation model word2vec, and semantic similar text representation is converted into
There will be lesser distance after vector, semantic dissimilar text representation there will be biggish distance, be based on this feature, knot
Closing word incorporation model word2vec will predict that theme label projects to term vector space, and further utilize k nearest neighbor algorithm search
K theme label similar with the prediction theme label is obtained to carry out as the final theme label of input picture or video clip
Mark.
In the present embodiment, prediction theme label is carried out using bluebeard compound incorporation model word2vec and k nearest neighbor algorithm
Extension, concrete operations include:
Prediction theme label is mapped to text vector space by bluebeard compound incorporation model word2vec；
Text vector is calculated at a distance from other vectors of corpus kind using the method for cosine similarity or Euclidean distance
Relationship；
It is obtained and the most similar K theme label of prediction theme label currently entered by k nearest neighbor method；
Using the prediction theme label of this K theme label and support vector machines model as image or video clip
Final theme label is labeled.
In the present embodiment, involved corpus is periodically updated, meanwhile, corpus is also to bluebeard compound incorporation model
Word2vec is updated.
Embodiment two:
In conjunction with attached drawing 2, the present embodiment proposes a kind of theme label recommendation tool based on deep learning, which includes deep
Spend network model 1, support vector machines model 2, bluebeard compound incorporation model word2vec 3, k nearest neighbor algoritic module 4.
In attached drawing 2, depth network model 1, support vector machines model 2, bluebeard compound incorporation model word2vec 3, K are close
Adjacent algoritic module 4 can directly acquire image or video clip without training, and export theme label.At this time:
Depth network model 1 carries out feature extraction to image or video clip；
The features of 2 pairs of support vector machines model extractions carry out the tagsort of theme label, and obtain about image or
One prediction theme label of video clip；
4 pairs of prediction theme labels of bluebeard compound incorporation model word2vec 3 and k nearest neighbor algoritic module are extended, and are obtained
With the most similar K theme label of prediction theme label currently entered；
K theme label of acquisition and the prediction theme label of support vector machines model 2word2vec as image or
The final theme label of video clip is labeled.
In conjunction with attached drawing 2, in order to guarantee to export the maximum correlation of theme label and image or video clip, we can be with
The image for being largely labeled with label or video clip are obtained in advance as training sample.At this point, theme label recommendation tool
It also needs to include the training module 8 for being trained the ability in feature extraction of depth network model 1.
Training module 8 includes collecting submodule 9 and judging submodule 10；
It collects image or video clip of the collection of submodule 9 with label and is stored in training set as training sample, training
The image or video clip of collection sequentially input depth network model 1；
Depth network model 1 carries out feature extraction to the image or video clip of training set, extracts feature as image spy
It levies vector and inputs support vector machines model 2, the feature of 2 pairs of support vector machines model extractions carries out the feature of theme label
Classification, and obtain close with K about a prediction theme label bluebeard compound incorporation model word2vec 3 of image or video clip
Adjacent algorithm obtains multiple theme labels after being extended to prediction theme label；
Judging submodule 10 judges whether the multiple theme labels for predicting to obtain after theme label and extension are related, and in phase
The next image or video clip directly exported in training set when pass is first corrected under exporting in training set again when uncorrelated
One image or video clip.
In training module 8 to depth network model 1, support vector machines model 2, bluebeard compound incorporation model
After the completion of word2vec3, the training of k nearest neighbor algoritic module 4, then the image directly acquired or video clip sequentially input, depth
Network model 1, support vector machines model 2, bluebeard compound incorporation model word2vec 3, k nearest neighbor algoritic module 4, i.e., it is exportable
Multiple theme labels relevant to input picture or video clip.
In the training process, involved training set is divided into three data sets；
There are three depth network models 1, is image classification model 5 based on CNN convolutional neural networks respectively, based on RNN
The image classification model 6 of Recognition with Recurrent Neural Network, the image classification model 7 based on DNN deep neural network；
Three image classification models carry out feature extraction to the image or video clip of three data sets respectively；
Three kinds of features that above three different images sorter network extracts are concatenated together to form new multidimensional image spy
Vector is levied, multidimensional image feature vector inputs support vector machines model 2, then, has so that image or video clip are original
Label is reference, generates a prediction theme label.
In the present embodiment, involved bluebeard compound incorporation model word2vec 3 and k nearest neighbor algorithm are to prediction theme label
The concrete operations being extended include:
Prediction theme label is mapped to text vector space by bluebeard compound incorporation model word2vec 3；
Text vector is calculated at a distance from other vectors of corpus kind using the method for cosine similarity or Euclidean distance
Relationship；
It is obtained and the most similar K theme label of prediction theme label currently entered by k nearest neighbor method；
Using the prediction theme label of this K theme label and support vector machines model 2 as image or video clip
Final theme label is labeled.
In summary, using a kind of theme label recommended method and tool based on deep learning of the invention, Ke Yiying
It uses in major social media network platform, the image or video uploaded by user automatically selects suitable theme label.Separately
Outside, when the theme label of image or video labeling and the matching degree of its content are higher, and more meet current institute it is popular if
When topic, image or video are concerned degree will be higher.
Use above specific case elaborates the principle of the present invention and embodiment, these embodiments are
It is used to help understand core of the invention technology contents, the protection scope being not intended to restrict the invention.Based on of the invention upper
State specific embodiment, those skilled in the art without departing from the principle of the present invention, to made by the present invention
Any improvement and modification, all shall fall within the protection scope of the present invention.
Claims (10)
1. a kind of theme label recommended method based on deep learning, which is characterized in that this method is based on zero-shot
Learning thought carries out feature extraction to image or video clip using the ability in feature extraction of depth network model, utilizes
Support vector machines model carries out the tagsort of theme label to the feature of extraction, and obtains about image or video clip
A prediction theme label, prediction theme label is expanded using bluebeard compound incorporation model word2vec and k nearest neighbor algorithm
Exhibition, and then obtain and the semantic relevant K theme label of prediction theme label, prediction theme label and K theme label conduct
The final theme label of input picture or video clip is labeled.
2. a kind of theme label recommended method based on deep learning according to claim 1, which is characterized in that using deeply
Before the ability in feature extraction of degree network model carries out feature extraction to image or video clip, need to depth network model
Ability in feature extraction is trained, concrete operations are as follows:
The image with label or video clip are collected as training set；
Feature extraction is carried out using image or video clip of the depth network model to training set；
The feature that depth network model is extracted inputs support vector machines model as image feature vector；
Support vector machines model carries out the tagsort of theme label to the feature of extraction, and obtains about image or video
One prediction theme label of segment；
Bluebeard compound incorporation model word2vec and k nearest neighbor algorithm obtain multiple theme marks after being extended to prediction theme label
Label；
Judge whether the multiple theme labels for predicting to obtain after theme label and extension are related；
If related, continue to train next image or video clip；
If uncorrelated, continue to train next image or video clip after correcting.
3. a kind of theme label recommended method based on deep learning according to claim 2, which is characterized in that the instruction
Practice collection and is divided into three data sets；
There are three the depth network models, and image classification model respectively based on CNN convolutional neural networks is followed based on RNN
The image classification model of ring neural network, the image classification model based on DNN deep neural network；
Three image classification models carry out feature extraction to the image or video clip of three data sets respectively；
By above three different images sorter network extract three kinds of features be concatenated together to be formed new multidimensional image feature to
Amount, multidimensional image feature vector input support vector machines model, are ginseng with the original label having of image or video clip
According to generating a prediction theme label.
4. a kind of theme label recommended method based on deep learning according to claim 1, which is characterized in that the knot
It closes word incorporation model word2vec and has modeled the correlative relationship between text, the bluebeard compound incorporation model word2vec will be literary
This expression is converted into term vector form, semantic similar text representation is converted into after vector will have it is lesser away from
From semantic dissimilar text representation will have biggish distance, be based on this feature, bluebeard compound incorporation model word2vec will be pre-
It surveys theme label and projects to term vector space, and is further similar to the prediction theme label using the acquisition of k nearest neighbor algorithm search
K theme label be labeled as the final theme label of input picture or video clip.
5. a kind of theme label recommended method based on deep learning according to claim 4, which is characterized in that utilize knot
It closes word incorporation model word2vec and k nearest neighbor algorithm to be extended prediction theme label, concrete operations include:
Prediction theme label is mapped to text vector space by bluebeard compound incorporation model word2vec；
The distance relation of text vector and other vectors of corpus kind is calculated using the method for cosine similarity or Euclidean distance；
It is obtained and the most similar K theme label of prediction theme label currently entered by k nearest neighbor method；
Using the prediction theme label of this K theme label and support vector machines model as the final of image or video clip
Theme label is labeled.
6. a kind of theme label recommended method based on deep learning according to claim 5, which is characterized in that institute's predicate
Material library is periodically updated, meanwhile, the corpus is also updated bluebeard compound incorporation model word2vec.
7. a kind of theme label recommendation tool based on deep learning, which is characterized in that the tool includes depth network model, branch
Hold vector machine SVM model, bluebeard compound incorporation model word2vec, k nearest neighbor algoritic module；
The depth network model carries out feature extraction to image or video clip；
The support vector machines model carries out the tagsort of theme label to the feature of extraction, and obtain about image or
One prediction theme label of video clip；
The bluebeard compound incorporation model word2vec and the k nearest neighbor algoritic module are extended prediction theme label, and obtain
It takes and the most similar K theme label of prediction theme label currently entered；
K theme label of acquisition and the prediction theme label of support vector machines model word2vec are as image or video
The final theme label of segment is labeled.
8. a kind of theme label recommendation tool based on deep learning according to claim 7, which is characterized in that the tool
It further include the training module being trained to the ability in feature extraction of depth network model；
The training module includes collecting submodule and judging submodule；
The collection submodule collects the image with label or video clip as training sample and is stored in training set, training set
Image or video clip sequentially input depth network model；
Depth network model carries out feature extraction to the image or video clip of training set, extracts feature as image feature vector
Support vector machines model is inputted, support vector machines model carries out the tagsort of theme label to the feature of extraction, and
It obtains about image or the prediction theme label bluebeard compound incorporation model word2vec and k nearest neighbor algorithm of video clip to pre-
It surveys after theme label is extended and obtains multiple theme labels；
The judging submodule judges whether the multiple theme labels for predicting to obtain after theme label and extension are related, and in correlation
When directly export training set in next image or video clip, when uncorrelated, first correct export again it is next in training set
Image or video clip.
9. a kind of theme label recommendation tool based on deep learning according to claim 8, which is characterized in that the instruction
Practice collection and is divided into three data sets；
There are three the depth network models, is image classification model based on CNN convolutional neural networks respectively, is followed based on RNN
The image classification model of ring neural network, the image classification model based on DNN deep neural network；
Three image classification models carry out feature extraction to the image or video clip of three data sets respectively；
By above three different images sorter network extract three kinds of features be concatenated together to be formed new multidimensional image feature to
Amount, multidimensional image feature vector inputs support vector machines model, then, with the original label having of image or video clip
For reference, a prediction theme label is generated.
10. a kind of theme label recommendation tool based on deep learning according to claim 8, which is characterized in that in conjunction with
Word incorporation model word2vec and k nearest neighbor algorithm include: to the concrete operations for predicting that theme label is extended
Prediction theme label is mapped to text vector space by bluebeard compound incorporation model word2vec；
The distance relation of text vector and other vectors of corpus kind is calculated using the method for cosine similarity or Euclidean distance；
It is obtained and the most similar K theme label of prediction theme label currently entered by k nearest neighbor method；
Using the prediction theme label of this K theme label and support vector machines model as the final of image or video clip
Theme label is labeled.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910587052.6A CN110297933A (en) | 2019-07-01 | 2019-07-01 | A kind of theme label recommended method and tool based on deep learning |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910587052.6A CN110297933A (en) | 2019-07-01 | 2019-07-01 | A kind of theme label recommended method and tool based on deep learning |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN110297933A true CN110297933A (en) | 2019-10-01 |
Family
ID=68029847
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910587052.6A Pending CN110297933A (en) | 2019-07-01 | 2019-07-01 | A kind of theme label recommended method and tool based on deep learning |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110297933A (en) |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111611805A (en) * | 2020-04-24 | 2020-09-01 | 平安科技（深圳）有限公司 | Auxiliary writing method, device, medium and equipment based on image |
| CN113139141A (en) * | 2021-04-22 | 2021-07-20 | 康键信息技术（深圳）有限公司 | User label extension labeling method, device, equipment and storage medium |
| CN114387118A (en) * | 2022-01-13 | 2022-04-22 | 重庆邮电大学 | Figure social network subject identification method and system based on particle computing |
| CN114580380A (en) * | 2022-03-07 | 2022-06-03 | 中国人民解放军国防科技大学 | Social media topic tag generation method based on topics |
| CN116156271A (en) * | 2022-12-14 | 2023-05-23 | 北京奇艺世纪科技有限公司 | Video title generation method and device, electronic equipment and readable storage medium |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2010086466A (en) * | 2008-10-02 | 2010-04-15 | Toyota Central R&D Labs Inc | Data classification device and program |
| CN105354307A (en) * | 2015-11-06 | 2016-02-24 | 腾讯科技（深圳）有限公司 | Image content identification method and apparatus |
| CN106682411A (en) * | 2016-12-22 | 2017-05-17 | 浙江大学 | Method for converting physical examination diagnostic data into disease label |
| CN108629373A (en) * | 2018-05-07 | 2018-10-09 | 苏州大学 | A kind of image classification method, system, equipment and computer readable storage medium |
| CN109271539A (en) * | 2018-08-31 | 2019-01-25 | 华中科技大学 | A kind of image automatic annotation method and device based on deep learning |
| CN109460475A (en) * | 2018-12-13 | 2019-03-12 | 北京百分点信息科技有限公司 | A kind of merging method of similar clue |
| CN109871444A (en) * | 2019-01-16 | 2019-06-11 | 北京邮电大学 | A kind of file classification method and system |
-
2019
- 2019-07-01 CN CN201910587052.6A patent/CN110297933A/en active Pending
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2010086466A (en) * | 2008-10-02 | 2010-04-15 | Toyota Central R&D Labs Inc | Data classification device and program |
| CN105354307A (en) * | 2015-11-06 | 2016-02-24 | 腾讯科技（深圳）有限公司 | Image content identification method and apparatus |
| CN106682411A (en) * | 2016-12-22 | 2017-05-17 | 浙江大学 | Method for converting physical examination diagnostic data into disease label |
| CN108629373A (en) * | 2018-05-07 | 2018-10-09 | 苏州大学 | A kind of image classification method, system, equipment and computer readable storage medium |
| CN109271539A (en) * | 2018-08-31 | 2019-01-25 | 华中科技大学 | A kind of image automatic annotation method and device based on deep learning |
| CN109460475A (en) * | 2018-12-13 | 2019-03-12 | 北京百分点信息科技有限公司 | A kind of merging method of similar clue |
| CN109871444A (en) * | 2019-01-16 | 2019-06-11 | 北京邮电大学 | A kind of file classification method and system |
Cited By (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111611805A (en) * | 2020-04-24 | 2020-09-01 | 平安科技（深圳）有限公司 | Auxiliary writing method, device, medium and equipment based on image |
| CN111611805B (en) * | 2020-04-24 | 2023-04-07 | 平安科技（深圳）有限公司 | Auxiliary writing method, device, medium and equipment based on image |
| CN113139141A (en) * | 2021-04-22 | 2021-07-20 | 康键信息技术（深圳）有限公司 | User label extension labeling method, device, equipment and storage medium |
| CN113139141B (en) * | 2021-04-22 | 2023-10-31 | 康键信息技术（深圳）有限公司 | User tag expansion labeling method, device, equipment and storage medium |
| CN114387118A (en) * | 2022-01-13 | 2022-04-22 | 重庆邮电大学 | Figure social network subject identification method and system based on particle computing |
| CN114387118B (en) * | 2022-01-13 | 2024-04-30 | 重庆邮电大学 | Character social network theme identification method and system based on pellet calculation |
| CN114580380A (en) * | 2022-03-07 | 2022-06-03 | 中国人民解放军国防科技大学 | Social media topic tag generation method based on topics |
| CN114580380B (en) * | 2022-03-07 | 2022-08-05 | 中国人民解放军国防科技大学 | Social media topic tag generation method based on topics |
| CN116156271A (en) * | 2022-12-14 | 2023-05-23 | 北京奇艺世纪科技有限公司 | Video title generation method and device, electronic equipment and readable storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110297933A (en) | A kind of theme label recommended method and tool based on deep learning | |
| Shao et al. | Find and focus: Retrieve and localize video events with natural language queries | |
| Yu et al. | Sketch me that shoe | |
| Tang et al. | Frame-wise cross-modal matching for video moment retrieval | |
| Zhang et al. | Rich visual knowledge-based augmentation network for visual question answering | |
| CN108765383B (en) | Video description method based on deep migration learning | |
| CN113536922A (en) | Video behavior identification method for weighting fusion of multiple image tasks | |
| Guan et al. | Multimodal compatibility modeling via exploring the consistent and complementary correlations | |
| Gökçe et al. | Score-level multi cue fusion for sign language recognition | |
| Liu et al. | Compact feature learning for multi-domain image classification | |
| Fu et al. | Learning semantic-aware spatial-temporal attention for interpretable action recognition | |
| Paul et al. | Text-based localization of moments in a video corpus | |
| CN109889923A (en) | Utilize the method for combining the layering of video presentation to summarize video from attention network | |
| CN115187910A (en) | Video classification model training method and device, electronic equipment and storage medium | |
| Zhao et al. | An end-to-end framework for clothing collocation based on semantic feature fusion | |
| CN115759293A (en) | Model training method, image retrieval device and electronic equipment | |
| Li et al. | Integrating language guidance into image-text matching for correcting false negatives | |
| Juyal et al. | Multilabel image classification using the CNN and DC-CNN model on Pascal VOC 2012 dataset | |
| CN114661951A (en) | Video processing method and device, computer equipment and storage medium | |
| Liu et al. | Dynamic self-attention with vision synchronization networks for video question answering | |
| Yuan et al. | MCIC: multimodal conversational intent classification for E-commerce customer service | |
| CN106022389B (en) | A kind of related feedback method actively selecting more example multiple labeling digital pictures | |
| CN111223014B (en) | Method and system for online generation of subdivision scene teaching courses from a large number of subdivision teaching contents | |
| Mi et al. | Dual-branch network with a subtle motion detector for microaction recognition in videos | |
| Hahn et al. | Learning to localize and align fine-grained actions to sparse instructions |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20191001 |
|
| RJ01 | Rejection of invention patent application after publication |