CN103440269B - A kind of video data search method based on study mutually - Google Patents

A kind of video data search method based on study mutually Download PDF

Info

Publication number
CN103440269B
CN103440269B CN201310332612.6A CN201310332612A CN103440269B CN 103440269 B CN103440269 B CN 103440269B CN 201310332612 A CN201310332612 A CN 201310332612A CN 103440269 B CN103440269 B CN 103440269B
Authority
CN
China
Prior art keywords
video data
matrix
feature
represent
similarity matrix
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Expired - Fee Related
Application number
CN201310332612.6A
Other languages
Chinese (zh)
Other versions
CN103440269A (en
Inventor
韩军伟
吉祥
郭雷
胡新韬
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Northwestern Polytechnical University
Original Assignee
Northwestern Polytechnical University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Northwestern Polytechnical University filed Critical Northwestern Polytechnical University
Priority to CN201310332612.6A priority Critical patent/CN103440269B/en
Publication of CN103440269A publication Critical patent/CN103440269A/en
Application granted granted Critical
Publication of CN103440269B publication Critical patent/CN103440269B/en
Expired - Fee Related legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Abstract

The present invention relates to a kind of video data search method based on study mutually, it is characterised in that: calculate the similarity matrix of variety classes video data feature, and utilize similarity matrix to calculate Laplacian Matrix; Calculate eigenvalue and the characteristic vector of variety classes video data Laplacian Matrix, with the characteristic vector corresponding to M eigenvalue of maximum front in Laplacian Matrix; Calculate the similarity matrix of variety classes video data characteristic vector, the corresponding element of the similarity matrix of characteristic vector is multiplied and obtains learning matrix; The corresponding element of learning matrix with the similarity matrix of every kind of feature is multiplied, the similarity matrix after being learnt; Utilize the similarity matrix after study that video data is ranked up, the video data after statistics several sequences front belongs to same category of video data quantity with inquiry target video data, obtains corresponding retrieval rate. The inventive method, retrieval rate is greatly improved than having had before study.

Description

A kind of video data search method based on study mutually
Technical field
The present invention relates to a kind of video data search method based on study mutually, it is possible to be applied in the middle of the retrieval of variety classes video data.
Background technology
Internet technology and numeral taking photo technology develop rapidly so that the video data on network gets more and more, video data retrieval also becomes the focus in a multimedia technology and difficulties. Chinese scholars proposes various features to carry out the retrieval of video data, these features are all based on greatly the color of video data, texture and shape, be referred to as low-level image feature, in recent years, scholar is had to propose the brain function feature cognitive based on human brain, increase than the retrieval rate of low-level image feature, but it have been found that, the speciality that different types of feature reflecting video data are different, if the blend of predominance of these features can be got up, then the accuracy rate of retrieval will get a greater increase.
Summary of the invention
Solve the technical problem that
In order to avoid the deficiencies in the prior art part, the present invention proposes a kind of video data search method based on study mutually, allow the advantage that low-level image feature and brain function feature learn the other side mutually, it is subsequently used in the retrieval of video data, result shows, the feature through study mutually can be greatly improved the accuracy rate of retrieval.
Technical scheme
A kind of video data search method based on study mutually, it is characterised in that step is as follows:
Step 1, calculate the feature X of N number of video data1,X2,...,XNSimilarity matrix W1And characteristic Y1,Y2,...,YNSimilarity matrix W2:
Adopt w i , j 1 = exp ( X i - X j ) T × ( X i - X j ) σ 2 Calculating obtains similarity matrix W1;
Adopt w i , j 2 = exp ( Y i - Y j ) T × ( Y i - Y j ) σ 2 Calculating obtains similarity matrix W2;
Wherein, X1,X2,...,XNRepresent the first feature of the 1st, 2 and N number of video data; Y1,Y2,...,YNRepresent the second feature of the 1st, 2 and N number of video data;Representing matrix W1I-th row jth column element;Representing matrix W2I-th row jth column element;Xi,XjRepresent the first feature of i-th and jth video data; Yi,YjRepresent the second feature of i-th and jth video data; Exp represents fetching number; I, j=1,2 ..., N; N > 0; �� > 0, for constant; Subscript T represents vector transposition;
Step 2: utilizeCalculate W1Laplacian Matrix L1; UtilizeCalculate W2Laplacian Matrix L2;
Wherein, D1Represent diagonal matrix, its element d i , j 1 = Σ t = 1 N w i , t 1 i = j 0 i ≠ j ; T=1,2 ..., N;Representing matrix W1I-th row t row element; D2Represent diagonal matrix, its element d i , j 2 = Σ t = 1 N w i , t 2 i = j 0 i ≠ j ; T=1,2 ..., N;Representing matrix W2I-th row t row element;
Step 3: calculate Laplacian Matrix L1And L2Eigenvalue and characteristic vector, then choose the characteristic vector U corresponding to front M eigenvalue of maximum respectively1,U2,...,UMAnd V1,V2,...,VM; Wherein, M >=1 represents constant; U1,U2,...,UMRepresent and belong to L1The characteristic vector being sized to N �� 1; V1,V2,...,VMRepresent and belong to L2The characteristic vector being sized to N �� 1;
Step 4: utilize characteristic vector U1,U2,...,UMAnd V1,V2,...,VMStructural matrix P=[U1U2...UM] and Q=[V1V2...VM]; Calculate [K1K2...KN]TSimilarity matrix S1[L1L2...LN]TSimilarity matrix S2,
S1Element computing formula be s i , j 1 = exp ( K i - K j ) T × ( K i - K j ) σ 2 ;
S2Element computing formula be s i , j 2 = exp ( L i - L j ) T × ( L i - L j ) σ 2 ;
Wherein, K1,K2,...,KNThe 1,2nd of representing matrix P ..., N row element; L1,L2,...,LNThe 1,2nd of representing matrix Q ..., N row element;
Step 5: by similarity matrix S1And S2Corresponding element be multiplied and obtain learning matrix S;
Step 6: by similarity matrix W1Be multiplied the similarity matrix E after being learnt with the corresponding element of learning matrix S1, by similarity matrix W2Be multiplied the similarity matrix E after being learnt with the corresponding element of learning matrix S2;
Step 7: utilize formula r=�� (I-�� E1)-1T and f=�� (I-�� E2)-1T calculates scores vector r and the f after two kinds of feature learnings of N number of video data, and is arranged from high to low according to mark size by N number of video data, the video data after being sorted;
Wherein, r=(r1,r2,...,rN) represent the first feature of N number of video data retrieve after score vector, r1,r2,...,rNRepresent the 1,2nd ..., the score of N number of video data; F=(f1,f2,...,fN) represent the second feature of N number of video data retrieve after score vector; f1,f2,...,fNRepresent the 1,2nd ..., the score of N number of video data; ��=1-�� represents constant; �� > 0 represents constant; T=[t1,...,tN]TRepresent query vector during retrieval, ti=1 represents that i-th video data is for inquiry target video data, otherwise ti=0.
Adopt after step 7 in the video data after front Q the sequence of statistics and belong to of a sort video data quantity C with inquiry target video data, calculate retrieval rate A=C/Q.
Beneficial effect
A kind of video data search method based on study mutually that the present invention proposes, first, calculates the similarity matrix of variety classes video data feature, and utilizes similarity matrix to calculate Laplacian Matrix; Secondly, calculate eigenvalue and the characteristic vector of variety classes video data Laplacian Matrix, find out the characteristic vector corresponding to M eigenvalue of maximum before in these Laplacian Matrixes respectively; 3rd, calculate the similarity matrix of variety classes video data characteristic vector respectively, the corresponding element of the similarity matrix of characteristic vector is multiplied and obtains learning matrix; 4th, learning matrix is utilized to be multiplied with the corresponding element of the similarity matrix of every kind of feature in the first step, similarity matrix after being learnt, 5th, to each inquiry target video data, utilize the similarity matrix after study, calculate the mark of each video data, and video data is arranged from high to low according to mark, video data quantity consistent with inquiry target video data in several video datas before statistics, calculate retrieval rate.
The method that the present invention proposes, it is possible to allow different types of video data feature mutually learn the advantage of the other side, compared with before study, substantially increases the accuracy rate of video data retrieval.
Accompanying drawing explanation
Fig. 1: the basic flow sheet of the inventive method
Fig. 2: the retrieval rate of the inventive method
Detailed description of the invention
In conjunction with embodiment, accompanying drawing, the invention will be further described:
Hardware environment for implementing is: AMDAthlon64 �� 25000+ computer, 2GB internal memory, 256M video card, and the software environment of operation is: Matlab2009a and WindowsXP. We achieve, with Matlab software, the method that the present invention proposes.
The present invention is embodied as follows:
Flow chart of the present invention is as shown in Figure 1. 1256 video datas for retrieving comprise three classes, are respectively as follows: 561 motion video data, 364 weather forecast video datas and 331 advertisement video data. Two kinds of features respectively brain function feature and low-level image features, specifically comprise the following steps that
1, the feature X of N number of video data is calculated1,X2,...,XNSimilarity matrix W1And characteristic Y1,Y2,...,YNSimilarity matrix W2, W1Element computing formula be:
In like manner calculate matrix W2, its element computing formula is w i , j 2 = exp ( Y i - Y j ) T × ( Y i - Y j ) σ 2 ;
Wherein, X1,X2,...,XNRepresent the first feature of the 1st, 2 and N number of video data; Y1,Y2,...,YNRepresent the second feature of the 1st, 2 and N number of video data;Representing matrix W1I-th row jth column element;Representing matrix W2I-th row jth column element; Xi,XjRepresent the first feature of i-th and jth video data; Yi,YjRepresent the second feature of i-th and jth video data; Exp represents fetching number; I, j=1,2 ..., N; N=1256; Subscript T represents vector transposition; ��=8 �� 10-6;
2, formula is utilizedCalculate W1Laplacian Matrix L1, in like manner calculateWherein, D1Represent diagonal matrix,
Its element d i , j 1 = Σ t = 1 N w i , t 1 i = j 0 i ≠ j ; T=1,2 ..., N;Representing matrix W1I-th row t row element; D2Represent diagonal matrix, its element d i , j 2 = Σ t = 1 N w i , t 2 i = j 0 i ≠ j ; T=1,2 ..., N;Representing matrix W2I-th row t row element;
3, Laplacian Matrix L is calculated1And L2Eigenvalue and characteristic vector, choose the characteristic vector U corresponding to front M eigenvalue of maximum1,U2,...,UMAnd V1,V2,...,VM;
Wherein, M >=1 represents constant; U1,U2,...,UMRepresent and belong to L1The characteristic vector being sized to N �� 1; V1,V2,...,VMRepresent and belong to L2The characteristic vector being sized to N �� 1;
4, characteristic vector U is utilized1,U2,...,UMAnd V1,V2,...,VMStructural matrix P=[U1U2...UM] and Q=[V1V2...VM]; Calculate [K1K2...KN]TSimilarity matrix S1[L1L2...LN]TSimilarity matrix S2, S1Element computing formula be:
In like manner calculate S2, S2Element computing formula be s i , j 2 = exp ( L i - L j ) T × ( L i - L j ) σ 2 ;
Wherein, K1,K2,...,KNThe 1,2nd of representing matrix P ..., N row element; L1,L2,...,LNThe 1,2nd of representing matrix Q ..., N row element;
5, by similarity matrix S1And S2Corresponding element be multiplied and obtain learning matrix S.
6, by similarity matrix W1Be multiplied the similarity matrix E after being learnt with the corresponding element of learning matrix S1, by similarity matrix W2Be multiplied the similarity matrix E after being learnt with the corresponding element of learning matrix S2
7, formula r=�� (I-�� E is utilized1)-1T and f=�� (I-�� E2)-1T calculates scores vector r and the f after two kinds of feature learnings of N number of video data, and is arranged from high to low according to mark size by N number of video data, the video data after being sorted.
Wherein, r=(r1,r2,...,rN) represent the first feature of N number of video data retrieve after score vector, r1,r2,...,rNRepresent the 1,2nd ..., the score of N number of video data; F=(f1,f2,...,fN) represent the second feature of N number of video data retrieve after score vector; f1,f2,...,fNRepresent the 1,2nd ..., the score of N number of video data; ��=1-�� represents constant; ��=0.99;T=[t1,...,tN] T query vector when representing retrieval, ti=1 represents that i-th video data is for inquiry target video data, otherwise ti=0;
8, the video data after front Q the sequence of statistics belongs to of a sort video data quantity C with inquiry target video data, calculate retrieval rate A=C/Q.
This algorithm is utilized to carry out video data retrieval, each video data in 1256 video datas is carried out primary retrieval as inquiry target video data, front 5, in video data after 10,15 and 20 sequences, statistics belongs to the percentage ratio shared by same category of video data with inquiry target video data respectively. The percentage ratio that 1256 video datas are inquired about gained is averaged, and obtains the average retrieval accuracy rate of 1256 videos. As shown in Figure 2. As a comparison, we also use brain function feature and low-level image feature individually to retrieve, retrieving does not learn mutually, the retrieval rate obtained is displayed that in fig 2, it can be seen that the retrieval rate of brain function feature after study and low-level image feature has had be greatly improved than before study. Wherein, improve 19.8% before the study of brain function aspect ratio, low-level image feature improves 27.5% than before study.

Claims (2)

1. the video data search method based on study mutually, it is characterised in that step is as follows:
Step 1, calculate the feature X of N number of video data1,X2,...,XNSimilarity matrix W1And characteristic Y1,Y2,...,YNSimilarity matrix W2:
AdoptCalculating obtains similarity matrix W1;
AdoptCalculating obtains similarity matrix W2;
Wherein, X1,X2,...,XNRepresent the 1st, 2 ... the first feature of N number of video data; Y1,Y2,...,YNRepresent the 1st, 2 ... the second feature of N number of video data, the first feature and the second feature respectively brain function feature and low-level image feature;Representing matrix W1I-th row jth column element;Representing matrix W2I-th row jth column element; Xi,XjRepresent the first feature of i-th and jth video data; Yi,YjRepresent the second feature of i-th and jth video data; Exp represents fetching number; I, j=1,2 ..., N; N > 0; �� > 0, for constant; Subscript T represents vector transposition;
Step 2: utilizeCalculate W1Laplacian Matrix L1; UtilizeCalculate W2Laplacian Matrix L2;
Wherein, D1Represent diagonal matrix, its element Representing matrix W1I-th row t row element; D2Represent diagonal matrix, its element Representing matrix W2I-th row t row element;
Step 3: calculate Laplacian Matrix L1And L2Eigenvalue and characteristic vector, then choose the characteristic vector U corresponding to front M eigenvalue of maximum respectively1,U2,...,UMAnd V1,V2,...,VM; Wherein, M >=1 represents constant; U1,U2,...,UMRepresent and belong to L1The characteristic vector being sized to N �� 1; V1,V2,...,VMRepresent and belong to L2The characteristic vector being sized to N �� 1;
Step 4: utilize characteristic vector U1,U2,...,UMAnd V1,V2,...,VMStructural matrix P=[U1U2...UM] and Q=[V1V2...VM]; Calculate [K1K2...KN]TSimilarity matrix S1[L1L2...LN]TSimilarity matrix S2,
S1Element computing formula be
S2Element computing formula be
Wherein, K1,K2,...,KNThe 1,2nd of representing matrix P ..., N row element; L1,L2,...,LNThe 1,2nd of representing matrix Q ..., N row element;
Step 5: by similarity matrix S1And S2Corresponding element be multiplied and obtain learning matrix S;
Step 6: by similarity matrix W1Be multiplied the similarity matrix E after being learnt with the corresponding element of learning matrix S1, by similarity matrix W2Be multiplied the similarity matrix E after being learnt with the corresponding element of learning matrix S2;
Step 7: utilize formula r=�� (I-�� E1)-1T and f=�� (I-�� E2)-1T calculates scores vector r and the f after two kinds of feature learnings of N number of video data, and is arranged from high to low according to mark size by N number of video data, the video data after being sorted;
Wherein, r=(r1,r2,...,rN) represent the first feature of N number of video data retrieve after score vector, r1,r2,...,rNRepresent the 1,2nd ..., the score of N number of video data; F=(f1,f2,...,fN) represent the second feature of N number of video data retrieve after score vector; f1,f2,...,fNRepresent the 1,2nd ..., the score of N number of video data; ��=1-�� represents constant; �� > 0 represents constant; T=[t1,...,tN]TRepresent query vector during retrieval, ti=1 represents that i-th video data is for inquiry target video data, otherwise ti=0.
2. the video data search method based on study mutually according to claim 1, it is characterized in that: adopt after step 7 in the video data after front Q the sequence of statistics and belong to of a sort video data quantity C with inquiry target video data, calculate retrieval rate A=C/Q.
CN201310332612.6A 2013-08-01 2013-08-01 A kind of video data search method based on study mutually Expired - Fee Related CN103440269B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201310332612.6A CN103440269B (en) 2013-08-01 2013-08-01 A kind of video data search method based on study mutually

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201310332612.6A CN103440269B (en) 2013-08-01 2013-08-01 A kind of video data search method based on study mutually

Publications (2)

Publication Number Publication Date
CN103440269A CN103440269A (en) 2013-12-11
CN103440269B true CN103440269B (en) 2016-06-08

Family

ID=49693961

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201310332612.6A Expired - Fee Related CN103440269B (en) 2013-08-01 2013-08-01 A kind of video data search method based on study mutually

Country Status (1)

Country Link
CN (1) CN103440269B (en)

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104111969B (en) * 2014-06-04 2017-06-20 百度移信网络技术(北京)有限公司 The method and system of a kind of similarity measurement

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101071439A (en) * 2007-05-24 2007-11-14 北京交通大学 Interactive video searching method based on multi-view angle
CN101650728A (en) * 2009-08-26 2010-02-17 北京邮电大学 Video high-level characteristic retrieval system and realization thereof
CN102142037A (en) * 2011-05-05 2011-08-03 西北工业大学 Video data search method based on functional magnetic resonance imaging

Family Cites Families (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US7076485B2 (en) * 2001-03-07 2006-07-11 The Mitre Corporation Method and system for finding similar records in mixed free-text and structured data

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101071439A (en) * 2007-05-24 2007-11-14 北京交通大学 Interactive video searching method based on multi-view angle
CN101650728A (en) * 2009-08-26 2010-02-17 北京邮电大学 Video high-level characteristic retrieval system and realization thereof
CN102142037A (en) * 2011-05-05 2011-08-03 西北工业大学 Video data search method based on functional magnetic resonance imaging

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
基于多特征距离学习的视频分类;李真超等;《计算机应用与软件》;20121231;第29卷(第12期);第10-12页 *

Also Published As

Publication number Publication date
CN103440269A (en) 2013-12-11

Similar Documents

Publication Publication Date Title
CN107358257B (en) Under a kind of big data scene can incremental learning image classification training method
CN103984959B (en) A kind of image classification method based on data and task-driven
CN107358293A (en) A kind of neural network training method and device
CN104156433B (en) Image retrieval method based on semantic mapping space construction
CN103793721B (en) Pedestrian repeat recognition method and system based on area related feedback
CN101989302B (en) Multilayer bitmap color feature-based image retrieval method
CN111242208A (en) Point cloud classification method, point cloud segmentation method and related equipment
CN109063719B (en) Image classification method combining structure similarity and class information
CN106095893A (en) A kind of cross-media retrieval method
CN103942749B (en) A kind of based on revising cluster hypothesis and the EO-1 hyperion terrain classification method of semi-supervised very fast learning machine
CN105809146A (en) Image scene recognition method and device
CN102999926B (en) A kind of image vision significance computational methods merged based on low-level image feature
CN103336835B (en) Image retrieval method based on weight color-sift characteristic dictionary
CN107609552A (en) Salient region detection method based on markov absorbing model
CN105320764A (en) 3D model retrieval method and 3D model retrieval apparatus based on slow increment features
CN106250555A (en) Vehicle retrieval method and devices based on big data
CN104252616A (en) Human face marking method, device and equipment
CN106780422A (en) A kind of notable figure fusion method based on Choquet integrations
CN102831161B (en) For the semi-supervised sequence learning method based on manifold regularization of image retrieval
CN107563324B (en) Hyperspectral image classification method and device of ultralimit learning machine with composite nuclear structure
CN113837191B (en) Cross-star remote sensing image semantic segmentation method based on bidirectional unsupervised domain adaptive fusion
CN105447100B (en) A kind of cloud atlas search method based on shape feature
CN103440269B (en) A kind of video data search method based on study mutually
CN103714178B (en) Automatic image marking method based on word correlation
CN116543250A (en) Model compression method based on class attention transmission

Legal Events

Date Code Title Description
C06 Publication
PB01 Publication
C10 Entry into substantive examination
SE01 Entry into force of request for substantive examination
C14 Grant of patent or utility model
GR01 Patent grant
CF01 Termination of patent right due to non-payment of annual fee

Granted publication date: 20160608

Termination date: 20190801

CF01 Termination of patent right due to non-payment of annual fee