CN104050247A - Method for realizing quick retrieval of mass videos - Google Patents
Method for realizing quick retrieval of mass videos Download PDFInfo
- Publication number
- CN104050247A CN104050247A CN201410245315.2A CN201410245315A CN104050247A CN 104050247 A CN104050247 A CN 104050247A CN 201410245315 A CN201410245315 A CN 201410245315A CN 104050247 A CN104050247 A CN 104050247A
- Authority
- CN
- China
- Prior art keywords
- video
- feature vector
- key feature
- image
- frame
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/70—Information retrieval; Database structures therefor; File system structures therefor of video data
- G06F16/71—Indexing; Data structures therefor; Storage structures
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/70—Information retrieval; Database structures therefor; File system structures therefor of video data
- G06F16/78—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
- G06F16/783—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually using metadata automatically derived from the content
- G06F16/7834—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually using metadata automatically derived from the content using audio features
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Multimedia (AREA)
- Library & Information Science (AREA)
- Data Mining & Analysis (AREA)
- Databases & Information Systems (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Software Systems (AREA)
- Image Analysis (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention relates to a method for realizing the quick retrieval of mass videos. The method comprises the following steps: respectively extracting spatial feature vectors from all frame video images in a video stream of a video library to obtain video feature sequences; extracting key feature vectors from the spatial feature vectors; establishing a distributed storage index database according to the key feature vectors of all video files in the video library; extracting key feature vector sets of videos to be retrieved and extracting video index files of the videos to be retrieved; performing the video similarity retrieving in the distributed storage index database according to the video index files of the videos to be retrieved and outputting video retrieval results of the video files with the similarity larger than the preset value of the system. Through the adoption of the method with the structure, representative visual words are adopted to replace key frames, video information is completely represented, a large amount of redundant of video information does not exist, the video information is very compact, the retrieval speed is increased, and the method has mass data concurrent processing capacity, and is wider in application range.
Description
Technical field
The present invention relates to multimedia information technique field, relate in particular to multimedia information retrieval, data mining and field of video processing, specifically refer to a kind of method that realizes magnanimity video quick-searching.
Background technology
Along with multimedia information technology develop rapidly, the appearance of video sharing website, internet video quantity increases rapidly, and becomes geometric series to rise.Issue, share by network the one way of life mode that becomes people with retrieve video.In the face of the multi-medium data of magnanimity, how retrieving fast same or analogous video becomes the focus of current industry and academia's research.
Traditional video retrieval method based on key frame mainly tends to the accuracy of video frequency searching, but computation complexity is high, spend some minutes and just can complete primary retrieval task.In the face of the networking video of magnanimity, traditional video comparison technology is not competent.The video retrieval technology of current Internet, use for reference the core concept of traditional text search engine, video features is regarded as to video word (visual word), build the inverted index of video file, realize the quick indexing to magnanimity video file.
Successfully mate the degree that the degree of enriching and the self information of retrieve video and reference video self information is expressed, describes that depends on.Internet video retrieval method is in extracting key frame, not often according to the conventional method, first carry out shot segmentation, then extract camera lens key frame, because the position of extracting key frame can be subject to the impact of the factor such as video frame rate, resolution, key frame can not be stablized, extract reliably.More simple possible method was video to be done to once sampling every 1 second, as key frame.The frequency that has in fact been equivalent to increase sampling, sample frequency is higher, and it is more abundant that original information is expressed, but calculated amount will be larger.The degree that increases information representation by increasing sample frequency, can cause like this, and existing information is overexpressed generation redundancy, has again information not given full expression to, and causes information dropout.And line sampling meeting makes the information of losing have randomness, because video information is not linear expression.The information of random loss can reduce the Stability and veracity of retrieval.In addition on the one hand, traditional extraction method of key frame, general information changes less place and extracts less key frame, became larger place extract more key frame in frame of video, can produce and relatively compact and more complete expressing information, its degree depends on cluster or the threshold value of cutting apart.Retrieve video and reference video tend to be subject to various noise, such as video resolution variation, Network Packet Loss, video frame losing, low frame per second, video inserts, video editings etc., can make original video information be mixed with noise, or cause partial information lose and no longer complete.Traditional video key frame extracting method is too idealized, a) do not consider the complicacy of external interference, suitably redundance is necessary, and b) it does not build for magnanimity retrieval tasks for the feature of extracting key frame, and relevant method is also improperly directly used for extracting key frame.The retrieval character that How to choose is appropriate, makes the number of frames of keyframe sequence that builds minimum, and video lens information representation relatively complete and have suitable redundancy becomes the key issue to be solved towards searching mass data technology assistant officer.
Summary of the invention
The object of the invention is to overcome the shortcoming of above-mentioned prior art, provide a kind of can realize adopt representational vision word replace key frame, not only without bulk redundancy, but also very compact, accelerate retrieval rate, there is mass data concurrent processing ability, there is the method that realizes magnanimity video quick-searching of broader applications scope.
To achieve these goals, the method that realizes magnanimity video quick-searching of the present invention has following formation:
This realizes the method for magnanimity video quick-searching, and its principal feature is that described method comprises the following steps:
(1) each frame video image in the video flowing of video library is extracted respectively to spatial signature vectors and obtain video features sequence;
(2) in the spatial signature vectors of described video features sequence, extract key feature vector;
(3) set up the distributed storage index database of all video files according to the key feature vector of all video files in video library;
(4) extract the key feature vector set of video to be retrieved and extract the video index file of this video to be retrieved;
(5) in described distributed storage index database, carry out video similarity retrieval according to the video index file of described video to be retrieved and export the video frequency searching result that similarity is greater than systemic presupposition value.
Preferably, described spatial signature vectors comprise gray space distribution characteristics and the texture space distribution characteristics of corresponding two field picture, described extracts respectively spatial signature vectors to each frame video image in the video flowing of video library, comprises the following steps:
(11) calculate gray level image and the Edge texture image of each frame video image in the video flowing of video library;
(12) calculate each frame video image gray level image central space feature and boundary space feature and obtain the gray space distribution characteristics of this frame video image being formed by described central space feature and boundary space feature;
(13) calculate the texture space distribution characteristics of the Edge texture image of each frame video image.
More preferably, the gray level image of each frame video image and Edge texture image in the described video flowing that calculates video library, comprise the following steps:
(111) the each frame video image in the video flowing of video library be divided into several onesize subimages and calculate gray-scale value and the texture number of edge points of each number of sub images;
(112) gray-scale value that calculates each number of sub images of each frame video image obtains the gray level image of this frame video image;
(113) the texture number of edge points of calculating each number of sub images of each frame video image obtains the Edge texture image of this frame video image.
More preferably, the central space feature of the gray level image of the described each frame video image of calculating and boundary space feature, be specially:
Calculate central space feature and the boundary space feature of the local binary patterns of the gray level image of each frame video image;
The texture space distribution characteristics of the Edge texture image of the described each frame video image of calculating, is specially:
Calculate the texture space distribution characteristics of the local binary patterns of the Edge texture image of each frame video image.
More preferably, described spatial signature vectors also comprises color histogram feature, and described extracts respectively spatial signature vectors to each frame video image in the video flowing of video library, further comprising the steps of:
(14) calculate the color histogram feature of each frame video image.
Preferably, described extracts key feature vector in the spatial signature vectors of described video features sequence, comprises the following steps:
(21) first spatial signature vectors of described video features sequence is defaulted as to key feature vector;
(22) calculate the mahalanobis distance of each spatial signature vectors and last key feature vector;
(23) corresponding the mahalanobis distance that is greater than systemic presupposition threshold value spatial signature vectors is extracted as to key feature vector.
Preferably, the described key feature vector according to all video files in video library is set up the distributed storage index database of all video files, comprises the following steps:
(31) set up the subspace projection histogram of key feature vector in described video features sequence and record each key feature vector the frequency that occurs in corresponding video;
(32) set up the inverted index file of all video files of video library;
(33) set up the distributed index database of all video files of video library.
More preferably, the described subspace projection histogram of setting up key feature vector in video features sequence, is specially:
By key feature vector projection in video features sequence in gray scale subspace, texture subspace and color sub-spaces and obtain the subspace projection histogram of each key feature vector.
Further, described record each key feature vector the frequency that occurs in corresponding video, be specially:
Record the eigenwert that represents this key feature vector frequency of occurrence in video in the corresponding subspace projection histogram of each key feature vector.
Further, the inverted index file of described all video files of setting up video library, comprises the following steps:
(321) in statistics video library, the corresponding key feature vector set of each video file merges the statistics key feature vector storehouse that forms this video library;
(322) set up the collection of document that has this key feature vector corresponding to each key feature vector in described statistics key feature vector storehouse;
(323) by the document of key feature vector set according to the quantity of contained key feature vector from how to sort to few;
(324) set up the inverted index file of all video files of video library according to each sub spaces.
Again further, the distributed index database of described all video files of setting up video library, comprises the following steps:
(331) utilize local sensitivity hash algorithm based on p-stable by the key feature DUAL PROBLEMS OF VECTOR MAPPING of each sub spaces to the one-dimensional space;
(332) adopt name_node to safeguard Hash table based on Hadoop distributed file system framework and adopt data_node to preserve the distributed index database that index data is all video files.
More preferably, described carries out video similarity retrieval according to the video index file of described video to be retrieved in described distributed storage index database, is specially:
(51) calculate in video subspace projection histogram to be retrieved and video library the histogrammic friendship of each video subspace projection as the similarity of each video in video to be retrieved and video library;
(52) reject according to the space-time structure consistance of the key feature vector of each video in video to be retrieved and video library the video file that does not meet space-time structure coherence request.
Further, described output similarity is greater than the video frequency searching result of systemic presupposition value, comprises the following steps:
(52) extract video to be retrieved key feature vector each subspace projection histogram and each key feature vector is mapped as to cryptographic hash in each sub spaces;
(53) choose similarity in distributed index database by described inverted index file and meet video file that systemic presupposition requires as output;
(54) the space-time structure consistance of calculating the key feature vector of each video in video to be retrieved and video library is also exported the video file that is greater than systemic presupposition value with the similarity of described video to be retrieved.
Adopt the method that realizes magnanimity video quick-searching in this invention, there is following beneficial effect:
The present invention, mainly for the selection problem that builds video index information integrity and index feature, has proposed a kind of subspace method based on video finger print, solve current towards mass data fast, the search problem of robust.First, this patent adopts novel extraction method of key frame, by the extraction replacement key-frame extraction of key feature vector, directly replace key frame with representational visual signature, be equivalent at feature space, original video be encoded, complete expression video information, both without bulk redundancy, very compact again, and overcome current key frame extracting parameter selection problem.Secondly, each visual signature is mapped to one dimension cryptographic hash, according to the cryptographic hash in-scope of visual signature, select suitable HDFS (Hadoop Distributed File System, Hadoop distributed file system) name_node (title node) and data_node (back end), accelerate retrieval rate, make it again to have the ability of mass data concurrent processing, there is range of application widely.
Brief description of the drawings
Fig. 1 is the process flow diagram of the method that realizes magnanimity video quick-searching of the present invention.
Fig. 2 is the process flow diagram that the method that realizes magnanimity video quick-searching of the present invention is applied to specific embodiment.
Fig. 3 is the process flow diagram that sequence of frames of video is mapped to video features sequence of the present invention.
Fig. 4 is the process flow diagram of calculating gray space distribution characteristics of the present invention.
Fig. 5 is the process flow diagram of extraction key feature vector of the present invention.
Embodiment
In order more clearly to describe technology contents of the present invention, conduct further description below in conjunction with specific embodiment.
The invention discloses a kind of magnanimity video method for quickly retrieving and system, wherein the method comprises: sequence of frames of video is mapped to the video features sequence of spatial signature vectors composition, extracts the key feature vector of wherein representative feature as video features sequence; Shine upon described key feature vector by hash function, the Hash bucket at the cryptographic hash place obtaining according to mapping, builds distributed index; According to the key feature vector set of video to be retrieved, calculate each corresponding cryptographic hash place Hash bucket numbering, extract the video index file of character pair, obtain candidate's video file by ballot mode, calculate the similarity of video to be retrieved and candidate's video file, output similarity be greater than certain threshold value as result for retrieval.
As shown in Figure 1, the method that realizes magnanimity video quick-searching of the present invention comprises the following steps:
(1) each frame video image in the video flowing of video library is extracted respectively to spatial signature vectors and obtain video features sequence;
One preferred embodiment in, described spatial signature vectors comprise gray space distribution characteristics and the texture space distribution characteristics of corresponding two field picture, therefore, described each frame video image in the video flowing of video library extracted respectively to spatial signature vectors obtain video features sequence, comprise the following steps:
(11) calculate gray level image and the Edge texture image of each frame video image in the video flowing of video library;
One preferred embodiment in, calculate gray level image and Edge texture image and can adopt following this mode, i.e. gray level image and the Edge texture image of each frame video image in the described video flowing that calculates video library, comprises the following steps:
(111) the each frame video image in the video flowing of video library be divided into several onesize subimages and calculate gray-scale value and the texture number of edge points of each number of sub images;
(112) gray-scale value that calculates each number of sub images of each frame video image obtains the gray level image of this frame video image;
(113) the texture number of edge points of calculating each number of sub images of each frame video image obtains the Edge texture image of this frame video image.
(12) calculate each frame video image gray level image central space feature and boundary space feature and obtain the gray space distribution characteristics of this frame video image being formed by described central space feature and boundary space feature;
Wherein central space feature and boundary space feature can be central space feature and the boundary space features based on local binary patterns.
(13) calculate the texture space distribution characteristics of the Edge texture image of each frame video image.
Wherein, texture space distribution characteristics can be the texture space distribution characteristics based on local binary patterns.
In the preferred embodiment of one, described spatial signature vectors can further include color histogram feature, make spatial signature vectors more can represent video features, described each frame video image in the video flowing of video library is extracted respectively to spatial signature vectors, further comprising the steps of:
(14) calculate the color histogram feature of each frame video image.
(2) in the spatial signature vectors of described video features sequence, extract key feature vector;
One preferred embodiment in, extract key feature vector and comprise the following steps:
(21) first spatial signature vectors of described video features sequence is defaulted as to key feature vector;
(22) calculate the mahalanobis distance of each spatial signature vectors and last key feature vector;
(23) corresponding the mahalanobis distance that is greater than systemic presupposition threshold value spatial signature vectors is extracted as to key feature vector.
(3) set up the distributed storage index database of all video files according to the key feature vector of all video files in video library;
One preferred embodiment in, set up distributed storage index database and comprise the following steps:
(31) set up the subspace projection histogram of key feature vector in described video features sequence and record each key feature vector the frequency that occurs in corresponding video;
Further, subspace can be gray scale subspace and texture subspace, can also comprise color sub-spaces, and the therefore described subspace projection histogram of setting up key feature vector in video features sequence, is specially:
By key feature vector projection in video features sequence in gray scale subspace, texture subspace and color sub-spaces and obtain the subspace projection histogram of each key feature vector.
Further, described record each key feature vector the frequency that occurs in corresponding video, be specially:
Record the eigenwert that represents this key feature vector frequency of occurrence in video in the corresponding subspace projection histogram of each key feature vector.
(32) set up the inverted index file of all video files of video library;
Further, the inverted index file of described all video files of setting up video library, comprises the following steps:
(321) in statistics video library, the corresponding key feature vector set of each video file merges the statistics key feature vector storehouse that forms this video library;
(322) set up the collection of document that has this key feature vector corresponding to each key feature vector in described statistics key feature vector storehouse;
(323) by the document of key feature vector set according to the quantity of contained key feature vector from how to sort to few;
(324) set up the inverted index file of all video files of video library according to each sub spaces.
(33) set up the distributed index database of all video files of video library.
Further, the distributed index database of described all video files of setting up video library, comprises the following steps:
(331) utilize local sensitivity hash algorithm based on p-stable by the key feature DUAL PROBLEMS OF VECTOR MAPPING of each sub spaces to the one-dimensional space;
(332) adopt name_node to safeguard Hash table based on Hadoop distributed file system framework and adopt data_node to preserve the distributed index database that index data is all video files.
(4) extract the key feature vector set of video to be retrieved and extract the video index file of this video to be retrieved; In specifically practicing, the key feature vector that extracts video to be retrieved herein can adopt as the key feature vector extracting method in step (1) and (2).
(5) in described distributed storage index database, carry out video similarity retrieval according to the video index file of described video to be retrieved and export the video frequency searching result that similarity is greater than systemic presupposition value.
One preferred embodiment in, described carries out video similarity retrieval according to the video index file of described video to be retrieved in described distributed storage index database, is specially:
(51) calculate in video subspace projection histogram to be retrieved and video library the histogrammic friendship of each video subspace projection as the similarity of each video in video to be retrieved and video library;
(52) reject according to the space-time structure consistance of the key feature vector of each video in video to be retrieved and video library the video file that does not meet space-time structure coherence request.
One preferred embodiment in, described output similarity is greater than the video frequency searching result of systemic presupposition value, comprises the following steps:
(52) extract video to be retrieved key feature vector each subspace projection histogram and each key feature vector is mapped as to cryptographic hash in each sub spaces;
(53) choose similarity in distributed index database by described inverted index file and meet video file that systemic presupposition requires as output;
(54) the space-time structure consistance of calculating the key feature vector of each video in video to be retrieved and video library is also exported the video file that is greater than systemic presupposition value with the similarity of described video to be retrieved.
Further set forth the method that realizes magnanimity video quick-searching of the present invention with a specific embodiment below, as shown in Figure 2, in concrete application, the method comprises the following steps:
(1) sdi video feature coding, is mapped to video features sequence by sequence of frames of video; As shown in Figure 3, specifically comprise following sub-step:
(11) read a frame video image from video flowing, image is divided into MxN onesize subimage, calculate each subimage gray-scale value and texture number of edge points;
(12) calculate LBP (the Local binary pattern of two types of gray level images, local binary patterns) space characteristics v_gray, central feature (f) and boundary characteristic (g) as shown in Figure 4, jointly formed the spatial distribution characteristic of the frame of video of 8 by central feature and boundary characteristic, see (h) in Fig. 4;
(13) the same, the LBP spatial distribution characteristic v_texture of edge calculation texture image, simple for the purpose of, number that can statistical picture piece internal edge is as the metric of Texture complication, its result of calculation is the same is one 8 bit space grain distribution features;
(14) in conjunction with v_gray and v_texture feature, construct polynary frame feature v=(v_gray, v_texture), we call a frame vision word (visual word) a frame feature v;
(15) except calculating the gray scale and texture LBP space characteristics of figure, can also add other frame features, such as 8 or the color histogram v_color_his_16 of 16bins, now v=(v_gray, v_texture, v_color_his_16); This frame feature constituted mode can overcome the defect that single proper subspace can not fine expression frame of video.
This patent is not considered temporal characteristics, because the temporal characteristics of retrieve video is subject to the impact of other disturbing factors such as low frame per second or scarce frame to have very large uncertainty, is probably wrong according to the space-time characteristic of the frame of time series structure.But in similar to search process the consistance of proving time order.
(2) video key feature extracts, and extracts the key feature vector of wherein representative feature as video features sequence; As shown in Figure 5, specifically comprise following sub-step:
(21) by first spatial signature vectors of video features sequence key feature vector by default;
(22) extract the spatial signature vectors v (n) of current n frame, if current feature v (n) and last key feature vector (v (m), m) mahalanobis distance is greater than threshold value thrsh, consider noise factor, 1<=thrsh<=2 herein, present frame is key feature vector, be designated as (v (n), n).
Two different proper vector v1 and v2 have expressed different video contents.The representational key feature vector of apparatus key vecotor replaces traditional key frame vector, has not only saved this step of key-frame extraction, expresses video content more directly, accurately and give birth to feature with source.Solve the selection problem of video index information integrity and index feature.
We are called vision word (visual word) crucial proper vector (key vector), and the set of visual word is called visual vocabulary table (visual vocabulary).The histogram of the set of eigenvectors of single video file is called feature histogram (vector histogram or visual word histogram).In order to make key vector there is abundant ability to express and abstract summarization, key vector is by different but sub vector Special composition gray distribution features Gray-LBP vector independently, spatial texture distribution characteristics Texture-LBP vector and color vector composition, can simply be expressed as key vector={Gray-LBP, Texture-LBP, Color}.Jointly forming by different abstract characteristics concept space the property taken advantage of describes space and has realized abundant ability to express and the abstract summarization of key vector.
This patent and other key-frame extraction differences be, the present invention directly extracts key feature instead of traditional key-frame extraction in video flowing.
Tradition key-frame extraction is to utilize Key-frame Extraction Algorithm to extract key frame, then utilize the key frame extracting to extract again retrieval character, extract key frame method used and extract the retrieval character calculating after key frame and be not exclusively equal to, sometimes widely different, can cause like this describing inaccurate; This is also one of not high enough reason of traditional retrieval character accuracy.
(3) sequence of frames of video, to the histogrammic mapping of video vision word, specifically comprises following sub-step:
(31) because vision word may have very high dimension, such as (f_gray, f_texture, f_color_his_16) dimension (8,8,16) totally 32 tie up, the nearly 1GB of its memory requirements, we heavily throw into respectively space, f_gray8 seat again 32 dimension spaces, and space, f_texture8 seat, in space, f_color_his_16 seat, add up respectively their histogram in subspace, its memory requirements significantly reduces, not enough 70MB, and the histogram size of single video file exceedes 10MB hardly;
(32) the histogrammic bin of subspace projection (certain sub spaces feature, such as 8 LBP features) numerical value represent the frequency that this feature occurs in video, in order to keep inner this feature of same bin in the distribution of time, record in the following way bin content:
The frequency that this feature of bin:(occurs is n1+n2+ ... + nk's and, frame number T1, continuously occurrence number n1, T2, n2 ..., Tk, nk).
(4) set up video file inverted index file, specifically comprise following sub-step:
(41) vision set of letters corresponding to each video in statistics video library, the statistics vision dictionary VwSet of formation video library.According to
Vw_i (i vision word in vision dictionary) set up have this vision word collection of document vf1, vf2, vf3 ..., vfni}, ni is collection of document size;
(42) document of vision word collection of document is by how many sequences from big to small of contained vision word;
(43) because higher-dimension vision word projects to low-dimensional proper subspace, set up inverted index file according to each subspace.
(5) set up distributed storage index database, specifically comprise the following steps:
(51) utilize local sensitivity hash algorithm (LSH) bundle space characteristics f_v based on p-stable, (such as f_colo_his_16) be mapped to the one-dimensional space [0-Range);
(52) adopt the HDFS file system framework of hadoop, safeguard LSH table with name_node, preserve index data with data_node.
(6) video similarity is calculated, and specifically comprises the following steps:
Retrieve video Vq subspace histogram be Bin_q_1, Bin_q_2 ..., Bin_q_M}, M is proper subspace size, video library video Vi histogram be Bin_i_1, Bin_i_2 ..., Bin_i_M}, in Bin_id_n, id is video unique number, and n is the sequence number of histogram bin
Bin_id_n is the number of times that this feature occurs;
(61) video similarity is histogrammic friendship,
(62) if similarity is greater than threshold value thrsh_sim, the relatively time series relation of vision word.Histogram time serial message keeps a record in step (32), and its algorithm is as follows:
Represent video according to retrieve video vision word in the order of the appearance of time, for example, (Vq_vw1, Bin_k1), (Vq_vw2, Bin_k2) ...,
(Vq_vwl, Bin_kl)) }, wherein vw1 is the vision word of first appearance in time, and vw2 occurs subsequently, and Bin_k1 represents that the sequence number of this vision word place histogram Bin is k1, the total number of kl histogram bin;
(63) if the vision word Vq_vw (x) in retrieve video occur time early than Vq_vw (y), x<y, in all sequences of the identical vision word that sequence number corresponding to video histogram of the coupling Bin that is Bin_kx comprises number, have at least one to be less than one of them sequence number that Bin_ky is corresponding; We think that the sequencing that retrieval of visual word occurs should be consistent with the sequencing that vision word same in similar video occurs, corresponding space-time structure has consistance, by a large amount of doubtful similar videos of the desirable removal of time sequencing.
(7) video is retrieved, specifically in the following ways:
Extract retrieve video vision word histogram, be cryptographic hash vision word feature at each subspace mapping, determine name_node and the data_nodes at access Hash bucket place, by inverted index video file, choose the most similar front 20% conduct output, then calculate the consistance of space-time structure, all video files that are retrieved that are greater than 0.7 by similarity size output similarity.
Adopt the method that realizes magnanimity video quick-searching in this invention, there is following beneficial effect:
The present invention, mainly for the selection problem that builds video index information integrity and index feature, has proposed a kind of subspace method based on video finger print, solve current towards mass data fast, the search problem of robust.First, this patent adopts novel extraction method of key frame, by the extraction replacement key-frame extraction of key feature vector, directly replace key frame with representational visual signature, be equivalent at feature space, original video be encoded, complete expression video information, both without bulk redundancy, very compact again, and overcome current key frame extracting parameter selection problem.Secondly, each visual signature is mapped to one dimension cryptographic hash, according to the cryptographic hash in-scope of visual signature, select suitable HDFS (Hadoop Distributed File System, Hadoop distributed file system) name_node (title node) and data_node (back end), accelerate retrieval rate, make it again to have the ability of mass data concurrent processing, there is range of application widely.
In this instructions, the present invention is described with reference to its specific embodiment.But, still can make various amendments and conversion obviously and not deviate from the spirit and scope of the present invention.Therefore, instructions and accompanying drawing are regarded in an illustrative, rather than a restrictive.
Claims (13)
1. a method that realizes magnanimity video quick-searching, is characterized in that, described method comprises the following steps:
(1) each frame video image in the video flowing of video library is extracted respectively to spatial signature vectors and obtain video features sequence;
(2) in the spatial signature vectors of described video features sequence, extract key feature vector;
(3) set up the distributed storage index database of all video files according to the key feature vector of all video files in video library;
(4) extract the key feature vector set of video to be retrieved and extract the video index file of this video to be retrieved;
(5) in described distributed storage index database, carry out video similarity retrieval according to the video index file of described video to be retrieved and export the video frequency searching result that similarity is greater than systemic presupposition value.
2. the method that realizes magnanimity video quick-searching according to claim 1, it is characterized in that, described spatial signature vectors comprise gray space distribution characteristics and the texture space distribution characteristics of corresponding two field picture, described extracts respectively spatial signature vectors to each frame video image in the video flowing of video library, comprises the following steps:
(11) calculate gray level image and the Edge texture image of each frame video image in the video flowing of video library;
(12) calculate each frame video image gray level image central space feature and boundary space feature and obtain the gray space distribution characteristics of this frame video image being formed by described central space feature and boundary space feature;
(13) calculate the texture space distribution characteristics of the Edge texture image of each frame video image.
3. the method that realizes magnanimity video quick-searching according to claim 2, is characterized in that, the gray level image of each frame video image and Edge texture image in the described video flowing that calculates video library, comprise the following steps:
(111) the each frame video image in the video flowing of video library be divided into several onesize subimages and calculate gray-scale value and the texture number of edge points of each number of sub images;
(112) gray-scale value that calculates each number of sub images of each frame video image obtains the gray level image of this frame video image;
(113) the texture number of edge points of calculating each number of sub images of each frame video image obtains the Edge texture image of this frame video image.
4. the method that realizes magnanimity video quick-searching according to claim 2, is characterized in that, the central space feature of the gray level image of the described each frame video image of calculating and boundary space feature, be specially:
Calculate central space feature and the boundary space feature of the local binary patterns of the gray level image of each frame video image;
The texture space distribution characteristics of the Edge texture image of the described each frame video image of calculating, is specially:
Calculate the texture space distribution characteristics of the local binary patterns of the Edge texture image of each frame video image.
5. the method that realizes magnanimity video quick-searching according to claim 2, it is characterized in that, described spatial signature vectors also comprises color histogram feature, and described extracts respectively spatial signature vectors to each frame video image in the video flowing of video library, further comprising the steps of:
(14) calculate the color histogram feature of each frame video image.
6. the method that realizes magnanimity video quick-searching according to claim 1, is characterized in that, described extracts key feature vector in the spatial signature vectors of described video features sequence, comprises the following steps:
(21) first spatial signature vectors of described video features sequence is defaulted as to key feature vector;
(22) calculate the mahalanobis distance of each spatial signature vectors and last key feature vector;
(23) corresponding the mahalanobis distance that is greater than systemic presupposition threshold value spatial signature vectors is extracted as to key feature vector.
7. the method that realizes magnanimity video quick-searching according to claim 1, is characterized in that, the described key feature vector according to all video files in video library is set up the distributed storage index database of all video files, comprises the following steps:
(31) set up the subspace projection histogram of key feature vector in described video features sequence and record each key feature vector the frequency that occurs in corresponding video;
(32) set up the inverted index file of all video files of video library;
(33) set up the distributed index database of all video files of video library.
8. the method that realizes magnanimity video quick-searching according to claim 7, is characterized in that, the described subspace projection histogram of setting up key feature vector in video features sequence, is specially:
By key feature vector projection in video features sequence in gray scale subspace, texture subspace and color sub-spaces and obtain the subspace projection histogram of each key feature vector.
9. the method that realizes magnanimity video quick-searching according to claim 8, is characterized in that, described record each key feature vector the frequency that occurs in corresponding video, be specially:
Record the eigenwert that represents this key feature vector frequency of occurrence in video in the corresponding subspace projection histogram of each key feature vector.
10. the method that realizes magnanimity video quick-searching according to claim 8, is characterized in that, the inverted index file of described all video files of setting up video library, comprises the following steps:
(321) in statistics video library, the corresponding key feature vector set of each video file merges the statistics key feature vector storehouse that forms this video library;
(322) set up the collection of document that has this key feature vector corresponding to each key feature vector in described statistics key feature vector storehouse;
(323) by the document of key feature vector set according to the quantity of contained key feature vector from how to sort to few;
(324) set up the inverted index file of all video files of video library according to each sub spaces.
11. methods that realize magnanimity video quick-searching according to claim 10, is characterized in that, the distributed index database of described all video files of setting up video library, comprises the following steps:
(331) utilize local sensitivity hash algorithm based on p-stable by the key feature DUAL PROBLEMS OF VECTOR MAPPING of each sub spaces to the one-dimensional space;
(332) adopt name_node to safeguard Hash table based on Hadoop distributed file system framework and adopt data_node to preserve the distributed index database that index data is all video files.
12. methods that realize magnanimity video quick-searching according to claim 7, is characterized in that, described carries out video similarity retrieval according to the video index file of described video to be retrieved in described distributed storage index database, is specially:
(51) calculate in video subspace projection histogram to be retrieved and video library the histogrammic friendship of each video subspace projection as the similarity of each video in video to be retrieved and video library;
(52) reject according to the space-time structure consistance of the key feature vector of each video in video to be retrieved and video library the video file that does not meet space-time structure coherence request.
13. methods that realize magnanimity video quick-searching according to claim 12, is characterized in that, described output similarity is greater than the video frequency searching result of systemic presupposition value, comprises the following steps:
(52) extract video to be retrieved key feature vector each subspace projection histogram and each key feature vector is mapped as to cryptographic hash in each sub spaces;
(53) choose similarity in distributed index database by described inverted index file and meet video file that systemic presupposition requires as output;
(54) the space-time structure consistance of calculating the key feature vector of each video in video to be retrieved and video library is also exported the video file that is greater than systemic presupposition value with the similarity of described video to be retrieved.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201410245315.2A CN104050247B (en) | 2014-06-04 | 2014-06-04 | The method for realizing massive video quick-searching |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201410245315.2A CN104050247B (en) | 2014-06-04 | 2014-06-04 | The method for realizing massive video quick-searching |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN104050247A true CN104050247A (en) | 2014-09-17 |
| CN104050247B CN104050247B (en) | 2017-08-08 |
Family
ID=51503079
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201410245315.2A Active CN104050247B (en) | 2014-06-04 | 2014-06-04 | The method for realizing massive video quick-searching |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN104050247B (en) |
Cited By (18)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104504121A (en) * | 2014-12-29 | 2015-04-08 | 北京奇艺世纪科技有限公司 | Video retrieval method and device |
| CN104504162A (en) * | 2015-01-21 | 2015-04-08 | 北京智富者机器人科技有限公司 | Video retrieval method based on robot vision platform |
| CN105095435A (en) * | 2015-07-23 | 2015-11-25 | 北京京东尚科信息技术有限公司 | Similarity comparison method and device for high-dimensional image features |
| CN105653700A (en) * | 2015-03-13 | 2016-06-08 | Tcl集团股份有限公司 | Video search method and system |
| CN106156284A (en) * | 2016-06-24 | 2016-11-23 | 合肥工业大学 | Video retrieval method is closely repeated based on random the extensive of various visual angles Hash |
| CN107748750A (en) * | 2017-08-30 | 2018-03-02 | 百度在线网络技术（北京）有限公司 | Similar video lookup method, device, equipment and storage medium |
| CN108780457A (en) * | 2016-02-09 | 2018-11-09 | 开利公司 | Multiple queries are executed in steady video search and search mechanism |
| CN109857908A (en) * | 2019-03-04 | 2019-06-07 | 北京字节跳动网络技术有限公司 | Method and apparatus for matching video |
| CN110032652A (en) * | 2019-03-07 | 2019-07-19 | 腾讯科技（深圳）有限公司 | Media file lookup method and device, storage medium and electronic device |
| CN110188098A (en) * | 2019-04-26 | 2019-08-30 | 浙江大学 | A kind of high dimension vector data visualization method and system based on the double-deck anchor point figure projection optimization |
| CN110275983A (en) * | 2019-06-05 | 2019-09-24 | 青岛海信网络科技股份有限公司 | The search method and device of traffic monitoring data |
| CN111294613A (en) * | 2020-02-20 | 2020-06-16 | 北京奇艺世纪科技有限公司 | Video processing method, client and server |
| CN111507260A (en) * | 2020-04-17 | 2020-08-07 | 重庆邮电大学 | Video similarity rapid detection method and detection device |
| CN112699348A (en) * | 2020-12-25 | 2021-04-23 | 中国平安人寿保险股份有限公司 | Method and device for verifying nuclear body information, computer equipment and storage medium |
| CN112861609A (en) * | 2020-12-30 | 2021-05-28 | 中国电子科技集团公司信息科学研究院 | Method for improving multi-thread content key frame identification efficiency |
| CN113779303A (en) * | 2021-11-12 | 2021-12-10 | 腾讯科技（深圳）有限公司 | Video set indexing method and device, storage medium and electronic equipment |
| CN113821704A (en) * | 2020-06-18 | 2021-12-21 | 华为技术有限公司 | Method and device for constructing index, electronic equipment and storage medium |
| CN115630191A (en) * | 2022-12-22 | 2023-01-20 | 成都纵横自动化技术股份有限公司 | Time-space data set retrieval method and device based on full-dynamic video and storage medium |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107943903A (en) * | 2017-11-17 | 2018-04-20 | 广州酷狗计算机科技有限公司 | Video retrieval method and device, computer equipment, storage medium |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO1996017313A1 (en) * | 1994-11-18 | 1996-06-06 | Oracle Corporation | Method and apparatus for indexing multimedia information streams |
| JP2002007479A (en) * | 2000-06-22 | 2002-01-11 | Ntt Communications Kk | Retrieving information displaying method, information retrieving system, retrieving server and recording medium of program for the server |
| CN101311947A (en) * | 2008-06-12 | 2008-11-26 | 浙江大学 | Real time intelligent control method based on natural video frequency |
| CN102436487A (en) * | 2011-11-03 | 2012-05-02 | 北京电子科技学院 | Optical flow method based on video retrieval system |
| CN102999640A (en) * | 2013-01-09 | 2013-03-27 | 公安部第三研究所 | Video and image retrieval system and method based on semantic reasoning and structural description |
-
2014
- 2014-06-04 CN CN201410245315.2A patent/CN104050247B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO1996017313A1 (en) * | 1994-11-18 | 1996-06-06 | Oracle Corporation | Method and apparatus for indexing multimedia information streams |
| JP2002007479A (en) * | 2000-06-22 | 2002-01-11 | Ntt Communications Kk | Retrieving information displaying method, information retrieving system, retrieving server and recording medium of program for the server |
| CN101311947A (en) * | 2008-06-12 | 2008-11-26 | 浙江大学 | Real time intelligent control method based on natural video frequency |
| CN102436487A (en) * | 2011-11-03 | 2012-05-02 | 北京电子科技学院 | Optical flow method based on video retrieval system |
| CN102999640A (en) * | 2013-01-09 | 2013-03-27 | 公安部第三研究所 | Video and image retrieval system and method based on semantic reasoning and structural description |
Cited By (32)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104504121A (en) * | 2014-12-29 | 2015-04-08 | 北京奇艺世纪科技有限公司 | Video retrieval method and device |
| CN104504162B (en) * | 2015-01-21 | 2018-12-04 | 北京智富者机器人科技有限公司 | A kind of video retrieval method based on robot vision platform |
| CN104504162A (en) * | 2015-01-21 | 2015-04-08 | 北京智富者机器人科技有限公司 | Video retrieval method based on robot vision platform |
| CN105653700A (en) * | 2015-03-13 | 2016-06-08 | Tcl集团股份有限公司 | Video search method and system |
| CN105653700B (en) * | 2015-03-13 | 2019-09-10 | Tcl集团股份有限公司 | Video retrieval method and system |
| RU2686590C1 (en) * | 2015-07-23 | 2019-04-29 | Бэйцзин Цзиндун Шанкэ Информейшн Текнолоджи Ко, Лтд. | Method and device for comparing similar elements of high-dimensional image features |
| WO2017012491A1 (en) * | 2015-07-23 | 2017-01-26 | 北京京东尚科信息技术有限公司 | Similarity comparison method and apparatus for high-dimensional image features |
| US11048966B2 (en) | 2015-07-23 | 2021-06-29 | Beijing Jingdong Shangke Information Technology Co., Ltd. | Method and device for comparing similarities of high dimensional features of images |
| CN105095435A (en) * | 2015-07-23 | 2015-11-25 | 北京京东尚科信息技术有限公司 | Similarity comparison method and device for high-dimensional image features |
| CN108780457A (en) * | 2016-02-09 | 2018-11-09 | 开利公司 | Multiple queries are executed in steady video search and search mechanism |
| CN106156284A (en) * | 2016-06-24 | 2016-11-23 | 合肥工业大学 | Video retrieval method is closely repeated based on random the extensive of various visual angles Hash |
| CN107748750A (en) * | 2017-08-30 | 2018-03-02 | 百度在线网络技术（北京）有限公司 | Similar video lookup method, device, equipment and storage medium |
| US10853416B2 (en) | 2017-08-30 | 2020-12-01 | Baidu Online Network Technology (Beijing) Co., Ltd. | Similar video lookup method and apparatus, device and storage medium |
| CN109857908A (en) * | 2019-03-04 | 2019-06-07 | 北京字节跳动网络技术有限公司 | Method and apparatus for matching video |
| CN109857908B (en) * | 2019-03-04 | 2021-04-09 | 北京字节跳动网络技术有限公司 | Method and apparatus for matching videos |
| CN110032652A (en) * | 2019-03-07 | 2019-07-19 | 腾讯科技（深圳）有限公司 | Media file lookup method and device, storage medium and electronic device |
| CN110032652B (en) * | 2019-03-07 | 2022-03-25 | 腾讯科技（深圳）有限公司 | Media file searching method and device, storage medium and electronic device |
| CN110188098A (en) * | 2019-04-26 | 2019-08-30 | 浙江大学 | A kind of high dimension vector data visualization method and system based on the double-deck anchor point figure projection optimization |
| CN110188098B (en) * | 2019-04-26 | 2021-02-19 | 浙江大学 | High-dimensional vector data visualization method and system based on double-layer anchor point map projection optimization |
| CN110275983A (en) * | 2019-06-05 | 2019-09-24 | 青岛海信网络科技股份有限公司 | The search method and device of traffic monitoring data |
| CN110275983B (en) * | 2019-06-05 | 2022-11-22 | 青岛海信网络科技股份有限公司 | Retrieval method and device of traffic monitoring data |
| CN111294613A (en) * | 2020-02-20 | 2020-06-16 | 北京奇艺世纪科技有限公司 | Video processing method, client and server |
| CN111507260A (en) * | 2020-04-17 | 2020-08-07 | 重庆邮电大学 | Video similarity rapid detection method and detection device |
| CN111507260B (en) * | 2020-04-17 | 2022-08-05 | 重庆邮电大学 | Video similarity rapid detection method and detection device |
| CN113821704A (en) * | 2020-06-18 | 2021-12-21 | 华为技术有限公司 | Method and device for constructing index, electronic equipment and storage medium |
| CN113821704B (en) * | 2020-06-18 | 2024-01-16 | 华为云计算技术有限公司 | Method, device, electronic equipment and storage medium for constructing index |
| CN112699348A (en) * | 2020-12-25 | 2021-04-23 | 中国平安人寿保险股份有限公司 | Method and device for verifying nuclear body information, computer equipment and storage medium |
| CN112861609A (en) * | 2020-12-30 | 2021-05-28 | 中国电子科技集团公司信息科学研究院 | Method for improving multi-thread content key frame identification efficiency |
| CN112861609B (en) * | 2020-12-30 | 2024-04-09 | 中国电子科技集团公司信息科学研究院 | Multithreading content key frame identification efficiency improvement method |
| CN113779303A (en) * | 2021-11-12 | 2021-12-10 | 腾讯科技（深圳）有限公司 | Video set indexing method and device, storage medium and electronic equipment |
| CN115630191A (en) * | 2022-12-22 | 2023-01-20 | 成都纵横自动化技术股份有限公司 | Time-space data set retrieval method and device based on full-dynamic video and storage medium |
| CN115630191B (en) * | 2022-12-22 | 2023-03-28 | 成都纵横自动化技术股份有限公司 | Time-space data set retrieval method and device based on full-dynamic video and storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN104050247B (en) | 2017-08-08 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN104050247A (en) | Method for realizing quick retrieval of mass videos | |
| CN108920720B (en) | Large-scale image retrieval method based on depth hash and GPU acceleration | |
| Picard et al. | Improving image similarity with vectors of locally aggregated tensors | |
| Jégou et al. | On the burstiness of visual elements | |
| RU2628192C2 (en) | Device for semantic classification and search in archives of digitized film materials | |
| CN110222218B (en) | Image retrieval method based on multi-scale NetVLAD and depth hash | |
| CN103336957B (en) | A kind of network homology video detecting method based on space-time characteristic | |
| CN104036012B (en) | Dictionary learning, vision bag of words feature extracting method and searching system | |
| CN105095435A (en) | Similarity comparison method and device for high-dimensional image features | |
| CN109783691B (en) | Video retrieval method for deep learning and Hash coding | |
| CN103218427B (en) | The extracting method of local description, image search method and image matching method | |
| CN104376003A (en) | Video retrieval method and device | |
| CN103186538A (en) | Image classification method, image classification device, image retrieval method and image retrieval device | |
| CN106557545A (en) | Video retrieval method and device | |
| CN110502664A (en) | Video tab indexes base establishing method, video tab generation method and device | |
| CN108335290B (en) | Image area copying and tampering detection method based on LIOP feature and block matching | |
| CN103473307A (en) | Cross-media sparse Hash indexing method | |
| CN112434553A (en) | Video identification method and system based on deep dictionary learning | |
| Jiang et al. | Learning segment similarity and alignment in large-scale content based video retrieval | |
| CN109918658A (en) | A kind of method and system obtaining target vocabulary from text | |
| CN111639230B (en) | Similar video screening method, device, equipment and storage medium | |
| CN111914950A (en) | Unsupervised cross-modal retrieval model training method based on depth dual variational hash | |
| CN105760875A (en) | Binary image feature similarity discrimination method based on random forest algorithm | |
| CN105183845A (en) | ERVQ image indexing and retrieval method in combination with semantic features | |
| CN110188625B (en) | Video fine structuring method based on multi-feature fusion |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| CB02 | Change of applicant information |
Address after: 200433, 101-10 floor, floor 1, building 127, Cathay Road, Cathay Road, Shanghai, Yangpu District Applicant after: SHANGHAI CERTUSNET INFORMATION TECHNOLOGY CO., LTD. Address before: 200433, room 1301, Fudan Science and technology building, 11 Guotai Road, Shanghai, Yangpu District Applicant before: Shanghai Meiqi Puyue Communication Technology Co., Ltd. |
|
| COR | Change of bibliographic data | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |