CN101799827A - Video database management method based on layering structure - Google Patents

Video database management method based on layering structure Download PDF

Info

Publication number
CN101799827A
CN101799827A CN 201010122083 CN201010122083A CN101799827A CN 101799827 A CN101799827 A CN 101799827A CN 201010122083 CN201010122083 CN 201010122083 CN 201010122083 A CN201010122083 A CN 201010122083A CN 101799827 A CN101799827 A CN 101799827A
Authority
CN
China
Prior art keywords
video
index
management method
method based
hierarchical structure
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN 201010122083
Other languages
Chinese (zh)
Inventor
王慧昱
童若峰
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Zhejiang University ZJU
Original Assignee
Zhejiang University ZJU
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Zhejiang University ZJU filed Critical Zhejiang University ZJU
Priority to CN 201010122083 priority Critical patent/CN101799827A/en
Publication of CN101799827A publication Critical patent/CN101799827A/en
Pending legal-status Critical Current

Links

Images

Abstract

The invention discloses a video database management method based on a layering structure, comprising the following specific steps of: (1) establishing a layering data model; (2) structuralizing videos; (3) indexing and retrieving the videos; and (4) designing various tables in a video database on the basis of the layering data model, video structuralization and video indexing and retrieval to realize the establishment of indexes. The method can be used for the layering management of video data and supporting the layering structure indexes and the annotated index of the videos. The invention can effectively support the retrieval of the video data and support the functions of high-efficient query, edition, labeling and the like of the video data under the conditions of large quantity and abundant source of the video data.

Description

Video material library management method based on hierarchical structure
Technical field:
The present invention relates to a kind of video material library management method based on hierarchical structure, the managing video material that this method can stratification is supported the hierarchical structure index and the note index of video.Big in video material quantity, under the situation that enrich in the source, can effectively support the video material retrieval, functions such as the efficient inquiry of support video material, editor, mark.
Background technology:
Along with the magnanimity growth of digital of digital video data amount, video material is playing a role in increasing the application.Video material is realistic strong, and the characteristics that the quarry is abundant have become the valuable source that virtual scene makes up.The quantity of video material constantly increases, and has increased us and has therefrom found the difficulty of the material of real needs.Compare with word content, video data has destructuring, magnanimity and contains semantic characteristics such as abundant, how efficiently to they organize, express, storage, management and retrieval, be challenge to conventional information retrieval and database technology proposition.For effective managing video material resource, such as associated scenario of extracting in the video and object, we need an effective video material management method.Video material library management method based on hierarchical structure is mainly analyzed from stratification data model, these two aspects of video structural index.
Data model is the core and the basis of Database Systems, and the performance of Database Systems is directly related with data model.The continuous development and the perfecting process of database model, the history of Database Systems development just.Traditional text database system because its data type and concern fairly simple, so its data model is also comparatively simple.In the video database field, also there is not the perfect data model that can generally use at present.When setting up the video model, though may represent that the general theory of video data and method all also do not form by otherwise achievement in research.The researcher inquires into the video data model from different directions separately at present, and the video data model of being set up generally also can only reflect or emphasize a certain class feature of video data, and it uses a certain field that also lays particular emphasis on.Because be subjected to the restriction of discipline development levels such as present image understanding, computer vision, artificial intelligence, also there are many difficulties in video data segmentation automatically and the senior semantic feature that extracts video, should be target with the model of setting up finite automaton and being applied to some specific area at present therefore.
As the most general and the most effective a kind of communication media, comprised other type of media in the video, abundant in content, thereby how effectively use and manage challenge is also arranged most.Video data itself is not supported user interactions, and the video use is a unidirectional experience of the process to a great extent, and the viewer is a passive role.Though abundant untapped content and the knowledge of the inner existence of video, the verified very difficult system management of this not structurized data.The effective video managerial demand extracts those undeveloped content and knowledge that contain in the video with certain automated manner as library system, provide the instrument of a similar card index catalogue to finish the video filing and seek required content whereby.Be that video must be followed a structured content index,, discharge rich knowledge resource in the video library, video conversion is become the manageable useful information that content is carried out fine granularity access and control based on index by creating the index that enriches of video content.
Summary of the invention:
The present invention relates to a kind of video material library management method based on hierarchical structure, this method is by definition scene entity, plot entity, particularly object entity, effectively represents the hierarchical structure of the video after the structuring.The stratification data model makes and merges work such as virtual scene structure in order to carry out video by user's positioning video object material apace for the index and the retrieval of video database efficiently provides a basis.In addition, data model is also supported the hierarchical semantic note of video, so that the user carries out semantic browsing or search at different video structure layers.
Video material library management method based on hierarchical structure comprises following concrete steps:
1) sets up the stratification data model; The stratification data model can the reflecting video data hierarchical structure, and contact and principal character between the reflecting video data structure are the bases of supporting video material structuring and video index and retrieval efficiently;
2) video structural; Video structural is exactly that the successive frame sequence in the video flowing is carried out cutting, and the difference by its content is launched is divided into some semantic paragraph unit to a continuous videos stream, mainly comprises following three steps; 2.1) video structural at first will be by the method based on the camera lens rim detection, video data stream is cut into a lot of plots; 2.2) feature of its less important extraction plot of video structural, representative frame and object; 2.3) video structural will be clustered into different scenes according to the difference of plot semanteme at last;
3) video index and retrieval comprise configuration index and note index based on video structure; Wherein, the configuration index of video structure refers to that we can set up an index structure for structurized video, comes effective organization and management video resource, user friendly retrieval and visit; The note index refers to can carry out note from three aspects at least based on a video sequence: the object video capture place, the video in event and the video, and we carry out the video annotation index to these three entities respectively;
4) based on the index and the retrieval of stratification data model, video structural and video, the various tables in the design video database are realized the foundation of index.
Compare with traditional video material management method, following characteristics arranged based on the video material library management method of hierarchical structure:
1. method is based on the video data relational model of stratification.This model has effectively been represented structurized video material and the corresponding note structure of the video structure with stratification.
2. on stratification video data model based, set up tree-shaped index structure, this configuration index is supported visit mutually between the object of top-down visit, bottom-up visit and same level.In addition, set up the note index, can on different levels, strengthen the fast access of semantic interpretation.
3. in our stratification video material management method, object video manages as important one deck.Effectively support the insertion and the editor of object video.
Description of drawings:
The invention will be further described below in conjunction with accompanying drawing and example.
Fig. 1 is the data model in the method used herein.
Fig. 2 is the process flow diagram of video structural in this method.
Fig. 3 is based on the video index of video structural and the structural drawing of retrieval
Fig. 4 is based on the exemplary plot of configuration index tree.
Fig. 5 is based on the database design figure of stratification video material library management method.
Embodiment:
With reference to accompanying drawing, the video material library management method based on hierarchical structure of the present invention, carry out according to the following steps:
1. set up stratification video data relational model shown in Figure 1.Three big basic video data objects have been introduced in this model: video documents (Video Document), the video data on corresponding the server; Frame sequence (Frame Sequence) is the interval that is made of series of frames in the video flowing, and the size of frame sequence can be from an independent frame to all videos document; Note (Annotation) can be the text description that each frame sequence distributes related content, to support content-based retrieval.Frame sequence can be refined as scene (Scene), plot (Story) and this three class of material object (Object), and present hierarchical relationship between the subclass.Video documents extracts the note that some describes its content from video material, note can be divided into three classes: the note of relevant object, the note of relevant incident and the note in relevant place.In addition, note can be subdivided into by the vocabulary of describing these semantic contents to be formed, and is respectively: object relative words (Object-Term), incident relative words (Event-Term) and place relative words (Location-Term).
2. shown in Fig. 2 process flow diagram, video is carried out structuring.Video structural is exactly that the successive frame sequence in the video flowing is carried out cutting, and the difference by its content is launched is divided into some semantic paragraph unit to a continuous videos stream.Can see that in process flow diagram frame sequence is subdivided into three kinds of entities: scene, plot and object.
3. video structural at first will carry out the camera lens rim detection.Polytype transient mode is arranged between the camera lens, and modal is " sudden change ", shows as the camera lens conversion of the mutability of adjacent two interframe generation.In addition, also there are some complicated transient modes, as fade in, fade out etc.The technology of video slicing has a variety of, and we utilize the otherness between the framing between adjacent camera lens, adopts the fixedly window " slip " on continuous video of frame number, thus the cut-point of differentiation camera lens.We adopt the in addition improved cutting method that can differentiate the shot transition of gradual change.Just formed plot by the video after the camera lens cutting.
4. object adopts Video SnapCut algorithm, adopts multinuclear to quicken and Cuda acceleration object rapid extraction.Adopt the key frame of first frame of plot as plot.
5. the process of video structure analysis is exactly the plot combination that semanteme is relevant, the process of cluster.Here, our plot that will have the identical semanteme procedure definition that is clustered into the scene of same kind is " scene classification ".
6. the index of video and retrieval mainly contain five kinds of methods: comprise method based on metadata, text based method, based on the method for audio frequency, content-based method and comprehensive method.Main comprehensively comprehensive based on the method for metadata, text based method and content-based method three of this method just belongs to comprehensive method.Our combined index and search method are based on video structural, and we mainly study based on the configuration index of video structure and note index, as shown in Figure 3.
7. configuration index is often expressed as tree construction, and the intermediate node in the index tree is the abstract of their child nodes.An index tree both can bottom-uply be constructed by abstract, also can top-downly construct by classifying.In the system, the index structure that we adopt combines top-down visit, bottom-up visit and visits these three kinds of methods together between the layer, as shown in Figure 4.Three types pointer is arranged in the index tree: the pointer, N layer that the N layer points to the N-1 layer points to from the 0th layer of pointer to the N-1 layer, the 3rd layer node and points to pointer its brotgher of node.
8. video sequence of note can carry out from three aspects at least: the object video capture place, the video in event and the video.These three aspects just in time can be mapped with our data model, i.e. scene in the data model, plot and object.Therefore, video annotation can be distinguished corresponding these three entities, and the note on these three kinds of levels is correlated with, and semantic relation is arranged, and can finally be combined into the note of video, more intactly describes video.
9. based on the stratification video data relational model of this chapter proposition and index, retrieval and the structured techniques of video, our database design as shown in Figure 5.Among the figure, Table VI DEO (videometer), table SCENE (scene table), table STORY (plot table) and table OBJECT (Object table) are the frame sequence objects in the data model.For fear of strong excessively coupling, we store these objects with four tables respectively.Similarly, table LOCATIONTERMS (place note vocabulary), table EVENTTERMS (Event Annotation vocabulary) and table OBJECTTERMS (object note vocabulary) corresponding the note vocabulary in the data model, store with three tables respectively.Same scene type also has a table (SCENEGENRE table) storage separately, and purpose is when the scene type is upgraded, and need not each row in the SCENE table be upgraded.Wherein, can see in the table of scene that a list item item GID is arranged, the major key GID of scene type table SCENEGENRE as the external key correspondence of expression scene type.In addition, the index relative between each is shown as can be seen by the line between table and the table satisfies the structure of the tree that we propose previously.For example, get in touch by OBJECTINCLUDED and object that all are included in this plot, represented index structure from top to bottom at STORY table; At OBJECT table by VIDEOREFID, STORYREFID, and SCENEREFID respectively with Table VI DEO, STORY, and the SCENE table is related, has represented a kind of index structure from the bottom up; Undertaken relatedly by OBJECTRELATEDID and object in the Same Scene, represented with the index structure between the layer.
What should be understood that is: the foregoing description is just to explanation of the present invention, rather than limitation of the present invention, and any innovation and creation that do not exceed in the connotation scope of the present invention all fall within protection scope of the present invention.

Claims (9)

1. based on the video material library management method of hierarchical structure, it is characterized in that comprising following concrete steps:
1) sets up the stratification data model;
2) video structural: the successive frame sequence in the video flowing is carried out cutting, and the difference by its content is launched is divided into some semantic paragraph unit to a continuous videos stream, mainly comprises following three steps:
2.1) by the method based on the camera lens rim detection, video data stream is cut into a lot of plots;
2.2) extract the feature of plot, representative frame and object;
2.3) according to the difference of plot semanteme, be clustered into different scenes;
3) video index and retrieval comprise configuration index and note index based on video structure; Wherein, the configuration index of video structure refers to set up an index structure for structurized video, comes effective organization and management video resource, user friendly retrieval and visit; The note index refers to can carry out note from three aspects at least based on a video sequence: the object video capture place, the video in event and the video, carry out the video annotation index to these three entities respectively.
4) based on the index and the retrieval of stratification data model, video structural and video, the various tables in the design video database are realized the foundation of index.
2. the video material library management method based on hierarchical structure as claimed in claim 1 is characterized in that in step 1) the basic video data object comprises: 1.1) video documents (Video Document), the video data on corresponding the server; 1.2) frame sequence (Frame Sequence), be the interval that constitutes by series of frames in the video flowing, the size of frame sequence from an independent frame to all videos document; 1.3) note (Annotation), for the text description of each frame sequence distribution related content, to support content-based retrieval.
3. the video material library management method based on hierarchical structure as claimed in claim 1, it is characterized in that in step 2.1) in, utilize the otherness between the framing between adjacent camera lens, the employing fixedly window of frame number is slided on continuous video, thereby differentiates the cut-point of camera lens.
4. the video material library management method based on hierarchical structure as claimed in claim 1, it is characterized in that in step 2.2) in, object adopts Video SnapCut algorithm, adopt multinuclear to quicken and Cuda acceleration object rapid extraction, and adopt the key frame of first frame of plot as plot.
5. the video material library management method based on hierarchical structure as claimed in claim 1 is characterized in that in step 2.3) in, the plot that will have identical semanteme is clustered into same kind, and the plot that semanteme is relevant makes up, is clustered into different scenes.
6. the video material library management method based on hierarchical structure as claimed in claim 1, it is characterized in that in step 3) the method for video index and retrieval comprises method based on metadata, text based method, based on the method for audio frequency, content-based method and comprehensive method.
7. the video material library management method based on hierarchical structure as claimed in claim 1 is characterized in that in step 3), configuration index is expressed as tree construction, and the intermediate node in the index tree is the abstract of their child nodes; Index tree is bottom-up constructs by abstract, perhaps top-downly constructs by classifying; Three types pointer is arranged in the index tree: the pointer, N layer that the N layer points to the N-1 layer points to from the 0th layer of pointer to the N-1 layer, the 3rd layer node and points to pointer its brotgher of node.
8. the video material library management method based on hierarchical structure as claimed in claim 1, it is characterized in that in step 3), video sequence of note carries out from three aspects at least: the object video capture place, the video in event and the video, these three aspects are corresponding with scene, plot and object difference in the data model.
9. the video material library management method based on hierarchical structure as claimed in claim 1, it is characterized in that in step 4), adopt Table VI DEO, table SCENE, table STORY and table OBJECT store the frame sequence object in the data model, adopt table LOCATIONTERMS, table EVENTTERMS and table OBJECTTERMS store the note vocabulary in the data model, adopt a table SCENEGENRE that scene type is stored separately, wherein, among the table SCENEGENRE list item item GID is arranged, the major key GID of scene type table SCENEGENRE as the external key correspondence of expression scene type; Represent each index relative between showing by table and the line between the table.
CN 201010122083 2010-03-11 2010-03-11 Video database management method based on layering structure Pending CN101799827A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN 201010122083 CN101799827A (en) 2010-03-11 2010-03-11 Video database management method based on layering structure

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN 201010122083 CN101799827A (en) 2010-03-11 2010-03-11 Video database management method based on layering structure

Publications (1)

Publication Number Publication Date
CN101799827A true CN101799827A (en) 2010-08-11

Family

ID=42595503

Family Applications (1)

Application Number Title Priority Date Filing Date
CN 201010122083 Pending CN101799827A (en) 2010-03-11 2010-03-11 Video database management method based on layering structure

Country Status (1)

Country Link
CN (1) CN101799827A (en)

Cited By (13)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102497542A (en) * 2011-12-27 2012-06-13 前卫视讯(北京)科技发展有限公司 Video positioning playback system and method thereof
CN102799684A (en) * 2012-07-27 2012-11-28 成都索贝数码科技股份有限公司 Video-audio file catalogue labeling, metadata storage indexing and searching method
CN103164491A (en) * 2011-12-19 2013-06-19 北大方正集团有限公司 Method and device for processing and retrieving data
CN104156484A (en) * 2014-08-27 2014-11-19 清新视界(北京)科技有限公司 Multisource relevant data playing method and system
CN104699666A (en) * 2015-01-30 2015-06-10 浙江大学 Method for learning hierarchical structure from book catalogue based on affinity propagation model
CN104781831A (en) * 2012-09-13 2015-07-15 通用电气公司 System and method for generating semantic annotations
CN105898204A (en) * 2014-12-25 2016-08-24 支录奎 Intelligent video recorder enabling video structuralization
CN106326382A (en) * 2016-08-16 2017-01-11 广东中星电子有限公司 Video data storage and search method and device based on layered code stream
CN106970958A (en) * 2017-03-17 2017-07-21 浙江大华技术股份有限公司 A kind of inquiry of stream file and storage method and device
CN107454437A (en) * 2016-06-01 2017-12-08 深圳市维杰乐思科技有限公司 A kind of video labeling method and its device, server
CN110543584A (en) * 2018-05-29 2019-12-06 腾讯科技(深圳)有限公司 method, device, processing server and storage medium for establishing face index
CN114189752A (en) * 2021-11-25 2022-03-15 杭州视在数科信息技术有限公司 Video storage scheme based on object storage service
CN117633297A (en) * 2024-01-26 2024-03-01 江苏瑞宁信创科技有限公司 Video retrieval method, device, system and medium based on annotation

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6941325B1 (en) * 1999-02-01 2005-09-06 The Trustees Of Columbia University Multimedia archive description scheme
US7143434B1 (en) * 1998-11-06 2006-11-28 Seungyup Paek Video description system and method
CN101650958A (en) * 2009-07-23 2010-02-17 中国科学院声学研究所 Extraction method and index establishment method of movie video scene clip

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US7143434B1 (en) * 1998-11-06 2006-11-28 Seungyup Paek Video description system and method
US6941325B1 (en) * 1999-02-01 2005-09-06 The Trustees Of Columbia University Multimedia archive description scheme
CN101650958A (en) * 2009-07-23 2010-02-17 中国科学院声学研究所 Extraction method and index establishment method of movie video scene clip

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
《2009 Second International Conference on Machine Vision》 20091230 Huiyu Wang,Ruofeng Tong Hierarchical Video Data Modeling and Indexing for Virtual Scene Construction 69-73 1-9 , 2 *

Cited By (20)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103164491B (en) * 2011-12-19 2016-03-30 北大方正集团有限公司 The method and apparatus of a kind of data processing and retrieval
CN103164491A (en) * 2011-12-19 2013-06-19 北大方正集团有限公司 Method and device for processing and retrieving data
CN102497542A (en) * 2011-12-27 2012-06-13 前卫视讯(北京)科技发展有限公司 Video positioning playback system and method thereof
CN102799684A (en) * 2012-07-27 2012-11-28 成都索贝数码科技股份有限公司 Video-audio file catalogue labeling, metadata storage indexing and searching method
CN102799684B (en) * 2012-07-27 2015-09-09 成都索贝数码科技股份有限公司 The index of a kind of video and audio file cataloguing, metadata store index and searching method
CN104781831B (en) * 2012-09-13 2019-04-30 通用电气公司 System and method for generative semantics mark
CN104781831A (en) * 2012-09-13 2015-07-15 通用电气公司 System and method for generating semantic annotations
CN104156484A (en) * 2014-08-27 2014-11-19 清新视界(北京)科技有限公司 Multisource relevant data playing method and system
CN105898204A (en) * 2014-12-25 2016-08-24 支录奎 Intelligent video recorder enabling video structuralization
CN104699666A (en) * 2015-01-30 2015-06-10 浙江大学 Method for learning hierarchical structure from book catalogue based on affinity propagation model
CN104699666B (en) * 2015-01-30 2017-09-01 浙江大学 Based on neighbour's propagation model from the method for library catalogue learning hierarchical structure
CN107454437A (en) * 2016-06-01 2017-12-08 深圳市维杰乐思科技有限公司 A kind of video labeling method and its device, server
CN107454437B (en) * 2016-06-01 2020-04-14 深圳市妙思创想教育科技有限公司 Video annotation method and device and server
CN106326382A (en) * 2016-08-16 2017-01-11 广东中星电子有限公司 Video data storage and search method and device based on layered code stream
CN106970958A (en) * 2017-03-17 2017-07-21 浙江大华技术股份有限公司 A kind of inquiry of stream file and storage method and device
CN106970958B (en) * 2017-03-17 2019-11-19 浙江大华技术股份有限公司 A kind of inquiry of stream file and storage method and device
CN110543584A (en) * 2018-05-29 2019-12-06 腾讯科技(深圳)有限公司 method, device, processing server and storage medium for establishing face index
CN110543584B (en) * 2018-05-29 2023-05-05 腾讯科技(深圳)有限公司 Method, device, processing server and storage medium for establishing face index
CN114189752A (en) * 2021-11-25 2022-03-15 杭州视在数科信息技术有限公司 Video storage scheme based on object storage service
CN117633297A (en) * 2024-01-26 2024-03-01 江苏瑞宁信创科技有限公司 Video retrieval method, device, system and medium based on annotation

Similar Documents

Publication Publication Date Title
CN101799827A (en) Video database management method based on layering structure
CN102799684B (en) The index of a kind of video and audio file cataloguing, metadata store index and searching method
CN104239501B (en) Mass video semantic annotation method based on Spark
CN101187938B (en) An uniform description and search method for multimedia element data
Eck et al. A semantic file system for integrated product data management
CN109344298A (en) A kind of method and device converting unstructured data to structural data
CN103310025A (en) Unstructured-data description method and device
CN112131449A (en) Implementation method of cultural resource cascade query interface based on elastic search
US10437872B2 (en) Computer implemented and computer controlled method, computer program product and platform for arranging data for processing and storage at a data storage engine
Obeso et al. Image annotation for Mexican buildings database
Li et al. Building a large annotation ontology for movie video retrieval
Lee Taking context seriously: a framework for contextual information in digital collections
Zhang Up-to-date summary of semantic-based visual information retrieval
Nguyen Bridging semantic gaps in information retrieval: Context-based approaches
Sun et al. Design of a Media Resource Management System for Colleges Based on Cloud Service
Li et al. Multimedia ontology modeling: An approach based on mpeg-7
Kim Web-based research collaboration service: Crowd lifelog research case study
Luo et al. Image retrieval in the unstructured data management system AUDR
Sebastine et al. Semantic web for content based video retrieval
KR100322300B1 (en) Method of retrieving image data by flow attribute tree and partial result matrix
Ashok Kumar et al. An efficient scene content-based indexing and retrieval on video lectures
Xu et al. A hierarchical video annotation system
Lang et al. The next-generation search engine: Challenges and key technologies
Zheng et al. An MPEG-7 compatible video retrieval system with support for semantic queries
Marins et al. Modeling provenance for semantic desktop applications

Legal Events

Date Code Title Description
C06 Publication
PB01 Publication
C10 Entry into substantive examination
SE01 Entry into force of request for substantive examination
C02 Deemed withdrawal of patent application after publication (patent law 2001)
WD01 Invention patent application deemed withdrawn after publication

Open date: 20100811