CN103714147A - Video resource data source processing method and system thereof - Google Patents
Video resource data source processing method and system thereof Download PDFInfo
- Publication number
- CN103714147A CN103714147A CN201310733513.9A CN201310733513A CN103714147A CN 103714147 A CN103714147 A CN 103714147A CN 201310733513 A CN201310733513 A CN 201310733513A CN 103714147 A CN103714147 A CN 103714147A
- Authority
- CN
- China
- Prior art keywords
- data
- video resource
- data source
- source
- video
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/70—Information retrieval; Database structures therefor; File system structures therefor of video data
- G06F16/71—Indexing; Data structures therefor; Storage structures
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/70—Information retrieval; Database structures therefor; File system structures therefor of video data
- G06F16/73—Querying
- G06F16/732—Query formulation
- G06F16/7335—Graphical querying, e.g. query-by-region, query-by-sketch, query-by-trajectory, GUIs for designating a person/face/object as a query predicate
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Multimedia (AREA)
- Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Databases & Information Systems (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Computational Linguistics (AREA)
- Mathematical Physics (AREA)
- Human Computer Interaction (AREA)
- Software Systems (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention discloses a video resource data source processing method and a system thereof. The method comprises the steps of obtaining data sources of multi-dimensional video source data, converting the data sources into data models established according to preset data structures, and storing the data models into materialized views. Through the method and the system, at reverse index setting time, only the uniform materialized views of the data models need facing, and a processing result can be obtained quickly at execution query time, so that the time for setting up the reverse index is greatly saved.
Description
Technical field
The present invention relates to information retrieval technique, relate in particular to a kind of disposal route and system thereof of video resource data source.
Background technology
Along with scientific and technological development, increasing user is by internet hunt and watch various videos.The video information providing due to internet is very abundant, and has the feature of continuous variation and renewal, has produced multiple search engine thereupon and has carried out Video Information Retrieval Techniques:.
In relational database system, index is the mode of retrieve data full blast.But for the video search engine of the whole network, can not meet its specific (special) requirements.What face due to search engine is the massive video data of the whole network, such as large-scale video website search engine indexs such as happy views, is all hundred million grades of even webpage quantity of several hundred billion, in the face of the video data of magnanimity like this, makes Database Systems be difficult to effectively management.
Inverted index is the very important indexed mode of search engine, the storage and retrieval that solves the video resource of magnanimity by inverted index.In practice, search engine conventionally will be in the face of the data source of different video resources, these data source types are various, source is complicated, if the data source of these various dimensions is not processed, cause the inverted index search efficiency of foundation low, can not meet the demand of search engine.
Known in sum, in prior art, the processing of the data source of video resource is not met the technical scheme of inverted index demand, be therefore necessary to propose improved technological means and address the above problem.
Summary of the invention
Fundamental purpose of the present invention is to provide a kind of disposal route and system thereof of video resource data source, does not meet the problem of inverted index demand to solve the processing to the data source of video resource that prior art exists.
In order to address the above problem, according to an aspect of the present invention, provide a kind of disposal route of video resource data source, it comprises: the data source of obtaining the video resource data of multiple dimension; Described data source is converted to the data model of setting up according to predetermined data-structure, and described data model is stored as to Materialized View.
Wherein, described data model comprises: basic data, it further comprises following information: video title, video profile, performer, director.
Wherein, described data model also comprises: growth data, it further comprises following information: platform properties, code stream information.
Wherein, the described step that described data source is converted to the data model of setting up according to predetermined data-structure, comprising: for the basic data of described data model, it adopts length-fixed structure, and described basic data is stored according to the mode of horizontal table; For the growth data of described data model, it adopts random length structure, and described growth data is stored according to the mode of list.
Wherein, the data source of obtaining the video resource data of multiple dimension described in comprises: according to the source of video resource data, divide described data source and comprise: file system, database; According to the terminal channel of video resource application, dividing described data source comprises: television terminal, mobile terminal; According to the file layout of video resource, dividing described data source comprises: extensible markup language document, text.
According to a further aspect in the invention, also provide a kind of disposal system of video resource data source, it comprises: acquisition module, for obtaining the data source of the video resource data of multiple dimension; Processing module, for described data source being converted to the data model of setting up according to predetermined data-structure, and is stored as Materialized View by described data model.
Wherein, described data model comprises: basic data, it further comprises following information: video title, video profile, performer, director.
Wherein, described data model also comprises: growth data, it further comprises following information: platform properties, code stream information.
Wherein, described processing module further comprises: the first processing module, and for the basic data for described data model, it adopts length-fixed structure, and described basic data is stored according to the mode of horizontal table; The second processing module, for the growth data for described data model, it adopts random length structure, and described growth data is stored according to the mode of list.
Wherein, the data source of obtaining the video resource data of multiple dimension described in comprises: according to the source of video resource data, divide described data source and comprise: file system, database; According to the terminal channel of video resource application, dividing described data source comprises: television terminal, mobile terminal; According to the file layout of video resource, dividing described data source comprises: extensible markup language document, text.
According to technical scheme of the present invention, by the data source of the video resource data of multiple dimension being converted to the data model of predetermined data-structure, and described data model is stored as to Materialized View, only need be in the face of the Materialized View of unified data model when setting up inverted index, when carrying out inquiry, can obtain rapidly result, thereby greatly save the time of setting up inverted index.
Accompanying drawing explanation
Accompanying drawing described herein is used to provide a further understanding of the present invention, forms the application's a part, and schematic description and description of the present invention is used for explaining the present invention, does not form inappropriate limitation of the present invention.In the accompanying drawings:
Fig. 1 is according to the process flow diagram of the disposal route of the video resource data source of the embodiment of the present invention;
Fig. 2 is according to the structured flowchart of the disposal system of the video resource data source of the embodiment of the present invention.
Embodiment
For making the object, technical solutions and advantages of the present invention clearer, below in conjunction with drawings and the specific embodiments, the present invention is described in further detail.
According to embodiments of the invention, provide a kind of disposal route of video resource data source.
Fig. 1 is that as shown in Figure 1, the method comprises according to the process flow diagram of the disposal route of the video resource data source of the embodiment of the present invention:
Step S102, obtains the data source of the video resource data of multiple dimension.
Above-mentioned data source refers to raw data, when obtaining or receiving the data source of video resource data for the first time, due to untreated, what search engine was faced is the data source with service logic, and this data source with service logic can not directly be set up the data structure of inverted index.
In actual applications, the data source of the video resource data that get is multiple dimensions, can have multiple dividing mode, for example: according to the source of video resource data, divide described data source and comprise: file system or database (DB); According to the terminal channel of video resource application, dividing described data source comprises: television terminal or mobile terminal; According to the file layout of video resource, dividing described data source comprises: extend markup language (XML) file or text (TXT).Certainly, the dimension of data source is not limited only to above-mentioned dividing mode, and the present invention does not limit for the dividing mode of other dimensions.
Step S104, is converted to described data source the data model of setting up according to predetermined data-structure, and described data model is stored as to Materialized View.
In fact Materialized View is exactly physics table, data model, based on database, is stored as Materialized View and data model is stored with the form of physics table, when being convenient in subsequent process search engine inquiry, calls.
The data source of different dimensions has feature separately, in order to shield the complicated service logic of multi-data source, the data source of various dimensions need to be converted to the data model of unified structure.The data model of predetermined data-structure comprises basic data and growth data.
Wherein, basic data is to search for the basic dimension data of being concerned about most, is to represent the requisite data of video (movie and television play).Such as comprising: information such as video title, video profile, performer's (protagonist), directors.Generally, video data is all with the applied logic attribute of off-line, and for example growth data comprises platform properties; In addition, also some video data, with self-defining functional attributes, comprises Platform Price, code stream information etc. such as growth data.Need explanation, above-mentioned is only exemplary illustration for example, is not limited to the present invention.
Data model, based on database, gets up basic data and growth data according to predetermined data structure storage.Particularly, basic data is fixed length, and basic data is according to horizontal extension, and each data is stored item by item; And growth data is random length, growth data is stored in the mode of row.This basic data adopts horizontal table mode, growth data to have higher dirigibility with the storage mode of list mode.
Then, the data model of predetermined data-structure is stored as to Materialized View, after only need be in the face of the Materialized View of unified data model while setting up inverted index, by Materialized View when carrying out inquiry, just can avoid carrying out operation consuming time, thereby obtain rapidly result, thereby greatly saved the time when setting up inverted index, for example, in the face of more than one hundred million data only need spend 1-2 minute, just complete rapidly and finish dealing with.
In actual applications, can, using the Materialized View of the data model storage of predetermined data-structure as basic view, according to this basic view, can set up the many views relevant to data structure, and set up inverted index according to a plurality of views.Thereby when carrying out inquiry, by the spreading parameter of inquiry, carry out inquiry, thereby obtain rapidly result.
According to embodiments of the invention, also provide a kind of disposal system of video resource data source.
Fig. 2 is according to the structured flowchart of the disposal system of the video resource data source of the embodiment of the present invention, and as shown in Figure 2, described system at least comprises: acquisition module 10 and processing module 20, describe structure and the annexation of each module below in detail.
Acquisition module 10, for obtaining the data source of the video resource data of multiple dimension.
Above-mentioned data source refers to raw data, when obtaining or receiving the data source of video resource data for the first time, due to untreated, what search engine was faced is the data source with service logic, and this data source with service logic can not directly be set up the data structure of inverted index.
In actual applications, the data source of the video resource data that get is multiple dimensions, can have multiple dividing mode, for example: according to the source of video resource data, divide described data source and comprise: file system or database (DB); According to the terminal channel of video resource application, dividing described data source comprises: television terminal or mobile terminal; According to the file layout of video resource, dividing described data source comprises: extend markup language (XML) file or text (TXT).Certainly, the dimension of data source is not limited only to above-mentioned dividing mode, and the present invention does not limit for the dividing mode of other dimensions.
Processing module 20 couples mutually with acquisition module 10, for described data source is converted to the data model according to predetermined data-structure, and described data model is stored as to Materialized View.
The data source of different dimensions has feature separately, in order to shield the complicated service logic of multi-data source, the data source of various dimensions need to be converted to the data model of unified structure.The data model of predetermined data-structure comprises basic data and growth data.
Wherein, basic data is to search for the basic dimension data of being concerned about most, is to represent the requisite data of video (movie and television play).Such as comprising: information such as video title, video profile, performer's (protagonist), directors.Generally, video data is all with the applied logic attribute of off-line, and for example growth data comprises platform properties; In addition, also some video data, with self-defining functional attributes, comprises Platform Price, code stream information etc. such as growth data.Need explanation, above-mentioned is only exemplary illustration for example, is not limited to the present invention.
Data model, based on database, gets up basic data and growth data according to predetermined data structure storage.Particularly, basic data is fixed length, and basic data is according to horizontal extension, and each data is stored item by item; And growth data is random length, growth data is stored in the mode of row.This basic data adopts horizontal table mode, growth data to have higher dirigibility with the storage mode of list mode.
In one embodiment of the invention, described processing module 20 further comprises: the first processing module (not shown), and for the basic data for described data model, it adopts length-fixed structure, and described basic data is stored according to the mode of horizontal table; The second processing module (not shown), for the growth data for described data model, it adopts random length structure, and described growth data is stored according to the mode of list.
Then, the data model of predetermined data-structure is stored as to Materialized View, after only need be in the face of the Materialized View of unified data model while setting up inverted index, by Materialized View when carrying out inquiry, just can avoid carrying out operation consuming time, thereby obtain rapidly result, thereby greatly saved the time when setting up inverted index, for example, in the face of more than one hundred million data only need spend 1-2 minute, just complete rapidly and finish dealing with.
In actual applications, can, using the Materialized View of the data model storage of predetermined data-structure as basic view, according to this basic view, can set up the many views relevant to data structure, and set up inverted index according to a plurality of views.Thereby when carrying out inquiry, by the spreading parameter of inquiry, carry out inquiry, thereby obtain rapidly result.
The operation steps of method of the present invention is corresponding with the architectural feature of system, can cross-reference, repeat no longer one by one.
In sum, according to technical scheme of the present invention, by the data source of the video resource data of multiple dimension being converted to the data model of predetermined data-structure, and described data model is stored as to Materialized View, only need be in the face of the Materialized View of unified data model when setting up inverted index, when carrying out inquiry, can obtain rapidly result, thereby greatly save the time of setting up inverted index.
The foregoing is only embodiments of the invention, be not limited to the present invention, for a person skilled in the art, the present invention can have various modifications and variations.Within the spirit and principles in the present invention all, any modification of doing, be equal to replacement, improvement etc., within all should being included in claim scope of the present invention.
Claims (10)
1. a disposal route for video resource data source, is characterized in that, comprising:
Obtain the data source of the video resource data of multiple dimension;
Described data source is converted to the data model of setting up according to predetermined data-structure, and described data model is stored as to Materialized View.
2. method according to claim 1, is characterized in that, described data model comprises: basic data, it further comprises following information: video title, video profile, performer, director.
3. method according to claim 2, is characterized in that, described data model also comprises: growth data, it further comprises following information: platform properties, code stream information.
4. method according to claim 3, is characterized in that, the described step that described data source is converted to the data model of setting up according to predetermined data-structure, comprising:
For the basic data of described data model, it adopts length-fixed structure, and described basic data is stored according to the mode of horizontal table;
For the growth data of described data model, it adopts random length structure, and described growth data is stored according to the mode of list.
5. method according to claim 1, is characterized in that, described in obtain the video resource data of multiple dimension data source comprise:
According to the source of video resource data, dividing described data source comprises: file system, database;
According to the terminal channel of video resource application, dividing described data source comprises: television terminal, mobile terminal;
According to the file layout of video resource, dividing described data source comprises: extensible markup language document, text.
6. a disposal system for video resource data source, is characterized in that, comprising:
Acquisition module, for obtaining the data source of the video resource data of multiple dimension;
Processing module, for described data source being converted to the data model of setting up according to predetermined data-structure, and is stored as Materialized View by described data model.
7. system according to claim 6, is characterized in that, described data model comprises: basic data, it further comprises following information: video title, video profile, performer, director.
8. system according to claim 7, is characterized in that, described data model also comprises: growth data, it further comprises following information: platform properties, code stream information.
9. system according to claim 8, is characterized in that, described processing module further comprises:
The first processing module, for the basic data for described data model, it adopts length-fixed structure, and described basic data is stored according to the mode of horizontal table;
The second processing module, for the growth data for described data model, it adopts random length structure, and described growth data is stored according to the mode of list.
10. system according to claim 5, is characterized in that, described in obtain the video resource data of multiple dimension data source comprise:
According to the source of video resource data, dividing described data source comprises: file system, database;
According to the terminal channel of video resource application, dividing described data source comprises: television terminal, mobile terminal;
According to the file layout of video resource, dividing described data source comprises: extensible markup language document, text.
Priority Applications (3)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201310733513.9A CN103714147A (en) | 2013-12-26 | 2013-12-26 | Video resource data source processing method and system thereof |
| PCT/CN2014/093176 WO2015096609A1 (en) | 2013-12-26 | 2014-12-05 | Method and system for creating inverted index file of video resource |
| US15/101,698 US20160306811A1 (en) | 2013-12-26 | 2014-12-05 | Method and system for creating inverted index file of video resource |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201310733513.9A CN103714147A (en) | 2013-12-26 | 2013-12-26 | Video resource data source processing method and system thereof |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN103714147A true CN103714147A (en) | 2014-04-09 |
Family
ID=50407122
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201310733513.9A Pending CN103714147A (en) | 2013-12-26 | 2013-12-26 | Video resource data source processing method and system thereof |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN103714147A (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2015096609A1 (en) * | 2013-12-26 | 2015-07-02 | 乐视网信息技术（北京）股份有限公司 | Method and system for creating inverted index file of video resource |
| CN106470138A (en) * | 2016-08-30 | 2017-03-01 | 成都科来软件有限公司 | A kind of method that corresponding time interval data is screened according to user's request |
-
2013
- 2013-12-26 CN CN201310733513.9A patent/CN103714147A/en active Pending
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2015096609A1 (en) * | 2013-12-26 | 2015-07-02 | 乐视网信息技术（北京）股份有限公司 | Method and system for creating inverted index file of video resource |
| CN106470138A (en) * | 2016-08-30 | 2017-03-01 | 成都科来软件有限公司 | A kind of method that corresponding time interval data is screened according to user's request |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109447277B (en) | Universal machine learning super-ginseng black box optimization method and system | |
| US11451863B1 (en) | Content versioning system | |
| US9298774B2 (en) | Changing the compression level of query plans | |
| CN107729399B (en) | Data processing method and device | |
| CN106294695A (en) | A kind of implementation method towards the biggest data search engine | |
| CN103678694A (en) | Method and system for establishing reverse index file of video resources | |
| WO2015096609A1 (en) | Method and system for creating inverted index file of video resource | |
| CN101963999A (en) | Music classified search engine system and music classified search method | |
| CN104573065A (en) | Report display engine based on metadata | |
| CN103020322A (en) | Query method | |
| CN103714158A (en) | Vertical search method and system for video websites | |
| EP2583195A1 (en) | Method and server for handling database queries | |
| CN102662986A (en) | System and method for microblog message retrieval | |
| CN102375827A (en) | Method for fast loading versioned electricity network model database | |
| CN103678715A (en) | Snapshot supporting metadata information management method for distributed file system | |
| CN105868170A (en) | Method for generating industrial data report in server | |
| CN104965903A (en) | Resource recommendation method and apparatus | |
| CN103714147A (en) | Video resource data source processing method and system thereof | |
| CN104679823A (en) | Semantic annotation-based association method and system of heterogeneous data | |
| CN102006156B (en) | Method and system for synchronizing configuration data among boards | |
| CN101909047A (en) | Method and device for acquiring multimedia programs | |
| KR101955376B1 (en) | Processing method for a relational query in distributed stream processing engine based on shared-nothing architecture, recording medium and device for performing the method | |
| CN112115206A (en) | Method and device for processing object storage metadata | |
| CN102467502A (en) | Retrieval method and system | |
| CN103699659A (en) | Method and system for managing word library of video resources |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| C41 | Transfer of patent application or patent right or utility model | ||
| TA01 | Transfer of patent application right |
Effective date of registration: 20151225 Address after: Room six, building 19, building 68, No. 100089 South Road, Haidian District, Beijing Applicant after: LETV CLOUD COMPUTING CO., LTD. Address before: Room six, building 19, building 68, No. 100089 South Road, Haidian District, Beijing Applicant before: LeTV Information Technology (Beijing) Co., Ltd. |
|
| AD01 | Patent right deemed abandoned |
Effective date of abandoning: 20180907 |
|
| AD01 | Patent right deemed abandoned |