CN103279474A - Video file index method and system - Google Patents
Video file index method and system Download PDFInfo
- Publication number
- CN103279474A CN103279474A CN2013101228824A CN201310122882A CN103279474A CN 103279474 A CN103279474 A CN 103279474A CN 2013101228824 A CN2013101228824 A CN 2013101228824A CN 201310122882 A CN201310122882 A CN 201310122882A CN 103279474 A CN103279474 A CN 103279474A
- Authority
- CN
- China
- Prior art keywords
- file
- hdfs
- fusion
- read
- piece
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Landscapes
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention discloses a video file index method and system. The method includes the steps of file merging and file extraction. According to the step of file extraction, data nodes of an HDFS extract corresponding file data from corresponding HDFS blocks according to offset and length information saved in an index table of a generated merged file. More specifically, the method includes the steps of merging small files into the large file, establishing an index for the merged file, storing the merged file onto the HDFS, acquiring a corresponding storage position at namenodes with the method of file mapping in the process of reading the merged file from the HDFS, and improving read efficiency by direct communication between an HDFS service client-side and the data nodes and by file prefetching at the HDFS service client-side. The video file index method and system can manage the small files in the HDFS efficiently, save internal storage resources of the namenodes of the HDFS, improve the efficiency in reading the small files, and save storage resources of the data nodes of the HDFS.
Description
Technical field
The present invention relates to the video file process field, in particular a kind of the indexing means of video file efficiently and system for Video service platform and portable terminal.
Background technology
HDFS (Hadoop Distributed File System distributed file system) is distributed system architecture of Hadoop() in a core element.Hadoop is a distributed system architecture that is provided support by the Apache foundation.HDFS is the file system of a design in order to store big file (each blocks of files is usually more than 64Mb), and has the data access patterns of streaming, and it may operate on the comparatively common hardware usually.The HDFS system architecture as shown in Figure 1, the client/server that the HDFS cluster is common mainly is made up of a title node (NameNode) and several data nodes (DataNode), the data node is with array format, array 1 as shown in Figure 1 and array 2; The HDFS system architecture also provides a secondary title node (Secondary NameNode) to be used for providing to the backup of title node and when the title node failure service usually; namenode is a topmost node in the HDFS framework, and it is managing metadata of all files.
The management of the small documents of HDFS at present exists very big deficiency; The title node can be that each HDFS piece and each file preserve a metadata, and be customer in response request more quickly and efficiently, when moving, the title node with all metadata store in internal memory.When a large amount of small documents (less than a HDFS piece, be defaulted as 64M) when being stored among the HDFS, and HDFS is with the piece managing memory source, each file takies a piece like this, illustrate: a metadata is approximately 150B, and the HDFS block size is 64M, then the file that is 1G for a size, storing its metadata needs about 2.55K, and storing its content needs about 1G; And for the file (1G altogether) that 1024 sizes are 1M, storing its metadata needs about 307.2K, and storing its content needs about 64G.It is big to take up space like this, and space availability ratio is low.
Therefore, prior art has yet to be improved and developed.
Summary of the invention
The technical problem to be solved in the present invention is, above-mentioned defective at prior art, a kind of indexing means of video file efficiently and system for Video service platform and portable terminal is provided, make up a kind of high-level efficiency and handle the index mechanism of small documents among the HDFS, overcome the problem of the wasting of resources that present HDFS storage small documents causes, can effectively improve the access efficiency to small documents.
The technical scheme that technical solution problem of the present invention adopts is as follows:
A kind of video file indexing means wherein, comprises step:
The step that A, file merge: a plurality of first files less than the first pre-sizing are fused into a fusion file that is no more than the second pre-sizing at the HDFS service client; And generate the beginning that there is the fusion file in a concordance list simultaneously, be used for side-play amount and the length of each composing document of record;
B, file mapping steps: merge filename according to the filename that will read file and its, obtain the positional information of file, comprise data node information and the HDFS block message at place;
The step that C, file are looked ahead: after obtaining to read the positional information of file, be a part that merges file if this will read file, the positional information that then this will be read all storage files in the HDFS piece of file place together reads the HDFS service client;
The step that D, file extract: side-play amount and the length information preserved in the concordance list according to the fusion file that generates, HDFS data node extracts the corresponding file data from corresponding HDFS piece.
Described video file indexing means, wherein, the size that merges file in the described steps A is no more than the size of HDFS piece.
Described video file indexing means, wherein, the pre-sizing of second in the described steps A is 64M.
Described video file indexing means, wherein, described steps A also comprises: after a plurality of first files being fused into the fusion file of a size that is no more than the HDFS piece, re-send among the HDFS.
Described video file indexing means, wherein, described step B also comprises: when HDFS merges file in storage, the positional information of file is comprised that the data node information at place and HDFS block message are saved in a concordance list and are placed on the title node, when file shines upon, safeguard that at the title node concordance list record merges the composing document name of file and the corresponding relation of HDFS memory location.
Described video file indexing means, wherein, described step B also comprises: use Hash table structure concordance list in the file mapping step.
Described video file indexing means, wherein, described step C also comprises: in the file pre-fetch step, when reading the composing document of a certain fusion file, all composing document information of this fusion file are together read the HDFS client.
A kind of video file directory system wherein, comprising:
The file integrated unit is used at the HDFS service client a plurality of first files less than the first pre-sizing being fused into a fusion file that is no more than the second pre-sizing; And generate the beginning that there is the fusion file in a concordance list simultaneously, be used for side-play amount and the length of each composing document of record;
The file map unit is used for merging filename according to the filename that will read file and its, obtains the positional information of file, is used for comprising data node information and the HDFS block message at place;
The file pre-fetch unit is used for after obtaining to read the positional information of file, is a part that merges file if this will read file, and the positional information that then this will be read all storage files in the HDFS piece of file place together reads the HDFS service client;
The file extracting unit, for side-play amount and the length information preserved according to the concordance list of the fusion file that generates, HDFS data node extracts the corresponding file data from corresponding HDFS piece.
Described video file directory system, wherein, the size of described fusion file is no more than the size of HDFS piece.
Described video file directory system, wherein, the described second pre-sizing is 64M.
The indexing means of video file efficiently and system for Video service platform and portable terminal provided by the present invention, the index mechanism of small documents among a kind of efficient management HDFS is provided, by a plurality of small documents being fused into a big file, then the file that merges being set up index and stored HDFS into; When reading file from HDFS, obtain corresponding memory location at the title node by file mapping, then by HDFS service client and the direct communication of data node and at the HDFS service client by the file raising reading efficiency of looking ahead.The present invention is the index mechanism of small documents among the efficient management HDFS, saves HDFS title node memory source, improves the small documents reading efficiency, has saved the storage resources of HDFS data nodes simultaneously.
Description of drawings
Fig. 1 is HDFS system architecture structural representation.
Fig. 2 is the process flow diagram of the preferred embodiment of video file indexing means of the present invention.
Fig. 3 is the processing procedure structured flowchart of the preferred embodiment of video file indexing means of the present invention.
The structural representation of the fusion file of the preferred embodiment of Fig. 4 video file indexing means of the present invention.
Fig. 5 reads document flowchart for the HDFS of embodiment of the invention video file directory system.
Fig. 6 is the video file directory system functional schematic block diagram of the embodiment of the invention.
Embodiment
For making purpose of the present invention, technical scheme and advantage clearer, clear and definite, below develop simultaneously with reference to accompanying drawing that the present invention is described in more detail for embodiment.Should be appreciated that specific embodiment described herein only in order to explaining the present invention, and be not used in restriction the present invention.
See also Fig. 2, Fig. 2 is the process flow diagram of the preferred embodiment of video file indexing means of the present invention.
The video file indexing means that the embodiment of the invention provides provides the index mechanism of small documents among a kind of efficient management HDFS, adopts a plurality of small documents are fused into a big file, then the file that merges is set up index and is stored HDFS into; When reading file from HDFS, obtain corresponding memory location at the title node by file mapping, then by HDFS service client and the direct communication of data node and at the HDFS service client by the file raising reading efficiency of looking ahead.
As shown in Figure 2, described method comprises the steps:
The step that S100, file merge: the fusion file that a plurality of first files less than the first pre-sizing is fused into one second pre-sizing at the HDFS service client: and generate the beginning that there is the fusion file in a concordance list simultaneously, be used for side-play amount and the length of each composing document of record.
In order to realize a plurality of small documents are fused into a big file in this step.Carry out at the HDFS service client, by the file that merges appointment a plurality of small documents less than the first pre-sizing (i.e. first file) are fused into a big file (i.e. the fusion file of the second pre-sizing), the size that merges file is no more than the size of HDFS piece, be generally 64M, meanwhile generate a concordance list and exist the beginning of merging file to record side-play amount and the length of each composing document.Corresponding processing procedure Organization Chart as shown in Figure 3 100, namely in the step that file merges, merge top of file and preserve one and be used for record composing document length and it is in the side-play amount that merges file.File structure after the fusion as shown in Figure 4
Described step S100 also comprises: after a plurality of first files being fused into the fusion file of a size that is no more than the HDFS piece, re-send among the HDFS.Be present embodiment in the step that file merges, send among the HDFS after a plurality of small documents (first file) being fused into the big file of a size that is no more than the HDFS piece.
S200, file mapping steps: merge filename according to the filename that will read file and its, the position that obtains file comprises data node information and the HDFS block message at place.
When HDFS merges file in storage, the positional information of file is comprised that the data node information at place and HDFS block message are saved in a concordance list and are placed on the title node, in the file mapping steps, safeguard that at the title node concordance list record merges the composing document name of file and the corresponding relation of HDFS memory location.
Corresponding processing procedure Organization Chart as shown in Figure 3 200, the file mapping, namely, filename and fusion filename thereof that the title node provides according to the request of receiving, in title node preservation metadata, search, the file location information that finds is comprised that the data node information at place and HDFS block message feed back to the HDFS client.
Preferably among the embodiment, in the file mapping step, use Hash table to construct concordance list.
The step that S300, file are looked ahead: after obtaining to read the positional information of file, be a part that merges file if this will read file, the positional information that then this will be read all storage files in the HDFS piece of file place together reads the HDFS service client;
Corresponding processing procedure Organization Chart as shown in Figure 3 300, file is looked ahead, namely, after obtaining to read the positional information of file, begin from the HDFS piece of storage file, to read file, if this document is a part that merges file, then the head of this HDFS piece together reads the HDFS client with side-play amount and the file size of the All Files stored in the concordance list form record block with this table.
In the file pre-fetch step, when reading the composing document of a certain fusion file, all composing document information of this fusion file are together read the HDFS client.
The step that S400, file extract: side-play amount and the length information preserved in the concordance list according to the fusion file that generates, HDFS data node extracts the corresponding file data from corresponding HDFS piece.
Corresponding processing procedure Organization Chart as shown in Figure 3 400: file extracts, namely, concordance list according to the head preservation of merging top of file that is HDFS piece, side-play amount and the file size of locating file correspondence in the HDFS piece, the side-play amount from the HDFS piece reads file length data in the concordance list then.
Embodiment further, step S100 in the file fusion steps, consider the relevance between the file, can adopt the file that correlation degree is high to merge, can improve the treatment effeciency of file.And control is lower than the size of described fusion file but as far as possible near the size of piece among the HDFS.
As seen, the embodiment of the invention provides a kind of indexing means of video file efficiently for Video service platform and portable terminal, make up a kind of high-level efficiency and handle the index mechanism of small documents among the HDFS, overcome the problem of the wasting of resources that present HDFS storage small documents causes, can effectively improve the access efficiency to small documents.
Based on above-described embodiment, the embodiment of the invention also provides a kind of video file directory system, and as shown in Figure 6, it comprises:
File integrated unit 510 is used at the HDFS service client a plurality of first files less than the first pre-sizing being fused into a fusion file that is no more than the second pre-sizing; And generate the beginning that there is the fusion file in a concordance list simultaneously, be used for side-play amount and the length of each composing document of record; Specifically as described in the above-mentioned step S100.
File pre-fetch unit 530 is used for after obtaining to read the positional information of file, is a part that merges file if this will read file, and the positional information that then this will be read all storage files in the HDFS piece of file place together reads the HDFS service client; Specifically as described in the above-mentioned step S300.
Further, after the pre-fetch unit of file described in the present embodiment 530 gets access to alternative document information, it is saved in this locality, request next time is these fileinfos of coupling earlier, as coupling, can directly read from the data node, as not match and send request to the title node again.
In the video file directory system of the embodiment of the invention, make up a kind of high-level efficiency and handled the index mechanism of small documents among the HDFS, employing comprises a plurality of small documents is fused into a big file integrated unit 510, filename is mapped to the file map unit 520 of HDFS memory location at the title node, obtain and read file in the file pre-fetch unit 530 of the alternative document information of same HDFS piece and the file extracting unit 540 that is used for reading file, described file integrated unit is in the HDFS client executing, a plurality of small documents is fused into a big file and the information of its composing document of record in merging file; Described file map unit 520, the filename that provides according to request and merge filename, the positional information of obtaining file at the title node comprises data node information and the HDFS block message at place, feeds back to the HDFS client then; Described file pre-fetch unit 530 when reading file after the positional information that obtains file, is preserved alternative document information with file place HDFS piece and is together read the HDFS client; Described file extracting unit 540 according to obtaining file location information, reads the HDFS client with file.
The specific implementation process of the system of the embodiment of the invention is as follows: file merges in the HDFS client executing, by the file that merges appointment a plurality of small documents are fused into a big file, but be no more than the size of HDFS piece, be generally 64M, meanwhile generate a concordance list and exist the beginning of merging file to record side-play amount and the length of each composing document, be submitted to HDFS then; Set up an index of metadata table that merges file at the title node of HDFS and record the positional information that merges file and composing document thereof.
Fig. 5 has provided the process flow diagram that adopts described index mechanism to read file from HDFS, when CLIENT PROGRAM opens file, title node to HDFS sends request, the filename that will open file is provided and merges filename, the title node is searched index of metadata table acquisition file with filename and is returned to CLIENT PROGRAM in the positional information of HDFS and with it, obtain the positional information of file storage, directly find corresponding HDFS piece with the communication of data node then, may be read into the side-play amount of file and length and together read client's node by the file alternative document information that this HDFS piece is preserved of looking ahead at the head of piece, set up the HDFS data input stream and from the data node, read file, read finish after the CLIENT PROGRAM close file.So far, adopt the HDFS of index mechanism of the present invention to read the document flow end.
In the preferred implementation of the present invention, client's node numerous small documents that the degree of association is high in the file fusion steps are fused into one near the fusion file of HDFS block size, in the file pre-fetch step will with read the alternative document information that file coexists in the HDFS piece and together read client's node and buffer memory.By index mechanism of the present invention, reducing the title node is the metadata that management document produces, reduce the HDFS piece that is used for storage file in the data node and takies simultaneously, in addition in CLIENT PROGRAM when continuing the operation alternative document, can from the file pre-fetch step, data in buffer directly find the memory location, need not search by the title node again, improve access efficiency.
In sum, in the embodiment of the invention, by a plurality of small documents being fused into a big file, and in merging file the information of its composing document of record, comprise side-play amount, file sizes etc. store HDFS then into; When reading file from HDFS, obtain corresponding memory location at the title node by file mapping, then by HDFS client and the direct communication of data node and in the HDFS client by the file raising reading efficiency of looking ahead.The present invention is the index mechanism of small documents among the efficient management HDFS, saves HDFS title node memory source, improves the small documents reading efficiency, has saved the storage resources of HDFS data nodes simultaneously.
Should be understood that application of the present invention is not limited to above-mentioned giving an example, for those of ordinary skills, can be improved according to the above description or conversion that all these improvement and conversion all should belong to the protection domain of claims of the present invention.
Claims (10)
1. a video file indexing means is characterized in that, comprises step:
The step that A, file merge: a plurality of first files less than the first pre-sizing are fused into a fusion file that is no more than the second pre-sizing at the HDFS service client; And generate the beginning that there is the fusion file in a concordance list simultaneously, be used for side-play amount and the length of each composing document of record;
B, file mapping steps: merge filename according to the filename that will read file and its, obtain the positional information of file, comprise data node information and the HDFS block message at place;
The step that C, file are looked ahead: after obtaining to read the positional information of file, be a part that merges file if this will read file, the positional information that then this will be read all storage files in the HDFS piece of file place together reads the HDFS service client;
The step that D, file extract: side-play amount and the length information preserved in the concordance list according to the fusion file that generates, HDFS data node extracts the corresponding file data from corresponding HDFS piece.
2. video file indexing means according to claim 1 is characterized in that, the size that merges file in the described steps A is no more than the size of HDFS piece.
3. video file indexing means according to claim 1 is characterized in that, the pre-sizing of second in the described steps A is 64M.
4. video file indexing means according to claim 1 is characterized in that, described steps A also comprises: after a plurality of first files being fused into the fusion file of a size that is no more than the HDFS piece, re-send among the HDFS.
5. video file indexing means according to claim 1, it is characterized in that, described step B also comprises: when HDFS merges file in storage, the positional information of file is comprised that the data node information at place and HDFS block message are saved in a concordance list and are placed on the title node, when file shines upon, safeguard that at the title node concordance list record merges the composing document name of file and the corresponding relation of HDFS memory location.
6. video file indexing means according to claim 5 is characterized in that, described step B also comprises: use Hash table structure concordance list in the file mapping step.
7. video file indexing means according to claim 1, it is characterized in that, described step C also comprises: in the file pre-fetch step, when reading the composing document of a certain fusion file, all composing document information of this fusion file are together read the HDFS client.
8. a video file directory system is characterized in that, comprising:
The file integrated unit is used at the HDFS service client a plurality of first files less than the first pre-sizing being fused into a fusion file that is no more than the second pre-sizing; And generate the beginning that there is the fusion file in a concordance list simultaneously, be used for side-play amount and the length of each composing document of record;
The file map unit is used for merging filename according to the filename that will read file and its, obtains the positional information of file, is used for comprising data node information and the HDFS block message at place;
The file pre-fetch unit is used for after obtaining to read the positional information of file, is a part that merges file if this will read file, and the positional information that then this will be read all storage files in the HDFS piece of file place together reads the HDFS service client;
The file extracting unit, for side-play amount and the length information preserved according to the concordance list of the fusion file that generates, HDFS data node extracts the corresponding file data from corresponding HDFS piece.
9. video file directory system according to claim 8 is characterized in that, the size of described fusion file is no more than the size of HDFS piece.
10. video file directory system according to claim 8 is characterized in that, the described second pre-sizing is 64M.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN2013101228824A CN103279474A (en) | 2013-04-10 | 2013-04-10 | Video file index method and system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN2013101228824A CN103279474A (en) | 2013-04-10 | 2013-04-10 | Video file index method and system |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN103279474A true CN103279474A (en) | 2013-09-04 |
Family
ID=49061994
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN2013101228824A Pending CN103279474A (en) | 2013-04-10 | 2013-04-10 | Video file index method and system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN103279474A (en) |
Cited By (13)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103577604A (en) * | 2013-11-20 | 2014-02-12 | 电子科技大学 | Image indexing structure for Hadoop distributed type environment |
| CN104572670A (en) * | 2013-10-15 | 2015-04-29 | 方正国际软件(北京)有限公司 | Small file storage, query and deletion method and system |
| CN104902022A (en) * | 2015-05-27 | 2015-09-09 | 北京集奥聚合科技有限公司 | Distributed file acquisition method and distributed file acquisition system |
| CN106294374A (en) * | 2015-05-15 | 2017-01-04 | 北京国双科技有限公司 | The method of small documents merging and data query system |
| CN106293518A (en) * | 2016-07-29 | 2017-01-04 | 成都神州数码索贝科技有限公司 | A kind of efficient security protection industry media data storage file system |
| CN106980578A (en) * | 2017-04-01 | 2017-07-25 | 广东浪潮大数据研究有限公司 | A kind of memory block management method and system |
| CN108108260A (en) * | 2016-11-25 | 2018-06-01 | 沈阳美行科技有限公司 | The method of calibration and device of a kind of resource file |
| CN109388613A (en) * | 2018-09-30 | 2019-02-26 | 联想(北京)有限公司 | A kind of file management method and electronic equipment |
| US10298709B1 (en) * | 2014-12-31 | 2019-05-21 | EMC IP Holding Company LLC | Performance of Hadoop distributed file system operations in a non-native operating system |
| CN110069451A (en) * | 2019-03-28 | 2019-07-30 | 浪潮卓数大数据产业发展有限公司 | A kind of method and device of HDFS storage small documents |
| CN110196841A (en) * | 2018-06-21 | 2019-09-03 | 腾讯科技（深圳）有限公司 | The storage method and device of file, querying method and device and server |
| CN112231293A (en) * | 2020-09-14 | 2021-01-15 | 杭州数梦工场科技有限公司 | File reading method and device, electronic equipment and storage medium |
| CN113242447A (en) * | 2021-05-10 | 2021-08-10 | 北京奇艺世纪科技有限公司 | Video data processing method and device |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102332027A (en) * | 2011-10-15 | 2012-01-25 | 西安交通大学 | Mass non-independent small file associated storage method based on Hadoop |
| CN102508880A (en) * | 2011-10-18 | 2012-06-20 | 广东威创视讯科技股份有限公司 | Method for joining files and method for splitting files |
-
2013
- 2013-04-10 CN CN2013101228824A patent/CN103279474A/en active Pending
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102332027A (en) * | 2011-10-15 | 2012-01-25 | 西安交通大学 | Mass non-independent small file associated storage method based on Hadoop |
| CN102508880A (en) * | 2011-10-18 | 2012-06-20 | 广东威创视讯科技股份有限公司 | Method for joining files and method for splitting files |
Non-Patent Citations (2)
| Title |
|---|
| XUHUI LIU等: "Implementing WebGIS on Hadoop: A Case Study of Improving Small File I/O Performance on HDFS", 《THE 2009 IEEE CONFERENCE ON CLUSTER COMPUTING AND WORKSHOPS》 * |
| 张春明等: "一种Hadoop小文件存储和读取的方法", 《计算机应用与软件》 * |
Cited By (20)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104572670A (en) * | 2013-10-15 | 2015-04-29 | 方正国际软件(北京)有限公司 | Small file storage, query and deletion method and system |
| CN104572670B (en) * | 2013-10-15 | 2019-07-23 | 方正国际软件(北京)有限公司 | A kind of storage of small documents, inquiry and delet method and system |
| CN103577604A (en) * | 2013-11-20 | 2014-02-12 | 电子科技大学 | Image indexing structure for Hadoop distributed type environment |
| US10298709B1 (en) * | 2014-12-31 | 2019-05-21 | EMC IP Holding Company LLC | Performance of Hadoop distributed file system operations in a non-native operating system |
| CN106294374A (en) * | 2015-05-15 | 2017-01-04 | 北京国双科技有限公司 | The method of small documents merging and data query system |
| CN106294374B (en) * | 2015-05-15 | 2019-06-04 | 北京国双科技有限公司 | The method and data query system that small documents merge |
| CN104902022A (en) * | 2015-05-27 | 2015-09-09 | 北京集奥聚合科技有限公司 | Distributed file acquisition method and distributed file acquisition system |
| CN104902022B (en) * | 2015-05-27 | 2019-02-26 | 北京集奥聚合科技有限公司 | Distributed file acquisition method and distributed file acquisition system |
| CN106293518B (en) * | 2016-07-29 | 2019-09-10 | 成都神州数码索贝科技有限公司 | A kind of efficient security industry media data storage file system |
| CN106293518A (en) * | 2016-07-29 | 2017-01-04 | 成都神州数码索贝科技有限公司 | A kind of efficient security protection industry media data storage file system |
| CN108108260A (en) * | 2016-11-25 | 2018-06-01 | 沈阳美行科技有限公司 | The method of calibration and device of a kind of resource file |
| CN108108260B (en) * | 2016-11-25 | 2022-05-31 | 沈阳美行科技股份有限公司 | Resource file verification method and device |
| CN106980578A (en) * | 2017-04-01 | 2017-07-25 | 广东浪潮大数据研究有限公司 | A kind of memory block management method and system |
| CN110196841A (en) * | 2018-06-21 | 2019-09-03 | 腾讯科技（深圳）有限公司 | The storage method and device of file, querying method and device and server |
| CN110196841B (en) * | 2018-06-21 | 2023-12-05 | 腾讯科技（深圳）有限公司 | File storage method and device, query method and device and server |
| CN109388613A (en) * | 2018-09-30 | 2019-02-26 | 联想(北京)有限公司 | A kind of file management method and electronic equipment |
| CN110069451A (en) * | 2019-03-28 | 2019-07-30 | 浪潮卓数大数据产业发展有限公司 | A kind of method and device of HDFS storage small documents |
| CN112231293A (en) * | 2020-09-14 | 2021-01-15 | 杭州数梦工场科技有限公司 | File reading method and device, electronic equipment and storage medium |
| CN113242447A (en) * | 2021-05-10 | 2021-08-10 | 北京奇艺世纪科技有限公司 | Video data processing method and device |
| CN113242447B (en) * | 2021-05-10 | 2022-05-17 | 北京奇艺世纪科技有限公司 | Video data processing method and device |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN103279474A (en) | Video file index method and system | |
| CN104133882A (en) | HDFS (Hadoop Distributed File System)-based old file processing method | |
| US10747951B2 (en) | Webpage template generating method and server | |
| CN104252536B (en) | A kind of internet log data query method and device based on hbase | |
| CN105183839A (en) | Hadoop-based storage optimizing method for small file hierachical indexing | |
| CN105718455A (en) | Data query method and apparatus | |
| KR101672349B1 (en) | File cloud service apparatus and method | |
| CN104778270A (en) | Storage method for multiple files | |
| CN104679898A (en) | Big data access method | |
| CN104753966A (en) | Resource file inquiry method and system based on server and client caches | |
| CN103595797B (en) | Caching method for distributed storage system | |
| CN104820714A (en) | Mass small tile file storage management method based on hadoop | |
| CN105303456A (en) | Method for processing monitoring data of electric power transmission equipment | |
| RU2009147457A (en) | FACILITATED BY SERVICE NODE AND SINGLE-RANGE SYNCHRONIZATION | |
| CN102479250A (en) | Disk caching method for embedded browser | |
| CN106484821A (en) | Hybrid cloud storage method under cloud computing architecture | |
| CN103279502B (en) | A kind of framework and method with the data de-duplication file system be combined with parallel file system | |
| CN104331428A (en) | Storage and access method of small files and large files | |
| CN109684282A (en) | A kind of method and device constructing metadata cache | |
| CN110399348A (en) | File deletes method, apparatus, system and computer readable storage medium again | |
| CN109284273B (en) | Massive small file query method and system adopting suffix array index | |
| KR20160067289A (en) | Cache Management System for Enhancing the Accessibility of Small Files in Distributed File System | |
| CN104348859B (en) | File synchronisation method, device, server, terminal and system | |
| CN103078936A (en) | Metadata hierarchical storage method and system for Global file system (GFS)-based distributed file system | |
| CN106446099A (en) | Distributed cloud storage method and system and uploading and downloading method thereof |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20130904 |