CN111949619B - Dynamic catalog generation method, system, electronic equipment and storage medium - Google Patents
Dynamic catalog generation method, system, electronic equipment and storage medium Download PDFInfo
- Publication number
- CN111949619B CN111949619B CN202010703826.XA CN202010703826A CN111949619B CN 111949619 B CN111949619 B CN 111949619B CN 202010703826 A CN202010703826 A CN 202010703826A CN 111949619 B CN111949619 B CN 111949619B
- Authority
- CN
- China
- Prior art keywords
- file
- storage
- directory
- dynamic
- generating
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 41
- 238000012216 screening Methods 0.000 claims description 22
- 238000001914 filtration Methods 0.000 claims description 7
- 239000002699 waste material Substances 0.000 abstract description 12
- 238000012549 training Methods 0.000 description 16
- 241000282472 Canis lupus familiaris Species 0.000 description 10
- 238000005516 engineering process Methods 0.000 description 6
- 238000010801 machine learning Methods 0.000 description 5
- 235000013162 Cocos nucifera Nutrition 0.000 description 4
- 244000060011 Cocos nucifera Species 0.000 description 4
- 238000010586 diagram Methods 0.000 description 3
- 241000282326 Felis catus Species 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 2
- 238000004891 communication Methods 0.000 description 2
- 238000011161 development Methods 0.000 description 2
- BQCADISMDOOEFD-UHFFFAOYSA-N Silver Chemical compound [Ag] BQCADISMDOOEFD-UHFFFAOYSA-N 0.000 description 1
- 238000004590 computer program Methods 0.000 description 1
- 238000010276 construction Methods 0.000 description 1
- 230000006870 function Effects 0.000 description 1
- 238000010295 mobile communication Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000012545 processing Methods 0.000 description 1
- 229910052709 silver Inorganic materials 0.000 description 1
- 239000004332 silver Substances 0.000 description 1
- 239000007787 solid Substances 0.000 description 1
- 230000007723 transport mechanism Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/10—File systems; File servers
- G06F16/17—Details of further file system functions
- G06F16/172—Caching, prefetching or hoarding of files
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/10—File systems; File servers
- G06F16/13—File access structures, e.g. distributed indices
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/10—File systems; File servers
- G06F16/17—Details of further file system functions
- G06F16/1737—Details of further file system functions for reducing power consumption or coping with limited storage space, e.g. in mobile devices
Abstract
The embodiment of the application discloses a method, a system, electronic equipment and a storage medium for generating a dynamic catalog, wherein the method for generating the dynamic catalog is used for storing an object file, the object file adopts an object storage form, an input file identification is obtained, the object file is screened according to the file identification to obtain a screened object file, index information is generated according to the object file and the file identification, and the dynamic catalog is generated according to the index information, so that the waste of storage space is effectively avoided, and the problem that the storage space is greatly occupied after multiple copies is solved.
Description
Technical Field
The embodiment of the application relates to the technical field of Internet, in particular to a method, a system, electronic equipment and a storage medium for generating a dynamic catalog.
Background
With the development of internet technology, people pay more and more attention to machine learning technology, in exploring a machine learning model, a large amount of training data, such as file data of audio, video, pictures, etc., is often required to be collected, and each file in the training data needs to be labeled. A document typically has multiple tags, such as a beach photograph containing a crowd of people and coconuts, possibly together with different tags such as "crowd", "beach", "coconuts", etc. When constructing different machine-learned training sets, files are copied into different training file directories according to the labels to form a training set.
However, the current method for constructing the training set according to the file label usually adopts manual copying or automatic copying through a script tool, and the method for constructing the training set occupies a large amount of storage space after multiple copying, so that the storage space is wasted.
Disclosure of Invention
The embodiment of the application aims to at least solve one of the technical problems existing in the prior art. Therefore, the embodiment of the application provides a method for generating the dynamic catalogue, which can effectively avoid the waste of storage space.
The embodiment of the application also provides a dynamic catalog generation system with the dynamic catalog generation method.
The embodiment of the application also provides the electronic equipment with the dynamic catalog generation method.
The embodiment of the application also provides a computer readable storage medium with the dynamic catalog generation method.
According to an embodiment of the first aspect of the present application, a method for generating an active directory includes: storing an object file, wherein the object file adopts an object storage form; acquiring a file identifier and screening the object file according to the file identifier to acquire a screened object file; generating index information according to the obtained object file and the file identification; and generating a dynamic catalog according to the index information.
The method for generating the dynamic catalogue has at least the following beneficial effects: the object files are stored, wherein the object files adopt an object storage form, the input file identifications are obtained, the object files are filtered according to the file identifications, the filtered object files are obtained, index information is generated according to the object files and the file identifications, and finally, the dynamic catalogue is generated according to the index information, so that the waste of storage space is effectively avoided, and the problem that the storage space is greatly occupied after multiple copies is solved.
According to some embodiments of the application, a category identification is determined for the object file.
According to some embodiments of the application, the file identification comprises: tag information for identifying a category of the object file; a directory name, the directory name identifying a name of the dynamic directory;
Correspondingly, the obtaining the file identifier and screening the object file according to the file identifier includes: and acquiring the tag information and the directory name, and screening the object file according to the tag information and the directory name.
According to some embodiments of the application, the index information includes: a file address identifying a storage location of the object file; a file name for identifying a name of the object file; index names, wherein the index names are generated according to the file identifications;
Correspondingly, the generating index information according to the obtained object file and the file identifier includes: and generating the file address, the file name and the index name according to the obtained object file and the file identification.
According to some embodiments of the application, the generating an active directory according to the index information includes: accessing the object file according to the index information; converting the object file from an object storage form to a file storage form and generating a file directory index; and generating a dynamic directory according to the file directory index.
An embodiment of the dynamic catalog generation system according to the second aspect of the present application comprises: the storage module is used for storing object files, and the object files adopt an object storage form; the screening module is used for acquiring the file identification and screening the object file according to the file identification so as to acquire a screened object file; the index module is used for generating index information according to the obtained object file and the file identification; and the generation module is used for generating the dynamic catalogue according to the index information.
The dynamic catalog generating system according to the present application has at least the following advantages: the object files are stored through the storage module, wherein the object files are in an object storage form, then the screening module is used for obtaining file identifications, the object files are screened according to the file identifications, so that screened object files are obtained, the obtained object files and the file identifications are used for generating index information through the index module, and finally the generation module is used for generating the dynamic catalogue according to the index information, so that the waste of storage space is effectively avoided, and the problem that the storage space is greatly occupied after multiple copies is solved.
According to some embodiments of the application, the filtering module is connected to the storage module through an object storage gateway to filter the object file; the object storage gateway is used for accessing the object files in the storage module.
According to some embodiments of the application, the generating module is further configured to: accessing an object file in the storage module through an object storage gateway; the object file is converted from an object storage form to a file storage form through an object storage-to-file storage protocol gateway, and a file directory index is generated; and generating the dynamic directory through the object storage-to-file storage protocol gateway by the file directory index.
An electronic device according to an embodiment of a third aspect of the present application includes: at least one processor, and a memory communicatively coupled to the at least one processor; wherein the memory stores instructions for execution by the at least one processor to cause the at least one processor to perform the method of generating an dynamic catalog as described in the first aspect when the instructions are executed.
The electronic equipment provided by the application has at least the following beneficial effects: by executing the method for generating the dynamic catalogue mentioned in the embodiment of the first aspect, the waste of the storage space is effectively avoided, and the problem that the storage space is greatly occupied after multiple copies is solved.
A computer-readable storage medium according to an embodiment of the fourth aspect of the present application stores computer-executable instructions for causing a computer to execute the dynamic catalog generation method according to the first aspect
The dynamic catalog generating system according to the present application has at least the following advantages: by executing the method for generating the dynamic catalogue mentioned in the embodiment of the first aspect, the waste of the storage space is effectively avoided, and the problem that the storage space is greatly occupied after multiple copies is solved.
Additional aspects and advantages of the application will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the application.
Drawings
FIG. 1 is a flow chart of a method for generating an active directory according to an embodiment of the present application;
fig. 2 is a specific flowchart of step S400 in the method for generating an active directory according to an embodiment of the present application;
FIG. 3 is a block diagram of a dynamic catalog generation system of an embodiment of the present application;
FIG. 4 is an exemplary block diagram of an application of the dynamic catalog generation system of an embodiment of the present application;
fig. 5 is a block diagram of an electronic device according to an embodiment of the present application.
Reference numerals:
A memory module 100; a screening module 200; an index module 300; a generation module 400; a processor 500; memory 600.
Detailed Description
Embodiments of the present application are described in detail below, examples of which are illustrated in the accompanying drawings, wherein like or similar reference numerals refer to like or similar elements or elements having like or similar functions throughout. The embodiments described below by referring to the drawings are exemplary only for explaining the embodiments of the present application and are not to be construed as limiting the embodiments of the present application.
In the description of the embodiments of the present application, several means one or more, and plural means two or more, and greater than, less than, exceeding, etc. are understood to exclude the present number, and above, below, within, etc. are understood to include the present number. The description of the first and second is for the purpose of distinguishing between technical features only and should not be construed as indicating or implying relative importance or implicitly indicating the number of technical features indicated or implicitly indicating the precedence of the technical features indicated.
With the development of internet technology, people pay more and more attention to machine learning technology, in exploring a machine learning model, a large amount of training data, such as file data of audio, video, pictures, etc., is often required to be collected, and each file in the training data needs to be labeled. A document typically has multiple tags, such as a beach photograph containing a crowd of people and coconuts, possibly together with different tags such as "crowd", "beach", "coconuts", etc. When constructing different machine-learned training sets, files are copied into different training file directories according to the labels to form a training set.
However, the current method for constructing the training set according to the file label usually adopts manual copying or automatic copying is completed through a script tool, and the method for constructing the training set is manually performed or performed through a program script, so that a large amount of storage space is occupied after files are copied for many times, and the waste of the storage space is caused.
Based on the above, the embodiment of the application provides a method, a system, an electronic device and a storage medium for generating a dynamic catalog, which can effectively avoid the waste of storage space and solve the problem that the storage space is greatly occupied after multiple copies
In a first aspect, an embodiment of the present application provides a method for generating a dynamic directory, by storing an object file, where the object file uses an object storage form, obtains an input file identifier, screens the object file according to the file identifier to obtain a screened object file, generates index information according to the object file and the file identifier, and finally generates the dynamic directory according to the index information, thereby effectively avoiding waste of storage space, and solving a problem that the storage space is greatly occupied after multiple copies.
Referring to fig. 1, a flow chart of a method for generating an active directory in an embodiment of the present application is shown. The method specifically comprises the following steps:
s100, storing an object file, wherein the object file adopts an object storage form;
s200, acquiring a file identifier and screening an object file according to the file identifier to acquire a screened object file;
s300, index information is generated according to the obtained object file and the file identification;
S400, generating a dynamic catalog according to the index information.
In step S100, the collected file data is stored as an object file in an object storage form, which is also called an object-based storage form, in which data and metadata are encapsulated into an object, the object file being stored in a flat structure or an address space. Each object file is assigned an object ID or unique identifier so that they may be retrieved from a single repository or pool. Metadata is data describing other data or information data about files stored within an object in an object-based storage form. In practical application, general file storage has the concept of directory tree, namely, the corresponding file can be found by searching the file directory, but by storing the object file in the form of object storage, the file is searched only by the unique address identification of the file data, so that the file data can be searched more quickly and conveniently.
The file data can be acquired according to actual demands, and specific acquisition modes comprise a software interface mode, a software robot acquisition mode, a web crawler mode, an open database mode and other acquisition modes. Data is collected through different collection modes for the construction of subsequent training sets.
In some embodiments, the collected file data is stored in the form of object storage, and corresponding category identifiers are determined for the stored object file according to the characteristics of object storage, for example, the stored object file determines corresponding category identifiers such as 'cat', 'dog', 'lawn', 'football' for a photo including cat, dog, lawn and football, and the content to be expressed by the object file is determined through determining the category identifiers, so that a training set of machine learning is also facilitated to be constructed.
In step S200, an input file identifier is obtained, a stored object file is screened according to the obtained file identifier, and an object file conforming to the file identifier is obtained according to the screening result, wherein the file identifier is used for determining information required for constructing the dynamic catalog, such as a category identifier of the object file required for constructing the dynamic catalog and a name of the dynamic catalog when constructing the dynamic catalog.
In some embodiments, the file identification includes tag information and a directory name, wherein the tag information is a category identification for identifying object files in the constructed dynamic directory; the directory name is then the name used to identify the constructed dynamic directory. Correspondingly, the object file is screened according to the obtained tag information and directory name, so as to obtain the screened object file. In practical application, the tag information of the input dynamic catalogue is dog and the name of the output dynamic catalogue is dog according to the practical requirement, the object files representing the objects to be screened are object files with the category identified as dog, and the object files are constructed into a dynamic catalogue, and the name of the dynamic catalogue is dog.
In step S300, index information is generated according to the object files obtained by filtering and the input file identifiers, wherein the index information is a data structure, metadata in the object files are saved, and a plurality of corresponding object files, such as locations where the object files are stored, names of the object files, and directory names constructed by the object files in the index information, are identified in the index information, can be obtained and accessed through the index information.
In some embodiments, the index information includes a file address, a file name, and an index name, where the file information is used to identify a storage location of the object file to which the index information points, and the file name refers to a name of the object file to which the index information points, and the index name is generated according to a file identifier, for example, according to a directory name in the file identifier. Correspondingly, a file address, a file name and an index name can be generated according to the obtained object file and the file identification, so as to be used for generating the following dynamic catalogue.
In practical application, the information generated in the index information is:
“DFDN:dogs
URL＝HTTP://objectService/Object1 FileName＝photo1.jpg
URL＝HTTP://objectService/Object3 FileName＝photo3.jpg
URL＝HTTP://objectService/Object6 FileName＝photo6.jpg”
here, "URL" refers to a file address of index information, which is a storage address stored in the object file, and "FileName" refers to a file name of the object file to which the index information points, and "DFDN" refers to a name of the constructed directory information.
In step S400, the index information is constructed into a dynamic directory, wherein the dynamic directory can be processed by an object storage to file storage protocol gateway, and the object storage to file storage protocol gateway can provide the object file to a user for access in a file storage mode. When the user accesses the object file provided by the gateway service, the gateway acquires the corresponding object file data from the object storage according to the index information and returns the corresponding object file data to the user for acquisition.
In some embodiments, referring to fig. 2, in step S400, the steps are further included:
s410, accessing the object file according to the index information;
s420, converting the object file from the object storage form to the file storage form and generating a file directory index;
S430, generating a dynamic catalog according to the file catalog index.
In step S410, the corresponding object file is accessed through the object store-to-file storage protocol gateway, for example, as mentioned in step S300 with respect to the example of index information by which the corresponding object file in the object store is accessed.
In step S420, the accessed object files are converted from the object storage form to the file storage form by the object storage-to-file storage protocol gateway, and file directory indexes corresponding to the respective object files are generated, where the file directory indexes refer to index information of the object files in the file storage form.
In step S430, a dynamic file is generated by the generated directory index information, and in practical application, taking an example of the index information as an example, the generated dynamic file directory includes the following information:
“dogs/photo1.jpg
dogs/photo3.jpg
dogs/photo6.jpg”。
It should be noted that, unlike the conventional file directory, the generated dynamic directory does not have to be stored in the local hard disk, and the object file included in the dynamic directory exists in the form of the index information of the object file, so that the local hard disk is not occupied, while the conventional file directory is constructed by copying the file, and the data of the included file directory is actually located in the local hard disk, so that the storage space of the local silver disk is occupied.
According to the method for generating the dynamic catalogue, the object files are stored, the object files are in the object storage mode, the input file identifications are obtained, the object files are screened according to the file identifications, the screened object files are obtained, index information is generated according to the object files and the file identifications, and finally the dynamic catalogue is generated according to the index information, so that waste of storage space is effectively avoided, and the problem that the storage space is occupied greatly after multiple copies is solved.
In a second aspect, an embodiment of the present application provides a system for generating a dynamic directory, where an object file is stored by a storage module, where the object file is in an object storage form, then a screening module obtains a file identifier, screens the object file according to the file identifier, so as to obtain a screened object file, then generates index information from the obtained object file and the file identifier by an index module, and finally generates the dynamic directory by the generation module according to the index information, thereby effectively avoiding waste of storage space and solving the problem that the storage space is greatly occupied after multiple copies.
In some embodiments, referring to FIG. 3, the dynamic catalog generation system comprises: the device comprises a storage module 100, a screening module 200, an index module 300 and a generation module 400, wherein the storage module 100 is connected with the screening module 200, the index module 300 is respectively connected with the screening module 200 and the storage module 100, and the generation module 400 is connected with the index module 300.
The storage module 100 is configured to store an object file acquired by a data acquisition manner such as big data, and store the object file in the storage module 100 in an object storage manner; the filtering module 200 inputs a corresponding file identifier to the object filter according to the actual requirement, and filters the object file in the storage module 100 according to the file identifier; the index module 300 generates index information according to the object file and the file identifier obtained by screening, and the specific generating manner is discussed in detail in the step S300 of the first aspect, so that details are not repeated; the generating module 400 generates an active directory according to the index information generated by the indexing module 300, so as to be accessed by the user.
In some embodiments, the filtering module 200 is connected to the storage module 100 through an object storage gateway, so as to access the storage module 100 to find the object file according to the object storage gateway, and filter the object file according to the input file identifier. The object storage gateway is used for providing interface service according to metadata of the object files, and is used for accessing the object files in the storage module 100 for screening according to the file identifications input by the object screener. The object filter accesses the object files meeting the conditions in the storage module 100 through the service interface of the object storage gateway according to the input file identifications, such as the label information and the directory name, so as to filter and obtain the corresponding object files.
In some embodiments, the generating module 400 is configured to access the object file in the storage module 100 through the object storage gateway, convert the accessed object file from the object storage form to the file storage form through the object storage to file storage protocol gateway, generate a file directory index, and generate the dynamic directory through the object storage to file storage protocol gateway according to the file directory index.
In a possible implementation example, referring to fig. 4, the filtering module 200 includes an object filter, a file identifier is input to the object filter according to actual requirements, the object filter accesses corresponding object files in the object storage module 100 through an object storage gateway according to the file identifier, including but not limited to, an object file 1, an object file 2 and an object file 3, generates index information through the index module 300 according to the accessed object file and the file identifier, then the index module 300 sends the generated index information to the generating module, the generating module accesses the corresponding object file in the storage module through the object storage gateway, and converts the object file from the object storage form to the file storage form through the object storage-to-file storage protocol gateway, generates a file directory index, and finally generates a dynamic directory according to the file directory index.
The dynamic catalog generating system of the embodiment of the application stores the object files through the storage module 100, wherein the object files adopt the object storage form, then obtains the file identifications through the screening module 200, screens the object files according to the file identifications to obtain screened object files, then generates index information through the index module 300 on the obtained object files and the file identifications, and finally generates the dynamic catalog through the generation module 400 according to the index information, thereby effectively avoiding the waste of storage space and solving the problem that the storage space is greatly occupied after multiple copies.
In a third aspect, an embodiment of the present application further provides an electronic device, including: at least one processor 500, and a memory 600 communicatively coupled to the at least one processor 500;
Wherein the processor 500 is configured to execute the method for generating an dynamic catalog in the embodiment of the first aspect by calling a computer program stored in the memory 600.
The memory 600 is used as a non-transitory computer readable storage medium for storing a non-transitory software program and a non-transitory computer executable program, such as the dynamic catalog generation method in the embodiment of the first aspect of the present application. The processor 500 implements the dynamic catalog generation method in the above-described first-aspect embodiment by running a non-transitory software program and instructions stored in the memory 600.
Memory 600 may include a storage program area that may store an operating system, at least one application program required for functionality, and a storage data area; the storage data area may store and execute the dynamic catalog generation method in the embodiment of the first aspect described above. In addition, memory 600 may include high-speed random access memory, and may also include non-transitory memory, such as at least one magnetic disk storage device, flash memory device, or other non-transitory solid state storage device. In some embodiments, memory 600 optionally includes memory 600 remotely located relative to processor 500, which may be connected to the terminal via a network. Examples of such networks include, but are not limited to, the internet, intranets, local area networks, mobile communication networks, and combinations thereof.
The non-transitory software programs and instructions required to implement the dynamic catalog generation method in the first-aspect embodiment described above are stored in the memory 600, which when executed by the one or more processors 500, perform the dynamic catalog generation method in the first-aspect embodiment described above.
In a fourth aspect, embodiments of the present application also provide a computer-readable storage medium storing computer-executable instructions for: the dynamic catalog generation method in the embodiment of the first aspect is performed.
In some embodiments, the computer-readable storage medium stores computer-executable instructions that are executed by one or more control processors 500, for example, by one processor 500 in an electronic device according to an embodiment of the third aspect, which may cause the one or more processors 500 to perform the method for generating an dynamic catalog in the embodiment of the first aspect.
The above described embodiments of the apparatus are only illustrative, wherein the units described as separate components may or may not be physically separate, i.e. may be located in one place, or may be distributed over a plurality of network elements. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.
Those of ordinary skill in the art will appreciate that all or some of the steps, systems, and methods disclosed above may be implemented as software, firmware, hardware, and suitable combinations thereof. Some or all of the physical components may be implemented as software executed by a processor, such as a central processing unit, digital signal processor, or microprocessor, or as hardware, or as an integrated circuit, such as an application specific integrated circuit. Such software may be distributed on computer readable media, which may include computer storage media (or non-transitory media) and communication media (or transitory media). The term computer storage media includes both volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules or other data, as known to those skilled in the art. Computer storage media includes, but is not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital Versatile Disks (DVD) or other optical disk storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other medium which can be used to store the desired information and which can be accessed by a computer. Furthermore, as is well known to those of ordinary skill in the art, communication media typically embodies computer readable instructions, data structures, program modules or other data in a modulated data signal such as a carrier wave or other transport mechanism and includes any information delivery media.
The embodiments of the present application have been described in detail above with reference to the accompanying drawings, but the embodiments of the present application are not limited to the above embodiments, and various changes can be made within the knowledge of those skilled in the art without departing from the spirit of the embodiments of the present application.
Claims (8)
1. The method for generating the dynamic catalogue is characterized by comprising the following steps:
storing an object file, and determining a category identification for the object file; wherein, the object file adopts an object storage form;
Acquiring a file identifier and screening the object file according to the file identifier to acquire a screened object file; wherein, the file identification includes:
Tag information for identifying the category identification of the object file;
A directory name, the directory name identifying a name of the dynamic directory;
the obtaining the file identifier and filtering the object file according to the file identifier includes:
Acquiring the tag information and the directory name and screening the object file according to the tag information and the directory name;
Generating index information according to the obtained object file and the file identification;
And generating the dynamic catalogue according to the index information.
2. The dynamic catalog generation method of claim 1, wherein the index information comprises:
A file address identifying a storage location of the object file;
a file name for identifying a name of the object file;
index names, wherein the index names are generated according to the file identifications;
The corresponding code is used to determine the position of the object,
The generating index information according to the obtained object file and the file identification comprises the following steps:
And generating the file address, the file name and the index name according to the obtained object file and the file identification.
3. The method of generating an active directory according to claim 2, wherein generating an active directory from the index information comprises:
Accessing the object file according to the index information;
Converting the object file from an object storage form to a file storage form and generating a file directory index;
And generating a dynamic directory according to the file directory index.
4. A dynamic catalog generation system, comprising:
The storage module is used for determining a category identifier for the object file; the storage module is used for storing an object file, and the object file adopts an object storage form;
The screening module is used for acquiring the file identification and screening the object file according to the file identification so as to acquire a screened object file; wherein, the file identification includes:
Tag information for identifying the category identification of the object file;
A directory name, the directory name identifying a name of the dynamic directory;
the obtaining the file identifier and filtering the object file according to the file identifier includes:
Acquiring the tag information and the directory name and screening the object file according to the tag information and the directory name;
The index module is used for generating index information according to the obtained object file and the file identification;
and the generation module is used for generating the dynamic catalogue according to the index information.
5. The system of claim 4, wherein the screening module is coupled to the storage module via an object storage gateway to screen the object file;
The object storage gateway is used for accessing the object files in the storage module.
6. The dynamic catalog generation system of claim 4 or 5, wherein the generation module is further configured to:
Accessing an object file in the storage module through an object storage gateway;
the object file is converted from an object storage form to a file storage form through an object storage-to-file storage protocol gateway, and a file directory index is generated;
and generating the dynamic directory through the object storage-to-file storage protocol gateway by the file directory index.
7. An electronic device, comprising:
At least one processor, and
A memory communicatively coupled to the at least one processor; wherein,
The memory stores instructions for execution by the at least one processor to cause the at least one processor to implement the dynamic catalog generation method of any one of claims 1 to 3 when executing the instructions.
8. A computer-readable storage medium storing computer-executable instructions for causing a computer to perform the dynamic catalog generation method of any one of claims 1 to 3.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010703826.XA CN111949619B (en) | 2020-07-21 | 2020-07-21 | Dynamic catalog generation method, system, electronic equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010703826.XA CN111949619B (en) | 2020-07-21 | 2020-07-21 | Dynamic catalog generation method, system, electronic equipment and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111949619A CN111949619A (en) | 2020-11-17 |
| CN111949619B true CN111949619B (en) | 2024-04-26 |
Family
ID=73340160
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010703826.XA Active CN111949619B (en) | 2020-07-21 | 2020-07-21 | Dynamic catalog generation method, system, electronic equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111949619B (en) |
Families Citing this family (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113157209A (en) * | 2021-04-09 | 2021-07-23 | 北京易华录信息技术股份有限公司 | Data reconstruction method and device from file system to object storage |
| CN113722271B (en) * | 2021-07-20 | 2023-11-21 | 湖南艾科诺维科技有限公司 | File management method, system and medium for data acquisition and playback |
| CN115858473B (en) * | 2023-01-29 | 2023-10-10 | 北京阿丘科技有限公司 | Data interaction method and device based on training system and object storage system |
Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US6647393B1 (en) * | 1996-11-22 | 2003-11-11 | Mangosoft Corporation | Dynamic directory service |
| CN101520804A (en) * | 2009-03-20 | 2009-09-02 | 深圳创维－Rgb电子有限公司 | File searching system and method based on SQL database |
| CN101763441A (en) * | 2010-01-13 | 2010-06-30 | 北京中加国道科技有限公司 | Technology organizing search results in active directory mode |
| US7752226B1 (en) * | 2002-12-20 | 2010-07-06 | Symantec Operating Corporation | Reverse pathname lookup by inode identifier |
| CN106874383A (en) * | 2017-01-10 | 2017-06-20 | 清华大学 | A kind of decoupling location mode of metadata of distributed type file system |
| CN110275864A (en) * | 2019-06-11 | 2019-09-24 | 武汉深之度科技有限公司 | Index establishing method, data query method and calculating equipment |
| CN110413771A (en) * | 2019-06-18 | 2019-11-05 | 平安科技（深圳）有限公司 | Classified index method, apparatus, equipment and storage medium based on solr |
| CN111125016A (en) * | 2019-12-24 | 2020-05-08 | 普世(南京)智能科技有限公司 | Magneto-optical hybrid file storage method and system based on label organization |
| CN111159106A (en) * | 2019-12-30 | 2020-05-15 | 亚信科技（中国）有限公司 | Data query method and device |
Family Cites Families (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US8719261B2 (en) * | 2011-12-02 | 2014-05-06 | Verizon Patent And Licensing Inc. | Dynamic catalog ranking |
| US10198445B2 (en) * | 2014-06-30 | 2019-02-05 | Google Llc | Automated archiving of user generated media files |
-
2020
- 2020-07-21 CN CN202010703826.XA patent/CN111949619B/en active Active
Patent Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US6647393B1 (en) * | 1996-11-22 | 2003-11-11 | Mangosoft Corporation | Dynamic directory service |
| US7752226B1 (en) * | 2002-12-20 | 2010-07-06 | Symantec Operating Corporation | Reverse pathname lookup by inode identifier |
| CN101520804A (en) * | 2009-03-20 | 2009-09-02 | 深圳创维－Rgb电子有限公司 | File searching system and method based on SQL database |
| CN101763441A (en) * | 2010-01-13 | 2010-06-30 | 北京中加国道科技有限公司 | Technology organizing search results in active directory mode |
| CN106874383A (en) * | 2017-01-10 | 2017-06-20 | 清华大学 | A kind of decoupling location mode of metadata of distributed type file system |
| CN110275864A (en) * | 2019-06-11 | 2019-09-24 | 武汉深之度科技有限公司 | Index establishing method, data query method and calculating equipment |
| CN110413771A (en) * | 2019-06-18 | 2019-11-05 | 平安科技（深圳）有限公司 | Classified index method, apparatus, equipment and storage medium based on solr |
| CN111125016A (en) * | 2019-12-24 | 2020-05-08 | 普世(南京)智能科技有限公司 | Magneto-optical hybrid file storage method and system based on label organization |
| CN111159106A (en) * | 2019-12-30 | 2020-05-15 | 亚信科技（中国）有限公司 | Data query method and device |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111949619A (en) | 2020-11-17 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111949619B (en) | Dynamic catalog generation method, system, electronic equipment and storage medium | |
| CN106484828B (en) | Distributed internet data rapid acquisition system and acquisition method | |
| CN102171660B (en) | Backing up and restoring selected versioned objects from a monolithic database backup | |
| US6199081B1 (en) | Automatic tagging of documents and exclusion by content | |
| JP6396897B2 (en) | Search for events by attendees | |
| US20150235236A1 (en) | Media dissemination system | |
| CN111859076B (en) | Data crawling method, device, computer equipment and computer readable storage medium | |
| CN108900547B (en) | Source returning control method and device | |
| US9870422B2 (en) | Natural language search | |
| US11556228B2 (en) | System and method for content management | |
| US20150088646A1 (en) | Systems and methods for generating intelligent account reconfiguration offers | |
| CN112084190A (en) | Big data based acquired data real-time storage and management system and method | |
| CN103927252A (en) | Cross-component log recording method, device and system | |
| CN104021170B (en) | A kind of information acquisition method and cloud server | |
| EP3282372B1 (en) | Method and apparatus for storing data | |
| WO2016206395A1 (en) | Weekly report information processing method and device | |
| US8732141B2 (en) | Passing information between server and client using a data package | |
| JPH1125125A (en) | Network information retrieving device, its method and storage medium | |
| CN112286868A (en) | Electronic file and record management based method | |
| WO2017088701A1 (en) | Mass picture management method and apparatus | |
| CN113515715B (en) | Buried point event code generation method, buried point event code processing method and related equipment | |
| CN109960695B (en) | Management method and device for database in cloud computing system | |
| CN110740046B (en) | Method and device for analyzing service contract | |
| CN110866165A (en) | Network video acquisition method and system | |
| CN104978339A (en) | File processing method and equipment, and network system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |