CN111694520B - Method and device for optimizing big data storage - Google Patents
Method and device for optimizing big data storage Download PDFInfo
- Publication number
- CN111694520B CN111694520B CN202010534111.6A CN202010534111A CN111694520B CN 111694520 B CN111694520 B CN 111694520B CN 202010534111 A CN202010534111 A CN 202010534111A CN 111694520 B CN111694520 B CN 111694520B
- Authority
- CN
- China
- Prior art keywords
- storage
- data
- node
- stored
- capacity
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0602—Interfaces specially adapted for storage systems specifically adapted to achieve a particular effect
- G06F3/0604—Improving or facilitating administration, e.g. storage management
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0628—Interfaces specially adapted for storage systems making use of a particular technique
- G06F3/0629—Configuration or reconfiguration of storage systems
- G06F3/0631—Configuration or reconfiguration of storage systems by allocating resources to storage systems
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0628—Interfaces specially adapted for storage systems making use of a particular technique
- G06F3/0638—Organizing or formatting or addressing of data
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0668—Interfaces specially adapted for storage systems adopting a particular infrastructure
- G06F3/0671—In-line storage system
- G06F3/0673—Single storage device
- G06F3/0674—Disk device
- G06F3/0676—Magnetic disk device
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0668—Interfaces specially adapted for storage systems adopting a particular infrastructure
- G06F3/0671—In-line storage system
- G06F3/0673—Single storage device
- G06F3/0682—Tape device
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Human Computer Interaction (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention relates to the technical field of computers, in particular to a method and a device for optimizing large data storage, wherein the device comprises a storage memory set or unit, a storage node management unit, a first recording unit, a first writing module, a second recording unit, a storage data receiving module, a decoding module, a compiling module and a processing module, and the method is based on the device and has the overall idea that the storage memory is divided into a plurality of storage nodes, the storage attributes and the storage rules of the storage nodes are edited, the data to be stored are decoded, the decoded data to be stored are written into the storage attributes according to the file attributes of the data to be stored and are classified according to the corresponding storage rules, and the processing module identifies redundant data in the data to be stored by using a redundant data identification method according to the selection of the storage rules and the storage capacity, and then writing the node code of the corresponding virtual position in the redundant data and storing the node code in the corresponding storage node.
Description
Technical Field
The invention relates to the technical field of computers, in particular to a data storage method, and specifically relates to a method and a device for optimizing big data storage.
Background
The data storage object comprises temporary files generated in the processing process of the data stream or information needing to be searched in the processing process. Data is recorded in a certain format on a storage medium inside or outside the computer. The data store is named, which is to reflect the constituent meaning of the information features. The data flow reflects data flowing in the system and shows the characteristics of dynamic data; the data store reflects data that is static in the system, characterizing static data.
With the fusion of big data, the data types are various, and the storage of the storage object firstly needs to ensure the correctness of data storage and be convenient for retrieval.
The existing data storage generally adopts block storage, but when data is retrieved after the block storage, the whole disk or tape under the same storage environment still needs to be searched in a traversing way, and even if a DAS storage mode (direct connection storage) is adopted, the efficiency is very low.
Disclosure of Invention
The present invention is directed to a method and an apparatus for optimizing big data storage, so as to solve the problems in the background art.
In order to achieve the purpose, the invention provides the following technical scheme:
a method for optimizing big data storage comprises
Acquiring a storage memory, dividing the storage memory into a plurality of storage nodes,
recording the virtual position and the storage capacity of each storage node;
setting the storage attribute of each storage node, recording the storage attribute and the storage rule,
setting and enabling data to be stored to be distributed to corresponding storage nodes by a storage rule only under the same storage attribute;
decoding the data to be stored, sending the decoded data to be stored to a compiling module, writing the storage attribute of the data to be stored into the compiling module according to the file attribute of the data to be stored, classifying the data according to corresponding storage rules,
respectively acquire the storage capacities of the storage nodes having the same classification,
according to the storage rule and the selection of the storage capacity, the processing module identifies the redundant data in the data to be stored by using a redundant data identification method, and then writes the node code corresponding to the virtual position in the redundant data and stores the node code in the corresponding storage node.
Further, the node code includes the following information: the virtual position of the corresponding storage node and the data capacity of the stored data;
wherein, the node code adopts binary writing.
Further, the method for selecting the storage capacity comprises the following steps:
acquiring the node code of the storage node in the same virtual position,
decompiling each node code to obtain the data capacity of each corresponding stored data,
and summing all data capacities of the storage nodes, and comparing the data capacities of the corresponding storage nodes to obtain the residual capacity or the storage capacity in the current state.
Furthermore, the compiling module writes the data to be compiled into the storage attributes, stores the storage attributes into a data structure of an encapsulation layer arranged in the processing module according to the time sequence,
and when the processing module writes the node code into the data structure of the packaging layer, a header field for writing the information of the node code data is formed.
Further, the storage rule simultaneously satisfies the following conditions:
under the same storage node, firstly,
(ii) having the same virtual position,
and the storage capacity of the storage nodes of the same classification is according to a priority value from small to large.
Further, the storage attribute comprises
Data classes or data structures having the same constants, or
A data class or data structure with variables.
Further, the node code satisfies the split storage of the tree structure information.
The invention also provides a big data storage optimization device, which comprises
A storage memory set or unit for large data storage,
a storage node management unit for dividing the storage memory set or unit according to a tree structure to form at least one storage node,
a first recording unit for recording and storing the virtual location and storage capacity of each storage node,
the first writing module writes each storage node into the same constant or writes each storage node into corresponding variables according to the corresponding data class or data structure body to form storage attributes,
a second recording unit for recording and storing the storage attribute and the storage rule,
a storage data receiving module for receiving data to be stored,
a decoding module for decoding the data to be stored,
the compiling module receives the decoded data to be stored, writes the storage attributes into the compiling module according to the file attributes of the data to be stored, classifies the data according to corresponding storage rules,
and the processing module is used for identifying redundant data in the data to be stored by using a redundant data identification method according to the storage rule and the selection of the storage capacity, writing the node code corresponding to the virtual position in the redundant data, and storing the node code in the corresponding storage node.
Further, the storage rules enable the data to be stored to be allocated to the corresponding storage nodes by the storage rules only under the same storage attributes.
The system further comprises a second writing module for writing the virtual position of the corresponding storage node and the data capacity of the storage data into the node code.
Compared with the prior art, the invention has the beneficial effects that:
firstly, the present invention is not limited to what kind of storage medium is adopted, and particularly, the DAS storage mode is optimized, and the present invention has the overall idea that a storage memory is divided into a plurality of storage nodes, only the storage attributes and the storage rules of the storage nodes need to be edited, when data storage is needed, data to be stored is decoded, the decoded data to be stored is written into the storage attributes according to the file attributes of the data to be stored, and is classified according to the corresponding storage rules, according to the selection of the storage rules and the storage capacity, a processing module identifies redundant data in the data to be stored by using a redundant data identification method, and then writes node codes corresponding to virtual positions into the redundant data and stores the node codes in the corresponding storage nodes.
And secondly, the storage attribute is divided, stored and written in, so that the storage rule of the storage data is classified, and tag search and classified search are conveniently established.
Thirdly, the storage capacity of each storage node is automatically calculated, and management of the storage nodes is facilitated.
Drawings
FIG. 1 is a flow chart of a method of the present invention;
FIG. 2 is a flow chart of a method of storage capacity selection in the present invention;
fig. 3 is a general block diagram of the system of the present invention.
Detailed Description
Detailed description of the preferred embodimentsreferring to fig. 1-3. The invention provides a method for optimizing big data storage, which comprises the following steps
Acquiring a storage memory, dividing the storage memory into a plurality of storage nodes,
recording the virtual position and the storage capacity of each storage node;
setting the storage attribute of each storage node, recording the storage attribute and the storage rule,
setting and enabling data to be stored to be distributed to corresponding storage nodes by a storage rule only under the same storage attribute;
decoding the data to be stored, sending the decoded data to be stored to a compiling module, writing the storage attribute of the data to be stored into the compiling module according to the file attribute of the data to be stored, classifying the data according to corresponding storage rules,
respectively acquire the storage capacities of the storage nodes having the same classification,
according to the storage rule and the selection of the storage capacity, the processing module identifies the redundant data in the data to be stored by using a redundant data identification method, and then writes the node code corresponding to the virtual position in the redundant data and stores the node code in the corresponding storage node.
In the above, the node code includes the following information: the virtual position of the corresponding storage node and the data capacity of the stored data;
wherein, the node code adopts binary writing.
In the above, the method of selecting the storage capacity includes:
acquiring the node code of the storage node in the same virtual position,
decompiling each node code to obtain the data capacity of each corresponding stored data,
and summing all data capacities of the storage nodes, and comparing the data capacities of the corresponding storage nodes to obtain the residual capacity or the storage capacity in the current state.
In the above, the compiling module writes the data to be compiled into the storage attributes, stores the storage attributes into the data structure of the encapsulation layer set in the processing module according to the time sequence,
and when the processing module writes the node code into the data structure of the packaging layer, a header field for writing the information of the node code data is formed.
In the above, the storage rule simultaneously satisfies the following conditions:
under the same storage node, firstly,
(ii) having the same virtual position,
and the storage capacity of the storage nodes of the same classification is according to a priority value from small to large.
In the above, the storage attribute comprises
Data classes or data structures having the same constants, or
A data class or data structure with variables.
In the above, the node code satisfies the split storage of the tree structure information.
The invention also provides a big data storage optimization device, which comprises
A storage memory set or unit for large data storage,
a storage node management unit for dividing the storage memory set or unit according to a tree structure to form at least one storage node,
a first recording unit for recording and storing the virtual location and storage capacity of each storage node,
the first writing module writes each storage node into the same constant or writes each storage node into corresponding variables according to the corresponding data class or data structure body to form storage attributes,
a second recording unit for recording and storing the storage attribute and the storage rule,
a storage data receiving module for receiving data to be stored,
a decoding module for decoding the data to be stored,
the compiling module receives the decoded data to be stored, writes the storage attributes into the compiling module according to the file attributes of the data to be stored, classifies the data according to corresponding storage rules,
and the processing module is used for identifying redundant data in the data to be stored by using a redundant data identification method according to the storage rule and the selection of the storage capacity, writing the node code corresponding to the virtual position in the redundant data, and storing the node code in the corresponding storage node.
In the above, the storage rule enables the data to be stored to be allocated to the corresponding storage node by the storage rule only under the same storage attribute.
In the foregoing, the apparatus further includes a second writing module, configured to write the virtual location of the corresponding storage node and the data capacity of the storage data into the node code.
Example 1.
The present invention is described in terms of a storage medium.
The storage medium is a magnetic disk or tape.
Taking a magnetic disk as an example, in the present invention,
a storage node may be one or more of the smallest units of storage of a disk.
The storage node may also be one or more disk units in a disk array.
The storage nodes may also be one or more disk arrays in the same storage environment.
The following takes one or more minimum storage units of a disk as an example.
A method for optimizing big data storage comprises
Obtaining the whole memory of the disk, dividing the whole memory of the disk into storage nodes with one or more minimum storage unit combination sets,
recording the virtual position and the storage capacity of each storage node; the virtual locations may be arranged in an edit according to a minimum set of combinations of storage units.
Writing each storage node into the same constant, or writing each storage node into corresponding variable according to the corresponding data class or data structure to form storage attribute, and satisfying the following storage rules, wherein the storage nodes are arranged under the same storage node, have the same virtual position, and the storage capacity of the storage nodes in the same classification is according to the priority value from small to large.
When the data is stored, the data to be stored is decoded, the decoded data is written into the storage attribute according to the file attribute of the data to be stored, the data is classified according to the corresponding storage rule, the processing module identifies the redundant data in the data to be stored by using a redundant data identification method according to the selection of the storage rule and the storage capacity, and then the node code corresponding to the virtual position is written into the redundant data and stored in the corresponding storage node.
In the retrieval, the search tag may be established according to a constant written in the storage node or a variable written in each storage node according to the corresponding data class or data structure.
Search tags may also be established according to virtual location.
Search tags may also be built in terms of headers.
In this embodiment, the storage attribute is divided and written in, so that the storage rule of the storage data is classified, and the tag search and the classification search are conveniently established. The storage capacity of each storage node is automatically calculated, and management of the storage nodes is facilitated.
Example 2.
The present embodiment is explained in terms of a storage manner.
The DAS storage mode has dispersed main geographical locations, and each branch has a data storage device, so that a plurality of data storage devices may be used as a storage set in the same area.
The main idea of the present invention is to arrange and divide the data storage devices in the same area, and during the division, one or several independent data storage devices can be used as one storage node. Then, in the same area, different storage data can be stored according to the storage nodes when being stored.
In practice, a server system is required to be established, and the server system is connected with the data storage device.
A method for optimizing big data storage comprises
The data storage devices in the same area are sorted and divided, and when the data storage devices are divided, one or more independent data storage devices can be used as a storage node,
recording the virtual position and the storage capacity of each storage node; the virtual locations may be arranged in an edit according to a minimum set of combinations of storage units.
Writing each storage node into corresponding variable according to corresponding data class or data structure to form storage attribute, and satisfying the following storage rule, firstly, under the same storage node, secondly, having the same virtual position, thirdly, storing capacity of the storage nodes of the same classification according to priority value from small to large.
When the data is stored, the data to be stored is decoded, the decoded data is written into the storage attribute according to the file attribute of the data to be stored, the data is classified according to the corresponding storage rule, the processing module identifies the redundant data in the data to be stored by using a redundant data identification method according to the selection of the storage rule and the storage capacity, and then the node code corresponding to the virtual position is written into the redundant data and stored in the corresponding storage node.
In the search, a search tag may be created according to a variable written in each storage node according to the corresponding data class or data structure.
Search tags may also be established according to virtual location.
Search tags may also be built in terms of headers.
Search tags may also be established by category.
In this embodiment, the storage attribute is divided and written in, so that the storage rule of the storage data is classified, and the tag search and the classification search are conveniently established. The storage capacity of each storage node is automatically calculated, and management of the storage nodes is facilitated.
The principles and embodiments of the present invention are explained herein using specific examples, which are presented only to assist in understanding the method and its core concepts of the present invention. The foregoing is only a preferred embodiment of the present invention, and it should be noted that there are objectively infinite specific structures due to the limited character expressions, and it will be apparent to those skilled in the art that a plurality of modifications, decorations or changes may be made without departing from the principle of the present invention, and the technical features described above may be combined in a suitable manner; such modifications, variations, combinations, or adaptations of the invention using its spirit and scope, as defined by the claims, may be directed to other uses and embodiments.
Claims (9)
1. A method for optimizing big data storage is characterized by comprising
Acquiring a storage memory, dividing the storage memory into a plurality of storage nodes,
recording the virtual position and the storage capacity of each storage node;
writing the same constant into each storage node, or writing the corresponding variable into each storage node according to the corresponding data class or data structure body to set the storage attribute of each storage node, recording the storage attribute and the storage rule,
setting and enabling data to be stored to be distributed to corresponding storage nodes by a storage rule only under the same storage attribute;
decoding the data to be stored, sending the decoded data to be stored to a compiling module, writing the storage attribute of the data to be stored into the compiling module according to the file attribute of the data to be stored, classifying the data according to corresponding storage rules,
respectively acquire the storage capacities of the storage nodes having the same classification,
according to the storage rule and the selection of the storage capacity, the processing module identifies redundant data in the data to be stored by using a redundant data identification method, then writes node codes corresponding to the virtual positions in the redundant data, and stores the node codes in the corresponding storage nodes;
and establishing a search tag according to the constant written into the storage node or the variable written into each storage node according to the corresponding data class or the data structure body, or establishing a search tag according to the virtual position, or establishing a search tag according to the field.
2. The big data storage optimization method according to claim 1, wherein the node code comprises the following information: the virtual position of the corresponding storage node and the data capacity of the stored data;
wherein, the node code adopts binary writing.
3. The big data storage optimization method according to claim 1, wherein the storage capacity selection method comprises:
acquiring the node code of the storage node in the same virtual position,
decompiling each node code to obtain the data capacity of each corresponding stored data,
and summing all data capacities of the storage nodes, and comparing the data capacities of the corresponding storage nodes to obtain the residual capacity or the storage capacity in the current state.
4. The big data storage optimization method according to claim 1, wherein the compiling module writes the data to be compiled into the storage attributes, stores the storage attributes into the data structure of the encapsulation layer in the processing module in time sequence,
and when the processing module writes the node code into the data structure of the packaging layer, a header field for writing the information of the node code data is formed.
5. The big data storage optimization method according to claim 1, wherein the storage rules simultaneously satisfy the following conditions:
under the same storage node, firstly,
(ii) having the same virtual position,
and the storage capacity of the storage nodes of the same classification is according to a priority value from small to large.
6. The big data storage optimization method according to claim 1 or 4, wherein the storage attribute comprises
Data classes or data structures having the same constants, or
A data class or data structure with variables.
7. The big data storage optimization method according to claim 1, wherein the node code satisfies split storage of tree structure information.
8. A big data storage optimization device is characterized by comprising
A storage memory set or unit for large data storage,
a storage node management unit for dividing the storage memory set or unit according to a tree structure to form at least one storage node,
a first recording unit for recording and storing the virtual location and storage capacity of each storage node,
the first writing module writes each storage node into the same constant or writes each storage node into corresponding variables according to the corresponding data class or data structure body to form storage attributes,
a second recording unit for recording and storing the storage attribute and the storage rule,
a storage data receiving module for receiving data to be stored,
a decoding module for decoding the data to be stored,
the compiling module receives the decoded data to be stored, writes the storage attributes into the compiling module according to the file attributes of the data to be stored, classifies the data according to corresponding storage rules,
the processing module is used for identifying redundant data in the data to be stored by using a redundant data identification method according to the storage rule and the selection of the storage capacity, writing node codes corresponding to the virtual positions in the redundant data, and storing the node codes in the corresponding storage nodes;
the storage rules enable the data to be stored to be distributed to the corresponding storage nodes by the storage rules only under the same storage attributes;
and establishing a search tag according to the constant written into the storage node or the variable written into each storage node according to the corresponding data class or the data structure body, or establishing a search tag according to the virtual position, or establishing a search tag according to the field.
9. The big data storage optimization device of claim 8, further comprising a second writing module to write the virtual location of the corresponding storage node and the data capacity of the storage data into the node code.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010534111.6A CN111694520B (en) | 2020-06-11 | 2020-06-11 | Method and device for optimizing big data storage |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010534111.6A CN111694520B (en) | 2020-06-11 | 2020-06-11 | Method and device for optimizing big data storage |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111694520A CN111694520A (en) | 2020-09-22 |
| CN111694520B true CN111694520B (en) | 2021-04-27 |
Family
ID=72480620
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010534111.6A Active CN111694520B (en) | 2020-06-11 | 2020-06-11 | Method and device for optimizing big data storage |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111694520B (en) |
Families Citing this family (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114185493B (en) * | 2022-02-15 | 2022-05-06 | 烟台市地理信息中心 | Method for dividing and storing surveying and mapping data |
| CN116244369B (en) * | 2023-05-12 | 2023-07-18 | 山东云力信息科技有限公司 | Data mining method and system based on big data visualization |
Family Cites Families (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US4318184A (en) * | 1978-09-05 | 1982-03-02 | Millett Ronald P | Information storage and retrieval system and method |
| CN100437586C (en) * | 2005-10-21 | 2008-11-26 | 中国科学院计算技术研究所 | A resource allocation method and system for shared memory |
| CN102147711B (en) * | 2010-12-31 | 2014-04-02 | 华为数字技术（成都）有限公司 | Storage method and device based on data content identification |
| CN109799948B (en) * | 2017-11-17 | 2023-05-16 | 航天信息股份有限公司 | Data storage method and device |
-
2020
- 2020-06-11 CN CN202010534111.6A patent/CN111694520B/en active Active
Also Published As
| Publication number | Publication date |
|---|---|
| CN111694520A (en) | 2020-09-22 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US5274807A (en) | Method for reducing magnetic storage volume for computer disk image backup | |
| US6915340B2 (en) | System and method for deriving future network configuration data from the current and previous network configuration data | |
| CN111694520B (en) | Method and device for optimizing big data storage | |
| US8166073B2 (en) | Information processing device, storage device and computer-readable medium for accepting description information of multi-media content including keywords and reference information indicative of duplicative occurrence of each keyword and retrieving location information in the content using the respective keywords and associated reference information | |
| US20060294101A1 (en) | Multi-strategy document classification system and method | |
| CN104461390A (en) | Method and device for writing data into imbricate magnetic recording SMR hard disk | |
| CN105224237A (en) | A kind of date storage method and device | |
| CN102147711A (en) | Storage method and device based on data content identification | |
| CN103955433A (en) | Shingled magnetic recording hard disk, and method and device for writing data in shingled magnetic recording hard disk | |
| CN114491172B (en) | Rapid retrieval method, device and equipment for tree structure nodes and storage medium | |
| CN110826342A (en) | Method, device, computer storage medium and terminal for realizing model management | |
| CN114943287A (en) | Computer big data acquisition and processing system, method, equipment and medium | |
| CN112597348A (en) | Method and device for optimizing big data storage | |
| CN111897828A (en) | Data batch processing implementation method, device, equipment and storage medium | |
| CN110569218B (en) | Offline modification method and device for EXT file system and storage medium | |
| FR2606182B1 (en) | INFORMATION STORAGE AND CATALOGING METHOD, AND COMPUTER COMPRISING A CLASSIFICATION SYSTEM IMPLEMENTING SAID METHOD | |
| CN114153394A (en) | Data storage management method, device and equipment | |
| CN103793391A (en) | Remote dynamic data processing and verifying method and system | |
| CN114281242B (en) | Memory load balancing method, device and equipment | |
| CN111752941A (en) | Data storage method, data access method, data storage device, data access device, server and storage medium | |
| US7672753B1 (en) | Optimized library device management | |
| CN115577113A (en) | Knowledge graph platform | |
| CN113946544A (en) | Metadata organization method and device of distributed file system | |
| CN117193674B (en) | Method and device for improving mass data access efficiency of Internet of things equipment | |
| CN110880005A (en) | Vector index establishing method and device and vector retrieval method and device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |