CN112799607A - Data storage method for partitioned storage according to data size - Google Patents

Data storage method for partitioned storage according to data size Download PDF

Info

Publication number
CN112799607A
CN112799607A CN202110385952.XA CN202110385952A CN112799607A CN 112799607 A CN112799607 A CN 112799607A CN 202110385952 A CN202110385952 A CN 202110385952A CN 112799607 A CN112799607 A CN 112799607A
Authority
CN
China
Prior art keywords
data
time
storage
content
distributed
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN202110385952.XA
Other languages
Chinese (zh)
Other versions
CN112799607B (en
Inventor
林岸华
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Liyang Guangdong Energy Saving Technology Co ltd
Original Assignee
Liyang Guangdong Energy Saving Technology Co ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Liyang Guangdong Energy Saving Technology Co ltd filed Critical Liyang Guangdong Energy Saving Technology Co ltd
Priority to CN202110385952.XA priority Critical patent/CN112799607B/en
Publication of CN112799607A publication Critical patent/CN112799607A/en
Application granted granted Critical
Publication of CN112799607B publication Critical patent/CN112799607B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601Interfaces specially adapted for storage systems
    • G06F3/0628Interfaces specially adapted for storage systems making use of a particular technique
    • G06F3/0638Organizing or formatting or addressing of data
    • G06F3/0643Management of files
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/30Information retrieval; Database structures therefor; File system structures therefor of unstructured textual data
    • G06F16/31Indexing; Data structures therefor; Storage structures
    • G06F16/316Indexing structures
    • G06F16/322Trees
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601Interfaces specially adapted for storage systems
    • G06F3/0628Interfaces specially adapted for storage systems making use of a particular technique
    • G06F3/0629Configuration or reconfiguration of storage systems
    • G06F3/0631Configuration or reconfiguration of storage systems by allocating resources to storage systems
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601Interfaces specially adapted for storage systems
    • G06F3/0668Interfaces specially adapted for storage systems adopting a particular infrastructure
    • G06F3/067Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]

Abstract

The invention relates to the technical field of data storage, in particular to a data storage method for partitioned storage according to the size of data. Which comprises the following steps: processing data; network coding; distributing data; data backup: the invention has high storage efficiency and high space utilization rate, can record the time and the content of data modification so as to be convenient for the subsequent viewing of the document contents in different modification time periods, compares the multiple modified contents stored in the content modification storage, conveniently inquires the modified contents in different time periods of data at any time, and conveniently and more intuitively views the modified contents of the data.

Description

Data storage method for partitioned storage according to data size
Technical Field
The invention relates to the technical field of data storage, in particular to a data storage method for partitioned storage according to the size of data.
Background
With the development of computer technology, sensor technology and internet of things technology, a large amount of data is created in the industrial field unprecedentedly, for the industrial field, while the big data brings potential value, one of the challenges of simultaneously bringing huge challenge to the big data is how to realize the rapid storage and management of the data while the data is rapidly generated, which plays an important role in the value mining of the data and has important significance for improving the manufacturing capability of equipment;
however, at present, when data is stored, the utilization rate of storage capacity is low, the contradiction between data and a memory cannot be effectively solved, the storage performance is low, the load is unbalanced, when some documents are stored in the data, the documents are modified and stored to replace the content before modification, the content data before modification is lost, the content modified in the previous period may be forgotten during reading, the modified content cannot be accurately compared, the modified content is inconvenient to check at different time, and the storage of document data is not facilitated.
Disclosure of Invention
The present invention is directed to a data storage method for partitioned storage according to data size, so as to solve the problems in the background art.
In order to achieve the above object, the present invention provides a data storage method for partitioned storage according to data size, comprising the following steps:
slicing the data transmission and filtering useless data;
the information exchange technology of the fusion routing and the encoding is used for carrying out linear or nonlinear processing on the information received on each channel on each node in the network and then forwarding the information to a lower node so as to improve the speed of data transmission;
the received original data is expanded and coded into a plurality of coded data blocks, the coded data blocks are distributed to different data storage nodes to store data in a distributed mode, when a user accesses the data, the data receiving nodes are used for obtaining the corresponding coded data blocks from the data storage nodes, and the original data is restored through calculation;
the coded data block allocation comprises the steps of:
executing a rule algorithm, collecting data required by the pre-partition area, and storing the data on each node in a rapid and balanced manner;
providing database access service, so that different types of data are stored in different partitions;
receiving data for distributed storage, and storing the data in an idle area in a distributed manner by adopting a binary sorting tree optimal adaptation algorithm;
recording data modification time and content, and storing the content and time of data backup so as to check document contents in different modification time periods in the following process, wherein as long as one of the nodes can be accessed, a user can obtain an original data file, the more data file copies are, the better the data availability is, the higher the reliability is, the multidata backup does not involve coding operation, the reading and writing efficiency is high, and the fault tolerance performance is good;
the data backup comprises the following steps:
storing the content of each time the data is changed;
recording the time of changing data storage;
comparing the stored multi-time modified contents, comparing the original data with the first modified contents, and comparing the first modified contents with the second modified contents, so that the data multi-time modified contents can be viewed more intuitively;
and searching the data document according to the modified content and the recorded time.
As a further improvement of the present technical solution, the slicing process includes the steps of:
collecting information stored by data;
analyzing the stored data to know the size of the data;
cutting the analyzed data into small data;
different types of data are processed in a standardized format.
As a further improvement of the technical solution, the acquiring of the information stored in the data storage includes the following steps:
acquiring metadata in a data life cycle, organizing the metadata, and writing the metadata into a database;
collecting data configuration conditions for subsequent partition storage;
and classifying the data of the same type.
As a further improvement of the technical solution, the information exchange technology employs a distributed erasure code technology, and the distributed erasure code technology includes the following gestures:
attitude one, random network coding, wherein the coefficient of the network node for operating the information is randomly selected;
and secondly, determining the network code, and calculating the network code through a polynomial time algorithm, wherein the polynomial time algorithm comprises the following steps:
firstly, constructing a path cluster from an information source s to each information sink node 1;
secondly, sequencing links on the path cluster according to topology;
and thirdly, searching the global coding vector of each link in sequence, so that the coding vector of the newest link on each path cluster can form a base, thereby ensuring the full rank of the finally formed transfer matrix.
As a further improvement of the technical solution, the random network code deciphering success rate calculation formula is as follows:
Figure 68232DEST_PATH_IMAGE002
where c is the minimum probability, q is the size of the encoded symbol field, d is the number of sink nodes,
Figure 637754DEST_PATH_IMAGE004
the number of links of the coding coefficient is randomly selected, the random network coding can play a role in information compression for linearly related information sources, in addition, the random network coding is the distributed realization of the network coding, the topological information of the whole network is not required to be known in advance, the random network coding is particularly suitable for a topological structure dynamic change or a large-scale network, and for the network with network nodes and link failures, the random network coding can utilize the residual capacity of the whole network to obtain the optimal capacity of the network, so that the robustness of multicast transmission is improved.
As a further improvement of the technical solution, the acquiring of the data required by the pre-partition includes the following gestures:
partitioning according to the parameter names and storing different parameter data in a partition mode according to the parameter names;
and secondly, partitioning according to time, and storing the same data content in a partition mode according to the modified time.
As a further improvement of the technical scheme, the distributed storage comprises file-level storage and file block-level storage, wherein the file-level storage is used for storing files with small data, the file block-level storage is used for cutting a large file into a plurality of small files to be stored in a centralized manner, and the phenomenon that the files are large in data and are damaged due to blocking during subsequent file searching is avoided.
As a further improvement of the technical solution, the binary tree optimal adaptation algorithm includes the following steps:
firstly, forming idle partitions into a binary sequencing tree;
secondly, searching idle partitions meeting regulation in sequence according to the property of the binary ordering tree, recombining the binary ordering tree after distribution to ensure that the structure of the binary ordering tree is not damaged, improving the efficiency problem of the existing optimal adaptation algorithm in the searching process, connecting all idle partition blocks in a tree structure according to the property of the binary ordering tree by adopting the optimal adaptation algorithm of the binary ordering tree, when allocating memory for a process, comparing from a root node according to the property of the binary ordering tree according to a request, adopting recursive search until the idle partition blocks meeting the conditions are found, then distributing, in the optimal adaptation algorithm adopting the binary ordering tree, if the capacities of two memory blocks are equal, inserting the left sub-tree of a node of a memory block inserted first according to an inserting sequence, and existing 6 memory partition blocks with the ordering sequence of R3, R5, R1, R6, R2 and R5, R4) and the capacity is R1 < R2 < R3 < R4 < R5 < R6), after a series of insertion operations are performed according to the algorithm, a binary sort tree can be generated, the algorithm adopts the binary sort tree to realize management, organization and query of the memory free blocks, the allocation speed of the free blocks is greatly improved, and thus the running speed of the process and the throughput of the system are improved.
Compared with the prior art, the invention has the beneficial effects that:
1. according to the data storage method stored in a partitioned mode according to the data size, the data modification time and content are recorded through data backup, the data backup content and time are stored in a distributed mode through the data, document contents in different modification time periods can be checked later, a user can obtain an original data file as long as one of the nodes can be accessed, the more data file copies are, the better data availability is, the higher reliability is, the multi-data backup does not relate to coding operation, the reading and writing efficiency is high, the fault tolerance performance is good, the content modified in the different data time periods can be inquired conveniently by comparing the multiple modified contents stored in the modified contents, and the content modified in the different data time periods can be checked conveniently and visually.
2. According to the data storage method stored in a partitioned mode according to the data size, original data received by network coding are expanded and coded into a plurality of coding data blocks through data distribution, the coding data blocks are distributed to different data storage nodes to store data in a distributed mode, management, organization and query of memory idle blocks are achieved, the distribution speed of the idle blocks is greatly improved, and therefore the running speed of a process and the throughput of a system are improved.
Drawings
FIG. 1 is an overall flow chart of example 1;
FIG. 2 is a block diagram showing a data processing flow in example 1;
FIG. 3 is a block diagram of a data preparation flow of example 1;
FIG. 4 is a block diagram showing a data distribution flow in embodiment 1;
fig. 5 is a block diagram of a data backup process in embodiment 1.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
Example 1
Referring to fig. 1 to 5, the present embodiment provides a data storage method for partitioned storage according to data size, including the following steps:
s1, data processing: slicing the data transmission and filtering useless data;
s2, network coding: the information exchange technology of the fusion routing and the encoding is used for carrying out linear or nonlinear processing on the information received on each channel on each node in the network and then forwarding the information to a lower node so as to improve the speed of data transmission;
s3, data allocation: the received original data is expanded and coded into a plurality of coded data blocks, the coded data blocks are distributed to different data storage nodes to store data in a distributed mode, when a user accesses the data, the data receiving nodes are used for obtaining the corresponding coded data blocks from the data storage nodes, and the original data is restored through calculation;
the coded data block allocation comprises the steps of:
s3.1, automatic pre-partitioning: executing a rule algorithm, collecting data required by the pre-partition area, and storing the data on each node in a rapid and balanced manner;
s3.2, service access: providing database access service, so that different types of data are stored in different partitions;
s3.3, distributed storage: receiving data for distributed storage, and storing the data in an idle area in a distributed manner by adopting a binary sorting tree optimal adaptation algorithm;
s4, data backup: recording data modification time and content, and storing the content and time of data backup so as to check document contents in different modification time periods in the following process, wherein as long as one of the nodes can be accessed, a user can obtain an original data file, the more data file copies are, the better the data availability is, the higher the reliability is, the multidata backup does not involve coding operation, the reading and writing efficiency is high, and the fault tolerance performance is good;
the data backup comprises the following steps:
s4.1, content modification and storage: storing the content of each time the data is changed;
s4.2, date recording: recording the time of changing data storage;
s4.3, comparison: comparing the stored multi-time modified contents, comparing the original data with the first modified contents, and comparing the first modified contents with the second modified contents, so that the data multi-time modified contents can be viewed more intuitively;
s4.4, data searching: and searching the data document according to the modified content and the recorded time.
In this embodiment, the slicing process includes the following steps:
s1.1, data preparation: collecting information stored by data;
s1.2, preprocessing data: analyzing the stored data to know the size of the data;
s1.3, data slicing: cutting the analyzed data into small data;
s1.4, data standardization: different types of data are processed in a standardized format.
Specifically, the collecting of the information stored in the data storage includes the following steps:
s1.11, metadata acquisition: acquiring metadata in a data life cycle, organizing the metadata, and writing the metadata into a database;
s1.12, configuration information acquisition: collecting data configuration conditions for subsequent partition storage;
s1.13, collecting cluster resource information: and classifying the data of the same type.
Further, the information exchange technology employs a distributed erasure code technology, which includes the following gestures:
attitude one, random network coding, wherein the coefficient of the network node for operating the information is randomly selected;
and secondly, determining the network code, and calculating the network code through a polynomial time algorithm, wherein the polynomial time algorithm comprises the following steps:
firstly, constructing a path cluster from an information source s to each information sink node 1;
secondly, sequencing links on the path cluster according to topology;
and thirdly, searching the global coding vector of each link in sequence, so that the coding vector of the newest link on each path cluster can form a base, thereby ensuring the full rank of the finally formed transfer matrix.
Specifically, the random network code deciphering success rate calculation formula is as follows:
Figure DEST_PATH_IMAGE006
where c is the minimum probability, q is the size of the encoded symbol field, d is the number of sink nodes,
Figure 273396DEST_PATH_IMAGE004
the number of links of the coding coefficient is randomly selected, the random network coding can play a role in information compression for linearly related information sources, in addition, the random network coding is the distributed realization of the network coding, the topological information of the whole network is not required to be known in advance, the random network coding is particularly suitable for a topological structure dynamic change or a large-scale network, and for the network with network nodes and link failures, the random network coding can utilize the residual capacity of the whole network to obtain the optimal capacity of the network, so that the robustness of multicast transmission is improved.
It should be noted that the collecting of the data required by the pre-partition includes the following gestures:
partitioning according to the parameter names and storing different parameter data in a partition mode according to the parameter names;
and secondly, partitioning according to time, and storing the same data content in a partition mode according to the modified time.
As a further improvement of the technical scheme, the distributed storage comprises file-level storage and file block-level storage, wherein the file-level storage is used for storing files with small data, the file block-level storage is used for cutting a large file into a plurality of small files to be stored in a centralized manner, and the phenomenon that the files are large in data and are damaged due to blocking during subsequent file searching is avoided.
In addition, the binary sorting tree optimal adaptation algorithm comprises the following steps:
firstly, forming idle partitions into a binary sequencing tree;
secondly, searching idle partitions meeting regulation in sequence according to the property of the binary ordering tree, recombining the binary ordering tree after distribution to ensure that the structure of the binary ordering tree is not damaged, improving the efficiency problem of the existing optimal adaptation algorithm in the searching process, connecting all idle partition blocks in a tree structure according to the property of the binary ordering tree by adopting the optimal adaptation algorithm of the binary ordering tree, when allocating memory for a process, comparing from a root node according to the property of the binary ordering tree according to a request, adopting recursive search until the idle partition blocks meeting the conditions are found, then distributing, in the optimal adaptation algorithm adopting the binary ordering tree, if the capacities of two memory blocks are equal, inserting the left sub-tree of a node of a memory block inserted first according to an inserting sequence, and existing 6 memory partition blocks with the ordering sequence of R3, R5, R1, R6, R2 and R5, R4, the capacity is R1 < R2 < R3 < R4 < R5 < R6, after a series of insertion operations are carried out according to the algorithm, a binary ordering tree can be generated, the algorithm adopts the binary ordering tree to realize the management, organization and query of the memory free blocks, the distribution speed of the free blocks is greatly improved, and therefore the running speed of the process and the throughput of the system are improved.
The foregoing shows and describes the general principles, essential features, and advantages of the invention. It will be understood by those skilled in the art that the present invention is not limited to the embodiments described above, and the preferred embodiments of the present invention are described in the above embodiments and the description, and are not intended to limit the present invention. The scope of the invention is defined by the appended claims and equivalents thereof.

Claims (8)

1. A data storage method for partitioned storage according to data size is characterized by comprising the following steps:
slicing the data transmission and filtering useless data;
the information exchange technology of the fusion routing and the encoding is used for carrying out linear or nonlinear processing on the information received on each channel on each node in the network and then forwarding the information to a lower node so as to improve the speed of data transmission;
the received original data is expanded and coded into a plurality of coded data blocks, the coded data blocks are distributed to different data storage nodes to store data in a distributed mode, when a user accesses the data, the data receiving nodes are used for obtaining the corresponding coded data blocks from the data storage nodes, and the original data is restored through calculation;
the coded data block allocation comprises the steps of: executing a rule algorithm, collecting data required by the pre-partition area, and storing the data on each node in a rapid and balanced manner; providing database access service, so that different types of data are stored in different partitions; receiving data for distributed storage, and storing the data in an idle area in a distributed manner by adopting a binary sorting tree optimal adaptation algorithm;
recording the time and content of data modification, and storing the content and time of data backup so as to view the document content in different modification time periods in the following;
the data backup comprises the following steps: storing the content of each time the data is changed; recording the time of changing data storage; comparing the stored multi-time modified contents, comparing the original data with the first modified contents, and comparing the first modified contents with the second modified contents, so that the data multi-time modified contents can be viewed more intuitively;
and searching the data document according to the modified content and the recorded time.
2. The data storage method according to claim 1, wherein: the slicing process comprises the following steps:
collecting information stored by data;
analyzing the stored data to know the size of the data;
cutting the analyzed data into small data;
different types of data are processed in a standardized format.
3. The data storage method according to claim 2, wherein: the collecting of the data storage information comprises the following steps:
acquiring metadata in a data life cycle, organizing the metadata, and writing the metadata into a database;
collecting data configuration conditions for subsequent partition storage;
and classifying the data of the same type.
4. The data storage method according to claim 1, wherein: the information exchange technology adopts a distributed erasure code technology, and the distributed erasure code technology comprises the following postures:
attitude one, random network coding, wherein the coefficient of the network node for operating the information is randomly selected;
and secondly, determining the network code, and calculating the network code through a polynomial time algorithm.
5. The data storage method according to claim 4, wherein: the random network code decoding success rate calculation formula is as follows:
Figure 349285DEST_PATH_IMAGE002
where c is the minimum probability, q is the size of the encoded symbol field, d is the number of sink nodes,
Figure DEST_PATH_IMAGE004
the number of links for which the coding coefficients are chosen randomly.
6. The data storage method according to claim 1, wherein: the acquisition of the data required by the pre-partition comprises the following gestures:
partitioning according to the parameter names and storing different parameter data in a partition mode according to the parameter names;
and secondly, partitioning according to time, and storing the same data content in a partition mode according to the modified time.
7. The data storage method according to claim 1, wherein: the distributed storage comprises file level storage and file block level storage.
8. The data storage method according to claim 1, wherein: the binary sorting tree optimal adaptation algorithm comprises the following steps:
firstly, forming idle partitions into a binary sequencing tree;
secondly, searching idle partitions meeting the adjustment in sequence according to the property of the binary tree, and recombining the binary tree after distribution.
CN202110385952.XA 2021-04-12 2021-04-12 Data storage method for partitioned storage according to data size Active CN112799607B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202110385952.XA CN112799607B (en) 2021-04-12 2021-04-12 Data storage method for partitioned storage according to data size

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202110385952.XA CN112799607B (en) 2021-04-12 2021-04-12 Data storage method for partitioned storage according to data size

Publications (2)

Publication Number Publication Date
CN112799607A true CN112799607A (en) 2021-05-14
CN112799607B CN112799607B (en) 2021-06-22

Family

ID=75816645

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202110385952.XA Active CN112799607B (en) 2021-04-12 2021-04-12 Data storage method for partitioned storage according to data size

Country Status (1)

Country Link
CN (1) CN112799607B (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113993163A (en) * 2021-10-26 2022-01-28 新华三信息安全技术有限公司 Service processing method and device

Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101840377A (en) * 2010-05-13 2010-09-22 上海交通大学 Data storage method based on RS (Reed-Solomon) erasure codes
CN106844060A (en) * 2017-03-10 2017-06-13 华中科技大学 The correcting and eleting codes archiving method and system of a kind of task based access control Load-aware
US20170364416A1 (en) * 2012-11-12 2017-12-21 Secured2 Corporation Systems and methods of transmitting data
JP2018156447A (en) * 2017-03-17 2018-10-04 日本電気株式会社 Storage array device, recovery method, and recovery program
CN110018783A (en) * 2018-01-09 2019-07-16 阿里巴巴集团控股有限公司 A kind of date storage method, apparatus and system
CN110531936A (en) * 2019-08-29 2019-12-03 西安交通大学 The crop type storage organization and method of distributed correcting and eleting codes mixing storage based on multi storage
CN110968255A (en) * 2018-09-29 2020-04-07 阿里巴巴集团控股有限公司 Data processing method, data processing device, storage medium and processor
CN112416941A (en) * 2020-11-30 2021-02-26 肖玉连 Block chain-based rapid data retrieval method and system

Patent Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101840377A (en) * 2010-05-13 2010-09-22 上海交通大学 Data storage method based on RS (Reed-Solomon) erasure codes
US20170364416A1 (en) * 2012-11-12 2017-12-21 Secured2 Corporation Systems and methods of transmitting data
CN106844060A (en) * 2017-03-10 2017-06-13 华中科技大学 The correcting and eleting codes archiving method and system of a kind of task based access control Load-aware
JP2018156447A (en) * 2017-03-17 2018-10-04 日本電気株式会社 Storage array device, recovery method, and recovery program
CN110018783A (en) * 2018-01-09 2019-07-16 阿里巴巴集团控股有限公司 A kind of date storage method, apparatus and system
CN110968255A (en) * 2018-09-29 2020-04-07 阿里巴巴集团控股有限公司 Data processing method, data processing device, storage medium and processor
CN110531936A (en) * 2019-08-29 2019-12-03 西安交通大学 The crop type storage organization and method of distributed correcting and eleting codes mixing storage based on multi storage
CN112416941A (en) * 2020-11-30 2021-02-26 肖玉连 Block chain-based rapid data retrieval method and system

Non-Patent Citations (3)

* Cited by examiner, † Cited by third party
Title
BRIJESH KUMAR RAI: "Adaptive erasure code based distributed storage systems", 《IEEE XPLORE》 *
李子天 等: "一种面向数据可用性和存储可靠性动态要求的自适应纠删码存储策略设计", 《小型微型计算机系统》 *
盛秀杰 等: "PetroV分布式数据存储与分析框架设计", 《石油地球物理勘探》 *

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113993163A (en) * 2021-10-26 2022-01-28 新华三信息安全技术有限公司 Service processing method and device
CN113993163B (en) * 2021-10-26 2023-07-25 新华三信息安全技术有限公司 Service processing method and device

Also Published As

Publication number Publication date
CN112799607B (en) 2021-06-22

Similar Documents

Publication Publication Date Title
CN110413611B (en) Data storage and query method and device
CN111552687B (en) Time sequence data storage method, query method, device, equipment and storage medium
US10620830B2 (en) Reconciling volumelets in volume cohorts
CN106874348B (en) File storage and index method and device and file reading method
CN108595664B (en) Agricultural data monitoring method in hadoop environment
CN102521386A (en) Method for grouping space metadata based on cluster storage
CN110825733B (en) Multi-sampling-stream-oriented time series data management method and system
CN107330094B (en) Bloom filter tree structure for dynamically storing key value pairs and key value pair storage method
CN113535732B (en) Verifiable query optimization method for reputation-behavior correlation-oriented double-block chain
CN112380383B (en) Fault-tolerant indexing method for real-time video stream data
CN112799607B (en) Data storage method for partitioned storage according to data size
CN111813840B (en) Data processing method, equipment and storage medium
CN113656397A (en) Index construction and query method and device for time series data
CN115858460A (en) File storage method and device, file reading method and device, medium and equipment
CN110018997B (en) Mass small file storage optimization method based on HDFS
CN111708894A (en) Knowledge graph creating method
CN111291085B (en) Hierarchical interest matching method, hierarchical interest matching device, computer equipment and storage medium
US20060015516A1 (en) Method and apparatus for adding supplemental information to PATRICIA tries
CN115495462A (en) Batch data updating method and device, electronic equipment and readable storage medium
CN105830063B (en) Coordinating small volumes in volume groups
Tasoulis et al. Novel approaches to unsupervised clustering through k-windows algorithm
Chazapis et al. Replica-aware, multi-dimensional range queries in distributed hash tables
CN113763099A (en) Data searching method, device, equipment and storage medium
CN105159966A (en) Method and apparatus for creating directory entity and directory entity processing system
CN110321349B (en) Self-adaptive data merging and storing method for data origin system

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant