CN111124295A - Agricultural data storage processing system and method based on ternary influence factor - Google Patents

Agricultural data storage processing system and method based on ternary influence factor Download PDF

Info

Publication number
CN111124295A
CN111124295A CN201911270374.4A CN201911270374A CN111124295A CN 111124295 A CN111124295 A CN 111124295A CN 201911270374 A CN201911270374 A CN 201911270374A CN 111124295 A CN111124295 A CN 111124295A
Authority
CN
China
Prior art keywords
data block
data
frequency
references
reading
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN201911270374.4A
Other languages
Chinese (zh)
Other versions
CN111124295B (en
Inventor
高燕
耿微
唐聃
曾琼
岳希
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Chengdu University of Information Technology
Original Assignee
Chengdu University of Information Technology
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Chengdu University of Information Technology filed Critical Chengdu University of Information Technology
Priority to CN201911270374.4A priority Critical patent/CN111124295B/en
Publication of CN111124295A publication Critical patent/CN111124295A/en
Application granted granted Critical
Publication of CN111124295B publication Critical patent/CN111124295B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601—Interfaces specially adapted for storage systems
    • G06F3/0602—Interfaces specially adapted for storage systems specifically adapted to achieve a particular effect
    • G06F3/0604—Improving or facilitating administration, e.g. storage management
    • G06F3/0607—Improving or facilitating administration, e.g. storage management by facilitating the process of upgrading existing storage systems, e.g. for improving compatibility between host and storage device
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F11/00—Error detection; Error correction; Monitoring
    • G06F11/07—Responding to the occurrence of a fault, e.g. fault tolerance
    • G06F11/08—Error detection or correction by redundancy in data representation, e.g. by using checking codes
    • G06F11/10—Adding special bits or symbols to the coded information, e.g. parity check, casting out 9's or 11's
    • G06F11/1008—Adding special bits or symbols to the coded information, e.g. parity check, casting out 9's or 11's in individual solid state devices
    • G06F11/1048—Adding special bits or symbols to the coded information, e.g. parity check, casting out 9's or 11's in individual solid state devices using arrangements adapted for a specific error detection or correction feature
    • G06F11/1056—Updating check bits on partial write, i.e. read/modify/write
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601—Interfaces specially adapted for storage systems
    • G06F3/0602—Interfaces specially adapted for storage systems specifically adapted to achieve a particular effect
    • G06F3/0608—Saving storage space on storage systems
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601—Interfaces specially adapted for storage systems
    • G06F3/0628—Interfaces specially adapted for storage systems making use of a particular technique
    • G06F3/0638—Organizing or formatting or addressing of data
    • G06F3/064—Management of blocks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601—Interfaces specially adapted for storage systems
    • G06F3/0628—Interfaces specially adapted for storage systems making use of a particular technique
    • G06F3/0638—Organizing or formatting or addressing of data
    • G06F3/0643—Management of files
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q50/00—Systems or methods specially adapted for specific business sectors, e.g. utilities or tourism
    • G06Q50/02—Agriculture; Fishing; Mining
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601—Interfaces specially adapted for storage systems
    • G06F3/0668—Interfaces specially adapted for storage systems adopting a particular infrastructure
    • G06F3/0671—In-line storage system
    • G06F3/0673—Single storage device

Abstract

The invention belongs to the technical field of data storage, and discloses an agricultural data storage processing system and method based on a ternary influence factor, wherein for a data block stored for the first time, the reference number of a file to the data block is 1, and according to the time locality principle of program access, the data block which is being accessed is likely to be accessed again in the near future; for the stored data block, dynamically updating the cold and hot degree of the calculated data block also needs to comprehensively consider three influence factors: the method comprises the steps that the number of references of a file to data blocks, the reading and writing frequency of the data blocks and a time interval period are set, different initial weights are set for the number of references and the reading frequency, and the weights dynamically change along with the time interval period. The invention can access the data with higher heat degree faster, and simultaneously adopts an erasure code strategy to carry out redundant storage on the cold data, thereby saving the storage space of the system. The number of references of a file to a data block indicates the number of times it is referenced by one or more files.

Description

Agricultural data storage processing system and method based on ternary influence factor
Technical Field
The invention belongs to the technical field of data storage, and particularly relates to an agricultural data storage processing system and method based on a ternary influence factor.
Background
Currently, the closest prior art: agricultural informatization's development has driven the demand of agricultural data management, to the various types of agricultural data of sensor information acquisition module collection, temperature data, humidity data, soil data, meteorological data and all kinds of data, need carry out the piecemeal to data to and the accurate cold and hot degree to the data block calculates, selects suitable storage strategy to save, effectively manages agricultural data, provides valuable guide and promotion effect for agricultural production and work. The performance of replica policy storage is high, but the storage overhead is also high. Erasure code strategies have the advantage of low redundancy and high disk utilization, but the cost of data recovery and data updating is very high. Therefore, in practical applications, it is desirable to store hot data using a copy policy and cold data using an erasure code policy. The hot data refers to data with high access frequency, that is, data with high user access frequency, and therefore a copy strategy with high storage performance is suitable for being adopted. The cold data is accessed less frequently, i.e., the user has less access to such data, such as agricultural system backup data. How to accurately classify data so that different storage schemes are employed for different types of data.
The existing method provides that two strategies, namely a copy strategy and an erasure code strategy, are simultaneously applied to carry out redundant storage on data during storage, the data are classified according to the size of a file, the erasure code strategy is adopted for a larger file, and the copy strategy is adopted for a smaller file. And determining the cold and hot degree of the data blocks by using the reference number of the file to the data blocks, judging the data blocks with higher reference number as hot data, performing redundant storage by adopting a copy strategy, judging the data blocks with lower reference number as cold data, and performing redundant storage by adopting an erasure code strategy. The cold and hot degree of the data block is judged by simultaneously using the reference number of the file to the data block and the reading and writing frequency of the data. The specific method comprises the steps that the reference number of the data blocks stored for the first time is 1, if the read frequency or the write frequency of the data blocks is high, the data blocks are judged to be hot data, and a copy strategy is adopted for redundant storage, otherwise, if the read frequency and the write frequency are low, the data blocks are judged to be cold data, and an erasure code strategy is adopted for redundant storage. Setting a reasonable threshold value for the reference number of the data blocks of the stored data blocks according to the system condition, when the reference number of the data blocks exceeds the threshold value, judging the data blocks to be hot data by the system, and performing redundant storage by adopting a copy strategy, and when the reference number of the data blocks does not reach the threshold value, judging the data blocks to be cold data by the system, and storing the data blocks by adopting an erasure code strategy.
The prior art has the following disadvantages: the first method has two drawbacks in classifying data according to the size of a file, and the first method is to store data in units of data blocks rather than in units of an entire file. Second, some large files may be accessed and updated frequently, and taking erasure coding policies for large files if one considers the file size on a single side may result in very large data recovery and update overhead. Therefore, a more rational approach to data classification is needed. The second method has obvious defects in determining the storage scheme of the file according to the reference number of the file to the data block. Firstly, the number of references of some data blocks is small, but the access frequency is high, and the importance of the data blocks is not negligible; second, for the first stored data block with a reference number of 1, the system determines that the data block is cold. Therefore, it is not reasonable to classify the data by considering only the number of references of the file to the data, and the determination criterion is too single. The third scheme is greatly improved compared with the former two data classification methods, but is still not comprehensive enough, the influence factor of time is not considered, the meaning of the reference quantity before one year and the reference quantity in one week are different, so that the real access frequency of a user to data cannot be accurately reflected only by considering the reference quantity and the read-write frequency, and the time factor is also required to be comprehensively considered to judge the cold and hot degree of the data block.
In summary, the problems of the prior art are as follows:
(1) the existing method only classifies data according to the size of a file, stores the data in a storage system by taking a data block as a unit, and does not store the data by taking the whole file as a unit; some large files may be accessed and updated frequently, and taking erasure coding strategies for large files if the file size is considered unilaterally may result in very large data recovery and update overhead.
(2) The existing method only determines that the storage scheme of the file has less reference number of data blocks according to the reference number of the file to the data blocks, but has higher access frequency and non-negligible importance; for the first stored data block with a reference number of 1, the system determines that the data block is cold. Therefore, the data is classified only by considering the number of references of the file to the data, and the determination criterion is too single.
(3) Compared with the former two data classification methods, the existing scheme is greatly improved, but the existing scheme is still not comprehensive enough, the influence factor of time is not considered, the meaning of the reference quantity before one year and the reference quantity in one week are different, so that the real access frequency of a user to data cannot be accurately reflected only by considering the reference quantity and the read-write frequency, and the time factor is also required to be comprehensively considered to judge the cold and hot degree of a data block.
The difficulty of solving the technical problems is as follows: the method solves the problems that firstly, the reference quantity, the read-write frequency, the latest updating time and the current time interval need to be recorded respectively, the cold and hot degrees of each data block are dynamically calculated and stored, and partial system resources need to be occupied when the cold and hot degrees of the data blocks are calculated.
The significance of solving the technical problems is as follows: if the cold and hot degrees of the data blocks are judged to be not accurate enough only by considering the reference quantity and the read-write frequency, the data access frequency of a user can not be reflected really, the cold and hot degrees of the data blocks are judged by comprehensively considering the time factor, the cold and hot degrees of one data block can be calculated accurately by dynamically changing the reference quantity and the read-write frequency according to the time interval period, and then the adaptive storage strategy is selected for storage, so that the access efficiency of the system can be effectively improved, and the storage space can be reasonably utilized.
Disclosure of Invention
Aiming at the problems in the prior art, the invention provides an agricultural data storage processing system and method based on a ternary influence factor.
The invention is realized in such a way that various agricultural data storage processing methods based on the ternary influence factors comprise the following steps:
step one, for a data block stored for the first time, the reference number of a file to the data block is 1, and according to the time locality principle of program access, the data block being accessed is likely to be accessed again in the near future;
step two, for the stored data block, dynamically updating and calculating the cold and hot degree of the data block also needs to comprehensively consider three influence factors: the method comprises the steps that the number of references of a file to data blocks, the reading and writing frequency of the data blocks and a time interval period are set, different initial weights are set for the number of references and the reading and writing frequency, and the weights dynamically change along with the time interval period.
Further, the first step calculates the cold and hot degree of the data block by comprehensively considering three influence factors: the method comprises the steps that the number of references of a file to data blocks, the reading and writing frequency of the data blocks and a time interval period are set, different initial weights are set for the number of references and the reading and writing frequency, and the weights dynamically change along with the time interval period; the weights for the newly deposited data block are set to: the read-write frequency of the data block is weighted most, and the number of references of the file to the data block is weighted second.
Further, the first step of the method for processing the data block stored for the first time includes:
first step, endowing data block with unique ID number and recording reference number C of data block ID1, the number of references is compared with C in the systemavgComparing to obtain a relative value of the reference amount;
Figure BDA0002312936010000041
secondly, counting the reading frequency and the writing frequency of the data blocks, and calculating the average value of the reading frequency and the average value of the writing frequency of all the data blocks, wherein the formula for calculating the average value of the reading frequency and the writing frequency is as follows:
Figure BDA0002312936010000042
Figure BDA0002312936010000043
Riis the read frequency, W, of the data blockiIs the write frequency, R, of the data blockavgIs the average of the read frequencies of all data blocks, WavgIs the average of all data block write frequencies;
comparing the read frequency of each data block with the average read frequency to obtain a relative value of the read frequency, comparing the write frequency of each data block with the average write frequency to obtain a relative value of the write frequency, wherein the formula for obtaining the relative values of the read frequency and the write frequency is as follows:
Figure BDA0002312936010000044
Figure BDA0002312936010000045
Figure BDA0002312936010000051
is the relative value of the data block read frequency,
Figure BDA0002312936010000052
is the relative value of the data block write frequency;
and fourthly, calculating the influence of the time interval period by adopting a time decay function of Newton's cooling law:
Figure BDA0002312936010000053
Figure BDA0002312936010000054
Figure BDA0002312936010000055
is the time-varying weight of the read-write frequency of the ith data block,
Figure BDA0002312936010000056
is a time-varying weight of the number of references of the ith data block, N1Is tiWeight when equal to 0, N2Is TiWeight when equal to 0, λ1And λ2Is the attenuation coefficient, tiIs the time interval between the latest change time of the ith data block reading frequency and the current time, TiIs the time interval between the latest change time of the reference quantity of the ith data block and the current time; for the stored data block, dynamically updating the cold and hot degree of the calculated data block also needs to comprehensively consider three influence factors: the method comprises the steps that the number of references of a file to data blocks, the reading and writing frequency of the data blocks and a time interval period are set, different initial weights are set for the number of references and the reading and writing frequency, and the weights dynamically change along with the time interval period; unlike storing a new block of data, there is a difference in the initial weight settings: initial weight N of number of references of file to data block2Maximum, initial weight N of read-write frequency of data block1Secondly, performing the following steps;
fifthly, setting different weights for the reference number and the read-write frequency of the file to the data blocks, the read-write frequency of the data blocks and the time interval period, and dynamically changing according to the respective time interval period; the weights for the newly deposited data block are set to: the read-write frequency weight of the data block is the maximum, and the weight of the reference number of the file to the data block is the second time;
sixthly, calculating a cold and hot degree formula of the ith data block as follows:
Figure BDA0002312936010000057
Figure BDA0002312936010000058
is the heat value of the ith data block, WcIs a weight of the number of references of a file to a data block, WrwIs the weight of the read-write frequency of the data block;
step seven, respectively comparing the cold and hot degree of each data block with the average hot degree of all the data blocks, if the hot degree of one data block is greater than the threshold value, judging the data block as hot data, and performing redundant storage by adopting a copy strategy; if the hot degree of a data block is less than the threshold value, the data block is judged to be cold data, and redundancy storage is carried out by adopting an erasure code strategy.
Furthermore, the reference quantity weight of the second-step file to the data block is the largest, and the read-write frequency of the data block is the second order.
Further, the second step includes, for the stored data block, the steps of:
the method comprises the following steps of firstly, counting the reference number of each data block, calculating the average value of the reference numbers of all the data blocks, comparing the reference number of each data block with the average reference number, and solving a reference number relative value, wherein the formula of the average value of the reference numbers and the reference number relative value of each data block is as follows:
Figure BDA0002312936010000061
Figure BDA0002312936010000062
Ciis the number of references to the ith data block, CavgIs the average of the number of references to all data blocks,
Figure BDA0002312936010000066
is the reference number relative value of the ith data block;
secondly, counting the reading frequency and the writing frequency of the data blocks, and calculating the average value of the reading frequency and the average value of the writing frequency of all the data blocks, wherein the formula for calculating the average value of the reading frequency and the writing frequency is as follows:
Figure BDA0002312936010000063
Figure BDA0002312936010000064
Riis the read frequency, W, of the ith data blockiIs the write frequency, R, of the ith data blockavgIs the average of the read frequencies of all data blocks, WavgIs the average of all data block write frequencies;
comparing the read frequency of each data block with the average read frequency to obtain a relative value of the read frequency, comparing the write frequency of each data block with the average write frequency to obtain a relative value of the write frequency, wherein the formula for obtaining the relative values of the read frequency and the write frequency is as follows:
Figure BDA0002312936010000065
Figure BDA0002312936010000071
Figure BDA0002312936010000072
is the relative value of the ith data block read frequency,
Figure BDA0002312936010000073
is the relative value of the writing frequency of the ith data block;
and fourthly, calculating the influence of the time interval period by adopting a time decay function of Newton's cooling law:
Figure BDA0002312936010000074
Figure BDA0002312936010000075
Figure BDA0002312936010000076
is the time-varying weight of the read-write frequency of the ith data block,
Figure BDA0002312936010000077
is a time-varying weight of the number of references of the ith data block, N1Is tiWeight when equal to 0, N2Is TiWeight when equal to 0, λ1And λ2Is the attenuation coefficient, tiIs the time interval between the latest change time of the ith data block reading frequency and the current time, TiIs the time interval between the latest change time of the reference quantity of the ith data block and the current time; for the stored data block, dynamically updating the cold and hot degree of the calculated data block also needs to comprehensively consider three influence factors: the method comprises the steps that the number of references of a file to data blocks, the reading and writing frequency of the data blocks and a time interval period are set, different initial weights are set for the number of references and the reading and writing frequency, and the weights dynamically change along with the time interval period; unlike storing a new block of data, there is a difference in the initial weight settings: initial weight N of number of references of file to data block2Maximum, initial weight N of read-write frequency of data block1Secondly, performing the following steps;
fifthly, calculating the cold and hot degree of the ith data block according to the values
Figure BDA0002312936010000078
Figure BDA0002312936010000079
Sixthly, calculating the average heat of all the storage data blocks, setting the average heat as a threshold value for judging cold and hot data, wherein the formula for calculating the average cold and hot degree is as follows:
Figure BDA00023129360100000710
wherein Havg_degreeRefers to the average heat of the system data block,
Figure BDA00023129360100000711
is the heat of the ith data block and n represents the number of data blocks in the system. Respectively comparing the cold and hot degree of each data block with the average hot degree of the data block, so as to dynamically adjust the redundancy strategy of the data block; if the heat degree of a data block is greater than the threshold value, judging that the data block is hot data, and performing redundant storage by adopting a copy strategy; if the hot degree of a data block is less than the threshold value, the data block is judged to be cold data, and redundancy storage is carried out by adopting an erasure code strategy.
The invention also aims to provide a ternary influence factor-based agricultural data storage and processing system for implementing the ternary influence factor-based agricultural data storage and processing method, and the ternary influence factor-based agricultural data storage and processing system comprises:
the first-time storage data block processing module is used for referring to the data block stored for the first time, the reference number of the file to the data block is 1, and the data block which is being accessed is likely to be accessed again in the near future according to the time locality principle of program access;
the stored data block processing module is used for dynamically updating and calculating the cold and hot degree of the data block for the stored data block, and three influence factors need to be comprehensively considered: the file comprises the reference number of the file to the data blocks, the reading and writing frequency of the data blocks and a time interval period, different initial weights are set for the reference number and the reading and writing frequency, and the weights dynamically change along with the time interval period.
The invention also aims to provide an information data processing terminal for realizing the agricultural data storage processing method based on the ternary influence factor.
Another object of the present invention is to provide a computer-readable storage medium, which includes instructions that, when executed on a computer, cause the computer to execute the agricultural data storage processing method based on ternary influence factors.
The invention also aims to provide a data storage processing terminal applying the agricultural data storage processing method based on the ternary influence factor.
The invention also aims to provide an agricultural information processing system applying the agricultural data storage processing method based on the ternary influence factor.
In summary, the advantages and positive effects of the invention are: the invention provides a method for accurately calculating and distinguishing cold and hot degrees of data blocks based on ternary influence factors, aiming at the defect that in the existing agricultural data storage management, the cold and hot data are distinguished only by judging from two indexes, namely file reference quantity and read-write frequency, and the consideration of time factors is lacked. The agricultural data storage scheme based on the ternary influence factors comprehensively considers three influence factors: the number of references of the file to the data blocks, the reading and writing frequency of the data blocks and the time interval period accurately divide and store the collected agricultural data in a cold and hot mode, and the storage efficiency and the access speed of the data are improved. The data with higher heat can be accessed more quickly, and meanwhile, the cold data is subjected to redundant storage by adopting an erasure code strategy, so that the storage space of the system is saved. The number of references of a file to a data block indicates the number of times it is referenced by one or more files.
Drawings
Fig. 1 is a flowchart of an agricultural data storage processing method based on a ternary influence factor according to an embodiment of the present invention.
FIG. 2 is a schematic structural diagram of an agricultural data storage and processing system based on a ternary influence factor according to an embodiment of the present invention;
in the figure: 1. a first storage data block processing module; 2. the data block processing module has been stored.
Fig. 3 is a schematic diagram of the number of references of a file to a data block according to an embodiment of the present invention.
Fig. 4 is a flow chart of calculating and storing the cooling and heating degree of a data block according to an embodiment of the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention more apparent, the present invention is further described in detail with reference to the following embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.
Aiming at the problems in the prior art, the invention provides an agricultural data storage processing system and method based on a ternary influence factor, and the invention is described in detail below with reference to the accompanying drawings.
As shown in fig. 1, the agricultural data storage processing method based on the ternary influence factor provided by the embodiment of the present invention includes the following steps:
s101: for the data block stored for the first time, the reference number of the file to the data block is 1, and according to the time locality principle of program access, the data block being accessed is likely to be accessed again in the near future;
s102: for the stored data block, dynamically updating the cold and hot degree of the calculated data block also needs to comprehensively consider three influence factors: the method comprises the steps that the number of references of a file to data blocks, the reading and writing frequency of the data blocks and a time interval period are set, different initial weights are set for the number of references and the reading and writing frequency, and the weights dynamically change along with the time interval period.
As shown in fig. 2, the agricultural data storage and processing system based on ternary influence factors provided by the embodiment of the present invention includes:
the first-time storage data block processing module 1 is used for storing the data block for the first time, the number of references of the file to the data block is 1, and the data block being accessed is likely to be accessed again in the near future according to the time locality principle of program access.
The stored data block processing module 2 is used for dynamically updating and calculating the cold and hot degree of the data block for the stored data block, and three influence factors need to be comprehensively considered: the method comprises the steps that the number of references of a file to data blocks, the reading and writing frequency of the data blocks and a time interval period are set, different initial weights are set for the number of references and the reading and writing frequency, and the weights dynamically change along with the time interval period.
The technical solution of the present invention is further described with reference to the following specific examples.
The agricultural data storage processing method based on the ternary influence factor provided by the embodiment of the invention comprises the following steps: when a CPU accesses a register, either access data or access instructions tend to be gathered in a contiguous piece of storage, which is called the locality principle. Thus, for a data block stored for the first time, the number of references to the data block by the file is 1, and it is likely that the data block being accessed will be accessed again in the near future according to the temporal locality principle of program access. When calculating the cold and hot degree of the data block, three influence factors need to be comprehensively considered: the method comprises the steps that the number of references of a file to data blocks, the reading and writing frequency of the data blocks and a time interval period are set, different initial weights are set for the number of references and the reading and writing frequency, and the weights dynamically change along with the time interval period; the weights for the newly deposited data block are set to: the read-write frequency of the data block is weighted most, and the number of references of the file to the data block is weighted second.
For the stored data block, dynamically updating the cold and hot degree of the calculated data block also needs to comprehensively consider three influence factors: the method comprises the steps that the number of references of a file to data blocks, the reading and writing frequency of the data blocks and a time interval period are set, different initial weights are set for the number of references and the reading and writing frequency, and the weights dynamically change along with the time interval period. Unlike storing a new block of data, there is a difference in the weight setting of the impact factors: the file has the largest weight of the number of references to the data blocks, and the read-write frequency of the data blocks is the second order.
For a data block that has been stored:
the method comprises the following steps of firstly, counting the reference number of each data block, calculating the average value of the reference numbers of all the data blocks, comparing the reference number of each data block with the average reference number, and solving a reference number relative value, wherein the formula of the average value of the reference numbers and the reference number relative value of each data block is as follows:
Figure BDA0002312936010000111
Figure BDA0002312936010000112
Ciis the number of references to the ith data block, CavgIs the average of the number of references to all data blocks,
Figure BDA0002312936010000113
is the reference number relative value of the ith data block;
secondly, counting the reading frequency and the writing frequency of the data blocks, and calculating the average value of the reading frequency and the average value of the writing frequency of all the data blocks, wherein the formula for calculating the average value of the reading frequency and the writing frequency is as follows:
Figure BDA0002312936010000114
Figure BDA0002312936010000115
Riis the read frequency, W, of the ith data blockiIs the write frequency, R, of the ith data blockavgIs the average of the read frequencies of all data blocks, WavgIs the average of all data block write frequencies;
comparing the read frequency of each data block with the average read frequency to obtain a relative value of the read frequency, comparing the write frequency of each data block with the average write frequency to obtain a relative value of the write frequency, wherein the formula for obtaining the relative values of the read frequency and the write frequency is as follows:
Figure BDA0002312936010000116
Figure BDA0002312936010000117
Figure BDA0002312936010000118
is the relative value of the ith data block read frequency,
Figure BDA0002312936010000119
is the relative value of the writing frequency of the ith data block;
and fourthly, calculating the influence of the time interval period by adopting a time decay function of Newton's cooling law:
Figure BDA0002312936010000121
Figure BDA0002312936010000122
Figure BDA0002312936010000123
is the time-varying weight of the read-write frequency of the ith data block,
Figure BDA0002312936010000124
is a time-varying weight of the number of references of the ith data block, N1Is tiWeight when equal to 0, N2Is TiWeight when equal to 0, λ1And λ2Is the attenuation coefficient, tiIs the time interval between the latest change time of the ith data block reading frequency and the current time, TiIs the time interval between the latest change time of the reference quantity of the ith data block and the current time; for the stored data block, dynamically updating the cold and hot degree of the calculated data block also needs to comprehensively consider three influence factors: number of references of file to data block,Setting different initial weights for the reference quantity and the read-write frequency of the data block and the time interval period, wherein the weights dynamically change along with the time interval period. Unlike storing a new block of data, there is a difference in the initial weight settings: initial weight N of number of references of file to data block2Maximum, initial weight N of read-write frequency of data block1Next, the method is described.
Fifthly, calculating the cold and hot degree of the ith data block according to the values
Figure BDA0002312936010000125
Figure BDA0002312936010000126
Sixthly, calculating the average heat of all the storage data blocks, setting the average heat as a threshold value for judging cold and hot data, wherein the formula for calculating the average cold and hot degree is as follows:
Figure BDA0002312936010000127
wherein Havg_degreeRefers to the average heat of the system data block,
Figure BDA0002312936010000128
is the heat of the ith data block and n represents the number of data blocks in the system. The data block redundancy strategy can be dynamically adjusted by respectively comparing the cold and hot degree of each data block with the average hot degree of the data block. If the heat degree of a data block is greater than the threshold value, judging that the data block is hot data, and performing redundant storage by adopting a copy strategy; if the hot degree of a data block is less than the threshold value, the data block is judged to be cold data, and redundancy storage is carried out by adopting an erasure code strategy.
For the first stored data block:
first step, endowing data block with unique ID number and recording reference number C of data block ID1, the number of references is compared with C in the systemavgMake comparison, get reference toThe relative value of the amount;
Figure BDA0002312936010000131
secondly, counting the reading frequency and the writing frequency of the data blocks, and calculating the average value of the reading frequency and the average value of the writing frequency of all the data blocks, wherein the formula for calculating the average value of the reading frequency and the writing frequency is as follows:
Figure BDA0002312936010000132
Figure BDA0002312936010000133
Riis the read frequency, W, of the data blockiIs the write frequency, R, of the data blockavgIs the average of the read frequencies of all data blocks, WavgIs the average of all data block write frequencies;
comparing the read frequency of each data block with the average read frequency to obtain a relative value of the read frequency, comparing the write frequency of each data block with the average write frequency to obtain a relative value of the write frequency, wherein the formula for obtaining the relative values of the read frequency and the write frequency is as follows:
Figure BDA0002312936010000134
Figure BDA0002312936010000135
Figure BDA0002312936010000136
is the relative value of the data block read frequency,
Figure BDA0002312936010000137
is the relative value of the data block write frequency;
and fourthly, calculating the influence of the time interval period by adopting a time decay function of Newton's cooling law:
Figure BDA0002312936010000138
Figure BDA0002312936010000139
Figure BDA00023129360100001310
is the time-varying weight of the read-write frequency of the ith data block,
Figure BDA00023129360100001311
is a time-varying weight of the number of references of the ith data block, N1Is tiWeight when equal to 0, N2Is TiWeight when equal to 0, λ1And λ2Is the attenuation coefficient, tiIs the time interval between the latest change time of the ith data block reading frequency and the current time, TiIs the time interval between the latest change time of the reference quantity of the ith data block and the current time; for the stored data block, dynamically updating the cold and hot degree of the calculated data block also needs to comprehensively consider three influence factors: the method comprises the steps that the number of references of a file to data blocks, the reading and writing frequency of the data blocks and a time interval period are set, different initial weights are set for the number of references and the reading and writing frequency, and the weights dynamically change along with the time interval period. Unlike storing a new block of data, there is a difference in the initial weight settings: initial weight N of number of references of file to data block2Maximum, initial weight N of read-write frequency of data block1Next, the method is described.
Fifthly, setting different weights for three influence factors of the number of references of the file to the data blocks, the reading and writing frequency of the data blocks and the time interval period; the weights for the newly deposited data block are set to: the read-write frequency weight of the data block is the maximum, and the weight of the reference number of the file to the data block is the second time;
sixthly, calculating a cold and hot degree formula of the ith data block as follows:
Figure BDA0002312936010000141
Figure BDA0002312936010000142
is the heat value of the ith data block, WcIs a weight of the number of references of a file to a data block, WrwIs a weight of the read and write frequency of the data block,
step seven, respectively comparing the cold and hot degree of each data block with the average hot degree of all the data blocks, if the hot degree of one data block is greater than the threshold value, judging the data block as hot data, and performing redundant storage by adopting a copy strategy; if the hot degree of a data block is less than the threshold value, the data block is judged to be cold data, and redundancy storage is carried out by adopting an erasure code strategy.
Referring to fig. 3, the meaning of the reference number of the file to the data block is visually shown. After the file is divided into a plurality of data blocks, 9 different data blocks are provided, wherein the number of the data blocks 2, 5 and 6 times of the three file references is 3. The quote amount may reflect to some extent the cool and hot level to which it is accessed by the user.
The technical solution of the present invention is further described with reference to the following specific examples.
Example 1: (calculating the Heat and average Heat of already stored data blocks)
The method comprises the following steps of firstly, counting the reference number of each data block, calculating the average value of the reference numbers of all the data blocks, and setting the reference numbers of three data blocks in the system as follows: c1＝5、C2＝6、C3Calculating to obtain an average value C of reference numbers of all data blocks according to (1) as 7avg＝6；
Secondly, comparing the reference number of each data block with the average reference number, calculating a relative value of the reference number, and respectively obtaining the relative reference number of each data block according to the (2) computerValue of
Figure BDA0002312936010000151
Thirdly, counting the reading frequency and the writing frequency of the data blocks, calculating the average value of the reading frequency and the average value of the writing frequency of all the data blocks, and setting the reading frequencies of the three data blocks in the system as follows: r1＝5、R2＝6、R3Write frequency W for 71＝3、W2＝4、W3The read-write frequency of the newly added data block is R respectively as 54＝6，W4Calculating according to (3) and (4) to obtain a reading frequency average value Ravg＝6，Wavg＝4；
Fourthly, respectively comparing the reading frequency of each data block with the average reading frequency to obtain relative values of the reading frequency, then respectively comparing the writing frequency of each data block with the average writing frequency to obtain relative values of the writing frequency, and according to the steps (5) and (6), calculating the relative values of the reading frequency of the four data blocks to be respectively
Figure BDA0002312936010000152
The writing frequency of the four data blocks is respectively
Figure BDA0002312936010000153
Fifthly, calculating the influence of the time interval period by adopting a time attenuation function of Newton's cooling law, and setting the weight N which initially changes along with time1＝3，N 25, attenuation coefficient λ1＝0.15，λ2The read-write frequency of the three data blocks is modified last time and the difference between the read-write frequency of the three data blocks and the current time is t1＝8、t2＝5、t310, the reference number of the three data blocks is modified last time and the difference between the reference number of the three data blocks and the current time is T1＝6、T2＝8、T3The weight of the read frequency of the three data blocks over time, calculated according to (7), is 12 (units are: days): 3e-1.2、3e-0.9、3e-1.5(ii) a Calculating three data blocks according to (8)The weights of the number of references over time are: 5e-0.6、5e-0.8、5e-1.2；
And sixthly, calculating the cold and hot degree of the ith data block according to the step (9) as follows:
Figure BDA0002312936010000154
Figure BDA0002312936010000161
and seventhly, calculating the average cold and hot degree of all the data blocks according to the step (10) to obtain: havg_degree＝3.9257；
Eighth step, respectively cooling and heating degree of each data block and average heating degree H of all data blocksavg_degreeComparing, if the heat degree of a data block is greater than the threshold value, judging the data block as hot data, and performing redundant storage by adopting a copy strategy; if the hot degree of a data block is less than the threshold value, the data block is judged to be cold data, and redundancy storage is carried out by adopting an erasure code strategy.
Example 2: (calculating the Heat of the first stored data Block)
First step, endowing data block with unique ID number and recording reference number C of data blockID＝1；
Secondly, calculating the reference number of the new data block and (1) to obtain an average value C of the reference numbers of all the data blocksavgBy comparison with 6, the computer obtains the relative value of the reference number of the new data block according to (2)
Figure BDA0002312936010000162
Thirdly, counting the reading frequency and the writing frequency of the data block, comparing the reading frequency and the writing frequency of the new data block with the average reading frequency and the average writing frequency of the system respectively to obtain relative values of the reading frequency and the writing frequency, wherein the reading frequency and the writing frequency of the newly added data block are R respectively4＝6，W 44, the average reading frequency of the system is 6, the average writing frequency of the system is 4, and the relative value of the reading frequency is calculated according to (5) and (6)
Figure BDA0002312936010000163
Relative value of write frequency
Figure BDA0002312936010000164
Fourthly, calculating the influence of the time interval period by adopting a time attenuation function of Newton's cooling law, and setting the weight N which initially changes along with the time1＝5，N 22, attenuation coefficient λ1＝0.15，λ2When the read-write frequency of the new data block is modified last time, the difference between the read-write frequency of the new data block and the current time is ti6 (unit is day), newly adding data block TiAnd (5) calculating the weight of the change of the reading frequency of the new data block along with the time according to (7) as follows: 5e-0.9(ii) a The weight of the change of the reference number of the new data block with time calculated according to (8) is as follows: 2;
and fifthly, calculating the cold and hot degree of the newly stored data block as follows:
Figure BDA0002312936010000165
sixthly, the cold and hot degrees of the new data block and the average hot degree H of all the data blocksavg_degreeAnd 3.9257, if the new data block is hot and is greater than the threshold, determining that the data block is hot, and performing redundant storage by adopting a copy strategy.
The three influence factors, the reference number of the file to the data block, the reading and writing frequency of the data block and the time interval period are comprehensively considered, the cold and hot degree of the data block is dynamically updated and calculated, more accurate calculation is carried out, a copy strategy is selected for the hot data block to carry out redundant storage, and the access response time of the hot data block can be effectively reduced.
It should be noted that the embodiments of the present invention can be realized by hardware, software, or a combination of software and hardware. Hardware portions may be implemented using dedicated logic and software portions may be stored in memory and executed by a suitable instruction execution system, such as a microprocessor or specially designed hardware. Those skilled in the art will appreciate that the apparatus and methods described above may be implemented using computer executable instructions and/or embodied in processor control code, such code being provided on a carrier medium such as a disk, CD-or DVD-ROM, programmable memory such as read only memory (firmware), or a data carrier such as an optical or electronic signal carrier, for example. The apparatus and its modules of the present invention may be implemented by hardware circuits such as very large scale integrated circuits or gate arrays, semiconductors such as logic chips, transistors, or programmable hardware devices such as field programmable gate arrays, programmable logic devices, etc., or by software executed by various types of processors, or by a combination of hardware circuits and software, e.g., firmware.
The above description is only for the purpose of illustrating the preferred embodiments of the present invention and is not to be construed as limiting the invention, and any modifications, equivalents and improvements made within the spirit and principle of the present invention are intended to be included within the scope of the present invention.

Claims (10)

1. A ternary influence factor-based agricultural data storage processing method is characterized by comprising the following steps:
step one, for a data block stored for the first time, the reference number of a file to the data block is 1, and according to the time locality principle of program access, the data block being accessed is likely to be accessed again in the near future;
step two, for the stored data block, dynamically updating and calculating the cold and hot degree of the data block also needs to comprehensively consider three influence factors: the method comprises the steps that the number of references of a file to data blocks, the reading and writing frequency of the data blocks and a time interval period are set, different initial weights are set for the number of references and the reading frequency, and the weights dynamically change along with the time interval period.
2. The agricultural data storage processing method based on ternary influence factors as claimed in claim 1, wherein the first step of calculating the cold and hot degree of the data block comprehensively considers three influence factors: the method comprises the steps that the number of references of a file to data blocks, the reading and writing frequency of the data blocks and a time interval period are set, different initial weights are set for the number of references and the reading frequency, and the weights dynamically change along with the time interval period; the weights for the newly deposited data block are set to: the read-write frequency of the data block is weighted most, and the number of references of the file to the data block is weighted second.
3. The method for storing and processing agricultural data based on ternary influence factors according to claim 1, wherein the step one comprises the following steps for processing the data blocks stored for the first time:
first step, endowing data block with unique ID number and recording reference number C of data blockID1, the number of references is compared with C in the systemavgComparing to obtain a relative value of the reference amount;
Figure FDA0002312932000000011
secondly, counting the reading frequency and the writing frequency of the data blocks, and calculating the average value of the reading frequency and the average value of the writing frequency of all the data blocks, wherein the formula for calculating the average value of the reading frequency and the writing frequency is as follows:
Figure FDA0002312932000000012
Figure FDA0002312932000000013
Riis the read frequency, W, of the data blockiIs the write frequency, R, of the data blockavgIs the average of the read frequencies of all data blocks, WavgIs the average of all data block write frequencies;
comparing the read frequency of each data block with the average read frequency to obtain a relative value of the read frequency, comparing the write frequency of each data block with the average write frequency to obtain a relative value of the write frequency, wherein the formula for obtaining the relative values of the read frequency and the write frequency is as follows:
Figure FDA0002312932000000021
Figure FDA0002312932000000022
Figure FDA0002312932000000023
is the relative value of the data block read frequency,
Figure FDA0002312932000000024
is the relative value of the data block write frequency;
the fourth step, the weights of the quote quantity and the reading frequency are calculated according to the variation of the time interval period by using the time decay function of Newton's law of cooling:
Figure FDA0002312932000000025
Figure FDA0002312932000000026
Figure FDA0002312932000000027
is the time-varying weight of the read-write frequency of the ith data block,
Figure FDA0002312932000000028
is a time-varying weight of the number of references of the ith data block, N1Is tiWeight when equal to 0, N2Is TiWeight when equal to 0, λ1And λ2Is the attenuation coefficient, tiIs the time interval between the latest change time of the ith data block reading frequency and the current time, TiIs the time interval between the latest change time of the reference quantity of the ith data block and the current time; for the stored data block, dynamically updating the cold and hot degree of the calculated data block also needs to comprehensively consider three influence factors: the method comprises the steps that the number of references of a file to data blocks, the reading and writing frequency of the data blocks and a time interval period are set, different initial weights are set for the number of references and the reading and writing frequency, and the weights dynamically change along with the time interval period; unlike storing a new block of data, there is a difference in the initial weight settings: initial weight N of number of references of file to data block2Maximum, initial weight N of read-write frequency of data block1Secondly, performing the following steps;
fifthly, setting different weights for the reference quantity of the file to the data blocks and the reading and writing frequency of the data blocks, wherein the weights change along with the time interval period; the weights for the newly deposited data block are set to: the read-write frequency weight of the data block is maximum, and the weight of the reference number of the file to the data block is minimum;
sixthly, calculating a cold and hot degree formula of the ith data block as follows:
Figure FDA0002312932000000031
Figure FDA0002312932000000032
is the heat value of the ith data block, WcIs a weight of the number of references of a file to a data block, WrwIs the weight of the read-write frequency of the data block;
step seven, respectively comparing the cold and hot degree of each data block with the average hot degree of all the data blocks, if the hot degree of one data block is greater than the threshold value, judging the data block as hot data, and performing redundant storage by adopting a copy strategy; if the hot degree of a data block is less than the threshold value, the data block is judged to be cold data, and redundancy storage is carried out by adopting an erasure code strategy.
4. The agricultural data storage processing method based on the ternary influence factor as claimed in claim 1, wherein the reference quantity weight of the second-step file to the data block is the largest, and the read-write frequency of the data block is the second order.
5. The method for storing and processing agricultural data based on ternary influence factors according to claim 4, wherein the second step comprises the steps of processing the stored data blocks:
the method comprises the following steps of firstly, counting the reference number of each data block, calculating the average value of the reference numbers of all the data blocks, comparing the reference number of each data block with the average reference number, and solving a reference number relative value, wherein the formula of the average value of the reference numbers and the reference number relative value of each data block is as follows:
Figure FDA0002312932000000033
Figure FDA0002312932000000034
Ciis the number of references to the ith data block, CavgIs the average of the number of references to all data blocks,
Figure FDA0002312932000000035
is the reference number relative value of the ith data block;
secondly, counting the reading frequency and the writing frequency of the data blocks, and calculating the average value of the reading frequency and the average value of the writing frequency of all the data blocks, wherein the formula for calculating the average value of the reading frequency and the writing frequency is as follows:
Figure FDA0002312932000000041
Figure FDA0002312932000000042
Riis the reading of the ith data blockFrequency, WiIs the write frequency, R, of the ith data blockavgIs the average of the read frequencies of all data blocks, WavgIs the average of all data block write frequencies;
comparing the read frequency of each data block with the average read frequency to obtain a relative value of the read frequency, comparing the write frequency of each data block with the average write frequency to obtain a relative value of the write frequency, wherein the formula for obtaining the relative values of the read frequency and the write frequency is as follows:
Figure FDA0002312932000000043
Figure FDA0002312932000000044
Figure FDA0002312932000000045
is the relative value of the ith data block read frequency,
Figure FDA0002312932000000046
is the relative value of the writing frequency of the ith data block;
the weights of the fourth step, the quote number and the reading frequency are calculated as a function of the time decay function of newton's law of cooling according to the variation of the period of the time interval:
Figure FDA0002312932000000047
Figure FDA0002312932000000048
Figure FDA0002312932000000049
is the time-varying weight of the read-write frequency of the ith data block,
Figure FDA00023129320000000410
is a time-varying weight of the number of references of the ith data block, N1Is tiWeight when equal to 0, N2Is TiWeight when equal to 0, λ1And λ2Is the attenuation coefficient, tiIs the time interval between the latest change time of the ith data block reading frequency and the current time, TiIs the time interval between the latest change time of the reference quantity of the ith data block and the current time; for the stored data block, dynamically updating the cold and hot degree of the calculated data block also needs to comprehensively consider three influence factors: the method comprises the steps that the number of references of a file to data blocks, the reading and writing frequency of the data blocks and a time interval period are set, different initial weights are set for the number of references and the reading and writing frequency, and the weights dynamically change along with the time interval period; unlike storing a new block of data, there is a difference in the initial weight settings: initial weight N of number of references of file to data block2Maximum, initial weight N of read-write frequency of data block1Secondly, performing the following steps;
fifthly, calculating the cold and hot degree of the ith data block according to the values
Figure FDA0002312932000000051
Figure FDA0002312932000000052
Sixthly, calculating the average heat of all the storage data blocks, setting the average heat as a threshold value for judging cold and hot data, wherein the formula for calculating the average cold and hot degree is as follows:
Figure FDA0002312932000000053
wherein Havg_degreeRefers to the average heat of the system data block,
Figure FDA0002312932000000054
is the heat of the ith data block, and n represents the number of data blocks in the system; respectively comparing the cold and hot degree of each data block with the average hot degree of the data block, so as to dynamically adjust the redundancy strategy of the data block; if the heat degree of a data block is greater than the threshold value, judging that the data block is hot data, and performing redundant storage by adopting a copy strategy; if the hot degree of a data block is less than the threshold value, the data block is judged to be cold data, and redundancy storage is carried out by adopting an erasure code strategy.
6. The agricultural data storage and processing system based on the ternary influence factor, which implements the agricultural data storage and processing method based on the ternary influence factor according to any one of claims 1 to 5, is characterized by comprising:
the first-time storage data block processing module is used for referring to the data block stored for the first time, the reference number of the file to the data block is 1, and the data block which is being accessed is likely to be accessed again in the near future according to the time locality principle of program access;
the stored data block processing module is used for dynamically updating and calculating the cold and hot degree of the data block for the stored data block, and three influence factors need to be comprehensively considered: the method comprises the steps that the number of references of a file to data blocks, the reading and writing frequency of the data blocks and a time interval period are set, different initial weights are set for the number of references and the reading and writing frequency, and the weights dynamically change along with the time interval period.
7. An information data processing terminal for realizing the agricultural data storage processing method based on the ternary influence factor according to any one of claims 1 to 5.
8. A computer-readable storage medium comprising instructions which, when executed on a computer, cause the computer to perform the method for agricultural data storage processing based on ternary influence factors as claimed in any one of claims 1 to 5.
9. A data storage processing terminal applying the agricultural data storage processing method based on the ternary influence factor according to any one of claims 1 to 5.
10. An agricultural informatization processing system applying the agricultural data storage and processing method based on the ternary influence factor according to any one of claims 1 to 5.
CN201911270374.4A 2019-12-11 2019-12-11 Agricultural data storage processing system and method based on ternary influence factor Active CN111124295B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201911270374.4A CN111124295B (en) 2019-12-11 2019-12-11 Agricultural data storage processing system and method based on ternary influence factor

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201911270374.4A CN111124295B (en) 2019-12-11 2019-12-11 Agricultural data storage processing system and method based on ternary influence factor

Publications (2)

Publication Number Publication Date
CN111124295A true CN111124295A (en) 2020-05-08
CN111124295B CN111124295B (en) 2021-06-04

Family

ID=70499195

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201911270374.4A Active CN111124295B (en) 2019-12-11 2019-12-11 Agricultural data storage processing system and method based on ternary influence factor

Country Status (1)

Country Link
CN (1) CN111124295B (en)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2023030227A1 (en) * 2021-08-31 2023-03-09 华为技术有限公司 Data processing method, apparatus and system
CN116204138A (en) * 2023-05-05 2023-06-02 成都三合力通科技有限公司 Efficient storage system and method based on hierarchical storage

Citations (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103838860A (en) * 2014-03-19 2014-06-04 华存数据信息技术有限公司 File storing system based on dynamic transcript strategy and storage method of file storing system
US9153247B1 (en) * 2015-01-21 2015-10-06 HGST Netherlands B.V. Far field interference mitigation by relative frequency ordering
CN105279166A (en) * 2014-06-20 2016-01-27 中国电信股份有限公司 File management method and system
CN106355031A (en) * 2016-09-21 2017-01-25 大连大学 Data value degree calculation method based on analytic hierarchy process
CN107102954A (en) * 2017-04-27 2017-08-29 华中科技大学 A kind of solid-state storage grading management method and system based on failure probability
CN108073472A (en) * 2017-12-12 2018-05-25 华中科技大学 A kind of memory correcting and eleting codes location mode perceived based on temperature
CN109144417A (en) * 2018-08-16 2019-01-04 广州杰赛科技股份有限公司 A kind of cloud storage method, system and equipment
CN109634779A (en) * 2018-11-19 2019-04-16 杭州电子科技大学 A kind of data incremental backup method based on data temperature self study
CN109669641A (en) * 2018-12-24 2019-04-23 深圳忆联信息系统有限公司 Reduce the data deposit method and device of the SSD bit error rate
CN109783016A (en) * 2018-12-25 2019-05-21 西安交通大学 A kind of elastic various dimensions redundancy approach in distributed memory system

Patent Citations (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103838860A (en) * 2014-03-19 2014-06-04 华存数据信息技术有限公司 File storing system based on dynamic transcript strategy and storage method of file storing system
CN105279166A (en) * 2014-06-20 2016-01-27 中国电信股份有限公司 File management method and system
US9153247B1 (en) * 2015-01-21 2015-10-06 HGST Netherlands B.V. Far field interference mitigation by relative frequency ordering
CN106355031A (en) * 2016-09-21 2017-01-25 大连大学 Data value degree calculation method based on analytic hierarchy process
CN107102954A (en) * 2017-04-27 2017-08-29 华中科技大学 A kind of solid-state storage grading management method and system based on failure probability
CN108073472A (en) * 2017-12-12 2018-05-25 华中科技大学 A kind of memory correcting and eleting codes location mode perceived based on temperature
CN109144417A (en) * 2018-08-16 2019-01-04 广州杰赛科技股份有限公司 A kind of cloud storage method, system and equipment
CN109634779A (en) * 2018-11-19 2019-04-16 杭州电子科技大学 A kind of data incremental backup method based on data temperature self study
CN109669641A (en) * 2018-12-24 2019-04-23 深圳忆联信息系统有限公司 Reduce the data deposit method and device of the SSD bit error rate
CN109783016A (en) * 2018-12-25 2019-05-21 西安交通大学 A kind of elastic various dimensions redundancy approach in distributed memory system

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
周阳: ""云存储中冷热数据的混合冗余方法研究"", 《移动通信》 *

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2023030227A1 (en) * 2021-08-31 2023-03-09 华为技术有限公司 Data processing method, apparatus and system
CN116204138A (en) * 2023-05-05 2023-06-02 成都三合力通科技有限公司 Efficient storage system and method based on hierarchical storage
CN116204138B (en) * 2023-05-05 2023-07-07 成都三合力通科技有限公司 Efficient storage system and method based on hierarchical storage

Also Published As

Publication number Publication date
CN111124295B (en) 2021-06-04

Similar Documents

Publication Publication Date Title
WO2021008220A1 (en) Systems and methods for data storage system
US9146688B2 (en) Advanced groomer for storage array
US8082388B2 (en) Optimizing operational requests of logical volumes
CN101788995B (en) Hotspot data identification method and device
CN111124295B (en) Agricultural data storage processing system and method based on ternary influence factor
CN110888600B (en) Buffer area management method for NAND flash memory
US10949115B2 (en) Flash memory polling
US11041763B2 (en) Adaptive throttling
WO2019100463A1 (en) Flash memory performance enhancement method, apparatus, and device, and storage medium
CN108399110A (en) Soft Inform ation management in high power capacity solid state drive
CN112786080B (en) Method and device for reducing error rate of flash memory and solid state disk
US20220165337A1 (en) Systems and methods for determining change of read threshold voltage
CN116700620A (en) Data storage method, device, equipment and storage medium
CN109669641B (en) Data storage method and device for reducing SSD bit error rate
CN114385092B (en) Erasing method of solid state disk flash memory array and solid state disk main control chip
CN115509454A (en) Solid state disk data writing method and device, electronic equipment and medium
CN111210858B (en) Method and system for relieving write interference of phase change memory
GB2514571A (en) Cache allocation in a computerized system
CN107357535A (en) A kind of method and device for determining the cold and hot grade of data
CN110580932B (en) Memory cell quality measurement method applied to wear leveling
US9846653B2 (en) Performing write operations on main memory
CN108984117B (en) Data reading and writing method, medium and equipment
KR20140041408A (en) Selecting storage locations for storing data based on storage location attributes and data usage statistics
US20240134727A1 (en) Storage device write-read error reduction system
CN112286843B (en) System and method for data storage system

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant