CN106095333B - Data storage system, data storage method and data storage center - Google Patents
Data storage system, data storage method and data storage center Download PDFInfo
- Publication number
- CN106095333B CN106095333B CN201610392193.9A CN201610392193A CN106095333B CN 106095333 B CN106095333 B CN 106095333B CN 201610392193 A CN201610392193 A CN 201610392193A CN 106095333 B CN106095333 B CN 106095333B
- Authority
- CN
- China
- Prior art keywords
- storage
- data
- target
- level
- target data
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0602—Interfaces specially adapted for storage systems specifically adapted to achieve a particular effect
- G06F3/0604—Improving or facilitating administration, e.g. storage management
- G06F3/0607—Improving or facilitating administration, e.g. storage management by facilitating the process of upgrading existing storage systems, e.g. for improving compatibility between host and storage device
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/22—Indexing; Data structures therefor; Storage structures
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0628—Interfaces specially adapted for storage systems making use of a particular technique
- G06F3/0629—Configuration or reconfiguration of storage systems
- G06F3/0634—Configuration or reconfiguration of storage systems by changing the state or mode of one or more devices
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0668—Interfaces specially adapted for storage systems adopting a particular infrastructure
- G06F3/0671—In-line storage system
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Human Computer Interaction (AREA)
- Software Systems (AREA)
- Data Mining & Analysis (AREA)
- Databases & Information Systems (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The embodiment of the invention provides a data storage system, a data storage method and a data storage center, wherein the system comprises the following components: n storage units with sequentially increased storage levels; the n memory cells include: the storage level is higher than the first storage level, and the storage level is sequentially increased by n-1 storage units; wherein, each storage unit stores data with a storage level not higher than the storage level of the storage unit; the data storage center is used for receiving a processing request for target data sent by the client; determining a target storage level corresponding to the target data; searching a target storage unit capable of processing the processing request from storage units with storage levels not lower than the target storage level; and triggering the target storage unit to process the target data according to the processing request. The embodiment of the invention can improve the data processing efficiency under the condition of considering the requirement of high storage capacity of mass data.
Description
Technical Field
The present invention relates to the field of storage technologies, and in particular, to a data storage system, a data storage method, and a data storage center.
Background
Along with the explosive growth of data, how the data storage system effectively performs query, writing and other processes on massive data becomes a research focus in the field of data storage.
The existing data storage mode mainly comprises the steps of storing mass data specially by arranging a large database, wherein the data storage mode can meet the requirement of high storage capacity of the mass data by the large database, but the efficiency of inquiring and writing certain data from the large database is greatly reduced, the data processing efficiency is sacrificed, and the data processing efficiency such as QPS (query per second) and other data inquiring and writing indexes are sacrificed; therefore, how to realize the storage and processing of the mass data under the condition of considering the data processing efficiency and the high storage requirement of the mass data becomes a problem which needs to be solved by the technicians in the field.
Disclosure of Invention
In view of the above, the embodiments of the present invention provide a data storage system, a data storage method, and a data storage center, so as to improve data processing efficiency under the condition of considering the requirement of high storage capacity of mass data.
In order to achieve the above object, the embodiment of the present invention provides the following technical solutions:
a data storage system, comprising: n storage units with sequentially increased storage levels; the n memory cells include: the storage level is higher than the first storage level, and the storage level is sequentially increased by n-1 storage units; wherein, each storage unit stores data with a storage level not higher than the storage level of the storage unit;
the data storage center is used for receiving a processing request for target data sent by the client; determining a target storage level corresponding to the target data; searching a target storage unit capable of processing the processing request from storage units with storage levels not lower than the target storage level; and triggering the target storage unit to process the target data according to the processing request.
The embodiment of the invention also provides a data storage method, which comprises the following steps:
receiving a processing request for target data sent by a client;
determining a target storage level corresponding to the target data;
searching a target storage unit capable of processing the processing request from storage units with storage levels not lower than the target storage level;
And triggering the target storage unit to process the target data according to the processing request.
The embodiment of the invention also provides a data storage center, which comprises:
the processing request receiving module is used for receiving a processing request aiming at target data sent by a client;
the target storage level determining module is used for determining a target storage level corresponding to the target data;
the target storage unit searching module is used for searching a target storage unit capable of processing the processing request from the storage units with the storage level not lower than the target storage level;
and the processing triggering module is used for triggering the target storage unit to process the target data according to the processing request.
Based on the above technical scheme, in the embodiment of the invention, the data storage center only selects the target storage unit from the storage units with the storage level not lower than the target storage level for processing the target data according to the processing request of the target data; therefore, the embodiment of the invention can adapt corresponding target storage units to process target data to be processed with different storage levels, reduce the range of the storage units capable of processing the target data, simultaneously enable the data access speed of the searched target storage units to be adapted to the storage levels of the target data to be processed as much as possible, and improve the processing efficiency of the target data.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings that are required to be used in the embodiments or the description of the prior art will be briefly described below, and it is obvious that the drawings in the following description are only embodiments of the present invention, and that other drawings can be obtained according to the provided drawings without inventive effort for a person skilled in the art.
FIG. 1 is a schematic diagram of a data storage system according to an embodiment of the present invention;
FIG. 2 is a schematic diagram illustrating an architecture application of a data storage system according to an embodiment of the present invention;
FIG. 3 is a flowchart of a data storage method according to an embodiment of the present invention;
FIG. 4 is a flowchart of a method for determining a target storage level of target data according to an embodiment of the present invention;
FIG. 5 is a flowchart of another method for determining a target storage level of target data according to an embodiment of the present invention;
FIG. 6 is a flowchart of another method for determining a target storage level of target data according to an embodiment of the present invention;
FIG. 7 is another flowchart of a data storage method according to an embodiment of the present invention;
FIG. 8 is a schematic diagram illustrating the processing of a data query according to an embodiment of the present invention;
FIG. 9 is a schematic diagram of another processing of a data query according to an embodiment of the present invention;
FIG. 10 is a flowchart illustrating a data storage method according to an embodiment of the present invention;
FIG. 11 is a block diagram of a data storage center according to an embodiment of the present invention;
FIG. 12 is a block diagram illustrating a target storage level determination module according to an embodiment of the present invention;
FIG. 13 is another block diagram illustrating a data storage center according to an embodiment of the present invention;
FIG. 14 is a block diagram of a data storage center according to an embodiment of the present invention;
fig. 15 is a schematic hardware configuration diagram of a data storage center according to an embodiment of the present invention.
Detailed Description
The following description of the embodiments of the present invention will be made clearly and completely with reference to the accompanying drawings, in which it is apparent that the embodiments described are only some embodiments of the present invention, but not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.
Fig. 1 is a schematic structural diagram of a data storage system according to an embodiment of the present invention, and referring to fig. 1, the data storage system may include: n storage units, wherein storage levels can be defined for each storage unit, and the storage levels of the n storage units are sequentially increased, namely, the storage level of the i storage unit is higher than the storage level of the i-1 storage unit, for example, the storage level of the first storage unit is a first storage level, the storage level of the second storage unit is a second storage level, and the second storage level is higher than the first storage level, and so on; n can be an integer greater than or equal to 2, and the specific value can be determined according to the actual data storage requirement;
Returning to FIG. 1, the n storage units may be further divided into a data storage center 10 of a first storage level, and n-1 storage units 20 of which storage levels are higher than the first storage level and storage levels are sequentially increased;
in an embodiment of the present invention, each of the n-1 storage units 20 may be considered as a storage system in the form of a database, and the data storage center 10 may be considered as a data processing device having data processing capability and buffering capability;
in the embodiment of the invention, the lower the storage level is, the faster the data access speed of the corresponding storage unit is, and the smaller the data storage capacity is; it can be understood that, the data storage center executes the data processing logic for the embodiment of the present invention, and interacts with each storage unit of the n-1 storage units 20 to schedule data, and has the data caching capability, so that the data access speed of the data storage center is the fastest, but the data storage capacity may be the smallest;
in n-1 storage units, according to the increasing sequence of the storage levels, the data access speed of the storage units is gradually reduced, and the data storage capacity is gradually increased until the data access speed of the nth storage unit of the highest storage level is lowest, and the data storage capacity is highest;
Optionally, each storage unit may store data with a storage level not higher than its own storage level; that is, the ith storage unit may store data having a storage level equal to or less than the ith storage level, e.g., the ith storage unit may store data of the ith storage level, and possibly data of storage levels 1 to i;
optionally, the storage unit with the highest storage level has the highest data storage capacity, so that in order to achieve the requirement of persistent storage of massive data, the embodiment of the invention can store all data in the storage unit with the highest storage level.
Architecture application of the data storage system shown in fig. 1 as shown in fig. 2, referring to fig. 2, the data storage system in fig. 2 is divided into three storage level structures, including: a first storage level data storage center 10, a second storage level key-value cache storage unit 21, and a third storage level relational database 22;
the key-value cache storage unit is a data storage unit which stores data in a memory in a key-value form, inputs a key character string and inquires and returns a value; namely, a key-value cache memory unit stores data of a second storage level in a key form;
The relational database is a database which adopts a relational model to organize data, the data is stored in a relational table form, a structured query statement SQL is input, and a data set is returned; namely, the data of the third storage level is stored in the form of a relational table in the relational database; furthermore, because the data storage capacity of the relational database is highest, in order to realize the requirement of persistent storage of mass data, the relational database can store data of a third storage level and data of a first storage level and a second storage level in a relational table form, namely, all the data can be stored in the relational database.
According to the embodiment of the invention, the storage units with different storage levels are arranged, so that the storage of the data with different storage levels in the data storage system can be realized, and the high storage requirement of mass data is met;
meanwhile, when data are subjected to processing such as writing and inquiring, the embodiment of the invention can receive the processing request for the target data sent by the client through the data storage center, so that the data storage center can determine the target storage level corresponding to the target data, and the target storage unit capable of processing the processing request is searched from the storage units with the storage level not lower than the target storage level, thereby the data storage center can trigger the target storage unit to process the target data according to the processing request;
Because the embodiment of the invention only selects the target storage unit from the storage units with the storage level not lower than the target storage level to process the target data; therefore, the embodiment of the invention can adapt corresponding target storage units to process target data to be processed with different storage levels, reduce the range of the storage units capable of processing the target data, simultaneously enable the data access speed of the searched target storage units to be adapted to the storage levels of the target data to be processed as much as possible, and improve the processing efficiency of the target data.
The embodiment of the invention realizes the processing of data query, writing and the like, mainly by triggering corresponding target storage units through the dispatching of a data storage center, and the data storage method provided by the embodiment of the invention is introduced from the angle of the data storage center.
FIG. 3 is a flowchart of a data storage method according to an embodiment of the present invention, where the method may be based on the data storage system shown in FIG. 1, that is, the data storage system may include: n storage units with sequentially increased storage levels; the n memory cells include: the storage level is higher than the first storage level, and the storage level is sequentially increased by n-1 storage units; wherein, each storage unit stores data with a storage level not higher than the storage level of the storage unit;
The method of FIG. 3 may be applied in a data storage center of a data storage system, and referring to FIG. 3, the method may include:
step S100, receiving a processing request for target data sent by a client;
alternatively, the processing request for the target data may be a query request for the target data, or a write request, etc.;
optionally, the processing request may carry an identifier of the target data.
Step S110, determining a target storage level corresponding to the target data;
optionally, the embodiment of the invention can set a storage level for the data according to the access attribute of the data; the access attribute can represent the access frequency of the data, and can be generally represented by information such as the read frequency, the write frequency and the like of the data;
the principle of setting the storage level for the data is mainly that the data with high access frequency corresponds to the storage unit with high access speed and low capacity, namely, the storage level corresponding to the data with higher access frequency is lower, and the storage level corresponding to the data with lower access frequency is higher;
the data storage center can acquire access attribute information (data representation of access attribute of target data) of the target data after receiving a processing request for the target data, so as to determine the access frequency of the target data based on the access attribute information of the target data, and determine a storage level corresponding to the determined access frequency as a target storage level;
Optionally, the embodiment of the invention can set the storage level corresponding to the higher access frequency to be lower;
specifically, the embodiment of the invention can set the corresponding access frequency ranges of different storage levels, and the lower the corresponding access frequency range of the storage level is, the larger the value of the corresponding access frequency range of the lower storage level is, so that after the access frequency of the target data is determined, the corresponding storage level of the access frequency range of the target data can be determined, and the determined storage level is taken as the corresponding target storage level of the target data;
optionally, the embodiment of the invention can acquire the access attribute information of the target data through the identification of the target data; the access attribute information of each data may be pre-collected and correspond to the collected access attribute information by identification of the data.
Step S120, searching a target storage unit capable of processing the processing request from the storage units with the storage levels not lower than the target storage level;
because each storage unit in the data storage system stores data with a storage level not higher than the storage level of the storage unit, after determining the target storage level of target data, if the target storage level is a query request for the target data, the target data may be stored in the storage unit with the storage level not lower than the target storage level, and the storage unit with the target data stored therein can be searched from the storage unit with the storage level not lower than the target storage level so as to realize the query of the target data;
If the writing request is directed against the target data, the target data is written in the storage units with the storage level not lower than the target storage level, so that the storage units capable of being written in the target data can be searched in the storage units with the storage level not lower than the target storage level, and the writing of the target data is realized;
it can be seen that, no matter the request for querying, writing, etc. of the target data, the data storage center needs to find the target storage unit capable of processing the processing request from the storage units with the storage level not lower than the target storage level of the target data, so as to facilitate the corresponding processing of the target data.
And step 130, triggering the target storage unit to process the target data according to the processing request.
After the target storage unit is determined, the data storage center can process target data through the target storage unit according to the processing request, so that the target data can be processed; if the processing request is a query request, the data storage center can call the target data content corresponding to the identification of the target data from the target storage unit and feed back the target data content to the client; if the processing request is a write request, the data storage center may write the target data into the target storage unit.
The data storage method provided by the embodiment of the invention is based on a data storage system, and the data storage system comprises the following steps: n storage units with sequentially increased storage levels; the n memory cells include: the storage level is higher than the first storage level, and the storage level is sequentially increased by n-1 storage units; wherein, each storage unit stores data with a storage level not higher than the storage level of the storage unit; according to the data storage system, the storage units with different storage levels are arranged, and each storage unit stores data with the storage level not higher than the storage level of the storage unit, so that the data with different storage levels can be stored in the data storage system, and the requirement of high storage capacity of mass data is met;
meanwhile, when the data storage method is executed, the data storage center only selects a target storage unit from storage units with storage levels not lower than the target storage level for data processing aiming at the processing request of the target data; therefore, the embodiment of the invention can adapt corresponding target storage units to process target data to be processed with different storage levels, reduce the range of the storage units capable of processing the target data, simultaneously enable the data access speed of the searched target storage units to be adapted to the storage levels of the target data to be processed as much as possible, and improve the processing efficiency of the target data.
Optionally, fig. 4 shows a flowchart of a method for determining a target storage level of target data according to an embodiment of the present invention, and referring to fig. 4, the flowchart may include:
step 200, acquiring the read frequency and the write frequency of target data;
the read frequency and the write frequency may be an alternative form of accessing the attribute information.
Step S210, determining a read-write ratio value of the target data according to the read frequency and the write frequency, and taking the read-write ratio value as the access frequency of the target data;
alternatively, the read-write ratio value of the target data may be a form of the access frequency of the target data, and the read-write ratio value of the target data may be a read frequency/write frequency.
And step S220, determining the storage level corresponding to the access frequency as a target storage level.
The smaller the read-write proportion value of the target data is, the lower the read frequency of the target data is, the higher the write frequency is, the lower the access frequency of the target data is, correspondingly, the target data is correspondingly in a storage unit with lower data access speed and higher data storage capacity, and therefore the storage level of the target data is higher;
the larger the read-write ratio value of the target data is, the higher the read frequency of the target data is, the lower the write frequency is, the higher the access frequency of the target data is, correspondingly, the target data should be correspondingly stored in a storage unit with higher data access speed and lower data storage capacity, and therefore, the storage level of the target data should be lower.
Optionally, the embodiment of the invention can realize the determination of the target storage level corresponding to the access frequency of the target data after determining the access frequency of the target data by setting the access frequency ranges corresponding to different storage levels;
optionally, fig. 5 shows a flowchart of another method for determining a target storage level of target data according to an embodiment of the present invention, and referring to fig. 5, the flowchart may include:
step S300, obtaining the read frequency and the write frequency of target data;
step S310, determining a read-write proportion value of the target data according to the read frequency and the write frequency;
step S320, taking the storage level corresponding to the read-write ratio value range where the read-write ratio value is located as a target storage level; one storage level corresponds to one read-write ratio value range, and the read-write ratio value ranges corresponding to the storage levels are different.
It should be noted that the read frequency and the write frequency are only an optional representation of the access attribute information of the target data, and the embodiment of the present invention does not exclude other forms of access attribute information, and the specific form of access attribute information may be set according to the actual situation, for example, only the read frequency or the write frequency may be selected as the access attribute information;
The read-write ratio value of the target data is also only an optional form of the access frequency of the target data, and the embodiment of the invention does not exclude other forms of access frequency, and the specific form of access frequency can be set according to actual situations, for example, the write frequency/read frequency, or the square of the read frequency/write frequency, etc. can be used as the access frequency.
Optionally, before determining the target storage level of the target data according to the access attribute information of the target data, the embodiment of the invention can also determine whether the data amount of the target data is too large or not through the data amount of the target data, and a storage unit with enough size is needed to store the target data, so that when determining that the data amount of the target data is too large and the storage unit with enough size is needed to store the target data, the embodiment of the invention determines that the target data needs to correspond to the storage unit corresponding to the data amount of the target data, and further takes the storage level of the storage unit corresponding to the data amount of the target data as the target storage level; meanwhile, when the data volume of the target data is not excessively large, determining the target storage level of the target data based on the access attribute information of the target data;
FIG. 6 is a flowchart of another method for determining a target storage level of target data according to an embodiment of the present invention, and referring to FIG. 6, the method may include:
Step S400, acquiring the data volume, the reading frequency and the writing frequency of target data;
step S410, judging whether the data volume of the target data is higher than a set data volume threshold, if so, executing step S420, otherwise, executing step S430;
step S420, taking a storage level corresponding to the data amount of the target data as a target storage level;
optionally, if the data amount of the target data is determined to be higher than the set data amount threshold, the embodiment of the invention may determine that the data amount of the target data is too large, and may use the storage level of the storage unit corresponding to the data amount as the target storage level; optionally, the implementation of the invention can set the corresponding data volume ranges of the storage units with different storage levels, so that when the data volume of the target data is determined to be higher than the set data volume threshold, the storage level of the storage unit corresponding to the data volume range where the data volume of the target data is located is used as the target storage level;
optionally, in the embodiment of the present invention, when the data amount of the target data is higher than the set data amount threshold, the highest storage level may be directly used as the target storage level of the target data.
Step S430, determining a read-write proportion value of the target data according to the read frequency and the write frequency;
It can be seen that, in the embodiment of the present invention, the data storage center triggers the execution of step S310 in fig. 5 when determining that the data size of the target data is lower than the set data size threshold.
Step S440, taking the storage level corresponding to the read-write proportion value range where the read-write proportion value is located as a target storage level; one storage level corresponds to one read-write ratio value range, and the read-write ratio value ranges corresponding to the storage levels are different.
Taking the data storage system shown in fig. 2 as an example, the embodiment of the invention can set a first access frequency range corresponding to a first storage level corresponding to a data storage center, a second access frequency range corresponding to a second storage level corresponding to a key-value cache storage unit, and a third access frequency range corresponding to a third storage level corresponding to a relational database; and the value of the first access frequency range > the value of the second access frequency range > the value of the third access frequency range;
further, after determining the access frequency of the target data, the embodiment of the invention determines that the first storage level is the target storage level of the target data if the access frequency is in the first access frequency range, determines that the second storage level is the target storage level of the target data if the access frequency is in the second access frequency range, and determines that the third storage level is the target storage level of the target data if the access frequency is in the third access frequency range.
After determining a target storage level of target data, the data storage center can search a target storage unit capable of processing the processing request from storage units with storage levels not lower than the target storage level so as to trigger the target storage unit to process the target data according to the processing request; the following will describe cases for a query request and a write request, respectively.
Fig. 7 is another flowchart of a data storage method according to an embodiment of the present invention, and referring to fig. 7, the method may include:
step S500, receiving a query request for target data sent by a client;
step S510, determining a target storage level corresponding to the target data;
step S520, starting from the storage units corresponding to the target storage level, sequentially inquiring the target data from the storage units which are not lower than the target storage level according to the sequentially-increasing sequence of the storage levels until the storage unit storing the target data is determined, and taking the first determined storage unit storing the target data as a target storage unit;
the embodiment of the invention can inquire whether the target data exists from the storage unit corresponding to the target storage level, if so, the storage unit corresponding to the target storage level is taken as the target storage unit, if not, the storage unit with the higher storage level is inquired whether the target data exists, and the like until the storage unit which is not lower than the target storage level and stores the target data is taken as the target storage unit.
Step S530, retrieving the target data from the target storage unit and feeding back the target data to the client.
After determining the target storage unit, the embodiment of the invention can call the target data from the target storage unit so as to feed back the target data to the client, thereby realizing the inquiry of the client on the target data.
Optionally, if the storage level of the target storage unit of the target data is higher than the target storage level of the target data, the target data is indicated to be not stored in the storage unit corresponding to the target storage level; based on the principle that the storage units store data with the storage level not higher than the storage level of the storage unit, the embodiment of the invention can perform the supplementary storage of the target data on the storage unit corresponding to the target storage level and the storage unit between the storage unit corresponding to the target storage level and the target storage unit, thereby improving the probability of inquiring the target data from the storage unit corresponding to the target storage level next time and improving the speed of inquiring the target data next time;
specifically, if the data storage center judges that the storage level of the target storage unit is higher than the target storage level, determining a transition storage level from the target storage level to the storage level of the target storage unit, and storing the target data in the storage units of the target storage level and each transition storage level respectively;
The transition storage level is a storage level between the target storage level and the storage level of the target storage unit, if the target storage level is a first storage level and the storage level of the target storage unit is a third storage level, the transition storage level may be a second storage level.
Taking the data storage system shown in fig. 2 as an example, the application process of the data storage method provided by the embodiment of the invention may be as follows:
(1) After the data storage center receives the query request of the target data, the target storage level of the target data can be determined;
(21) If the target storage level is the first storage level, judging whether target data is queried in the local cache of the corresponding data storage center, if so, invoking the target data from the local cache of the data storage center and feeding back to the client, and if not, sending a request for querying the target data to a key value cache storage unit of the second storage level by the data storage center;
if the key value cache storage unit inquires the target data, the data storage center retrieves the target data from the key value cache storage unit and feeds the target data back to the client, and meanwhile, the target data is cached in a local cache of the data storage center; if the key value cache storage unit does not inquire the target data, the data storage center sends a request for inquiring the target data to a relational database of a third storage level;
Because the storage level of the relational database is highest, all data are stored in a lasting mode, target data can be inquired in the relational database and fed back to the client; meanwhile, the data storage center can cache target data in a local cache, or respectively cache target data in a local cache and a key value cache storage unit of the data storage center;
(22) If the target storage level is the second storage level, the data storage center sends a request for inquiring the target data to the corresponding key value cache storage unit;
if the key value cache storage unit inquires the target data, the data storage center retrieves the target data from the key value cache storage unit and feeds the target data back to the client, and if the key value cache storage unit does not inquire the target data, the data storage center sends a request for inquiring the target data to a relational database of a third storage level;
inquiring target data in the relational database and feeding back the target data to the client; meanwhile, the data storage center caches target data in the key value cache storage unit;
(23) If the target storage level is the third storage level, the data storage center sends a request for inquiring target data to the corresponding relational database, and the target data is inquired from the relational database and fed back to the client.
In connection with the above application process description, fig. 8 and 9 show a schematic process of data query, which is specifically as follows:
1. the data storage center receives the query request of the target data, checks whether the requested target data uses a local cache (the data storage level < = 1), and if so, goes to 2;
2. the data storage center queries the local cache, if the query is successful, the query is turned to 7, otherwise, the query is turned to 3;
3. the data storage center sends a query request of target data to the KV cache storage unit; if the query is successful, checking whether the target data needs to be stored in a local cache (the data storage level < =1), if so, turning to 6, otherwise, turning to 7; if the query fails to go to 4;
4. the data storage center sends a query request of target data to a relational Database (DB), if the query is successful, the data storage center checks whether the target data needs to be stored in a KV cache storage unit (data storage level < = 2), if so, the data storage center goes to 5, otherwise, the data storage center goes to 7;
5. the data storage center stores the target data into the KV cache storage unit, checks whether the target data needs to be stored into a local cache, and if yes, goes to 6, otherwise goes to 7;
6. the data storage center stores the target data to a local cache;
7. The data storage center returns the query result to the client.
It should be noted that if the target data cannot be queried from the relational database of the highest storage level, it indicates that the target data is not stored in the data storage system, and cannot be queried, and the data storage center can feed back the corresponding result to the client.
According to the embodiment of the invention, the hot spot data with high access frequency can be stored in the corresponding storage unit with higher access speed and lower capacity, so that the overall QPS for data reading is improved; in addition, by persisting all data into the highest storage level storage unit (e.g., relational database), the mass storage, complex structured query, and data persistence problems can be solved.
Fig. 10 is a further flowchart of a data storage method according to an embodiment of the present invention, and referring to fig. 10, the method may include:
step S600, receiving a write request for target data sent by a client;
step S610, determining a target storage level corresponding to the target data;
step S620, taking all the storage units with the storage level not lower than the target storage level as target storage units;
optionally, in the implementation of the present invention, the target data may be written into a storage unit having a storage level greater than or equal to the target storage level, so that all storage units having a storage level not lower than the target storage level may be used as target storage units to write the target data; this way, consistency of the same data between storage units of each storage level can be ensured.
Step S630, storing the target data in a target storage unit.
Taking the data storage system shown in fig. 2 as an example, the application process of the data storage method provided by the embodiment of the invention may be as follows:
after receiving the write request of the target data, the data storage center can determine the target storage level of the target data;
if the target storage level is the first storage level, respectively writing target data into a data storage center corresponding to the first storage level, a key value cache storage unit with the storage level higher than the first storage level and a relational database;
if the target storage level is the second storage level, respectively writing target data into a key value cache storage unit corresponding to the second storage level and a relational database with the storage level higher than the second storage level;
and if the target storage level is the third storage level, writing target data in a relational database corresponding to the third storage level.
Alternatively, the data storage center may trigger the storage unit of the highest storage level to store all data, on the basis that the storage unit of the highest storage level may store all data.
In the embodiment of the present invention, n-1 memory cells other than the data storage center among the n memory cells may include: and a key value cache storage unit of the second storage level and a relational database of the third storage level.
The data storage method provided by the embodiment of the invention can improve the data processing efficiency under the condition of considering the requirement of high storage capacity of mass data.
The data storage center provided by the embodiment of the invention is introduced below, and the data storage center described below can be referred to correspondingly with the data storage method described above.
FIG. 11 is a block diagram illustrating a data storage center according to an embodiment of the present invention, where the data storage center may be located in a data storage system, where the data storage system may include: n storage units with sequentially increased storage levels; the n memory cells include: the data storage center of the first storage level and n-1 storage units with storage levels higher than the first storage level and sequentially increased storage levels; wherein, each storage unit stores data with a storage level not higher than the storage level of the storage unit;
returning to fig. 11, the data storage center may include:
a processing request receiving module 100, configured to receive a processing request for target data sent by a client;
a target storage level determining module 200, configured to determine a target storage level corresponding to the target data;
a target storage unit searching module 300, configured to search a target storage unit capable of processing the processing request from storage units with a storage level not lower than the target storage level;
And the processing triggering module 400 is configured to trigger the target storage unit to process the target data according to the processing request.
Optionally, fig. 12 illustrates an optional structure of the target storage level determining module 200 provided in an embodiment of the present invention, and referring to fig. 12, the target storage level determining module 200 may include:
an access attribute information acquisition unit 210 for acquiring access attribute information of the target data;
an access frequency determining unit 220 configured to determine an access frequency of the target data according to the access attribute information;
and a target storage level corresponding unit 230, configured to determine a storage level corresponding to the access frequency as a target storage level, where a storage level corresponding to a higher access frequency is lower.
Alternatively, the access attribute information obtaining unit 210 may be specifically configured to obtain a read frequency and a write frequency of the target data.
The access frequency determining unit 220 may specifically be configured to determine, according to the read frequency and the write frequency, a read-write ratio value of the target data, and use the read-write ratio value as the access frequency of the target data.
Optionally, the target storage level corresponding unit 230 is specifically configured to use a storage level corresponding to a read-write ratio value range in which the read-write ratio value is located as the target storage level; one storage level corresponds to one read-write ratio value range, and the read-write ratio value ranges corresponding to the storage levels are different.
Optionally, the access attribute information obtaining unit 210 may be further configured to obtain a data amount of the target data when obtaining a read frequency and a write frequency of the target data;
accordingly, fig. 13 is a block diagram illustrating another structure of a data storage center according to the embodiment of the present invention, and in combination with fig. 11 and fig. 13, the data storage center may further include:
the first judgment triggering module 500 is configured to, if it is judged that the data amount of the target data is higher than the set data amount threshold, take a storage level corresponding to the data amount of the target data as a target storage level;
and a second judgment triggering module 600, configured to trigger the access frequency determining unit to determine a read-write ratio value of the target data according to the read frequency and the write frequency if it is judged that the data amount of the target data is lower than a set data amount threshold.
Optionally, in the case of performing a query request on the target data, the processing request receiving module 100 is specifically configured to receive a query request for the target data sent by the client;
the target storage unit searching module 300 is specifically configured to, starting from a storage unit corresponding to the target storage level, sequentially query the target data from the storage units not lower than the target storage level according to the sequentially increasing order of the storage levels until determining that the storage unit storing the target data is reached, and using the first determined storage unit storing the target data as a target storage unit;
The processing triggering module 400 is specifically configured to retrieve the target data from the target storage unit and feed back the target data to the client.
Optionally, fig. 14 is a block diagram illustrating still another structure of a data storage center according to an embodiment of the present invention, and in conjunction with fig. 11 and fig. 14, the data storage center may further include:
and the target data supplementing and storing module 700 is configured to determine a transition storage level from the target storage level to the storage level of the target storage unit if the storage level of the target storage unit is higher than the target storage level, and store the target data in the storage units of the target storage level and each transition storage level, respectively.
In the case of performing a write request on target data, the processing request receiving module 100 is specifically configured to receive a write request for target data sent by a client;
the target storage unit searching module 300 is specifically configured to use all storage units with storage levels not lower than the target storage level as target storage units;
the process trigger module 400 is specifically operable to store the target data in a target storage unit.
Optionally, the data storage center may be further configured to trigger the storage unit of the highest storage level to store all data.
Optionally, the n-1 memory cells may include: and a key value cache storage unit of the second storage level and a relational database of the third storage level.
Fig. 15 is a schematic diagram showing a hardware configuration of a data storage center according to an embodiment of the present invention, and referring to fig. 15, the data storage center may include: a processor 1, a communication interface 2, a memory 3 and a communication bus 4;
wherein the processor 1, the communication interface 2 and the memory 3 complete the communication with each other through the communication bus 4;
alternatively, the communication interface 2 may be an interface of a communication module, such as an interface of a GSM module;
a processor 1 for executing a program;
a memory 3 for storing a program;
the program may include program code including computer operating instructions.
The processor 1 may be a central processing unit CPU or a specific integrated circuit ASIC (Application Specific Integrated Circuit) or one or more integrated circuits configured to implement embodiments of the present invention.
The memory 3 may comprise a high-speed RAM memory or may further comprise a non-volatile memory, such as at least one disk memory.
Wherein, the procedure can be specifically used for:
Receiving a processing request for target data sent by a client;
determining a target storage level corresponding to the target data;
searching a target storage unit capable of processing the processing request from storage units with storage levels not lower than the target storage level;
and triggering the target storage unit to process the target data according to the processing request.
Alternatively, the data storage center may be located in the data storage system described above.
The following describes functions of the data storage system provided by the embodiment of the present invention; the structure of the data storage system provided in the embodiment of the present invention may be as shown in fig. 1, including: n storage units with sequentially increased storage levels; the n memory cells include: the storage level is higher than the first storage level, and the storage level is sequentially increased by n-1 storage units; wherein, each storage unit stores data with a storage level not higher than the storage level of the storage unit;
the data storage center can be used for receiving a processing request for target data sent by the client; determining a target storage level corresponding to the target data; searching a target storage unit capable of processing the processing request from storage units with storage levels not lower than the target storage level; and triggering the target storage unit to process the target data according to the processing request.
Optionally, when determining the target storage level corresponding to the target data, the data storage center may be specifically configured to obtain access attribute information of the target data, determine an access frequency of the target data according to the access attribute information, and determine a storage level corresponding to the access frequency as the target storage level, where the higher the access frequency, the lower the storage level corresponding to the access frequency.
Optionally, the data storage center may be specifically configured to obtain a read frequency and a write frequency of the target data when obtaining access attribute information of the target data;
the data storage center is specifically configured to determine a read-write ratio value of the target data according to the read frequency and the write frequency when determining the access frequency of the target data according to the access attribute information, and take the read-write ratio value as the access frequency of the target data.
Optionally, when determining the storage level corresponding to the access frequency as the target storage level, the data storage center is specifically configured to use the storage level corresponding to the read-write ratio value range in which the read-write ratio value is located as the target storage level; one storage level corresponds to one read-write ratio value range, and the read-write ratio value ranges corresponding to the storage levels are different.
Optionally, the data storage center may further acquire a data amount of the target data when acquiring a read frequency and a write frequency of the target data;
correspondingly, the data storage center is further used for taking the storage level corresponding to the data amount of the target data as a target storage level if the data amount of the target data is judged to be higher than the set data amount threshold; and triggering the step of determining the read-write proportion value of the target data according to the read frequency and the write frequency if the data quantity of the target data is judged to be lower than the set data quantity threshold value.
Optionally, when receiving a processing request for target data sent by a client, the data storage center is specifically configured to receive a query request for target data sent by the client;
the data storage center finds a target storage unit capable of processing the processing request from storage units with storage levels not lower than the target storage level, and the data storage center is specifically applicable to:
starting from the storage units corresponding to the target storage level, sequentially inquiring the target data from the storage units which are not lower than the target storage level according to the sequentially-increasing sequence of the storage levels until the storage unit storing the target data is determined, and taking the first determined storage unit storing the target data as a target storage unit;
The data storage center is specifically configured to retrieve the target data from the target storage unit and feed the target data back to the client when triggering the target storage unit to process the target data according to the processing request.
Optionally, the data storage center may be further configured to determine a transition storage level from the target storage level to the storage level of the target storage unit if the storage level of the target storage unit is higher than the target storage level, and store the target data in the storage units of the target storage level and each transition storage level, respectively.
Optionally, when receiving a processing request for target data sent by a client, the data storage center is specifically configured to receive a write request for target data sent by the client;
the data storage center searches for a target storage unit capable of processing the processing request from storage units with storage levels not lower than the target storage level, and the data storage center is particularly applicable to taking the storage units with the storage levels not lower than the target storage level as target storage units;
the data storage center is specifically configured to store the target data in the target storage unit when triggering the target storage unit to process the target data according to the processing request.
Optionally, the data storage center may also trigger the storage unit of the highest storage level to store all data.
In the present specification, each embodiment is described in a progressive manner, and each embodiment is mainly described in a different point from other embodiments, and identical and similar parts between the embodiments are all enough to refer to each other. For the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant points refer to the description of the method section.
Those of skill would further appreciate that the various illustrative elements and algorithm steps described in connection with the embodiments disclosed herein may be implemented as electronic hardware, computer software, or combinations of both, and that the various illustrative elements and steps are described above generally in terms of functionality in order to clearly illustrate the interchangeability of hardware and software. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the solution. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present invention.
The steps of a method or algorithm described in connection with the embodiments disclosed herein may be embodied directly in hardware, in a software module executed by a processor, or in a combination of the two. The software modules may be disposed in Random Access Memory (RAM), memory, read Only Memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art.
The previous description of the disclosed embodiments is provided to enable any person skilled in the art to make or use the present invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the generic principles defined herein may be applied to other embodiments without departing from the spirit or scope of the invention. Thus, the present invention is not intended to be limited to the embodiments shown herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.
Claims (13)
1. A data storage system, comprising: n storage units with sequentially raised local cache and storage levels;
the n memory cells include: the storage system comprises a data storage center with a first storage level and n-1 storage units with storage levels higher than the first storage level and sequentially increased, wherein the n-1 storage units comprise: a key value buffer memory unit for storing data in the form of key values at a second storage level, and a relational database for storing data in the form of a relational database table at a third storage level;
According to the ascending order of the storage levels, the data access speed of the storage units is gradually reduced, the data storage capacity is gradually increased, and the storage level corresponding to the data with high access frequency is low so as to be correspondingly stored in the storage units with high access speed and low capacity;
wherein, each storage unit stores data with a storage level not higher than the storage level of the storage unit, and the storage unit with the highest storage level persistently stores data with all storage levels;
the data storage center is used for receiving a processing request for target data sent by the client; determining the access frequency of the target data, determining an access frequency range in which the access frequency of the target data is located according to the access frequency, and taking a storage level corresponding to the access frequency range as a target storage level of the target data based on a relation that the value of the access frequency range corresponding to the lower storage level is larger;
when the processing request is a query request for target data, starting from a storage unit corresponding to the target storage level, and querying whether the target data exists; if the storage unit corresponding to the target storage level exists, the storage unit corresponding to the target storage level is used as a target storage unit; if the target data does not exist, inquiring whether the target data exists in the storage unit with a higher storage level, and so on until the storage unit storing the target data is determined, taking the first determined storage unit storing the target data as a target storage unit, calling the target data from the target storage unit and feeding back to the client;
When the processing request is a writing request for the target data, taking all storage units with storage levels not lower than the target storage level as target storage units, and writing all the target data into the target storage units with storage levels not lower than the target storage level;
and triggering the target storage unit to process the target data according to the processing request.
2. The data storage system according to claim 1, wherein the data storage center is specifically configured to obtain access attribute information of the target data, and determine an access frequency of the target data according to the access attribute information.
3. A data storage system according to any one of claims 1-2 wherein,
the data storage center is specifically configured to use all storage units with storage levels not lower than the target storage level as target storage units;
the data storage center is specifically configured to store the target data in the target storage unit when triggering the target storage unit to process the target data according to the processing request.
4. The data storage system of claim 2, wherein the key cache storage unit stores data of the second storage level in the form of a key value;
The relational database stores data of a third storage level in a relational table form, or the relational database stores data of the third storage level and data of the first storage level and the second storage level in a relational table form.
5. A data storage method, wherein the data storage method is applied to a data storage system;
the data storage system includes at least: n storage units with sequentially raised local cache and storage levels; the n memory cells include: the storage system comprises a data storage center with a first storage level and n-1 storage units with storage levels higher than the first storage level and sequentially increased, wherein the n-1 storage units comprise: a key value buffer memory unit for storing data in the form of key values at a second storage level, and a relational database for storing data in the form of a relational database table at a third storage level; according to the ascending order of the storage levels, the data access speed of the storage units is gradually reduced, the data storage capacity is gradually increased, and the storage level corresponding to the data with high access frequency is low so as to be correspondingly stored in the storage units with high access speed and low capacity;
Wherein, each storage unit stores data with a storage level not higher than the storage level of the storage unit, and the storage unit with the highest storage level persistently stores data with all storage levels;
the data storage method comprises the following steps:
receiving a processing request for target data sent by a client;
determining the access frequency of the target data, determining an access frequency range in which the access frequency of the target data is located according to the access frequency, and taking a storage level corresponding to the access frequency range as a target storage level of the target data based on a relation that the value of the access frequency range corresponding to the lower storage level is larger;
when the processing request is a query request for target data, starting from a storage unit corresponding to the target storage level, and querying whether the target data exists; if the storage unit corresponding to the target storage level exists, the storage unit corresponding to the target storage level is used as a target storage unit; if the target data does not exist, inquiring whether the target data exists in the storage unit with a higher storage level, and so on until the storage unit storing the target data is determined, taking the first determined storage unit storing the target data as a target storage unit, calling the target data from the target storage unit and feeding back to the client;
When the processing request is a writing request for the target data, taking all storage units with storage levels not lower than the target storage level as target storage units, and writing all the target data into the target storage units with storage levels not lower than the target storage level;
and triggering the target storage unit to process the target data according to the processing request.
6. The data storage method according to claim 5, wherein the determining the access frequency of the target data includes: and acquiring access attribute information of the target data, and determining the access frequency of the target data according to the access attribute information.
7. The data storage method according to claim 6, wherein the acquiring access attribute information of the target data includes: acquiring the read frequency and the write frequency of the target data;
the determining the access frequency of the target data according to the access attribute information comprises: and determining a read-write proportion value of the target data according to the read frequency and the write frequency, and taking the read-write proportion value as the access frequency of the target data.
8. The data storage method according to claim 7, wherein after the determining the read-write ratio value of the target data and taking the read-write ratio value as the access frequency of the target data, the method further comprises:
Taking a storage level corresponding to the read-write proportion value range in which the read-write proportion value is positioned as a target storage level; one storage level corresponds to one read-write ratio value range, and the read-write ratio value ranges corresponding to the storage levels are different.
9. The data storage method of claim 7, wherein the method further comprises:
acquiring the data volume of the target data when acquiring the read frequency and the write frequency of the target data;
if the data amount of the target data is judged to be higher than a set data amount threshold, taking a storage level corresponding to the data amount of the target data as a target storage level;
and triggering the read-write ratio value of the target data according to the read frequency and the write frequency if the data quantity of the target data is judged to be lower than the set data quantity threshold value.
10. The data storage method of claim 5, wherein the method further comprises:
and if the storage level of the target storage unit is higher than the target storage level, determining a transition storage level from the target storage level to the storage level of the target storage unit, and storing the target data in the storage units of the target storage level and each transition storage level respectively.
11. A data storage center for use in a data storage system, the data storage system comprising: n storage units with sequentially raised local cache and storage levels;
the data storage center includes:
the processing request receiving module is used for receiving a processing request aiming at target data sent by a client;
a target storage level determining module, configured to determine an access frequency of the target data, determine an access frequency range in which the access frequency of the target data is located according to the access frequency, and use a storage level corresponding to the access frequency range as a target storage level of the target data based on a relationship that a value of the access frequency range corresponding to a lower storage level is greater;
the target storage unit searching module is used for searching whether the target data exists or not from the storage unit corresponding to the target storage level when the processing request is a search request aiming at the target data; if the storage unit corresponding to the target storage level exists, the storage unit corresponding to the target storage level is used as a target storage unit; if the target data does not exist, inquiring whether the target data exists in the storage unit with a higher storage level, and so on until the storage unit storing the target data is determined, taking the first determined storage unit storing the target data as a target storage unit, calling the target data from the target storage unit and feeding back to the client; when the processing request is a writing request for the target data, taking all storage units with storage levels not lower than the target storage level as target storage units, and writing all the target data into the target storage units with storage levels not lower than the target storage level;
The n memory cells include: the data storage center of the first storage level, and n-1 storage units with storage levels higher than the first storage level and storage levels sequentially increased, wherein the n-1 storage units comprise: a key value buffer memory unit for storing data in the form of key values at a second storage level, and a relational database for storing data in the form of a relational database table at a third storage level;
according to the ascending order of the storage levels, the data access speed of the storage units is gradually reduced, the data storage capacity is gradually increased, and the storage level corresponding to the data with high access frequency is low so as to be correspondingly stored in the storage units with high access speed and low capacity;
wherein, each storage unit stores data with a storage level not higher than the storage level of the storage unit, and the storage unit with the highest storage level persistently stores data with all storage levels;
and the processing triggering module is used for triggering the target storage unit to process the target data according to the processing request.
12. A computer readable storage medium, characterized in that the storage medium has stored therein a computer program for executing the data storage method of any one of claims 5-10.
13. A data storage center, comprising: a memory and a processor;
the memory is used for storing a computer program;
the processor is used for executing the computer program stored in the memory;
the computer program for performing the data storage method of any of claims 5-10.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201610392193.9A CN106095333B (en) | 2016-06-02 | 2016-06-02 | Data storage system, data storage method and data storage center |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201610392193.9A CN106095333B (en) | 2016-06-02 | 2016-06-02 | Data storage system, data storage method and data storage center |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN106095333A CN106095333A (en) | 2016-11-09 |
| CN106095333B true CN106095333B (en) | 2023-09-26 |
Family
ID=57447838
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201610392193.9A Active CN106095333B (en) | 2016-06-02 | 2016-06-02 | Data storage system, data storage method and data storage center |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN106095333B (en) |
Families Citing this family (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107367017B (en) * | 2017-07-04 | 2020-06-26 | 重庆美的通用制冷设备有限公司 | Control system of water chilling unit |
| CN109542961B (en) * | 2018-10-19 | 2024-05-03 | 中国平安财产保险股份有限公司 | Data storage method, device, computer equipment and storage medium |
| CN112825023A (en) * | 2019-11-20 | 2021-05-21 | 上海商汤智能科技有限公司 | Cluster resource management method and device, electronic equipment and storage medium |
| CN116431082B (en) * | 2023-06-14 | 2023-11-21 | 深圳嘉华众力科技有限公司 | Method, apparatus, computer device and storage medium for data storage |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1828556A (en) * | 2005-03-05 | 2006-09-06 | 腾讯科技（深圳）有限公司 | Method, system and apparatus for data memory |
| CN103064914A (en) * | 2012-12-20 | 2013-04-24 | 曙光信息产业(北京)有限公司 | Data processing system and method |
| CN103250143A (en) * | 2012-12-28 | 2013-08-14 | 华为技术有限公司 | Data storage method and storage device |
| CN104462240A (en) * | 2014-11-18 | 2015-03-25 | 浪潮(北京)电子信息产业有限公司 | Method and system for realizing hierarchical storage and management in cloud storage |
| CN105183389A (en) * | 2015-09-15 | 2015-12-23 | 北京金山安全软件有限公司 | Data hierarchical management method and device and electronic equipment |
| CN105205014A (en) * | 2015-09-28 | 2015-12-30 | 北京百度网讯科技有限公司 | Data storage method and device |
-
2016
- 2016-06-02 CN CN201610392193.9A patent/CN106095333B/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1828556A (en) * | 2005-03-05 | 2006-09-06 | 腾讯科技（深圳）有限公司 | Method, system and apparatus for data memory |
| CN103064914A (en) * | 2012-12-20 | 2013-04-24 | 曙光信息产业(北京)有限公司 | Data processing system and method |
| CN103250143A (en) * | 2012-12-28 | 2013-08-14 | 华为技术有限公司 | Data storage method and storage device |
| CN104462240A (en) * | 2014-11-18 | 2015-03-25 | 浪潮(北京)电子信息产业有限公司 | Method and system for realizing hierarchical storage and management in cloud storage |
| CN105183389A (en) * | 2015-09-15 | 2015-12-23 | 北京金山安全软件有限公司 | Data hierarchical management method and device and electronic equipment |
| CN105205014A (en) * | 2015-09-28 | 2015-12-30 | 北京百度网讯科技有限公司 | Data storage method and device |
Also Published As
| Publication number | Publication date |
|---|---|
| CN106095333A (en) | 2016-11-09 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN106095333B (en) | Data storage system, data storage method and data storage center | |
| US9672235B2 (en) | Method and system for dynamically partitioning very large database indices on write-once tables | |
| CN108710639B (en) | Ceph-based access optimization method for mass small files | |
| US7117222B2 (en) | Pre-formatted column-level caching to improve client performance | |
| CN107368527B (en) | Multi-attribute index method based on data stream | |
| CN109766318B (en) | File reading method and device | |
| CN107436910B (en) | Data query method and device | |
| CN109597829B (en) | Middleware method for realizing searchable encryption relational database cache | |
| CN106569963A (en) | Buffering method and buffering device | |
| US10838875B2 (en) | System and method for managing memory for large keys and values | |
| CN104123356A (en) | Method for increasing webpage response speed under large data volume condition | |
| CN107766258B (en) | Memory storage method and device and memory query method and device | |
| CN108009250B (en) | Multi-classification event data cache establishing and querying method and device | |
| CN113835639B (en) | I/O request processing method, device, equipment and readable storage medium | |
| CN108763458B (en) | Content characteristic query method, device, computer equipment and storage medium | |
| EP3507699B1 (en) | Method and systems for master establishment using service-based statistics | |
| US9760836B2 (en) | Data typing with probabilistic maps having imbalanced error costs | |
| CN100476815C (en) | Webpage mark extracting method | |
| CN111552438A (en) | Object writing method, device, server and storage medium | |
| CN107967306B (en) | Method for rapidly mining association blocks in storage system | |
| CN115964395A (en) | Data reading method and device and electronic equipment | |
| CN113722274A (en) | Efficient R-tree index remote sensing data storage model | |
| CN107679093B (en) | Data query method and device | |
| CN111159175A (en) | Incomplete database Skyline query method based on index | |
| CN112860712B (en) | Block chain-based transaction database construction method, system and electronic equipment |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |