CN105183389A - Data hierarchical management method and device and electronic equipment - Google Patents
Data hierarchical management method and device and electronic equipment Download PDFInfo
- Publication number
- CN105183389A CN105183389A CN201510587108.XA CN201510587108A CN105183389A CN 105183389 A CN105183389 A CN 105183389A CN 201510587108 A CN201510587108 A CN 201510587108A CN 105183389 A CN105183389 A CN 105183389A
- Authority
- CN
- China
- Prior art keywords
- data
- storage unit
- target data
- visiting frequency
- client
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Landscapes
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The embodiment of the invention discloses a data hierarchical management method, a data hierarchical management device and electronic equipment, relates to the technical field of computer application, and solves the problem that the overall storage cost of data is high in the prior art. The data grading management method comprises the following steps: receiving a data acquisition request sent by a client, wherein the data acquisition request comprises file information of target data requested to be acquired by the client; acquiring a data unification mark of target data according to file information of the target data requested to be acquired by the client; determining a storage unit where the target data is currently located according to the data unified tags of the target data, wherein different storage units store data with different access frequency levels; and calling the target data from the storage unit where the target data is currently located, and sending the target data to the client. The embodiment of the invention is mainly used for carrying out hierarchical management on data of different access frequency levels.
Description
Technical field
The present invention relates to Computer Applied Technology field, particularly relate to a kind of data staging management method, device and electronic equipment.
Background technology
Cloud storage is in the conceptive extension of cloud computing and the new concept of development out one, it is a kind of emerging Network storage technology, refer to by functions such as cluster application, network technology or distributed file systems, various dissimilar memory device a large amount of in network is gathered collaborative work by application software, jointly externally provides data to store and Operational Visit function.At present, cloud stores has become a kind of infrastructure service, Internet user none initiatively or passively do not using cloud stores service, complete the backup of various data, recovery, synchronous, upgrading etc.
At present, for the manufacturer of cloud stores service is provided, when managing data, all data being stored in same storage unit, the data of user's request access, all transferring from this storage unit.
Realizing in process of the present invention, inventor finds at least there is following technical matters in prior art:
The data of different access frequency rank are all stored in same storage unit, in order to ensure the response speed of the data of all visiting frequency ranks, this storage unit needs to arrange according to the data of high visiting frequency rank, and higher for the cost of the storage unit of the data that store high visiting frequency rank, thus make the global storage cost of data higher.
Summary of the invention
In view of this, the data staging management method that the embodiment of the present invention provides and device, can solve the problem of the global storage cost how reducing data.
First aspect, the embodiment of the present invention provides a kind of data staging management method, comprising:
Receive the data acquisition request that client sends, described data acquisition request comprises the fileinfo of the target data that described client-requested obtains;
According to the fileinfo of the target data that described client-requested obtains, obtain the unified mark of data of described target data;
According to the unified mark of the data of described target data, determine the storage unit at the current place of described target data, wherein, different cell stores has the data of different access frequency rank;
Transfer described target data in storage unit by the current place of described target data, be sent to described client.
Optionally, the unified mark of the described data according to described target data, determine that the storage unit at the current place of described target data comprises:
According to the unified mark of the data of described target data, send data inquiry request to slip condition database, wherein, described slip condition database records data unification and marks the corresponding relation with data place storage unit;
The Query Result that accepting state database returns, described Query Result shows the storage unit at the current place of described target data.
Optionally, described method also comprises:
Regularly the data in each storage unit are scanned;
If the visiting frequency that the visiting frequency in the given time of the one or more data in storage unit is arranged lower than the storage unit at current place, then described one or more data are migrated to other storage unit of the low one-level of visiting frequency by the storage unit at current place;
If the visiting frequency that the visiting frequency in the given time of the one or more data in storage unit is arranged higher than the storage unit at current place, then described one or more data are migrated to other storage unit of visiting frequency high one-level by the storage unit at current place;
The corresponding relation of the unified mark of data in described slip condition database and data place storage unit is upgraded.
Optionally, described method also comprises:
The visiting frequency of new data is assessed, according to the visiting frequency rank belonging to the visiting frequency of the new data evaluated, described new data is deposited in storage unit corresponding to the visiting frequency rank belonging to described.
Optionally, at the fileinfo of the described target data obtained according to described client-requested, before obtaining the unified mark of data of described target data, described method also comprises:
Described target data whether is stored in data query buffer memory;
If store described target data in described data buffer storage, then the address of described target data place data buffer storage is sent to described client, to make described client target data according to the address acquisition of described target data place data buffer storage.
Second aspect, the embodiment of the present invention provides a kind of data staging management devices, comprising:
Receiving element, for receiving the data acquisition request that client sends, described data acquisition request comprises the fileinfo of the target data that described client-requested obtains;
Acquiring unit, for the fileinfo of target data obtained according to described client-requested, obtains the unified mark of data of described target data;
Determining unit, for according to the unified mark of the data of described target data, determines the storage unit at the current place of described target data;
At least two storage unit, for storing the data of different access frequency rank respectively;
Transferring and transmitting element, for transferring described target data in the storage unit by the current place of described target data, being sent to described client.
Optionally, described determining unit comprises:
Sending module, for according to the unified mark of the data of described target data, sends data inquiry request to slip condition database, and wherein, described slip condition database records data unification and marks the corresponding relation with data place storage unit;
Receiver module, for the Query Result that accepting state database returns, described Query Result shows the storage unit at the current place of described target data.
Optionally, described device also comprises:
Scanning element, scans the data in each storage unit for regular;
Migration units, for when the visiting frequency that the visiting frequency in the given time of the one or more data in storage unit is arranged lower than the storage unit at current place, described one or more data are migrated to other storage unit of the low one-level of visiting frequency by the storage unit at current place, and when the visiting frequency that the visiting frequency in the given time of the one or more data in storage unit is arranged higher than the storage unit at current place, described one or more data are migrated to other storage unit of visiting frequency high one-level by the storage unit at current place;
Updating block, after described one or more data being migrated to the low rank of visiting frequency or other storage unit of visiting frequency high one-level in described migration units by the storage unit at current place, the corresponding relation of the unified mark of data in described slip condition database and data place storage unit is upgraded.
Optionally, described device also comprises:
Assessment unit, for assessing the visiting frequency of new data, according to the visiting frequency rank belonging to the visiting frequency of the new data evaluated, deposits described new data in storage unit corresponding to the visiting frequency rank belonging to described.
Optionally, described device also comprises:
Whether query unit, for storing described target data in data query buffer memory;
Transmitting element, for when storing described target data in described data buffer storage, the address of described target data place data buffer storage is sent to described client, to make described client target data according to the address acquisition of described target data place data buffer storage;
When not storing described target data in described data buffer storage, described in transfer and transfer target data with transmitting element by the storage unit at the current place of described target data, be sent to described client.
The third aspect, the embodiment of the present invention additionally provides a kind of electronic equipment, can comprise:
One or more processor;
Storer;
One or more module, one or more module stores described in which memory, proceeds as follows when being performed by one or more processor described:
Receive the data acquisition request that client sends, described data acquisition request comprises the fileinfo of the target data that described client-requested obtains;
According to the fileinfo of the target data that described client-requested obtains, obtain the unified mark of data of described target data;
According to the unified mark of the data of described target data, determine the storage unit at the current place of described target data, wherein, different cell stores has the data of different access frequency rank;
Transfer described target data in storage unit by the current place of described target data, be sent to described client.
Fourth aspect, the embodiment of the present invention additionally provides a kind of storage medium, for storing application program, a kind of data staging management method that described application program provides for performing the embodiment of the present invention.
5th aspect, the embodiment of the present invention additionally provides a kind of application program, for performing a kind of data staging management method that the embodiment of the present invention provides.
In the present invention, according to the fileinfo of the target data that client-requested obtains, obtain the unified mark of data of target data, and determine the storage unit at the current place of target data accordingly, wherein, different cell stores has the data of different access frequency rank, transfers described target data in the storage unit by the current place of described target data, is sent to described client.Compared with prior art, the data of different access frequency rank are stored in different storage unit by the present invention respectively, and transfer by carrying out data in the storage unit at the current place of these data according to the data acquisition request of client, thus while the response speed of data ensureing different access frequency rank, the global storage cost of data can be reduced.
Accompanying drawing explanation
In order to be illustrated more clearly in the technical scheme in the embodiment of the present invention, below the accompanying drawing used required in describing embodiment is briefly described, apparently, accompanying drawing in the following describes is only some embodiments of the present invention, for those of ordinary skill in the art, under the prerequisite not paying creative work, other accompanying drawing can also be obtained according to these accompanying drawings.
Fig. 1 is the schematic flow sheet of data staging management method one embodiment of the present invention;
Fig. 2 is the schematic flow sheet of another embodiment of data staging management method of the present invention;
Fig. 3 is the schematic flow sheet of another embodiment of data staging management method of the present invention;
Fig. 4 is the schematic flow sheet of another embodiment of data staging management method of the present invention;
Fig. 5 is the structural representation of data staging management devices one embodiment of the present invention;
Fig. 6 is the structural representation of another embodiment of data staging management devices of the present invention;
Fig. 7 is the structural representation of another embodiment of data staging management devices of the present invention.
Embodiment
Below in conjunction with the accompanying drawing in the embodiment of the present invention, be clearly and completely described the technical scheme in the embodiment of the present invention, obviously, described embodiment is only the present invention's part embodiment, instead of whole embodiments.Based on the embodiment in the present invention, those of ordinary skill in the art, not making other embodiments all obtained under creative work prerequisite, belong to the scope of protection of the invention.
Referring to Fig. 1, the embodiment of the present invention provides a kind of data staging management method, comprising:
The data acquisition request that S11, reception client send, described data acquisition request comprises the fileinfo of the target data that described client-requested obtains;
Wherein, described fileinfo comprises the relevant information representing described target data, can be filename, URL (UniformResourceLocator, uniform resource locator) address, but be not limited only to this.
S12, the fileinfo of target data obtained according to described client-requested, obtain the unified mark of data of described target data;
Wherein, the unified mark of data of described target data is generated by the whole world unified mark maker, fileinfo regardless of described target data is filename, address, or other marks, this whole world unified mark maker all can according to the fileinfo of described target data, the unified and unified mark of uniquely identified data of production form.
S13, according to the unified mark of the data of described target data, determine the storage unit at the current place of described target data, wherein, different cell stores has the data of different access frequency rank;
Transfer described target data in S14, storage unit by the current place of described target data, be sent to described client.
The data staging management method that the embodiment of the present invention provides, according to the fileinfo of the target data that client-requested obtains, obtain the unified mark of data of target data, and determine the storage unit at the current place of target data accordingly, wherein, different cell stores has the data of different access frequency rank, transfers described target data in the storage unit by the current place of described target data, is sent to described client.Compared with prior art, the data of different access frequency rank are stored in different storage unit by the present invention respectively, and transfer by carrying out data in the storage unit at the current place of these data according to the data acquisition request of client, thus while the response speed of data ensureing different access frequency rank, the global storage cost of data can be reduced.
Alternatively, in data staging management method one embodiment of the present invention, the unified mark of the described data according to described target data, determine that the storage unit (S13) at the current place of described target data can comprise:
S131, according to the unified mark of the data of described target data, send data inquiry request to slip condition database, wherein, described slip condition database records the corresponding relation of the unified mark of data and data place storage unit.
The Query Result that S132, accepting state database return, described Query Result shows the storage unit at the current place of described target data.
Wherein, slip condition database can adopt the corresponding relation of the unified mark of the form record data of form and data place storage unit, according to the unified mark of the data of target data, slip condition database is concentrated with the corresponding relation of data place storage unit at the unified mark of described data, search the storage unit that described target data is corresponding, obtain Query Result.
Alternatively, in data staging management method one embodiment of the present invention, described method also can comprise:
S15, regularly the data in each storage unit to be scanned.
Particularly, each data in each storage unit are scanned, determine the frequency that each data are accessed.
If the visiting frequency that the visiting frequency in the given time of the one or more data in S16 storage unit is arranged lower than the storage unit at current place, then described one or more data are migrated to other storage unit of the low one-level of visiting frequency by the storage unit at current place.
Wherein, the described schedule time is pre-set by system, if the visiting frequency that the visiting frequency in the given time of the one or more data in storage unit is arranged lower than the storage unit at current place, then other storage unit of the low one-level of visiting frequency is enough to meet the storage to these one or more data in response speed, therefore, these one or more data are migrated to other storage unit of the low one-level of visiting frequency, to reduce carrying cost by the storage unit at current place.
If the visiting frequency that the visiting frequency in the given time of the one or more data in S17 storage unit is arranged higher than the storage unit at current place, then described one or more data are migrated to other storage unit of visiting frequency high one-level by the storage unit at current place.
Wherein, if the visiting frequency that the visiting frequency in the given time of the one or more data in storage unit is arranged higher than the storage unit at current place, then this storage unit can not meet the storage of these one or more data in response speed, therefore, these one or more data are migrated to other storage unit of visiting frequency high one-level, to meet the response speed of these one or more data by the storage unit at current place.
S18, the corresponding relation of the unified mark of data in described slip condition database and data place storage unit to be upgraded.
Alternatively, in data staging management method one embodiment of the present invention, described method also can comprise:
S19, the visiting frequency of new data to be assessed, according to the visiting frequency rank belonging to the visiting frequency of the new data evaluated, described new data is deposited in storage unit corresponding to the visiting frequency rank belonging to described.
Alternatively, in data staging management method one embodiment of the present invention, at the fileinfo of the described target data obtained according to described client-requested, before obtaining the unified mark (S12) of data of described target data, described method also can comprise:
Described target data whether is stored in S20, data query buffer memory;
If store described target data in the described data buffer storage of S21, then the address of described target data place data buffer storage is sent to described client, to make described client target data according to the address acquisition of described target data place data buffer storage;
If do not store described target data in described data buffer storage, then perform step S12.
Wherein, data buffer storage is arranged in internal memory, can temporary transient store data, but the data deposited can be discharged after a period of time; And storage unit is arranged in hard disk, can permanent memory.Client is greater than by called data in hard disk by the speed of called data in data buffer storage, therefore, first judge whether store target data in data buffer storage, if had, then client is directly transferred from data buffer storage, thus can improve the speed that client obtains data; If no, client is transferred from the storage unit hard disk.
Referring to Fig. 2, the embodiment of the present invention provides a kind of data staging management method.In the present embodiment, three storage unit can be provided with: dsc data space, warm data space and cold data space, wherein,
Dsc data space stores dsc data, and described dsc data refers to the data that within 4 hours, user necessarily accesses;
Temperature data space stores warm data, and described temperature data refer to the data that within a week, user necessarily accesses;
Cold data space stores cold data, and described cold data refer to the data that within month, user not necessarily accesses.
Referring to Fig. 2, described data staging management method comprises:
1.1, client sends data acquisition request to data mode detector, and described data acquisition request comprises the filename of the target data that described client-requested obtains.
1.2, to the whole world, unified mark maker sends data mode inquiry request to data mode detector, and described data mode inquiry request comprises ID and the fileinfo of the target data that client-requested obtains.
1.3, the unified mark in whole world maker generates the unified mark of data, and sends data inquiry request to slip condition database.Wherein, described slip condition database records the corresponding relation of the unified mark of data and data place data space.
Wherein, after the whole world unified mark maker receives data mode inquiry request, whether ID and the fileinfo of the target data first checking client-requested to obtain be legal, only has and just generate the unified mark of data when legal.
1.4, slip condition database is according to the unified mark of described data, inquires the storage space at corresponding target data place, and query results is returned to data mode detector, hard objectives data current place data space.
1.5, data mode detector forms data call strategy, and data scheduling request is sent to data scheduler.
Wherein, described data call strategy instruction calls this target data by which data space.
Particularly, described data call strategy comprises: if store this target data in data buffer storage, then preferentially from data buffer storage, call this target data; If do not store this target data in data buffer storage, then according to the unified mark of the data of this target data, send data inquiry request to slip condition database, wherein, described slip condition database records data unification and marks the corresponding relation with data place data space.Wherein, slip condition database can adopt the corresponding relation of the unified mark of the form record data of form and data place data space, according to the unified mark of the data of target data, concentrate with the corresponding relation of data place data space at the unified mark of described data, search the data space that described target data is corresponding, and call this target data from the data space at this target data place.
If 1.6.1 this target data is present in data buffer storage, the address of this target data place data buffer storage is sent to client by data scheduler.
1.6.2, client obtains described data according to the link of this target data place data buffer storage.
Further, as shown in Figure 3, after step 1.1 is to 1.5, described data staging management method can also comprise:
If do not store this target data in 2.1 data buffer storages, then data scheduler again inquires about the data space at this target data place from slip condition database.
Particularly, data scheduler, according to the unified mark of the data of this target data, sends data inquiry request to slip condition database, and wherein, described slip condition database records data unification and marks the corresponding relation with data place data space.Wherein, slip condition database can adopt the corresponding relation of the unified mark of the form record data of form and data place data space, according to the unified mark of the data of target data, data scheduler is concentrated with the corresponding relation of data place data space at the unified mark of described data, search the data space that described target data is corresponding, and call this target data from the data space at this target data place.
If 2.2 these data are in dsc data space, then data scheduler reads this data from dsc data space.
2.3, these data are returned to client by data scheduler.
Further, as shown in Figure 4, described data staging management method can also comprise:
3.1, data scheduler regularly scans dsc data space.
If the one or more data in 3.2 dsc data spaces are not visited in the given time, then data scheduler initiates instruction, by corresponding data collection from dsc data spatial migration to warm data space.
3.3, the corresponding relation of data scheduler to the unified mark of data in slip condition database and data place data space upgrades.
Further, as shown in Figure 4, described data staging management method can also comprise:
4.1, data scheduler regularly scans warm data space.
If the one or more data in 4.2 warm data spaces are not visited in the given time, then data scheduler initiates instruction, and corresponding data collection is moved to cold data space from warm data space.
4.3, the corresponding relation of data scheduler to the unified mark of data in slip condition database and data place data space upgrades.
Further, as shown in Figure 4, described data staging management method can also comprise:
5.1, data scheduler regularly verifies the data in cold data space, ensures that the data in cold data space can be used.
Wherein, describedly verification is carried out to the data in cold data space be specifically as follows: calculate the data acquisition checking algorithm in cold data space, draw proof test value, the proof test value that this proof test value is corresponding with these data itself compares, if the two is consistent, then show that these data are available; If the two is inconsistent, then show that these data are unavailable.Wherein, described checking algorithm can verify for md5, CRC (CyclicRedundancyCheck, superfluous She of circulating verify) etc., and the proof test value of these data correspondence itself is deposited in cold data space together with these data.
5.2, the corresponding relation of data scheduler to the unified mark of data in slip condition database and data place data space upgrades.
Referring to Fig. 5, the embodiment of the present invention provides a kind of data staging management devices, and described data staging management devices comprises:
Receiving element 11, for receiving the data acquisition request that client sends, described data acquisition request comprises the fileinfo of the target data that described client-requested obtains;
Wherein, described fileinfo comprises the relevant information representing described target data, can be filename, URL address, but be not limited only to this.
Acquiring unit 12, for the fileinfo of target data obtained according to described client-requested, obtains the unified mark of data of described target data;
Wherein, the unified mark of data of described target data is generated by the whole world unified mark maker, fileinfo regardless of described target data is filename, address, or other marks, this whole world unified mark maker all can according to the fileinfo of described target data, the unified and unified mark of uniquely identified data of production form.
Determining unit 13, for according to the unified mark of the data of described target data, determines the storage unit at the current place of described target data;
At least two storage unit 14, for storing the data of different access frequency rank respectively;
Transferring and transmitting element 15, for transferring described target data in the storage unit by the current place of described target data, being sent to described client.
The data staging management devices that the embodiment of the present invention provides, according to the fileinfo of the target data that client-requested obtains, obtain the unified mark of data of target data, and determine the storage unit at the current place of data accordingly, wherein, different cell stores has the data of different access frequency rank, transfers described target data in the storage unit by the current place of described target data, is sent to described client.Compared with prior art, the data of different access frequency rank are stored in different storage unit by the present invention respectively, and transfer by carrying out data in the storage unit at the current place of these data according to the data acquisition request of client, thus while the response speed of data ensureing different access frequency rank, the global storage cost of data can be reduced.
Alternatively, in data staging management devices one embodiment of the present invention, described determining unit 13 can comprise:
Sending module 131, for according to the unified mark of the data of described target data, sends data inquiry request to slip condition database, and wherein, described slip condition database records data unification and marks the corresponding relation with data place storage unit;
Receiver module 132, for the Query Result that accepting state database returns, described Query Result shows the storage unit at the current place of described target data.
Wherein, slip condition database can adopt the corresponding relation of the unified mark of the form record data of form and data place storage unit, according to the unified mark of the data of target data, slip condition database is concentrated with the corresponding relation of data place storage unit at the unified mark of described data, search the storage unit that described target data is corresponding, obtain Query Result.
Referring to Fig. 6, alternatively, in data staging management devices one embodiment of the present invention, described data staging management devices also can comprise:
Scanning element 16, scans the data in each storage unit for regular;
Migration units 17, for when the visiting frequency that the visiting frequency in the given time of the one or more data in storage unit is arranged lower than the storage unit at current place, described one or more data are migrated to other storage unit of the low one-level of visiting frequency by the storage unit at current place, and when the visiting frequency that the visiting frequency in the given time of the one or more data in storage unit is arranged higher than the storage unit at current place, described one or more data are migrated to other storage unit of visiting frequency high one-level by the storage unit at current place;
Wherein, the described schedule time is pre-set by system, if the visiting frequency that the visiting frequency in the given time of the one or more data in storage unit is arranged lower than the storage unit at current place, then other storage unit of the low one-level of visiting frequency is enough to meet the storage to these one or more data in response speed, therefore, these one or more data are migrated to other storage unit of the low one-level of visiting frequency, to reduce carrying cost by the storage unit at current place; If the visiting frequency that the visiting frequency in the given time of the one or more data in storage unit is arranged higher than the storage unit at current place, then this storage unit can not meet the storage of these one or more data in response speed, therefore, these one or more data are migrated to other storage unit of visiting frequency high one-level, to meet the response speed of these one or more data by the storage unit at current place.
Updating block 18, after described one or more data being migrated to the low rank of visiting frequency or other storage unit of visiting frequency high one-level in described migration units 17 by the storage unit at current place, the corresponding relation of the unified mark of data in described slip condition database and data place storage unit is upgraded.
Referring to Fig. 6, alternatively, in data staging management devices one embodiment of the present invention, described data staging management devices also can comprise:
Assessment unit 19, for assessing the visiting frequency of new data, according to the visiting frequency rank belonging to the visiting frequency of the new data evaluated, deposits described new data in storage unit corresponding to the visiting frequency rank belonging to described.
Referring to Fig. 7, alternatively, in data staging management devices one embodiment of the present invention, described data staging management devices also can comprise:
Whether query unit 20, for storing described target data in data query buffer memory;
Transmitting element 21, for when storing described target data in described data buffer storage, the address of described target data place data buffer storage is sent to described client, to make described client target data according to the address acquisition of described target data place data buffer storage;
When not storing described target data in described data buffer storage, described in transfer and transfer target data with transmitting element 15 by the storage unit at the current place of described target data, be sent to described client.
Wherein, data buffer storage is arranged in internal memory, can temporary transient store data, but the data deposited can be discharged after a period of time; And storage unit is arranged in hard disk, can permanent memory.Client is greater than by called data in hard disk by the speed of called data in data buffer storage, therefore, first judge whether store target data in data buffer storage, if had, then client is directly transferred from data buffer storage, thus can improve the speed that client obtains data; If no, client is transferred from the storage unit hard disk.
Embodiment of the present invention data staging management method and device, go for carrying out differentiated control to the data of different access frequency rank, reduces the global storage cost of data.
The embodiment of the present invention additionally provides a kind of electronic equipment, can comprise:
One or more processor;
Storer;
One or more module, one or more module stores described in which memory, proceeds as follows when being performed by one or more processor described:
Receive the data acquisition request that client sends, described data acquisition request comprises the fileinfo of the target data that described client-requested obtains;
According to the fileinfo of the target data that described client-requested obtains, obtain the unified mark of data of described target data;
According to the unified mark of the data of described target data, determine the storage unit at the current place of described target data, wherein, different cell stores has the data of different access frequency rank;
Transfer described target data in storage unit by the current place of described target data, be sent to described client.
The embodiment of the present invention additionally provides a kind of storage medium, for storing application program, and a kind of data staging management method that described application program provides for performing the embodiment of the present invention.
The embodiment of the present invention additionally provides a kind of application program, for performing a kind of data staging management method that the embodiment of the present invention provides.
In flow charts represent or in this logic otherwise described and/or step, such as, the sequencing list of the executable instruction for realizing logic function can be considered to, may be embodied in any computer-readable medium, for instruction execution system, device or equipment (as computer based system, comprise the system of processor or other can from instruction execution system, device or equipment instruction fetch and perform the system of instruction) use, or to use in conjunction with these instruction execution systems, device or equipment.With regard to this instructions, " computer-readable medium " can be anyly can to comprise, store, communicate, propagate or transmission procedure for instruction execution system, device or equipment or the device that uses in conjunction with these instruction execution systems, device or equipment.The example more specifically (non-exhaustive list) of computer-readable medium comprises following: the electrical connection section (electronic installation) with one or more wiring, portable computer diskette box (magnetic device), random access memory (RAM), ROM (read-only memory) (ROM), erasablely edit ROM (read-only memory) (EPROM or flash memory), fiber device, and portable optic disk ROM (read-only memory) (CDROM).In addition, computer-readable medium can be even paper or other suitable media that can print described program thereon, because can such as by carrying out optical scanning to paper or other media, then carry out editing, decipher or carry out process with other suitable methods if desired and electronically obtain described program, be then stored in computer memory.
Should be appreciated that each several part of the present invention can realize with hardware, software, firmware or their combination.In the above-described embodiment, multiple step or method can with to store in memory and the software performed by suitable instruction execution system or firmware realize.Such as, if realized with hardware, the same in another embodiment, can realize by any one in following technology well known in the art or their combination: the discrete logic with the logic gates for realizing logic function to data-signal, there is the special IC of suitable combinational logic gate circuit, programmable gate array (PGA), field programmable gate array (FPGA) etc.
Those skilled in the art are appreciated that realizing all or part of step that above-described embodiment method carries is that the hardware that can carry out instruction relevant by program completes, described program can be stored in a kind of computer-readable recording medium, this program perform time, step comprising embodiment of the method one or a combination set of.
For convenience of description, describing above device is be divided into various units/modules to describe respectively with function.Certainly, the function of each units/modules can be realized in same or multiple software and/or hardware when implementing of the present invention.
As seen through the above description of the embodiments, those skilled in the art can be well understood to the mode that the present invention can add required general hardware platform by software and realizes.Based on such understanding, technical scheme of the present invention can embody with the form of software product the part that prior art contributes in essence in other words, this computer software product can be stored in storage medium, as ROM/RAM, magnetic disc, CD etc., comprising some instructions in order to make a computer equipment (can be personal computer, server, or the network equipment etc.) perform the method described in some part of each embodiment of the present invention or embodiment.
The above; be only the specific embodiment of the present invention, but protection scope of the present invention is not limited thereto, is anyly familiar with those skilled in the art in the technical scope that the present invention discloses; the change that can expect easily or replacement, all should be encompassed within protection scope of the present invention.Therefore, protection scope of the present invention should be as the criterion with the protection domain of claim.
Claims (10)
1. a data staging management method, is characterized in that, comprising:
Receive the data acquisition request that client sends, described data acquisition request comprises the fileinfo of the target data that described client-requested obtains;
According to the fileinfo of the target data that described client-requested obtains, obtain the unified mark of data of described target data;
According to the unified mark of the data of described target data, determine the storage unit at the current place of described target data, wherein, different cell stores has the data of different access frequency rank;
Transfer described target data in storage unit by the current place of described target data, be sent to described client.
2. data staging management method according to claim 1, is characterized in that, the unified mark of the described data according to described target data, determines that the storage unit at the current place of described target data comprises:
According to the unified mark of the data of described target data, send data inquiry request to slip condition database, wherein, described slip condition database records data unification and marks the corresponding relation with data place storage unit;
The Query Result that accepting state database returns, described Query Result shows the storage unit at the current place of described target data.
3. data staging management method according to claim 2, is characterized in that, described method also comprises:
Regularly the data in each storage unit are scanned;
If the visiting frequency that the visiting frequency in the given time of the one or more data in storage unit is arranged lower than the storage unit at current place, then described one or more data are migrated to other storage unit of the low one-level of visiting frequency by the storage unit at current place;
If the visiting frequency that the visiting frequency in the given time of the one or more data in storage unit is arranged higher than the storage unit at current place, then described one or more data are migrated to other storage unit of visiting frequency high one-level by the storage unit at current place;
The corresponding relation of the unified mark of data in described slip condition database and data place storage unit is upgraded.
4. data staging management method according to claim 3, is characterized in that, described method also comprises:
The visiting frequency of new data is assessed, according to the visiting frequency rank belonging to the visiting frequency of the new data evaluated, described new data is deposited in storage unit corresponding to the visiting frequency rank belonging to described.
5. data staging management method according to claim 1, is characterized in that, at the fileinfo of the described target data obtained according to described client-requested, before obtaining the unified mark of data of described target data, described method also comprises:
Described target data whether is stored in data query buffer memory;
If store described target data in described data buffer storage, then the address of described target data place data buffer storage is sent to described client, to make described client target data according to the address acquisition of described target data place data buffer storage.
6. a data staging management devices, is characterized in that, comprising:
Receiving element, for receiving the data acquisition request that client sends, described data acquisition request comprises the fileinfo of the target data that described client-requested obtains;
Acquiring unit, for the fileinfo of target data obtained according to described client-requested, obtains the unified mark of data of described target data;
Determining unit, for according to the unified mark of the data of described target data, determines the storage unit at the current place of described target data;
At least two storage unit, for storing the data of different access frequency rank respectively;
Transferring and transmitting element, for transferring described target data in the storage unit by the current place of described target data, being sent to described client.
7. data staging management devices according to claim 6, is characterized in that, described determining unit comprises:
Sending module, for according to the unified mark of the data of described target data, sends data inquiry request to slip condition database, and wherein, described slip condition database records data unification and marks the corresponding relation with data place storage unit;
Receiver module, for the Query Result that accepting state database returns, described Query Result shows the storage unit at the current place of described target data.
8. data staging management devices according to claim 7, is characterized in that, described device also comprises:
Scanning element, scans the data in each storage unit for regular;
Migration units, for when the visiting frequency that the visiting frequency in the given time of the one or more data in storage unit is arranged lower than the storage unit at current place, described one or more data are migrated to other storage unit of the low one-level of visiting frequency by the storage unit at current place, and when the visiting frequency that the visiting frequency in the given time of the one or more data in storage unit is arranged higher than the storage unit at current place, described one or more data are migrated to other storage unit of visiting frequency high one-level by the storage unit at current place;
Updating block, after described one or more data being migrated to the low rank of visiting frequency or other storage unit of visiting frequency high one-level in described migration units by the storage unit at current place, the corresponding relation of the unified mark of data in described slip condition database and data place storage unit is upgraded.
9. data staging management devices according to claim 8, is characterized in that, described device also comprises:
Assessment unit, for assessing the visiting frequency of new data, according to the visiting frequency rank belonging to the visiting frequency of the new data evaluated, deposits described new data in storage unit corresponding to the visiting frequency rank belonging to described.
10. an electronic equipment, is characterized in that, comprising:
One or more processor;
Storer;
One or more module, one or more module stores described in which memory, proceeds as follows when being performed by one or more processor described:
Receive the data acquisition request that client sends, described data acquisition request comprises the fileinfo of the target data that described client-requested obtains;
According to the fileinfo of the target data that described client-requested obtains, obtain the unified mark of data of described target data;
According to the unified mark of the data of described target data, determine the storage unit at the current place of described target data, wherein, different cell stores has the data of different access frequency rank;
Transfer described target data in storage unit by the current place of described target data, be sent to described client.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201510587108.XA CN105183389A (en) | 2015-09-15 | 2015-09-15 | Data hierarchical management method and device and electronic equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201510587108.XA CN105183389A (en) | 2015-09-15 | 2015-09-15 | Data hierarchical management method and device and electronic equipment |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN105183389A true CN105183389A (en) | 2015-12-23 |
Family
ID=54905498
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201510587108.XA Pending CN105183389A (en) | 2015-09-15 | 2015-09-15 | Data hierarchical management method and device and electronic equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN105183389A (en) |
Cited By (14)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106095333A (en) * | 2016-06-02 | 2016-11-09 | 腾讯科技（深圳）有限公司 | A kind of data-storage system, date storage method and data storage center |
| CN106156331A (en) * | 2016-07-06 | 2016-11-23 | 益佳科技（北京）有限责任公司 | Cold and hot temperature data server system and processing method thereof |
| CN106331148A (en) * | 2016-09-14 | 2017-01-11 | 郑州云海信息技术有限公司 | Cache management method and cache management device for data reading by clients |
| CN107451152A (en) * | 2016-05-31 | 2017-12-08 | 阿里巴巴集团控股有限公司 | Computing device, data buffer storage and the method and device of lookup |
| CN107580060A (en) * | 2017-09-14 | 2018-01-12 | 商客通尚景科技江苏有限公司 | Banked cache method is divided in a kind of mobile terminal |
| WO2018019310A1 (en) * | 2016-07-27 | 2018-02-01 | 中兴通讯股份有限公司 | Big data system data backup and recovery methods and devices, and computer storage medium |
| CN107729182A (en) * | 2017-10-11 | 2018-02-23 | 苏州乐麟无线信息科技有限公司 | The method and device of data storage and access |
| CN108270834A (en) * | 2016-12-30 | 2018-07-10 | 北京国双科技有限公司 | Data transmission method and device |
| CN108538323A (en) * | 2018-04-04 | 2018-09-14 | 北京蓝杞数据科技有限公司天津分公司 | A kind of data transferring method, system, electronic equipment and readable storage medium storing program for executing |
| CN110290171A (en) * | 2019-05-15 | 2019-09-27 | 陈菡 | A kind of data managing method and device, electronic equipment |
| CN111614730A (en) * | 2020-04-28 | 2020-09-01 | 北京金山云网络技术有限公司 | File processing method and device of cloud storage system and electronic equipment |
| CN112825023A (en) * | 2019-11-20 | 2021-05-21 | 上海商汤智能科技有限公司 | Cluster resource management method and device, electronic equipment and storage medium |
| CN113010514A (en) * | 2021-03-01 | 2021-06-22 | 中国工商银行股份有限公司 | Hot loading method and device |
| CN113296696A (en) * | 2021-03-02 | 2021-08-24 | 阿里巴巴新加坡控股有限公司 | Data access method, computing device and storage medium |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20030139193A1 (en) * | 2002-01-29 | 2003-07-24 | Buckley David J. | Wireless device hub system and method |
| CN101034340A (en) * | 2003-03-27 | 2007-09-12 | 株式会社日立制作所 | Date storage method and data storage system |
| US20080187345A1 (en) * | 2007-02-01 | 2008-08-07 | Sharp Kabushiki Kaisha | Image history storage method, image history storage system, image history storage program, and recording medium |
| CN102456004A (en) * | 2010-10-18 | 2012-05-16 | 金蝶软件（中国）有限公司 | Query control method and system of page data |
| CN104583929A (en) * | 2012-10-12 | 2015-04-29 | 株式会社日立制作所 | Storage apparatus and data management method |
-
2015
- 2015-09-15 CN CN201510587108.XA patent/CN105183389A/en active Pending
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20030139193A1 (en) * | 2002-01-29 | 2003-07-24 | Buckley David J. | Wireless device hub system and method |
| CN101034340A (en) * | 2003-03-27 | 2007-09-12 | 株式会社日立制作所 | Date storage method and data storage system |
| US20080187345A1 (en) * | 2007-02-01 | 2008-08-07 | Sharp Kabushiki Kaisha | Image history storage method, image history storage system, image history storage program, and recording medium |
| CN102456004A (en) * | 2010-10-18 | 2012-05-16 | 金蝶软件（中国）有限公司 | Query control method and system of page data |
| CN104583929A (en) * | 2012-10-12 | 2015-04-29 | 株式会社日立制作所 | Storage apparatus and data management method |
Cited By (18)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107451152A (en) * | 2016-05-31 | 2017-12-08 | 阿里巴巴集团控股有限公司 | Computing device, data buffer storage and the method and device of lookup |
| CN106095333A (en) * | 2016-06-02 | 2016-11-09 | 腾讯科技（深圳）有限公司 | A kind of data-storage system, date storage method and data storage center |
| CN106095333B (en) * | 2016-06-02 | 2023-09-26 | 腾讯科技（深圳）有限公司 | Data storage system, data storage method and data storage center |
| CN106156331A (en) * | 2016-07-06 | 2016-11-23 | 益佳科技（北京）有限责任公司 | Cold and hot temperature data server system and processing method thereof |
| WO2018019310A1 (en) * | 2016-07-27 | 2018-02-01 | 中兴通讯股份有限公司 | Big data system data backup and recovery methods and devices, and computer storage medium |
| CN106331148A (en) * | 2016-09-14 | 2017-01-11 | 郑州云海信息技术有限公司 | Cache management method and cache management device for data reading by clients |
| CN108270834A (en) * | 2016-12-30 | 2018-07-10 | 北京国双科技有限公司 | Data transmission method and device |
| CN108270834B (en) * | 2016-12-30 | 2021-08-10 | 北京国双科技有限公司 | Data transmission method and device |
| CN107580060A (en) * | 2017-09-14 | 2018-01-12 | 商客通尚景科技江苏有限公司 | Banked cache method is divided in a kind of mobile terminal |
| CN107729182B (en) * | 2017-10-11 | 2020-12-04 | 苏州乐麟无线信息科技有限公司 | Data storage and access method and device |
| CN107729182A (en) * | 2017-10-11 | 2018-02-23 | 苏州乐麟无线信息科技有限公司 | The method and device of data storage and access |
| CN108538323A (en) * | 2018-04-04 | 2018-09-14 | 北京蓝杞数据科技有限公司天津分公司 | A kind of data transferring method, system, electronic equipment and readable storage medium storing program for executing |
| CN110290171A (en) * | 2019-05-15 | 2019-09-27 | 陈菡 | A kind of data managing method and device, electronic equipment |
| CN112825023A (en) * | 2019-11-20 | 2021-05-21 | 上海商汤智能科技有限公司 | Cluster resource management method and device, electronic equipment and storage medium |
| CN111614730A (en) * | 2020-04-28 | 2020-09-01 | 北京金山云网络技术有限公司 | File processing method and device of cloud storage system and electronic equipment |
| CN113010514A (en) * | 2021-03-01 | 2021-06-22 | 中国工商银行股份有限公司 | Hot loading method and device |
| CN113010514B (en) * | 2021-03-01 | 2024-02-20 | 中国工商银行股份有限公司 | Thermal loading method and device |
| CN113296696A (en) * | 2021-03-02 | 2021-08-24 | 阿里巴巴新加坡控股有限公司 | Data access method, computing device and storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN105183389A (en) | Data hierarchical management method and device and electronic equipment | |
| US8719818B2 (en) | Cloud-based test system utilizing cloud systems that include cloud servers to simulate virtual test machines to test an electrical device | |
| US10154089B2 (en) | Distributed system and data operation method thereof | |
| CN102185934B (en) | System and method for writing MAC (media access control) address | |
| CN108011929A (en) | Data request processing method, apparatus, computer equipment and storage medium | |
| JP6492123B2 (en) | Distributed caching and cache analysis | |
| CN103475682B (en) | File transfer method and file transfer equipment | |
| US9654415B2 (en) | Information processing system, management server group, and server management program | |
| CN104731516A (en) | Method and device for accessing files and distributed storage system | |
| CN103607424A (en) | Server connection method and server system | |
| CN104618482A (en) | Cloud data access method, server, traditional storage device and architecture | |
| CN104823170A (en) | Distributed caching cluster management | |
| US9984079B1 (en) | Managing data storage using storage policy specifications | |
| CN105068758A (en) | Distributed file system data I/O optimization method for parallel data collection | |
| CN109491645A (en) | Point methods, device, computer equipment and storage medium are buried based on interface element | |
| CN105335170A (en) | Distributed system and incremental data updating method | |
| CN111444157B (en) | Distributed file system and data access method | |
| CN108347459A (en) | A kind of high in the clouds data quick storage method and device | |
| US9380127B2 (en) | Distributed caching and cache analysis | |
| CN103986779A (en) | Application data transmission method, device and terminal equipment | |
| CN103905441A (en) | Data acquisition method and device | |
| CN103327036B (en) | The identification method of internet browsing equipment and Cookie server | |
| CN112839076A (en) | Data storage method, data reading method, gateway, electronic equipment and storage medium | |
| CN109144782A (en) | A kind of data reconstruction method and device | |
| CN106294842A (en) | A kind of data interactive method, platform and distributed file system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20151223 |
|
| RJ01 | Rejection of invention patent application after publication |