CN109669644A - A kind of method and apparatus of data storage - Google Patents
A kind of method and apparatus of data storage Download PDFInfo
- Publication number
- CN109669644A CN109669644A CN201910002082.6A CN201910002082A CN109669644A CN 109669644 A CN109669644 A CN 109669644A CN 201910002082 A CN201910002082 A CN 201910002082A CN 109669644 A CN109669644 A CN 109669644A
- Authority
- CN
- China
- Prior art keywords
- data
- target
- stored
- storage
- mapping ruler
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 36
- 238000013500 data storage Methods 0.000 title claims abstract description 34
- 238000013507 mapping Methods 0.000 claims abstract description 94
- 230000008569 process Effects 0.000 claims description 5
- 235000013399 edible fruits Nutrition 0.000 claims 1
- 238000013506 data mapping Methods 0.000 description 7
- 238000010586 diagram Methods 0.000 description 6
- 230000006870 function Effects 0.000 description 6
- 238000011161 development Methods 0.000 description 4
- 238000012545 processing Methods 0.000 description 4
- 238000005516 engineering process Methods 0.000 description 3
- 230000004048 modification Effects 0.000 description 3
- 238000012986 modification Methods 0.000 description 3
- 238000012795 verification Methods 0.000 description 3
- 108010001267 Protein Subunits Proteins 0.000 description 2
- 238000012217 deletion Methods 0.000 description 2
- 230000037430 deletion Effects 0.000 description 2
- 230000004044 response Effects 0.000 description 2
- 238000000547 structure data Methods 0.000 description 2
- 230000009471 action Effects 0.000 description 1
- 230000006399 behavior Effects 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 230000000052 comparative effect Effects 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 239000002360 explosive Substances 0.000 description 1
- 230000006855 networking Effects 0.000 description 1
- 238000007781 pre-processing Methods 0.000 description 1
- 230000000750 progressive effect Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0602—Interfaces specially adapted for storage systems specifically adapted to achieve a particular effect
- G06F3/0604—Improving or facilitating administration, e.g. storage management
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0668—Interfaces specially adapted for storage systems adopting a particular infrastructure
- G06F3/067—Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/10—Protocols in which an application is distributed across nodes in the network
- H04L67/1097—Protocols in which an application is distributed across nodes in the network for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Human Computer Interaction (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
This application discloses a kind of method and apparatus of data storage, this method comprises: firstly, obtaining the first data to be stored of target data source, wherein target data source corresponds to target mapping ruler and target structural data template；Secondly, obtaining the second data to be stored that data structure is consistent with target structural data template based on the first data to be stored；Then, the second data to be stored is mapped into corresponding target storage server according to target mapping ruler；Finally, carrying out cloud storage to the second data to be stored in target storage server.It can be seen that, first obtain the second data to be stored that data structure is consistent with target structural data template, it is mapped again based on the corresponding target mapping ruler of data source so as to subsequent cloud storage, target structural data template and target mapping ruler can meet the storage demand of the data source most possibly, improve storage efficiency and storage speed.
Description
Technical field
This application involves the method and apparatus that computer data processing technology field more particularly to a kind of data store.
Background technique
With the development of science and technology, computer is more and more common in daily life and social work at this stage,
It is more and more important.Information-based, networking trend, so that more and more enterprises, businessman, government bodies and individual etc. pass through meter
Calculation machine obtains information, processing information, while a large amount of information being saved in a computer in the form of data file.
The fast development of information-intensive society, more and more information by digitization, especially along with internet, cloud computing,
The development of the technologies such as Internet of Things, data are in explosive growth.Based on this, mass data brings new choose to data storage storage
War, for example, in face of magnanimity, various data, server needs efficiently and rapidly to store these data.
That is, since current development trend is that information content is more and more abundant, data volume is increasing, many magnanimity
Data platform gradually occur data storage bottleneck the problems such as, therefore, people pay close attention to be data how efficient storage.
Summary of the invention
The method and apparatus that technical problems to be solved in this application are to provide a kind of data storage, being capable of maximum possible
The storage demand of ground with meeting data source data source improves storage efficiency and storage speed.
In a first aspect, the embodiment of the present application provides a kind of method of data storage, this method comprises:
The first data to be stored of target data source is obtained, the target data source corresponds to target mapping ruler and target knot
Structure data template；
The second data to be stored, the data structure of second data to be stored are obtained based on first data to be stored
It is consistent with the target structural data template；
Second data to be stored is mapped into corresponding target storage server according to the target mapping ruler；
Cloud storage is carried out to second data to be stored in the target storage server.
Optionally, further includes:
According to target data de-duplication mode to the second data to be stored mapped in the target storage server into
Row data de-duplication, the target data de-duplication mode correspond to the target data source；
It is accordingly, described that cloud storage is carried out to second data to be stored in the target storage server, specifically:
Cloud storage is carried out to the second data to be stored for completing data de-duplication in the target storage server.
Optionally, further includes:
According to the data storage requirement of the target data source, the corresponding target mapping ruler is preset；
According to the target mapping ruler, the corresponding target structural data template is pre-generated；
According to the characteristic of the target structural data template, the corresponding target data de-duplication side is preset
Formula.
Optionally, the target mapping ruler includes DATATYPES TO rule, data source mapping ruler, data structure
Mapping ruler, data time mapping ruler or data category mapping ruler.
Optionally, the target data de-duplication mode includes the data de-duplication mode of fixed size block or becomes larger
The data de-duplication mode of fritter.
It is optionally, described that second data to be stored is obtained based on first data to be stored, comprising:
Judge whether the data structure of first data to be stored is consistent with the target structural data template；
If so, first data to be stored is determined as second data to be stored；
If it is not, pre-processing according to the target structural data template to first data to be stored, institute is obtained
State the second data to be stored.
Optionally, the data structure of judgement first data to be stored is with the target structural data template
It is no to be consistent, comprising:
According to the target data source, the corresponding target structural data template is obtained；
The data structure of first data to be stored is compared with the target structural data template, according to than
Relatively result judges whether to be consistent.
Optionally, described second data to be stored is mapped into corresponding target according to the target mapping ruler to deposit
Store up server, comprising:
According to the target data source, the corresponding target mapping ruler is obtained；
Second data to be stored is mapped into corresponding target storage server according to the target mapping ruler.
Optionally, further includes:
According to the mathematical logic relationship of the target storage server, data access interface is determined.
Second aspect, the embodiment of the present application provide a kind of device of data storage, which includes:
Acquiring unit, for obtaining the first data to be stored of target data source, the target data source corresponds to target and reflects
Penetrate rule and target structural data template；
Obtaining unit, for obtaining the second data to be stored based on first data to be stored, described second wait store
The data structure of data is consistent with the target structural data template；
Map unit, for second data to be stored to be mapped to corresponding target according to the target mapping ruler
Storage server；
Storage unit, for carrying out cloud storage to second data to be stored in the target storage server.
Compared with prior art, the application has at least the following advantages:
Using the technical solution of the embodiment of the present application, firstly, obtaining the first data to be stored of target data source, wherein
Target data source corresponds to target mapping ruler and target structural data template；Secondly, obtaining number based on the first data to be stored
The second data to be stored being consistent according to structure with target structural data template；Then, according to target mapping ruler by second to
Storing data maps to corresponding target storage server；Finally, being carried out in target storage server to the second data to be stored
Cloud storage.It can be seen that first obtaining the second data to be stored that data structure is consistent with target structural data template, then it is based on
The corresponding target mapping ruler of data source is mapped so as to subsequent cloud storage, target structural data template and target mapping rule
The storage demand of the data source can then be met most possibly, improve storage efficiency and storage speed.
Detailed description of the invention
Technical solution in ord to more clearly illustrate embodiments of the present application, below will be to required in the embodiment of the present application description
Attached drawing to be used is briefly described, it should be apparent that, the accompanying drawings in the following description is only more as described in this application
Embodiment for those of ordinary skill in the art without creative efforts, can also be attached according to these
Figure obtains other attached drawings.
Fig. 1 is system framework schematic diagram involved in application scenarios a kind of in the embodiment of the present application；
Fig. 2 is a kind of flow diagram of the method for data storage provided by the embodiments of the present application；
Fig. 3 is the flow diagram of the method for another data storage provided by the embodiments of the present application；
Fig. 4 is a kind of structural schematic diagram of the device of data storage provided by the embodiments of the present application.
Specific embodiment
In order to make those skilled in the art more fully understand application scheme, below in conjunction in the embodiment of the present application
Attached drawing, the technical scheme in the embodiment of the application is clearly and completely described, it is clear that described embodiment is only this
Apply for a part of the embodiment, instead of all the embodiments.Based on the embodiment in the application, those of ordinary skill in the art exist
Every other embodiment obtained under the premise of creative work is not made, shall fall in the protection scope of this application.
Computer is more and more common in daily life and social work at this stage, also more and more important.Information-based, net
The trend of network, so that more and more enterprises, businessman, government bodies and individual etc. obtain information, processing by computer
Information, while a large amount of information being saved in a computer in the form of data file.Inventor has found that with letter
Breath amount is more and more abundant, data volume is increasing, and the problems such as data storage bottleneck gradually occur in many mass data platforms, that is,
How server will efficiently and rapidly store magnanimity, various data.
In order to solve this problem, in the embodiment of the present application, firstly, obtaining the first number to be stored of target data source
According to, wherein target data source corresponds to target mapping ruler and target structural data template；Secondly, being based on the first number to be stored
The second data to be stored being consistent according to data structure is obtained with target structural data template；Then, according to target mapping ruler
Second data to be stored is mapped into corresponding target storage server；Finally, in target storage server to second wait store
Data carry out cloud storage.It can be seen that first obtaining the second number to be stored that data structure is consistent with target structural data template
According to, then mapped so as to subsequent cloud storage based on the corresponding target mapping ruler of data source, target structural data template and
Target mapping ruler can meet the storage demand of the data source most possibly, improve storage efficiency and storage speed.
For example, one of the scene of the embodiment of the present application, can be applied in scene as shown in Figure 1, the scene
Including processor 101 and user terminal 102, wherein processor 101 is interacted with user terminal 102, and user terminal 102 can
To be PC machine, it is also possible to other mobile terminals, such as mobile phone or platform computer.User carries out first in user terminal 102
The cloud storage of data to be stored operates, and processor 101 obtains in response to the operation and carrys out user terminal 102 derived from target data
First data to be stored in source, the target data source correspond to target mapping ruler and target structural data template；Processor
101 obtain the second data to be stored, the data structure of second data to be stored and institute based on first data to be stored
Target structural data template is stated to be consistent；Processor 101 reflects second data to be stored according to the target mapping ruler
It is incident upon corresponding target storage server；Processor 101 the target storage server to second data to be stored into
It racks storage.Processor 101 returns to the successful information of cloud storage to the user terminal 102, so that user is based on user terminal
102 clear first data to be stored cloud storage successes.
It is understood that in above-mentioned application scenarios, although by the action description of the application embodiment by server
101 execute, and still, the application is unrestricted in terms of executing subject, move disclosed in the application embodiment as long as performing
Work.
It is understood that above-mentioned scene is only a Sample Scenario provided by the embodiments of the present application, the embodiment of the present application
It is not limited to this scene.
With reference to the accompanying drawing, the method and apparatus that data store in the embodiment of the present application are described in detail by embodiment
Specific implementation.
Illustrative methods
Referring to fig. 2, a kind of flow diagram of the method for data storage in the embodiment of the present application is shown.In the present embodiment
In, the method for example may comprise steps of:
Step 201: obtaining the first data to be stored of target data source, the target data source corresponds to target mapping ruler
With target structural data template.
It is understood that being operated in response to user in the cloud storage of user terminal, the available number to be stored of processor
According to, that is, the first data to be stored specifies the data source of first data to be stored, that is, target data source, so as to subsequent bright
Really what corresponding mode is cloud storage is carried out using.
It should be noted that the source due to various data is different, that is, various data have different
The data storage requirement of data source, different data sources may be different, in order to facilitate data storage, should preset each data
The corresponding mapping ruler of the data storage requirement in source, and different mapping rulers corresponds to and different are suitble to and are easy to carry out data
The structural data template of mapping.Specifically, for target data source, correspondence is preset based on its data storage requirement
Mapping ruler as target mapping ruler；Corresponding structural data template is pre-generated based on target mapping ruler again, is made
For the target structural data template of corresponding target data source.Therefore, in some embodiments of the present embodiment, such as may be used also
With the following steps are included:
Step A: according to the data storage requirement of the target data source, the corresponding target mapping rule are preset
Then.
Step B: according to the target mapping ruler, the corresponding target structural data template is pre-generated.
Wherein, it should be noted that mapping ruler is what the data storage requirement based on data source was set, common data
Storage demand for example can be and carry out storage demand based on data type, carried out storage demand based on data source, based on data
Structure is carried out storage demand, storage demand is carried out based on data time and carries out storage demand etc. based on data category, then base
Can be to be stored based on any one of the above data in the target mapping ruler that the data storage requirement of target data source is set is needed
Ask setting.Therefore, in some embodiments of the present embodiment, the target mapping ruler includes DATATYPES TO rule
Then, data source mapping ruler, data structure mapping ruler, data time mapping ruler or data category mapping ruler.
Step 202: the second data to be stored is obtained based on first data to be stored, second data to be stored
Data structure is consistent with the target structural data template.
It is understood that the data that data structure is consistent with target structural data template are just most suitable for target mapping rule
Then, it is easy to carry out data mapping using target mapping ruler, then is needed before carrying out data mapping first based on first wait store
Data obtain the second data to be stored that data structure is consistent with target structural data template, to improve follow-up data mapping
Performance and efficiency.
Specifically, it is necessary first to judge the data structure of the first data to be stored and target structural data template whether phase
Symbol, result are to be consistent or be not consistent.When the two is consistent, show the data structure of the first data to be stored with target knot
Structure data template is consistent, and does not need to carry out other data processing operations, can be determined directly as data structure and target structural
The second data to be stored that data template is consistent.When the two is not consistent, show that the data structure of the first data to be stored does not conform to
It is suitable, it needs to carry out pretreatment operation to the first data to be stored, that is, it is pre-processed based on target structural data template, with
Just the second data to be stored that data structure is consistent with target structural data template is obtained.Therefore, in some of the present embodiment
In embodiment, the step 202 for example be may comprise steps of:
Step C: judge the data structure of first data to be stored and the target structural data template whether phase
Symbol；If so, executing step D；If it is not, executing step E.
Wherein, for step C, the target structural data template of corresponding target data source should be obtained first, so
Afterwards, compare the data structure and target structural data template of the first data to be stored, to judge whether based on comparative result
The two is consistent.Therefore, in some embodiments in the present embodiment, the step C for example be may comprise steps of:
Step C1: according to the target data source, the corresponding target structural data template is obtained；
Step C2: the data structure of first data to be stored and the target structural data template are compared
Compared with judging whether to be consistent according to comparison result.
Step D: first data to be stored is determined as second data to be stored.
Step E: pre-processing first data to be stored according to the target structural data template, obtains institute
State the second data to be stored.
Step 203: second data to be stored being mapped into corresponding target according to the target mapping ruler and is stored
Server.
It is understood that step 202 obtain that data structure is consistent with target structural data template second wait deposit
It after storing up data, needs to carry out data mapping based on corresponding target mapping ruler, to map to corresponding target storage clothes
It is engaged on device, so that the data storage requirement for meeting target data source of maximum possible, improves data storage efficiency.
Specifically, for step 203, firstly, the target mapping ruler of corresponding target data source should be obtained, so
Afterwards, data mapping is carried out using target mapping ruler to the second data to be stored.Therefore, in some embodiments of the present embodiment
In, the step 202 for example may comprise steps of:
Step F: according to the target data source, the corresponding target mapping ruler is obtained；
Step G: second data to be stored is mapped into corresponding target storage clothes according to the target mapping ruler
Business device.
It should be noted that being mapped in second on target memory wait store after step 203 completes data mapping
Data may with stored data on target memory disk there are Data duplications the case where, it is superfluous in order to avoid repeated data
It is remaining, the storage operation of repeated data is reduced, data space utilization rate is effectively improved, needs to show target storage server
The second data to be stored penetrated carries out data de-duplication using corresponding target data de-duplication mode.Therefore, in this reality
It applies in some embodiments of example, after step 203, such as can also include step H: according to target data de-duplication side
Formula carries out data de-duplication, the target repeated data to the second data to be stored mapped in the target storage server
Deletion mode corresponds to the target data source.
Similarly, the characteristic of different structural data templates is different, then corresponding data de-duplication mode is also just different,
That is, the second data to be stored that data structure is consistent with target structural data template, there is corresponding target repeated data to delete
Except mode, which is to be in advance based on the property settings of target structural data template.Therefore, exist
It in some embodiments of the present embodiment, such as can also include step I: according to the spy of the target structural data template
Property, preset the corresponding target data de-duplication mode.
It should be noted that since target memory disk is divided into various data blocks (can also abbreviation block), target repeat number
According to deletion mode be used to delete the second data to be stored for being mapped on target memory relative on target memory disk
Duplicate data in the data of storage, specifically both comparisons whether there is repeated data and carry out the mistake of data de-duplication
Journey, then target data de-duplication mode can be the data de-duplication mode of fixed size block, be also possible to the fritter that becomes larger
Data de-duplication mode.Therefore, in some embodiments of the present embodiment, the target data de-duplication mode packet
Include the data de-duplication mode of fixed size block or the data de-duplication mode for the fritter that becomes larger.
It should also be noted that, step H execute can by the data de-duplication module in target storage server into
It is capable, data de-duplication module use a hash function for data block calculate verification and, the hash function is with very high general
Rate uniquely identifies data.For example, the probability that the Hash of hash function SHA256 collides is about 2-256.Data de-duplication
Data block is recorded in a tables of data by module, and the tables of data establishes the verification from data block and the storage to data block
The mapping of position, and the mapping of the verification from data block and the reference number to data block.When data block takes in the storage
In business device when data with existing backup, only increase the reference number of the data block.When data high redundancy, with backup services
The mode of device, backup virtual machine image or backup source code library reduces space consuming at double.Certainly, data de-duplication can also be
File or byte level carry out.
Step 204: cloud storage being carried out to second data to be stored in the target storage server.
It is understood that after step 203 completes data mapping, it can be directly by the second data to be stored in target
Storage server carries out cloud storage, and data storage is completed.
Certainly, when after step 203 according to target data de-duplication mode to mapping in target storage server
After second data to be stored carries out data de-duplication, the second data to be stored for completing data de-duplication should be stored, that is,
In some embodiments of the present embodiment, the step 204 is for example specifically as follows: in the target storage server to complete
Cloud storage is carried out at the second data to be stored of data de-duplication.
To sum up, it when the first data to be stored of target data source needs to carry out data storage, can be based on shown in Fig. 3
A kind of schematic illustration of data storage, carries out the detailed description of data storage.Wherein, target data source is data source 1, data
The corresponding target mapping mode in source 1 is mapping ruler 1, and the target structural data template that mapping ruler 1 generates is structuring number
According to template 1.The first data to be stored for obtaining data source 1, judges the data structure and structural data of the first data to be stored
Whether template 1 is consistent, if so, the first data to be stored is determined as the second data to be stored；If it is not, according to structural data mould
Plate 1 pre-processes the first data to be stored, obtains the second data to be stored.According to mapping ruler 1 by the second number to be stored
According to mapping to corresponding target storage server, that is, distributing the second data to be stored to storage server 1, storage server 2
In storage server 3.Storage server 1, storage server 2 and storage server 3 respectively to mapping data on it into
It racks storage.
It should also be noted that, after step 204 completes data storage, it is contemplated that the subsequent data access behaviour that can be carried out
Make, it should determine that the first data to be stored of target data source is stored in the data access interface of target storage server, the number
According to the mathematical logic relationship determination that access interface is based on target storage server.Therefore, in some implementations of the present embodiment
It after step 204, such as further include step J in mode: according to the mathematical logic relationship of the target storage server, really
Determine data access interface.
The various embodiments provided through this embodiment, firstly, the first data to be stored of target data source is obtained,
In, target data source corresponds to target mapping ruler and target structural data template；Secondly, being obtained based on the first data to be stored
The second data to be stored that data structure is consistent with target structural data template；Then, according to target mapping ruler by second
Data to be stored maps to corresponding target storage server；Finally, target storage server to the second data to be stored into
It racks storage.It can be seen that first obtaining the second data to be stored that data structure is consistent with target structural data template, then base
It is mapped in the corresponding target mapping ruler of data source so as to subsequent cloud storage, target structural data template and target mapping
Rule can meet the storage demand of the data source most possibly, improve storage efficiency and storage speed.
Example devices
Referring to fig. 4, a kind of structural schematic diagram of the device of data storage in the embodiment of the present application is shown.In the present embodiment
In, described device for example can specifically include:
Acquiring unit 401, for obtaining the first data to be stored of target data source, the target data source corresponds to target
Mapping ruler and target structural data template；
Obtaining unit 402, for obtaining the second data to be stored based on first data to be stored, described second wait deposit
The data structure of storage data is consistent with the target structural data template；
Map unit 403, it is corresponding for mapping to second data to be stored according to the target mapping ruler
Target storage server；
Storage unit 404, for carrying out cloud storage to second data to be stored in the target storage server.
Optionally, described device further include:
Delete unit, for according to target data de-duplication mode to second mapped in the target storage server
Data to be stored carries out data de-duplication, and the target data de-duplication mode corresponds to the target data source；
Accordingly, the storage unit 404 is specifically used for:
Cloud storage is carried out to the second data to be stored for completing data de-duplication in the target storage server.
Optionally, described device further include:
First setup unit is preset corresponding described for the data storage requirement according to the target data source
Target mapping ruler；
Generation unit, for pre-generating the corresponding target structural data mould according to the target mapping ruler
Plate；
Second setup unit is preset corresponding described for the characteristic according to the target structural data template
Target data de-duplication mode.
Optionally, the target mapping ruler includes DATATYPES TO rule, data source mapping ruler, data structure
Mapping ruler, data time mapping ruler or data category mapping ruler.
Optionally, the target data de-duplication mode includes the data de-duplication mode of fixed size block or becomes larger
The data de-duplication mode of fritter.
Optionally, the obtaining unit 402 includes judgment sub-unit, determines subelement and obtain subelement:
The judgment sub-unit, for judge first data to be stored data structure and the target structural number
Whether it is consistent according to template；If so, into the determining subelement；If it is not, into the acquisition subelement；
The determining subelement, for first data to be stored to be determined as second data to be stored；
The acquisition subelement, for being carried out according to the target structural data template to first data to be stored
Pretreatment obtains second data to be stored.
Optionally, the judgment sub-unit includes obtaining module and comparison module:
The acquisition module, for obtaining the corresponding target structural data template according to the target data source；
The comparison module, for by the data structure of first data to be stored and the target structural data mould
Plate is compared, and judges whether to be consistent according to comparison result.
Optionally, the map unit 403 includes obtaining subelement and mapping subelement:
The acquisition subelement, for obtaining the corresponding target mapping ruler according to the target data source；
The mapping subelement, for second data to be stored to be mapped to correspondence according to the target mapping ruler
Target storage server.
Optionally, described device further include:
Determination unit determines data access interface for the mathematical logic relationship according to the target storage server.
The various embodiments provided through this embodiment, acquiring unit are used to obtain the first of target data source wait store
Data, the target data source correspond to target mapping ruler and target structural data template；Obtaining unit is used for based on described
First data to be stored obtains the second data to be stored, the data structure and the target structural of second data to be stored
Data template is consistent；Map unit is used to map to second data to be stored according to the target mapping ruler corresponding
Target storage server；Storage unit is used to carry out cloud to second data to be stored in the target storage server to deposit
Storage.It can be seen that first obtaining the second data to be stored that data structure is consistent with target structural data template, then it is based on data
The corresponding target mapping ruler in source is mapped so as to subsequent cloud storage, target structural data template and target mapping ruler energy
Enough storage demands for meeting the data source most possibly, improve storage efficiency and storage speed.
Each embodiment in this specification is described in a progressive manner, the highlights of each of the examples are with other
The difference of embodiment, the same or similar parts in each embodiment may refer to each other.For device disclosed in embodiment
For, since it is corresponded to the methods disclosed in the examples, so being described relatively simple, related place is said referring to method part
It is bright.
Professional further appreciates that, unit described in conjunction with the examples disclosed in the embodiments of the present disclosure
And algorithm steps, can be realized with electronic hardware, computer software, or a combination of the two, in order to clearly demonstrate hardware and
The interchangeability of software generally describes each exemplary composition and step according to function in the above description.These
Function is implemented in hardware or software actually, the specific application and design constraint depending on technical solution.Profession
Technical staff can use different methods to achieve the described function each specific application, but this realization is not answered
Think beyond scope of the present application.
It should be noted that, in this document, relational terms such as first and second and the like are used merely to a reality
Body or operation are distinguished with another entity or operation, are deposited without necessarily requiring or implying between these entities or operation
In any actual relationship or order or sequence.The terms "include", "comprise" or its any other variant are intended to non-row
His property includes, so that the process, method, article or equipment for including a series of elements not only includes those elements, and
And further include other elements that are not explicitly listed, or further include for this process, method, article or equipment institute it is intrinsic
Element.In the absence of more restrictions, the element limited by sentence "including a ...", it is not excluded that including institute
State in the process, method, article or equipment of element that there is also other identical elements.
The above is only the preferred embodiment of the application, not makes any form of restriction to the application.Though
Right the application has been disclosed in a preferred embodiment above, however is not limited to the application.It is any to be familiar with those skilled in the art
Member, in the case where not departing from technical scheme ambit, all using the methods and technical content of the disclosure above to the application
Technical solution makes many possible changes and modifications or equivalent example modified to equivalent change.Therefore, it is all without departing from
The content of technical scheme, any simple modification made to the above embodiment of the technical spirit of foundation the application are equal
Variation and modification, still fall within technical scheme protection in the range of.
Claims (10)
1. a kind of method of data storage characterized by comprising
The first data to be stored of target data source is obtained, the target data source corresponds to target mapping ruler and target structural
Data template；
The second data to be stored, the data structure of second data to be stored and institute are obtained based on first data to be stored
Target structural data template is stated to be consistent；
Second data to be stored is mapped into corresponding target storage server according to the target mapping ruler；
Cloud storage is carried out to second data to be stored in the target storage server.
2. the method according to claim 1, wherein further include:
Weight is carried out to the second data to be stored mapped in the target storage server according to target data de-duplication mode
Complex data is deleted, and the target data de-duplication mode corresponds to the target data source；
It is accordingly, described that cloud storage is carried out to second data to be stored in the target storage server, specifically:
Cloud storage is carried out to the second data to be stored for completing data de-duplication in the target storage server.
3. according to the method described in claim 2, it is characterized by further comprising:
According to the data storage requirement of the target data source, the corresponding target mapping ruler is preset；
According to the target mapping ruler, the corresponding target structural data template is pre-generated；
According to the characteristic of the target structural data template, the corresponding target data de-duplication mode is preset.
4. according to the method described in claim 3, it is characterized in that, the target mapping ruler includes DATATYPES TO rule
Then, data source mapping ruler, data structure mapping ruler, data time mapping ruler or data category mapping ruler.
5. according to the method described in claim 2, it is characterized in that, the target data de-duplication mode includes fixed size
The data de-duplication mode of block or the data de-duplication mode for the fritter that becomes larger.
6. the method according to claim 1, wherein it is described based on first data to be stored obtain second to
Storing data, comprising:
Judge whether the data structure of first data to be stored is consistent with the target structural data template；
If so, first data to be stored is determined as second data to be stored；
If it is not, pre-process according to the target structural data template to first data to be stored, described the is obtained
Two data to be stored.
7. the method according to claim 1, wherein the data structure of judgement first data to be stored
Whether it is consistent with the target structural data template, comprising:
According to the target data source, the corresponding target structural data template is obtained；
The data structure of first data to be stored is compared with the target structural data template, is tied according to comparing
Fruit judges whether to be consistent.
8. the method according to claim 1, wherein it is described according to the target mapping ruler by described second to
Storing data maps to corresponding target storage server, comprising:
According to the target data source, the corresponding target mapping ruler is obtained；
Second data to be stored is mapped into corresponding target storage server according to the target mapping ruler.
9. the method according to claim 1, wherein further include:
According to the mathematical logic relationship of the target storage server, data access interface is determined.
10. a kind of device of data storage characterized by comprising
Acquiring unit, for obtaining the first data to be stored of target data source, the target data source corresponds to target mapping rule
Then with target structural data template；
Obtaining unit, for obtaining the second data to be stored, second data to be stored based on first data to be stored
Data structure be consistent with the target structural data template；
Map unit is stored for second data to be stored to be mapped to corresponding target according to the target mapping ruler
Server；
Storage unit, for carrying out cloud storage to second data to be stored in the target storage server.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910002082.6A CN109669644A (en) | 2019-01-02 | 2019-01-02 | A kind of method and apparatus of data storage |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910002082.6A CN109669644A (en) | 2019-01-02 | 2019-01-02 | A kind of method and apparatus of data storage |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN109669644A true CN109669644A (en) | 2019-04-23 |
Family
ID=66146714
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910002082.6A Pending CN109669644A (en) | 2019-01-02 | 2019-01-02 | A kind of method and apparatus of data storage |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109669644A (en) |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111061743A (en) * | 2019-12-25 | 2020-04-24 | 北京百度网讯科技有限公司 | Data processing method and device and electronic equipment |
| CN112905122A (en) * | 2021-02-20 | 2021-06-04 | 炬芯科技股份有限公司 | Data storage method and device |
| CN115562911A (en) * | 2022-12-07 | 2023-01-03 | 中科方德软件有限公司 | Virtual machine data backup method, device, system, electronic equipment and storage medium |
| CN117726170A (en) * | 2023-12-18 | 2024-03-19 | 北京罗格数据科技有限公司 | Data storage method and device |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102436408A (en) * | 2011-10-10 | 2012-05-02 | 上海交通大学 | Data storage cloud and cloud backup method based on Map/Dedup |
| CN104765574A (en) * | 2015-04-23 | 2015-07-08 | 成都博元时代软件有限公司 | Data cloud storage method |
| CN106227769A (en) * | 2016-07-15 | 2016-12-14 | 北京奇虎科技有限公司 | Date storage method and device |
-
2019
- 2019-01-02 CN CN201910002082.6A patent/CN109669644A/en active Pending
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102436408A (en) * | 2011-10-10 | 2012-05-02 | 上海交通大学 | Data storage cloud and cloud backup method based on Map/Dedup |
| CN104765574A (en) * | 2015-04-23 | 2015-07-08 | 成都博元时代软件有限公司 | Data cloud storage method |
| CN106227769A (en) * | 2016-07-15 | 2016-12-14 | 北京奇虎科技有限公司 | Date storage method and device |
Cited By (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111061743A (en) * | 2019-12-25 | 2020-04-24 | 北京百度网讯科技有限公司 | Data processing method and device and electronic equipment |
| CN111061743B (en) * | 2019-12-25 | 2023-10-31 | 北京百度网讯科技有限公司 | Data processing method and device and electronic equipment |
| CN112905122A (en) * | 2021-02-20 | 2021-06-04 | 炬芯科技股份有限公司 | Data storage method and device |
| CN112905122B (en) * | 2021-02-20 | 2024-04-09 | 炬芯科技股份有限公司 | Method and device for storing data |
| CN115562911A (en) * | 2022-12-07 | 2023-01-03 | 中科方德软件有限公司 | Virtual machine data backup method, device, system, electronic equipment and storage medium |
| CN117726170A (en) * | 2023-12-18 | 2024-03-19 | 北京罗格数据科技有限公司 | Data storage method and device |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109669644A (en) | A kind of method and apparatus of data storage | |
| CN106980623B (en) | Data model determination method and device | |
| WO2021036810A1 (en) | Evidence verification method, system, apparatus and device, and readable storage medium | |
| CN109597822B (en) | User data storage and query method and user data processing device | |
| CN104881466B (en) | The processing of data fragmentation and the delet method of garbage files and device | |
| CN102708183B (en) | Method and device for data compression | |
| CN109145051A (en) | The data summarization method and device and electronic equipment of distributed data base | |
| CN106874348A (en) | File is stored and the method for indexing means, device and reading file | |
| CN110599169A (en) | Data processing method, device, terminal and medium | |
| WO2022048359A1 (en) | Data archiving method and apparatus, and electronic device and storage medium | |
| CN106685741B (en) | Dynamic management method for network interface of cloud host | |
| CN108959573A (en) | Data migration method, device, electronic equipment and storage medium based on desktop cloud | |
| CN105678625A (en) | Method and equipment for determining identity information of user | |
| CN111209061B (en) | User information filling method, device, computer equipment and storage medium | |
| CN113806300A (en) | Data storage method, system, device, equipment and storage medium | |
| US11321318B2 (en) | Dynamic access paths | |
| CN109450731A (en) | A kind of test data generating method of application layer communication protocol | |
| CN114781654A (en) | Federal transfer learning method, device, computer equipment and medium | |
| CN112445776A (en) | Presto-based dynamic barrel dividing method, system, equipment and readable storage medium | |
| CN116842012A (en) | Method, device, equipment and storage medium for storing Redis cluster in fragments | |
| CN110472972B (en) | Synchronous snapshot method and system for block chain storage data | |
| CN109981755A (en) | Image-recognizing method, device and electronic equipment | |
| CN113626438B (en) | Data table management method, device, computer equipment and storage medium | |
| CN114936198A (en) | Data migration method and device | |
| CN104866535A (en) | Compression method and device of number segment records |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20190423 |