CN105892953A - Distributed data processing method and distributed data processing device - Google Patents
Distributed data processing method and distributed data processing device Download PDFInfo
- Publication number
- CN105892953A CN105892953A CN201610260783.6A CN201610260783A CN105892953A CN 105892953 A CN105892953 A CN 105892953A CN 201610260783 A CN201610260783 A CN 201610260783A CN 105892953 A CN105892953 A CN 105892953A
- Authority
- CN
- China
- Prior art keywords
- storage
- data
- stored
- identification code
- random number
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0668—Interfaces specially adapted for storage systems adopting a particular infrastructure
- G06F3/0671—In-line storage system
- G06F3/0683—Plurality of storage devices
- G06F3/0685—Hybrid storage combining heterogeneous device types, e.g. hierarchical storage, hybrid arrays
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0602—Interfaces specially adapted for storage systems specifically adapted to achieve a particular effect
- G06F3/0604—Improving or facilitating administration, e.g. storage management
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0602—Interfaces specially adapted for storage systems specifically adapted to achieve a particular effect
- G06F3/062—Securing storage systems
- G06F3/0622—Securing storage systems in relation to access
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0628—Interfaces specially adapted for storage systems making use of a particular technique
- G06F3/0629—Configuration or reconfiguration of storage systems
- G06F3/0631—Configuration or reconfiguration of storage systems by allocating resources to storage systems
Abstract
The invention discloses a distributed data processing method and a distributed data processing device. The method comprises the following steps of: dividing a storage space of a distributed system into a plurality of storage layers, wherein each storage layer comprises a plurality of storage devices; setting a layer code for each storage layer and a device code for each storage device; distributing a storage identification code for each storage device according to the layer code and the device code; when receiving the data to be stored, generating a random number corresponding to the storage identification code form; and storing the data to be stored to the storage device corresponding to the storage identification code with the highest match degree with the random number. By adopting the method and device, the sufficient utilization of the storage resources in the distributed system is realized.
Description
Technical field
The present invention relates to distributed system technical field, particularly relate to a kind of distributed data processing method and
Device.
Background technology
Carrying out in a deep going way along with informationalized, data produced by the operation system of all trades and professions get more and more,
Due to traditional centralized access plan, (centralized access refers to as data formulation one to be stored specific
Accessing zone, such as data central host, all access all in this region) by equal for all of data
Exist and extract data, if data center's main frame is out of order, the most whole system for concentrating on data center's main frame
System cannot store and extract data, and the most centralized access plan is compared to distributed data access, and data are steady
It is qualitative relatively low, so distributed data access mode is arisen at the historic moment.
Existing distributed data access mode is usually storage under utilized for data to be stored write
In equipment；But, owing to seldom being moved after data write storage device, and along with storage device
Extension and updating, most data are all written to increase newly, in the less new storage device of data, thus
Cause newly-increased storage device storage resource occupation storage device too much, old storage underutilization of resources, it is impossible to
Make full use of the storage resource in distributed system.
Summary of the invention
Present invention is primarily targeted at a kind of distributed data processing method and device of offer, it is intended to solve
Existing distributed system data storage can not make full use of in system the technical problem storing resource.
For achieving the above object, a kind of distributed data processing method that the present invention provides, this distributed number
Include according to processing method:
The memory space of distributed system is divided into multiple accumulation layer, and each accumulation layer includes that some storages set
Standby；
Arrange layer coding for each accumulation layer, each storage device arranges device coding；
It is that each storage device distributes a storage identification code according to layer coding and device coding；
When receiving data to be stored, generate a random number corresponding with described storage identification code form；
Described data to be stored are stored in corresponding to the storage identification code the highest with described nonces match degree
Storage device in.
Preferably, described is that each storage device distributes a storage identification according to layer coding and device coding
Also include after the step of code:
When receiving data and extracting instruction, obtain the target storage identification code that these data are extracted in instruction；
The storage data of corresponding storage device in corresponding accumulation layer are extracted according to described target storage identification code.
Preferably, described for each accumulation layer arrange layer coding, each storage device device coding is set
Step includes:
According to the information security grade of storage data, one layer coding is set for each accumulation layer；
According to the position of same accumulation layer internal memory devices, one device coding is set for each storage device.
Preferably, described described data to be stored are stored in the storage the highest with described nonces match degree
Also include before the step in storage device corresponding to identification code:
Resolve described data to be stored, obtain the information security demand levels of data to be stored；
Obtain information security grade and meet the accumulation layer of described information security demand levels, this accumulation layer conduct
Accumulation layer to be selected, to determine storage device based on random number from accumulation layer to be selected.
Described described data to be stored are stored in the storage identification code institute the highest with described nonces match degree
The corresponding step in storage device includes:
Described random number is mated with the storage identification code of each storage device in accumulation layer to be selected,
Go out each storage device matching degree about own storage identification code Yu random number；
Treat that deposit data is stored in the storage device that described matching degree is the highest by described.
Preferably, described when receiving data to be stored, generate one and described storage identification code form
The step of corresponding random number includes:
When receiving data to be stored, verify the legitimacy of these data to be stored；
If described data to be stored are legal, then the real random number generator controlling to preset generates one with described
The random number that storage identification code form is corresponding.
Preferably, described when receiving data and extracting instruction, obtain these data and extract the target in instruction
The step of storage identification code includes:
When receiving data and extracting instruction, generate a checking random number, and this checking random number is sent out
Deliver to data and extract the transmitting terminal of instruction；
Obtain described transmitting terminal checking information based on checking generating random number, and to described checking information and
Transmitting terminal verifies；
When described checking information and transmitting terminal are deposited by verification, the then target obtained in the extraction instruction of these data
Storage identification code.
The present invention also provides for a kind of distributed data processing device, and this distributed data processing device includes:
Storage of hierarchically module, for being divided into multiple accumulation layer, Mei Gecun by the memory space of distributed system
Reservoir includes some storage devices；
Encoding setting module, for arranging layer coding for each accumulation layer, each storage device arranges equipment
Coding；
Distribution module, is that one storage of each storage device distribution is known for encoding with device coding according to layer
Other code；
Random number module, for when receiving data to be stored, generates one and described storage identification code
The random number that form is corresponding；
Memory module, for being stored in described data to be stored and described the highest the depositing of nonces match degree
In storage storage device corresponding to identification code.
Preferably, described distributed data processing device also includes:
Identification code acquisition module, for when receiving data and extracting instruction, obtains these data and extracts instruction
In target storage identification code；
Data extraction module, deposits for extracting correspondence in corresponding accumulation layer according to described target storage identification code
The storage data of storage equipment.
Preferably, described encoding setting module includes:
Safety distribution unit, for the information security grade according to storage data, is arranged for each accumulation layer
One layer coding；
Encoding setting unit, for the position according to same accumulation layer internal memory devices, sets for each storage
For arranging a device coding.
Preferably, described distributed data processing device also includes:
Parsing module, for, before storing data to be stored, resolving described data to be stored, obtain and wait to deposit
The information security demand levels of storage data；
Layer acquisition module, meets the storage of described information security demand levels for obtaining information security grade
Layer, this accumulation layer is as accumulation layer to be selected, to determine storage device based on random number from accumulation layer to be selected；
Described memory module includes:
Matching unit, for by described random number and the storage identification of each storage device in accumulation layer to be selected
Code mates, and draws each storage device matching degree about own storage identification code Yu random number；
By described, memory element, for treating that deposit data is stored in the storage device that described matching degree is the highest.
Preferably, described random number module includes:
Authentication unit, for when receiving data to be stored, verifies the legitimacy of these data to be stored；
Random number generation unit, if legal for described data to be stored, then controls the true random number preset
Generator generates a random number corresponding with described storage identification code form.
Preferably, described identification code acquisition module includes:
Signal generating unit, for when receiving data and extracting instruction, generates a checking random number, and will
This checking random number sends the transmitting terminal to data extraction instruction；
Verification unit is for obtaining described transmitting terminal checking information based on checking generating random number and right
Described checking information and transmitting terminal verify；
Acquiring unit, for when described checking information and transmitting terminal are by verification, then obtaining these data and extract
Target storage identification code in instruction.
The present invention by being first divided into multiple accumulation layer, each accumulation layer bag by the memory space of distributed system
Include some storage devices；Then arrange layer coding for each accumulation layer, each storage device arranges equipment and compiles
Code；It is that each storage device distributes a storage identification code further according to layer coding and device coding；Work as reception
During to data to be stored, generate a random number corresponding with storage identification code form；Finally by be stored
Data are stored in the storage the highest with nonces match degree storage device corresponding to identification code, thus root
According to the random number generated, data to be stored are stored in each storage device, because of each and storage identification code
The probability that corresponding random number occurs every time is equal, so data to be stored store each storage and set
Standby probability is also identical, and when data to be stored are more, data to be stored can be divided the most uniformly
It is distributed in each storage device, makes the storage resource of storage device new, old all be fully used.
Accompanying drawing explanation
Fig. 1 is the schematic flow sheet of distributed data processing method first embodiment of the present invention；
Fig. 2 is the schematic flow sheet of distributed data processing method one preferred embodiment of the present invention；
Fig. 3 is in distributed data processing method the second embodiment of the present invention when receiving data to be stored,
Generate the refinement schematic flow sheet of a random number corresponding with storage identification code form；
Fig. 4 is when receiving data extraction instructions in distributed data processing method the 3rd embodiment of the present invention
Time, obtain the refinement schematic flow sheet that these data extract the target storage identification code of instruction；
Fig. 5 is the high-level schematic functional block diagram of distributed data processing device first embodiment of the present invention；
Fig. 6 is the high-level schematic functional block diagram of distributed data processing device one preferred embodiment of the present invention；
Fig. 7 is the refinement function of random number module in distributed data processing device the second embodiment of the present invention
Module diagram；
Fig. 8 is the refinement of identification code acquisition module in distributed data processing device the 3rd embodiment of the present invention
High-level schematic functional block diagram.
The realization of the object of the invention, functional characteristics and advantage will in conjunction with the embodiments, do referring to the drawings further
Explanation.
Detailed description of the invention
Should be appreciated that specific embodiment described herein, only in order to explain the present invention, is not used to limit
Determine the present invention.
The present invention provides a kind of distributed data processing method, in distributed data processing method of the present invention
In first embodiment, with reference to Fig. 1, this distributed data processing method includes:
Step S10, is divided into multiple accumulation layer by the memory space of distributed system, if each accumulation layer includes
Dry storage device；
According to factors such as information security grade, storage size, data types, by distributed system
Memory space is divided into multiple accumulation layer, and such as, information security grade is divided into rudimentary, intermediate and senior, deposits
Reservoir also includes rudimentary accumulation layer, middle rank accumulation layer and advanced storage layer, and each accumulation layer includes multiple
Storage device, in same accumulation layer, each storage device is same or similar in ardware feature and software feature.
Step S20, for each accumulation layer arrange layer coding, each storage device device coding is set；
Step S30, is that each storage device distributes a storage identification code according to layer coding and device coding；
One uniquely identified layer coding is set for each accumulation layer, is also that each storage device is arranged simultaneously
One uniquely identified device coding,
Preferably, step S20 includes:
Step S21, according to the information security grade of storage data, arranges a layer coding for each accumulation layer；
Step S22, according to the position of same accumulation layer internal memory devices, arranges one for each storage device
Device coding.
System has different requirements to the information security grade of storage data, such as secret in finance, business
The storage data such as close are private information, have high-grade information security requirement；Such as exploitation, OK
The storage data such as industry specification are public information, have low-grade information security requirement.According to storage data
Information security grade, for each accumulation layer, one layer coding be set, this layer of coding is also corresponding different
Information security grade.Then according to the position of same accumulation layer internal memory devices, set for each storage device
Put a device coding.
The most rudimentary accumulation layer respective layer coding 01, middle rank accumulation layer respective layer coding 02, advanced storage layer
Respective layer coding 03, in rudimentary accumulation layer the device coding of storage device include 0001,0002,0003,
0004,0005, in middle rank accumulation layer the device coding of storage device include 0001,0002,0003,0004,
0005, in advanced storage layer, the device coding of storage device includes 0001,0002,0003,0004,0005.
Then encode according to layer and device coding is that each storage device distributes a storage identification code, the most rudimentary
In storage the storage identification code of storage device be respectively 010001,010002,010003,010004,
010005, thus corresponding storage device can be found quickly and accurately according to storage identification code.
Step S40, when receiving data to be stored, generate one with storage identification code form corresponding with
Machine number；
When receiving data to be stored, the real random number generator controlling to carry generates a random number,
This random number is corresponding or identical with storage identification code form, and such as storage identification code form is six natural numbers,
And front two represents accumulation layer, latter four represent storage device, and in like manner, the random number of generation is also six
Natural number.
Data to be stored are stored in corresponding to the storage identification code the highest with nonces match degree by step S50
Storage device in.
The storage device corresponding to the storage identification code of coupling is searched, the most at random according to the random number generated
Number is 020003, then finding device coding in middle rank accumulation layer is the storage device of 0003, and by be stored
Data storage (i.e. write) is to storing in the storage device that identification code is 020003.
Preferably, with reference to Fig. 1, also include after step S30:
Step S60, when receiving all data extraction instructions, obtains these data and extracts the target storage knowledge of instruction
Other code；
It is that data extract end transmission that data extract instruction, is used for pointing out data storage end to provide data；Number
According to extraction instruction comprising extraction data-triggered information, the storage identification code of data, checking information need to be extracted
Deng, data are extracted the transmitting terminal of instruction and are verified or/and data are extracted instruction by distributed access device
Integrity verify, if described transmitting terminal and data extract instruction all by verifying, then obtain data
Extract the target storage identification code in instruction.
Step S70, extracts the storage number of corresponding storage device in corresponding accumulation layer according to target storage identification code
According to.
The target identifying code extracted data in instruction is analyzed, and first determines the accumulation layer of correspondence, then
In this accumulation layer determined, find the storage device of correspondence, thus extract data and extract instruction correspondence
Storage data, it is achieved to storage data quickly the most respectively and extract, thus can extract in instruction according to data
Storage identification code reversely extract storage data, it is achieved thereby that the filling of the storage resource in distributed system
Divide and utilize.
In the present embodiment, by first the memory space of distributed system being divided into multiple accumulation layer, each
Accumulation layer includes some storage devices；Then arrange layer coding for each accumulation layer, each storage device sets
Put device coding；It is that each storage device distributes a storage identification code further according to layer coding and device coding；
When receiving data to be stored, generate a random number corresponding with storage identification code form；To wait to deposit
Storage data are stored in the storage the highest with nonces match degree storage device corresponding to identification code, thus
Data to be stored are stored in each storage device by the random number according to generating, because each identifies with storage
The probability that the random number that code is corresponding occurs every time is equal, so data to be stored store each storage
The probability of equipment is also identical, and when data to be stored are more, data to be stored can be the most uniform
It is distributed in each storage device, makes the storage resource of storage device new, old all be fully used.
Preferably, with reference to Fig. 2, before step S50, distributed data processing method of the present invention also includes:
Step S80, resolves data to be stored, obtains the information security demand levels of data to be stored；
Step S90, obtains information security grade and meets the accumulation layer of information security demand levels, this accumulation layer
As accumulation layer to be selected, to determine storage device based on random number from accumulation layer to be selected.
After generating the random number, first data to be stored can be resolved, obtain these data to be stored
Information security demand levels, it is then determined that the information security grade institute mated with this information security demand levels
Corresponding accumulation layer, the accumulation layer this determined is as accumulation layer to be selected.
Further, step S50 includes:
Step S51, is carried out the storage identification code of each storage device in random number and accumulation layer to be selected
Join, draw each storage device matching degree about own storage identification code Yu random number；
Step S52, will treat that deposit data is stored in the storage device that matching degree is the highest.
After determining accumulation layer to be selected according to data to be stored, according to random number first from accumulation layer to be selected
Determine layer coding and the highest accumulation layer of nonces match degree, then proceed to according to random number from matching degree
The storage device selecting device coding in high accumulation layer the highest with nonces match degree sets as storage to be selected
Standby, will mate with the storage identification code of each storage device in accumulation layer to be selected by random number, draw
Each storage device is about the matching degree of own storage identification code Yu random number, and it is the highest to select matching degree
Storage device to be selected, finally will treat that deposit data is stored in matching degree the highest storage device (storage the most to be selected
Equipment) in, and can reversely extract storage data according to the storage identification code that data are extracted in instruction, from
And targetedly data to be stored are stored different storage devices according to information security demand levels
In, on the premise of ensureing data information security to be stored, also achieve and achieve in distributed system
Making full use of of storage resource.
Further, on the basis of distributed data processing method first embodiment of the present invention, propose to divide
Cloth data processing method the second embodiment, with reference to Fig. 3, step S40 includes:
Step S41, when receiving data to be stored, verifies the legitimacy of these data to be stored；
Step S42, if data to be stored are legal, then control the real random number generator preset generate one with
The random number that storage identification code form is corresponding.
When receiving data to be stored, need to the legitimacy of these data of data to be stored own (the most whether
Completely, truly) verify with the legitimacy of data receiver identity, only checking treat deposit data this
When body is legal and data receiver identity is legal, i.e. when deposit data is legal, just control preset truly random
Number generator generates a random number corresponding with storage identification code form, thus is guaranteeing data to be stored
For time legal, just generation random number is in case the storage device of subsequently selected correspondence stores, thus enters one
Step improves the safety of distributed data access of the present invention.
Further, on the basis of distributed data processing method first embodiment of the present invention, propose to divide
Cloth data processing method the 3rd embodiment, with reference to Fig. 4, step S60 includes:
Step S61, when receiving data and extracting instruction, generates a checking random number, and by this checking
Random number sends the transmitting terminal to data extraction instruction；
Step S62, obtain transmitting terminal based on checking generating random number checking information, and to checking information and
Transmitting terminal verifies；
Step S63, when checking information and transmitting terminal are by verification, then obtain these data and extracts the mesh in instruction
Mark storage identification code.
When receiving data and extracting instruction, distributed data processing device occurs based on carrying true random number
Device (utilizing physical event rather than the generator of software algorithm generation random number) generates a checking random number,
And this checking random number is sent the transmitting terminal to data extraction instruction, data extract the transmitting terminal root of instruction
The AES according to this checking random number and specified and key, generate corresponding checking information, and data
This checking information is sent to distributed data processing device by the transmitting terminal extracting instruction；At distributed data
Checking information and transmitting terminal, based on the checking information received, are verified by reason device, if checking is described
Checking information transmitting terminal is legal and checking information is correct, then obtain target storage and identify.
In the present embodiment, when receiving data and extracting instruction, first extract instruction transmitting terminal to data and send out
Sending a checking random number, this checking random number is true random number, i.e. the change of this checking random number is
Real-time, irregular, thus the checking information obtained also is change real-time, irregular, improves
The safety of checking information, and only by checking information and the verification of transmitting terminal, just obtain these data
Extract the target storage identification code in instruction, to carry out follow-up data extraction procedure, thus determine number
After identity according to the transmitting terminal extracting instruction, just allow it to carry out data extraction, improve whole distribution
The safety of formula system.
The present invention also provides for a kind of distributed data processing device, real at distributed data processing device first
Executing in example, with reference to Fig. 5, this distributed data processing device includes:
Storage of hierarchically module 10, for the memory space of distributed system is divided into multiple accumulation layer, each
Accumulation layer includes some storage devices；
Storage of hierarchically module 10 according to factors such as information security grade, storage size, data types,
The memory space of distributed system is divided into multiple accumulation layer, such as, information security grade be divided into rudimentary,
Intermediate and senior, accumulation layer also includes rudimentary accumulation layer, middle rank accumulation layer and advanced storage layer, Mei Gecun
Reservoir includes multiple storage device, and in same accumulation layer, each storage device is special in ardware feature and software
In property same or similar.
Encoding setting module 20, for arranging layer coding for each accumulation layer, each storage device sets
Standby coding；
Distribution module 30, is that each storage device distributes a storage for encoding with device coding according to layer
Identification code；
Encoding setting module 20 arranges a uniquely identified layer coding for each accumulation layer, is also every simultaneously
Individual storage device arranges a uniquely identified device coding, and distribution module 30 is compiled according to layer coding and equipment
Code is that each storage device distributes a storage identification code.
Preferably, encoding setting module 20 includes:
Safety distribution unit 21, for the information security grade according to storage data, sets for each accumulation layer
Put a layer coding；
Encoding setting unit 22, for the position according to same accumulation layer internal memory devices, for each storage
Equipment arranges a device coding.
System has different requirements to the information security grade of storage data, such as secret in finance, business
The storage data such as close are private information, have high-grade information security requirement；Such as exploitation, OK
The storage data such as industry specification are public information, have low-grade information security requirement.Safety distribution unit
21, according to the information security grade of storage data, arrange layer coding for each accumulation layer, and this layer encodes
Also corresponding different information security grade.Then encoding setting unit 22 stores according in same accumulation layer
The position of equipment, arranges a device coding for each storage device.
The most rudimentary accumulation layer respective layer coding 01, middle rank accumulation layer respective layer coding 02, advanced storage layer
Respective layer coding 03, in rudimentary accumulation layer the device coding of storage device include 0001,0002,0003,
0004,0005, in middle rank accumulation layer the device coding of storage device include 0001,0002,0003,0004,
0005, in advanced storage layer, the device coding of storage device includes 0001,0002,0003,0004,0005.
Then encode according to layer and device coding is that each storage device distributes a storage identification code, the most rudimentary
In storage the storage identification code of storage device be respectively 010001,010002,010003,010004,
010005, thus corresponding storage device can be found quickly and accurately according to storage identification code.
Random number module 40, for when receiving data to be stored, generates one and storage identification code lattice
The random number that formula is corresponding；
When receiving data to be stored, the real random number generator that random number module 40 controls to carry generates
One random number, this random number is corresponding or identical with storage identification code form, such as storage identification code form
Being six natural numbers, and front two represents accumulation layer, latter four represent storage device, in like manner, generation
Random number is also six natural numbers.
Memory module 50, for being stored in the storage identification the highest with nonces match degree by data to be stored
In storage device corresponding to Ma；
Memory module 50 is searched the storage corresponding to storage identification code of coupling and is set according to the random number generated
Standby, such as random number is 020003, then finding device coding in middle rank accumulation layer is the storage device of 0003,
And data to be stored are stored (i.e. write) to storing in the storage device that identification code is 020003.
Preferably, with reference to Fig. 1, distributed data processing device also includes:
Identification code acquisition module 60, for when receiving data and extracting instruction, obtains the extraction of these data and refers to
Target storage identification code in order；
It is that data extract end transmission that data extract instruction, is used for pointing out data storage end to provide data；Number
According to extraction instruction comprising extraction data-triggered information, the storage identification code of data, checking information need to be extracted
Deng, data are extracted the transmitting terminal of instruction and are verified or/and extract data and refer to by identification code acquisition module 60
The integrity of order is verified, if described transmitting terminal and data extract instruction all by checking, then obtains number
According to the target storage identification code extracted in instruction.
Data extraction module 70, for extracting in corresponding accumulation layer corresponding according to described target storage identification code
The storage data of storage device.
The target identifying code that data are extracted in instruction by data extraction module 70 is analyzed, and first determines correspondence
Accumulation layer, in this accumulation layer determined, then find the storage device of correspondence, thus extract data
Extract the storage data that instruction is corresponding, it is achieved to storage data the most respectively and extract, thus can basis
The storage identification code that data are extracted in instruction reversely extracts storage data, it is achieved thereby that in distributed system
The making full use of of storage resource.
In the present embodiment, first the memory space of distributed system is divided into many by storage of hierarchically module 10
Individual accumulation layer, each accumulation layer includes some storage devices；Then encoding setting module 20 is each storage
Layer arranges layer coding, each storage device arranges device coding；Distribution module 30 is further according to layer coding and sets
A storage identification code is distributed for being encoded to each storage device；When receiving data to be stored, at random
Digital-to-analogue block 40 generates a random number corresponding with storage identification code form；Last memory module 50 will be treated
Storage data are stored in the storage the highest with nonces match degree storage device corresponding to identification code, from
And according to the random number generated, data to be stored are stored in each storage device, because each is known with storage
The probability that the random number that other code is corresponding occurs every time is equal, deposits so data to be stored store each
The probability of storage equipment is also identical, and when data to be stored are more, data to be stored can be the most uniform
Be distributed in each storage device, even if newly, the storage resource of old storage device is all fully used.
Preferably, with reference to Fig. 6, distributed data processing device also includes:
Parsing module 80, for, before storing data to be stored, resolving data to be stored, obtain to be stored
The information security demand levels of data；
Layer acquisition module 90, meets the accumulation layer of information security demand levels for obtaining information security grade,
This accumulation layer is as accumulation layer to be selected, to determine storage device based on random number from accumulation layer to be selected.
After generating the random number, data to be stored can first be resolved by parsing module 60, obtains this and treats
The information security demand levels of storage data, then layer acquisition module 70 determines and this information security demand etc.
The accumulation layer corresponding to information security grade of level coupling, the accumulation layer this determined is as accumulation layer to be selected.
Further, memory module 50 includes:
Matching unit 51, for by random number and the storage identification code of each storage device in accumulation layer to be selected
Mate, draw each storage device matching degree about own storage identification code Yu random number；
Memory element 52, for treating that deposit data is stored in the storage device that matching degree is the highest.
After determining accumulation layer to be selected according to data to be stored, matching unit 51 is first from treating according to random number
Select the accumulation layer determining in accumulation layer that layer coding is the highest with nonces match degree, then proceed to according to random number
The storage device conduct that device coding is the highest with nonces match degree is selected from the accumulation layer that matching degree is the highest
Storage device to be selected, i.e. matching unit 51 are by random number and the storage of each storage device in accumulation layer to be selected
Identification code mates, and draws each storage device matching degree about own storage identification code Yu random number,
And select the storage device to be selected that matching degree is the highest, last memory element 52 will treat that deposit data is stored in coupling
Spend in the highest storage device (storage device the most to be selected), and depositing in instruction can be extracted according to data
Storage identification code reversely extracts storage data, thus will wait targetedly according to information security demand levels to deposit
Storage data store in different storage devices, on the premise of ensureing data information security to be stored, also
Achieve making full use of of the storage resource in distributed system.
Further, on the basis of distributed data processing device first embodiment of the present invention, propose to divide
Cloth data processing equipment the second embodiment, with reference to Fig. 7, in a second embodiment, random number module 40
Including:
Authentication unit 41, for when receiving data to be stored, verifies the legitimacy of these data to be stored；
Random number generation unit 42, if legal for data to be stored, then the true random number controlling to preset is sent out
Raw device generates a random number corresponding with storage identification code form.
When receiving data to be stored, authentication unit 41 need to legal to these data of data to be stored own
Property (the most complete, true) and the legitimacy of data receiver identity verify, only verifying
Legal until deposit data itself and time data receiver identity is legal, i.e. when deposit data is legal, random number is raw
Become unit 42 just controls preset real random number generator generation one with store identification code form corresponding with
Machine number, thus when guaranteeing that data to be stored are legal, just generates random number in case subsequently selected correspondence
Storage device stores, thus further increases the safety of distributed data access of the present invention.
Further, on the basis of distributed data processing device first embodiment of the present invention, propose to divide
Cloth data processing equipment the 3rd embodiment, with reference to Fig. 8, identification code acquisition module 60 includes:
Signal generating unit 61, for when receiving data and extracting instruction, generates a checking random number, and
This checking random number is sent the transmitting terminal to data extraction instruction；
Verification unit 62, for obtaining described transmitting terminal checking information based on checking generating random number, and
Described checking information and transmitting terminal are verified；
Acquiring unit 63, for when described checking information and transmitting terminal are by verification, then obtaining these data and carry
Target storage identification code in instruction fetch.
When receiving data and extracting instruction, signal generating unit 61 (utilizes based on carrying real random number generator
Physical event rather than software algorithm generate the generator of random number) generate a checking random number, verification is single
This checking random number is sent and extracts, to data, the transmitting terminal instructed by unit 62, and data extract the transmitting terminal of instruction
The AES according to this checking random number and specified and key, generate corresponding checking information, and number
According to the transmitting terminal extracting instruction, this checking information is sent to verification unit 62；Verification unit 62 is based on reception
The checking information arrived, verifies checking information and transmitting terminal, if verifying described checking information transmitting terminal
Legal and checking information is correct, then acquiring unit 63 obtains target storage and identifies.
In the present embodiment, when receiving data and extracting instruction, signal generating unit 61 is first extracted to data and is referred to
Making transmitting terminal send a checking random number, this checking random number is true random number, i.e. this checking random number
The change of value is real-time, irregular, thus the checking information obtained also is change real-time, irregular
, improve the safety of checking information, and only by checking information and the verification of transmitting terminal, obtain
Take unit 63 and just obtain the target storage identification code that these data are extracted in instruction, carry carrying out follow-up data
Take process, thus after determining the identity that data extract the transmitting terminal instructed, just allow it to carry out data
Extract, improve the safety of whole distributed system.
Through the above description of the embodiments, those skilled in the art is it can be understood that arrive above-mentioned
Embodiment method can add the mode of required general hardware platform by software and realize, naturally it is also possible to logical
Cross hardware, but a lot of in the case of the former is more preferably embodiment.Based on such understanding, the present invention's
The part that prior art is contributed by technical scheme the most in other words can be with the form body of software product
Revealing to come, this computer software product is stored in a storage medium (such as ROM/RAM, magnetic disc, light
Dish) in, including some instructions with so that a station terminal equipment (can be mobile phone, computer, service
Device, air-conditioner, or the network equipment etc.) perform each embodiment of the present invention method.
These are only the preferred embodiments of the present invention, not thereby limit the scope of the claims of the present invention, every
Utilize equivalent structure or equivalence flow process conversion that description of the invention and accompanying drawing content made, or directly or
Connect and be used in other relevant technical fields, be the most in like manner included in the scope of patent protection of the present invention.
Claims (10)
1. a distributed data processing method, it is characterised in that this distributed data processing method includes:
The memory space of distributed system is divided into multiple accumulation layer, and each accumulation layer includes that some storages set
Standby；
Arrange layer coding for each accumulation layer, each storage device arranges device coding；
It is that each storage device distributes a storage identification code according to layer coding and device coding；
When receiving data to be stored, generate a random number corresponding with described storage identification code form；
Described data to be stored are stored in corresponding to the storage identification code the highest with described nonces match degree
Storage device in.
2. distributed data processing method as claimed in claim 1, it is characterised in that described according to layer
Coding and device coding are also to include after each storage device distributes a step storing identification code:
When receiving data and extracting instruction, obtain the target storage identification code that these data are extracted in instruction；
The storage data of corresponding storage device in corresponding accumulation layer are extracted according to described target storage identification code.
3. distributed data processing method as claimed in claim 2, it is characterised in that described for each
Accumulation layer arranges layer coding, each storage device arranges the step of device coding and includes:
According to the information security grade of storage data, one layer coding is set for each accumulation layer；
According to the position of same accumulation layer internal memory devices, one device coding is set for each storage device.
4. distributed data processing method as claimed in claim 3, it is characterised in that described by described
Data to be stored are stored in the storage the highest with described nonces match degree storage device corresponding to identification code
In step before also include:
Resolve described data to be stored, obtain the information security demand levels of data to be stored；
Obtain information security grade and meet the accumulation layer of described information security demand levels, this accumulation layer conduct
Accumulation layer to be selected, to determine storage device based on random number from accumulation layer to be selected.
Described described data to be stored are stored in the storage identification code institute the highest with described nonces match degree
The corresponding step in storage device includes:
Described random number is mated with the storage identification code of each storage device in accumulation layer to be selected,
Go out each storage device matching degree about own storage identification code Yu random number；
Treat that deposit data is stored in the storage device that described matching degree is the highest by described.
5. the distributed data processing method as described in Claims 1-4 any one, it is characterised in that
Described when receiving data to be stored, generate a random number corresponding with described storage identification code form
Step include:
When receiving data to be stored, verify the legitimacy of these data to be stored；
If described data to be stored are legal, then the real random number generator controlling to preset generates one with described
The random number that storage identification code form is corresponding.
6. distributed data processing method as claimed in claim 5, it is characterised in that described when receiving
During to data extraction instruction, the step obtaining the target storage identification code in the extraction instruction of these data includes:
When receiving data and extracting instruction, generate a checking random number, and this checking random number is sent out
Deliver to data and extract the transmitting terminal of instruction；
Obtain described transmitting terminal checking information based on checking generating random number, and to described checking information and
Transmitting terminal verifies；
When described checking information and transmitting terminal are deposited by verification, the then target obtained in the extraction instruction of these data
Storage identification code.
7. a distributed data processing device, it is characterised in that this distributed data processing device includes:
Storage of hierarchically module, for being divided into multiple accumulation layer, Mei Gecun by the memory space of distributed system
Reservoir includes some storage devices；
Encoding setting module, for arranging layer coding for each accumulation layer, each storage device arranges equipment
Coding；
Distribution module, is that one storage of each storage device distribution is known for encoding with device coding according to layer
Other code；
Random number module, for when receiving data to be stored, generates one and described storage identification code
The random number that form is corresponding；
Memory module, for being stored in described data to be stored and described the highest the depositing of nonces match degree
In storage storage device corresponding to identification code.
8. distributed data processing device as claimed in claim 7, it is characterised in that described distributed
Data processing equipment also includes:
Identification code acquisition module, for when receiving data and extracting instruction, obtains these data and extracts instruction
In target storage identification code；
Data extraction module, deposits for extracting correspondence in corresponding accumulation layer according to described target storage identification code
The storage data of storage equipment.
9. distributed data processing device as claimed in claim 8, it is characterised in that described coding sets
Put module to include:
Safety distribution unit, for the information security grade according to storage data, is arranged for each accumulation layer
One layer coding；
Encoding setting unit, for the position according to same accumulation layer internal memory devices, sets for each storage
For arranging a device coding.
10. distributed data processing device as claimed in claim 9, it is characterised in that described distribution
Formula data processing equipment also includes:
Parsing module, for, before storing data to be stored, resolving described data to be stored, obtain and wait to deposit
The information security demand levels of storage data；
Layer acquisition module, meets the storage of described information security demand levels for obtaining information security grade
Layer, this accumulation layer is as accumulation layer to be selected, to determine storage device based on random number from accumulation layer to be selected；
Described memory module includes:
Matching unit, for by described random number and the storage identification of each storage device in accumulation layer to be selected
Code mates, and draws each storage device matching degree about own storage identification code Yu random number；
By described, memory element, for treating that deposit data is stored in the storage device that described matching degree is the highest.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201610260783.6A CN105892953B (en) | 2016-04-25 | 2016-04-25 | Distributed data processing method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201610260783.6A CN105892953B (en) | 2016-04-25 | 2016-04-25 | Distributed data processing method and device |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN105892953A true CN105892953A (en) | 2016-08-24 |
| CN105892953B CN105892953B (en) | 2019-07-26 |
Family
ID=56705521
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201610260783.6A Active CN105892953B (en) | 2016-04-25 | 2016-04-25 | Distributed data processing method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN105892953B (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111461620A (en) * | 2020-04-09 | 2020-07-28 | 中振区块链(深圳)有限公司 | Block chain-based distributed storage method and device for logistics data |
| CN112015810A (en) * | 2020-08-04 | 2020-12-01 | 珍岛信息技术（上海）股份有限公司 | Distributed database information inquiry management system based on internet |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101193031A (en) * | 2006-11-24 | 2008-06-04 | 曲锐 | Data processing method and network based on distributed hash table |
| US20120323864A1 (en) * | 2011-06-17 | 2012-12-20 | Inventec Corporation | Distributed de-duplication system and processing method thereof |
| CN103583030A (en) * | 2011-05-25 | 2014-02-12 | 阿尔卡特朗讯公司 | Method and apparatus for achieving data security in a distributed cloud computing environment |
| CN103778255A (en) * | 2014-02-25 | 2014-05-07 | 深圳市中博科创信息技术有限公司 | Distributed file system and data distribution method thereof |
| CN105959350A (en) * | 2016-04-22 | 2016-09-21 | 深圳市永兴元科技有限公司 | Distributed data storage method and device |
-
2016
- 2016-04-25 CN CN201610260783.6A patent/CN105892953B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101193031A (en) * | 2006-11-24 | 2008-06-04 | 曲锐 | Data processing method and network based on distributed hash table |
| CN103583030A (en) * | 2011-05-25 | 2014-02-12 | 阿尔卡特朗讯公司 | Method and apparatus for achieving data security in a distributed cloud computing environment |
| US20120323864A1 (en) * | 2011-06-17 | 2012-12-20 | Inventec Corporation | Distributed de-duplication system and processing method thereof |
| CN103778255A (en) * | 2014-02-25 | 2014-05-07 | 深圳市中博科创信息技术有限公司 | Distributed file system and data distribution method thereof |
| CN105959350A (en) * | 2016-04-22 | 2016-09-21 | 深圳市永兴元科技有限公司 | Distributed data storage method and device |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111461620A (en) * | 2020-04-09 | 2020-07-28 | 中振区块链(深圳)有限公司 | Block chain-based distributed storage method and device for logistics data |
| CN112015810A (en) * | 2020-08-04 | 2020-12-01 | 珍岛信息技术（上海）股份有限公司 | Distributed database information inquiry management system based on internet |
| CN112015810B (en) * | 2020-08-04 | 2024-03-19 | 珍岛信息技术（上海）股份有限公司 | Internet-based distributed database information query management system |
Also Published As
| Publication number | Publication date |
|---|---|
| CN105892953B (en) | 2019-07-26 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN102497635B (en) | Server, terminal and account password acquisition method | |
| CN101038619B (en) | Radio frequency recognition system privacy identification method | |
| CN103491084B (en) | The authentication method of a kind of client and device | |
| CN109191649B (en) | Method and device for sharing control authority of intelligent door lock | |
| CN103916366A (en) | Login method, maintenance terminal, data management service equipment and login system | |
| CN105516055B (en) | Data access method, access device, target device and management server | |
| CN103530785A (en) | Double anti-counterfeiting system and method based on near field communication (NFC) function | |
| CN105488875B (en) | A kind of gate inhibition's verification method and device | |
| CN103973711A (en) | Verification method and device | |
| CN112202747A (en) | Target device binding method and device, storage medium and electronic device | |
| CN104836786A (en) | Automatic login system and automatic login method | |
| CN104580235A (en) | Authentication method and authentication system for equipment connection | |
| CN105959350A (en) | Distributed data storage method and device | |
| CN108920919A (en) | Control method, the device and system of interactive intelligence equipment | |
| CN105022939A (en) | Information verification method and device | |
| CN105892953A (en) | Distributed data processing method and distributed data processing device | |
| CN105792095A (en) | Secret key negotiation method and system for MTC (Machine Type Communication) packet communication and network entity | |
| CN104469736A (en) | Data processing method, server and terminal | |
| CN103139136B (en) | The management process of a kind of password and equipment | |
| CN104270754B (en) | A kind of Subscriber Identity Module method for authenticating and device | |
| CN105224541A (en) | The uniqueness control method of data, information storage means and device | |
| CN102426592B (en) | Method for initializing database based on dynamic password | |
| CN102983969A (en) | Security login system and security login method for operating system | |
| CN105872013A (en) | Cloud computing system | |
| CN104580262A (en) | Safety method applicable to wireless internet of things |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| CB02 | Change of applicant information |
Address after: 518057 Shenzhen Software Park, Nanshan District high tech Industrial Park, Guangdong, China,, 6 401-402 Applicant after: Yongxing Shenzhen Polytron Technologies Inc Address before: 518057 Shenzhen Software Park, Nanshan District high tech Industrial Park, Guangdong, China,, 6 401-402 Applicant before: Shenzhen Longrise Technology Co., Ltd. |
|
| COR | Change of bibliographic data | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |