CN104780038B - A kind of distributed collaboration encryption method and device - Google Patents
A kind of distributed collaboration encryption method and device Download PDFInfo
- Publication number
- CN104780038B CN104780038B CN201410017811.2A CN201410017811A CN104780038B CN 104780038 B CN104780038 B CN 104780038B CN 201410017811 A CN201410017811 A CN 201410017811A CN 104780038 B CN104780038 B CN 104780038B
- Authority
- CN
- China
- Prior art keywords
- data
- encryption
- warehouse
- fairground
- source
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Abstract
The invention discloses a kind of distributed collaboration encryption method and device, methods described to include:In the loading procedure to the ETL of source data, when determining that loaded source data includes private data according to the scope of the private data, the encryption policy configured is read, the private data is encrypted according to the encryption policy;Source data after encryption is sent to the data warehouse, the source data after the encryption is deposited in the caching of the data warehouse;Corresponding target data objects in target data fairground and the target data fairground are determined in Data Mart more than one according to the mapping relations;Data synchronization request message is sent to the target data fairground;Source data after the encryption is write in the disk of the data warehouse;After receiving the data sync response message that the target data fairground is sent, the source data after the encryption that is deposited in the caching is handled according to the data sync response message.
Description
Technical field
The present invention relates to the encryption technology in big data field, more particularly to a kind of distributed collaboration encryption method and device.
Background technology
Available data cipher mode includes single system cipher mode and mirror-image copies mode, and both is suitable for forms data and put down
Platform or the less situation of data scale;Wherein, because data are stored in a manner of table, mirror image cipher mode is substantially table level
The mapping of data object is synchronous, is not suitable for multi-platform isomerous environment model.Encryption for mass data, prior art are general
In the presence of two kinds of solutions:
A solution is to use the isomeric data plateform system being made up of Distributed Data Warehouse and other databases,
In the isomeric data plateform system, the private data guard between synchronous each platform is using secondary cipher mode, i.e. data
Warehouse is encrypted and the encryption of other databases is carried out at twice:First, data warehouse is according to the security strategy being set
Private data in storehouse is encrypted;Afterwards, other database roots are encrypted again according to the security strategy of itself;When using hidden
Include AES with respective security strategy during private data and key is reduced.
Another solution is in data extraction, conversion and loading(ETL, Extraction Transformation
Loading)During use one-time pad encryption, then asynchronous transmission is to data warehouse and Data Mart.The problem of this scheme
It is that can not ensure data warehouse due to when other reasonses change, the consistency synchronization of data, such as after key changes,
Encryption data will update again, and ETL can not be responsible for increasing simultaneously ETL management load, so as to reduce the speed of data loading.
Secondary cipher mode causes same data source information to be encrypted twice used by prior art, and consumption is a large amount of
Computing resource, while secondary encryption can cause the algorithm of separated encryption, key may be inconsistent, customer privacy data may produce
Raw encrypted result difference;Asynchronous Transfer Mode can realize one-time pad encryption used by prior art, but can not ensure different
Walk in transmitting procedure due to network failure, loss of data, incorrect order caused by Buffer Overflow, the data quality problem brought.
Both the above solution can not all solve isomery big data plateform system during operation, master data warehouse by
After key version updating etc. causes customer privacy data ciphertext to change, data warehouse and client in other databases are hidden
The associated synchronisation and consistency problem of private data.
The content of the invention
In view of this, the embodiment of the present invention provides a kind of distributed collaboration for solution problems of the prior art and added
Decryption method and device, while one-time pad encryption is realized, it can solve the problem that privacy between data warehouse and other interior databases of system
The coordinated management problem of data.
What the technical scheme of the embodiment of the present invention was realized in:
A kind of distributed collaboration encryption method, applied to isomeric data plateform system, the isomeric data plateform system bag
Include data extraction, conversion and load ETL servers, data warehouse and more than one Data Mart, in the data warehouse
Mapping relations are respectively present between each second data object in first data object and one Data Mart above;Match somebody with somebody
Put the scope and private attribute of the private data in source data, the private attribute comprise at least encryption first version information and
Encryption policy, methods described include:
The ETL servers determine institute during being loaded to source data, according to the scope of the private data
When the source data of loading includes private data, the encryption policy of loaded source data configuration is read as, according to the encryption plan
Slightly the private data is encrypted, the source data after being encrypted;
Source data after the encryption is sent to the data warehouse by the ETL servers;
Source data after the encryption is deposited in caching by the data warehouse;
The data warehouse determines target data set according in Data Mart of the mapping relations more than one
Corresponding target data objects in city and the target data fairground;
The data warehouse sends data synchronization request message to the target data fairground, and by the source after the encryption
Data are write in the disk of the data warehouse;Wherein, the data synchronization request message includes the first edition of the encryption
This information.
A kind of distributed collaboration encryption method, data extraction, conversion and loading ETL applied to isomeric data plateform system
Server, the isomeric data plateform system include ETL servers and data warehouse;
The private attribute of private data in the ETL servers configuration source data, the private attribute, which comprises at least, to be added
Close strategy, methods described include:
During being loaded to source data, the ETL servers determine institute according to the scope of the private data
When the source data of loading includes private data, the encryption policy of loaded source data configuration is read as, the encryption policy is made
For the execution input parameter of function in dynamic link library;
The private data is encrypted according to the encryption policy for the ETL servers, the source number after being encrypted
According to;
Source data after the encryption is sent to the data warehouse by the ETL servers.
A kind of distributed collaboration encryption method, applied to isomeric data plateform system, the isomeric data plateform system bag
Include data extraction, conversion and load ETL servers, data warehouse and more than one Data Mart, in the data warehouse
Mapping relations are respectively present between each second data object in first data object and one Data Mart above;
Methods described includes:
The data warehouse receives the source data after the encryption that ETL servers are sent, and by the source data after the encryption
It is deposited in the caching of the data warehouse;
The data warehouse determines target data set in Data Mart more than one according to the mapping relations
Corresponding target data objects in city and the target data fairground;
The data warehouse writes the source data after the encryption in the disk of the data warehouse;
After the data warehouse receives the data sync response message that the target data fairground is sent, according to the data
Synchronous response message is handled the source data after the encryption that is deposited in the caching.
A kind of distributed collaboration encryption device, applied to isomeric data plateform system, the isomeric data plateform system bag
Include data extraction, conversion and load ETL servers, data warehouse and more than one Data Mart, in the data warehouse
Mapping relations are respectively present between each second data object in first data object and one Data Mart above;Institute
Stating device includes dispensing unit, ciphering unit, memory cell, the first determining unit, writing unit and first processing units, its
In:
The dispensing unit, for configuring the private attribute of the private data in source data, the private attribute is at least wrapped
Include encryption version information and encryption policy;
The ciphering unit, for the extraction to source data, conversion and load ETL loading procedure in, when according to institute
When stating the scope of private data and determining that loaded source data includes private data, encryption policy that reading has configured, according to institute
State encryption policy the private data is encrypted, the source data after being encrypted, and the source data after the encryption is sent out
Give the data warehouse;
The memory cell, for the source data after the encryption to be deposited in the caching of the data warehouse;
First determining unit, for determining mesh in the Data Mart more than one according to the mapping relations
Mark corresponding target data objects in Data Mart and the target data fairground;
Said write unit, for sending data synchronization request message, the data syn-chronization to the target data fairground
Request message includes encryption version information, the source data after the encryption is write on the disk of the data warehouse;
The first processing units, after the data sync response message sent for receiving the target data fairground, root
The source data after the encryption that is deposited in the caching is handled according to the data sync response message.
A kind of distributed collaboration encryption device, data extraction, conversion and loading ETL applied to isomeric data plateform system
Server, the isomeric data plateform system include ETL servers and data warehouse;
Described device includes dispensing unit, reading unit, the first ciphering unit and the first transmitting element, wherein:
The dispensing unit, for configuring the private attribute of the private data in source data, the private attribute is at least wrapped
Include encryption policy;
The reading unit, for the extraction to source data, conversion and load ETL loading procedure in, when it is determined that institute
When the source data of loading includes private data, the encryption policy configured is read, the encryption policy is as in dynamic link library
The execution input parameter of function;
First ciphering unit, for the private data to be encrypted according to the encryption policy, encrypted
Source data afterwards;
First transmitting element, for the source data after the encryption to be sent into the data warehouse.
A kind of distributed collaboration encryption device, applied to isomeric data plateform system, described device includes memory cell, the
One determining unit, writing unit and processing unit, wherein:
The memory cell, the source data after encryption for receiving the transmission of ETL servers, and by the source after the encryption
Data register is in the caching of the data warehouse;
First determining unit, for determining mesh in the Data Mart more than one according to the mapping relations
Mark corresponding target data objects in Data Mart and the target data fairground;
Said write unit, for the source data after the encryption to be write in the disk of the data warehouse;
The processing unit, after the data sync response message sent for receiving the target data fairground, according to institute
Data sync response message is stated to handle the source data after the encryption that is deposited in the caching.
In the embodiment of the present invention, in the loading procedure to the ETL of source data, when the scope according to the private data is true
When fixed loaded source data includes private data, the encryption policy configured is read, according to the encryption policy to described hidden
Private data are encrypted;Source data after encryption is sent to the data warehouse, the source data after the encryption is deposited at institute
In the caching for stating data warehouse;Target data fairground in Data Mart more than one is determined according to the mapping relations
With corresponding target data objects in the target data fairground;Data synchronization request is sent to the target data fairground to disappear
Breath;Source data after the encryption is write in the disk of the data warehouse;Receive the number that the target data fairground is sent
After synchronous response message, according to the data sync response message to the source number after the encryption that is deposited in the caching
According to being handled;In this way, while one-time pad encryption is realized, can solve the problem that hidden between other databases in data warehouse and system
The coordinated management problem of private data.
Brief description of the drawings
Fig. 1 is the composition structural representation of the isomeric data plateform system of the embodiment of the present invention one;
Fig. 2 is the implementation process schematic diagram of the distributed collaboration encryption method of the embodiment of the present invention two;
Fig. 3 is the implementation process signal that the distributed collaboration encryption method of the embodiment of the present invention three is deployed on data warehouse
Figure;
Fig. 4 is the implementation process schematic diagram that the embodiment of the present invention cooperates with encryption method;
Fig. 5 is the implementation process schematic diagram that the target data fairground of the embodiment of the present invention four determines synchronous response message;
Fig. 6 implementation processs of data synchronization process between the data warehouse of the embodiment of the present invention five and target data fairground are shown
It is intended to;
Fig. 7 is the implementation process schematic diagram of the distributed collaboration encryption method of the embodiment of the present invention seven;
Fig. 8 is the implementation process schematic diagram of the distributed collaboration encryption method of the embodiment of the present invention eight;
Fig. 9 is the implementation process schematic diagram of the distributed collaboration encryption method of the embodiment of the present invention nine;
Figure 10 is the composition structural representation of the distributed collaboration encryption device of the embodiment of the present invention ten;
Figure 11 is the composition structural representation of the distributed collaboration encryption device of the embodiment of the present invention 11;
Figure 12 is the composition structural representation of the distributed collaboration encryption device of the embodiment of the present invention 12;
Figure 13 is the composition structural representation of the distributed collaboration encryption device of the embodiment of the present invention 13.
Embodiment
Embodiment one
A kind of distributed collaboration encryption method and device that the embodiment of the present invention one provides, applied to isomeric data platform system
System, Fig. 1 is the composition structural representation of the isomeric data plateform system of the embodiment of the present invention one, as shown in figure 1, the isomeric data is put down
Platform system includes privacy information protection layer equipment 11, obtains layer equipment 12, data Layer equipment 13 and application layer equipment 14, wherein:
The privacy information protection layer equipment 11 includes removing privacyization processing engine and privacy reduction engine, goes at privacy
It can be decipher to manage engine, for the decryption to privacy information;Correspondingly, privacy reduction engine can be encryption equipment, be used for
Encryption to privacy information.
The layer equipment 12 that obtains is used to pass through ETL to the source data of acquisition and goes privacyization to handle, and the acquisition layer is set
Standby 12 can be ETL servers.
The data Layer equipment 13 includes data warehouse(DW or DWH, Data Warehouse)With each Data Mart;By
It is described to obtain the data input after layer equipment 12 is handled to the DW, then it is synchronized to each Data Mart by judgement.
Application layer equipment 14 by the data input that the data Layer equipment 13 stores into corresponding application, such as some numbers
Reduced according to without privacy, be then directly inputted to and be not required in the application of reduction;Some data need privacy to reduce, then by privacy also
It is input to after original in the application for needing privacy to reduce;Application layer equipment 14 also includes the function of key management.
Here, the ETL servers are used to extract source data, changed and be loaded, to be stored in data warehouse
In.Data warehouse is also known as data master depot, the first data object in the data warehouse with it is one more than data
Mapping relations are respectively present between each second data object in fairground;The mapping relations are reflected including first, second, and third
Relation is penetrated, first mapping relations are used to show the corresponding relation between data warehouse and Data Mart, second mapping
Correspondence in the first table and Data Mart of the relation for showing data storage in data warehouse between the second table of data storage
Relation, the second number that the 3rd mapping relations are used to show in the first data object and second table in first table
According to the corresponding relation between object, first data object and second data object are all indicated with behavior unit.
This mapping relations in the embodiment of the present invention between data warehouse and Data Mart can navigate to capable level, can be more accurate
Ground synchronizes to data, synchronous so as to breach the mapping of the table level of traditional encryption.
Embodiment two
The embodiment of the present invention provides a kind of distributed collaboration encryption method, described different applied to isomeric data plateform system
Structure data platform system includes ETL servers, data warehouse and more than one Data Mart, and first in the data warehouse
Mapping relations are respectively present between each second data object in data object and one Data Mart above;Fig. 2 is
The implementation process schematic diagram of the distributed collaboration encryption method of the embodiment of the present invention two, as shown in Fig. 2 the distributed collaboration encryption side
Method comprises the following steps:
Step 201, the scope and private attribute of the private data in source data are configured;
Here, the source data can come from business intelligence(BI, Business Intelligence)Deng source database,
When isomeric data plateform system is used for the communications field, the source data may be customer data and single etc. in detail, customer data and in detail
The privacy information of client may be included in list, i.e. the source data includes private data.
Here, the private attribute comprises at least private attribute type, the first version information of encryption and encryption policy;Its
In, the private attribute type includes phone number, opposite-terminal number, name, address and passport NO.;The private attribute is set
Put:The operation increased to private attribute, delete, change, looked into.The encryption policy includes AES, the initial value of key inputs,
Start encrypted location and encryption length;The AES is at least one of following AES:It is character type AES, visible
Single byte AES and numerical value AES.
Step 202, in loading procedure is carried out to source data, when what is loaded according to the determination of the scope of the private data
When source data includes private data, the encryption policy that loaded source data has configured is read as, according to the encryption policy pair
The private data is encrypted, the source data after being encrypted;Source data after the encryption is sent to the data bins
Storehouse;
Here, the step 201 and step 202 can be completed on ETL servers, and the encryption policy is as dynamic
The execution input parameter of function in chained library;The first crypto engine is provided with ETL servers, first crypto engine being capable of root
The source data converted is encrypted according to the encryption policy configured.It should be noted that privacy in the embodiment of the present invention
Data scope and private attribute configuration can also be completed on other servers, ETL servers can needs when
Wait the scope and private attribute for reading the private data configured in the server.In the embodiment of the present invention, the encryption behaviour of source data
Work is completed on ETL servers, and consumption is to consume ETL server resources, the cryptographic operation and data warehouse of source data
It is unrelated with Data Mart.
Step 203, the source data after the encryption is deposited in the caching of the data warehouse;Closed according to the mapping
System determines in Data Mart more than one corresponding number of targets in target data fairground and the target data fairground
According to object, data synchronization request message is sent to the target data fairground;Source data after the encryption is write into the number
According in the disk in warehouse;
Here, the target data fairground is the more than one Data Mart, and the target data objects are encryption
The part in source data afterwards;The data synchronization request message includes the first version information of the encryption;
Step 204, after receiving the data sync response message that the target data fairground is sent, according to the data syn-chronization
Response message is handled the source data after the encryption that is deposited in the caching.
Here, the step 203 and step 204 can be completed on data warehouse, and data warehouse receives to be serviced from ETL
Source data after the encryption of device, first it can typically be stored on caching;It is then determined that whether the source data after the encryption needs together
Walk on Data Mart, when needing to be synchronized on target data fairground, the source data after will this be encrypted is sent to target
Data Mart;It is that is, synchronous using data between data warehouse and target data fairground in the embodiment of the present invention
The mode do not landed, so, save the cpu resource of data warehouse.
In the embodiment of the present invention, methods described also includes step B1, step B2 and step B3, wherein:
Step B1, the target data fairground receive the data synchronization request message, the data synchronization request message
Include the identity of sender;
Step B2, according to communication body of the identity of sender in the data synchronization request message to described sender
Part is confirmed, when confirming successfully, data sync response message is sent to the data warehouse;
Step B3, according to communication body of the identity of sender in the data synchronization request message to described sender
Part is confirmed;When confirming failure, data sync response message is not sent to the data warehouse.
Above-mentioned steps B1 to B3 is completed on target data fairground, is provided by above-mentioned steps in transmitting procedure
Communications identities confirm, and the data syn-chronization between follow-up data warehouse and target data fairground provides guaranteed reliability.
In the embodiment of the present invention, methods described also includes step C1, step C2 and step C3, wherein:
Step C1, after the target data objects receive the target data objects, target data fairground root
Integrality is carried out to received data object according to the data synchronization request message and uniformity confirms;The data syn-chronization please
Message is asked to include the original position and size of target data objects;
Step C2, when confirming successfully, sent to the data warehouse for showing that reception data object successful first is true
Recognize message;
Step C3, when confirming failure, sent to the data warehouse for representing that receive data object failure second is true
Recognize message.
Above-mentioned steps B1 to B3 pages is completed on target data fairground, is provided and is being transmitted by above-mentioned steps
Rear verification and check.
Embodiment three
Step 201 and step 202 in the embodiment of the present invention two are that the first crypto engine is provided with ETL servers,
Can also be so as to which newly-increased source data be encrypted according to the scope of private data and encryption policy, in the embodiment of the present invention
Second crypto engine is set on data warehouse, second crypto engine is identical with the first crypto engine, except that, the
One crypto engine is encrypted for newly-increased source data, and the second crypto engine be then in the scope expansion of private data,
The first data object to be stored on data warehouse is encrypted, and specifically, Fig. 3 is the distributed association of the embodiment of the present invention three
The implementation process schematic diagram being deployed in encryption method on data warehouse, as shown in figure 3, the distributed collaboration encryption method includes
Following steps:
Step 301, when confirming that the scope of private data expands, the data warehouse is according to the encryption policy to newly-increased
First data object is encrypted, and the first data object after encryption is deposited in caching;
Step 302, target data fairground in Data Mart more than one is determined according to the mapping relations;
Step 303, data synchronization request message is sent to the target data fairground, in the data synchronization request message
Including encryption version information, the source data after the encryption is write in the disk of the data warehouse;
Step 304, after receiving the data sync response message that the target data fairground is sent, according to the data syn-chronization
Response message is handled the source data after the encryption that is deposited in the caching.
The step 304 in step 204 and the embodiment of the present invention three in the embodiment of the present invention two, it is described according to the number
The source data after the encryption that is deposited in the caching is handled according to synchronous response message, including step F1:
Step F1, determine that the target data fairground need not receive the data pair according to the data sync response message
As when, source data after the encryption that will be deposited in the caching is removed.
Step 204 in the embodiment of the present invention, it is described according to the data sync response message to being deposited at the caching
In the encryption after source data handled, including step F2 and step F3, wherein:
Step F2, determine that the target data fairground needs to receive the number of targets according to the data sync response message
During according to object, the target data objects being deposited in the caching of the data warehouse are sent to the target data set
City;
Step F3, receives the first confirmation message that the target data fairground is sent, and first confirmation message is used for table
Bright reception data success, the source data after the encryption that will be deposited in the caching are removed.
Fig. 4 is the implementation process schematic diagram that the embodiment of the present invention cooperates with encryption method, as shown in figure 4, wherein, step 401
It is the first ciphering process on ETL servers to 403, step 404 to step 406 is encrypted for second on data warehouse
Journey, wherein the first ciphering process is step 201 is to as described in step 203 in the embodiment of the present invention two, the second ciphering process such as this hair
Bright embodiment step 301 is to described in 304.Wherein, privacy identification encryption identification module can be carried out according to the scope of private data
Set, during specific implementation, above-mentioned encryption function can be by increasing User-Defined Functions in data warehouse(UDF,
User Define Function)To realize.
Example IV
It is described to be disappeared according to the data sync response in the step 204 of the embodiment of the present invention two and embodiment three in step 304
The data sync response message in handling the source data after the encryption that is deposited in the caching is ceased, is by target
Data Mart is sent to data warehouse, and target data fairground can carry out a series of sentencing when receiving data synchronization request message
It is disconnected, when judging to need synchronous target data objects, the first confirmation will be carried in data sync response message;When sentencing
It is disconnected when not needing synchronous target data objects, the second confirmation can be just carried in data sync response message.Fig. 5 is this hair
Bright example IV target data fairground determines the implementation process schematic diagram of synchronous response message, as shown in figure 5, target data fairground
Determine that synchronous response message comprises the following steps:
Step 501, synchronization policy is configured for one Data Mart above;
Here, the synchronization policy includes synchronizing cycle;It is described to configure synchronous plan for one Data Mart above
Slightly can be that user configuration or data warehouse or Data Mart are automatically configured according to the setting of user,
Those skilled in the art can realize according to various prior arts, repeat no more here.
Step 502, the data synchronization request message that the data warehouse is sent is received;
Here, the data synchronization request message includes the first version information of currently used AES;
Step 503, judge whether the second edition information of AES of the first version information with being stored is consistent,
When consistent, step 504 is performed, when inconsistent, performs step 505;
Step 504, the data sync response message for carrying the first confirmation is sent to the data warehouse;
Here, the data sync response message includes the first confirmation, and first confirmation shows to make described
Data warehouse sends target data objects when receiving the data sync response message, to the target data fairground;
Step 505, continue to judge whether the reception time of the data synchronization request message and the synchronizing cycle are consistent,
When consistent, step 504 is performed, when inconsistent, performs step 506;
Step 506, the data sync response message for carrying the second confirmation is sent to the data warehouse;
Here, the data sync response message includes the second confirmation, and second confirmation shows to make described
For data warehouse when receiving the data sync response message, not sent to the target data fairground needs synchronous target
Data object.
Embodiment five
In the embodiment of the present invention two and three, when target data fairground judges to need synchronous target data objects, target
Data Mart sends the data sync response message for carrying the first confirmation to data warehouse, and data warehouse is receiving carrying
When having the data sync response message of the first confirmation message, target data objects can be sent to target data fairground, start data
Transmitting procedure;In data transmission procedure is implemented, application programming can be established between data warehouse and Data Mart and connect
Mouthful(API, Application Programming Interface).
The data syn-chronization support full dose method of synchronization and increment synchronization mode two between data warehouse and target data fairground
The kind method of synchronization, wherein, the full dose method of synchronization is a kind of method of synchronization of cover type, and increment synchronization mode is the same of additional formula
Step mode;When target data fairground receives target data objects, the second mapping relations inquiry pair in mapping relations
The second table answered, then the 3rd mapping relations in mapping relations inquire the associated row in the second table, then according to same
Step mode synchronizes to the associated row in the second table.It should be noted that after the completion of ETL encryptions, can also be to the encryption
Source data afterwards creates encrypted indexes, and the encrypted indexes are with behavior unit;In this manner it is possible to it will be added according to the 3rd mapping relations
Source data after close is synchronized in target data fairground from data warehouse, can also by the newly-increased data after private data scope from
Data warehouse is synchronized in target data fairground.
Target data objects can also be deposited in caching by target data fairground, when complete data integrality with it is consistent
Property examine after, the associated row that is written to target data objects according to mapping relations in the second table.Can also be when the number of caching
When reaching certain threshold value according to amount, the associated row that is written to target data objects according to mapping relations in the second table.Target data
Fairground is when carrying out data syn-chronization, it is also necessary to which record data changes situation to form change daily record, and change daily record includes encryption
Strategy;If failure of data synchronization, data recovery is carried out according to change daily record.
Above-mentioned data synchronization process can be by increasing User-Defined Functions(UDF, User Define
Function)To realize;The reality of Fig. 6 data synchronization process between the data warehouse of the embodiment of the present invention five and target data fairground
Existing schematic flow sheet, as shown in fig. 6, the data synchronization process comprises the following steps:
Step 601, data warehouse sends data synchronization request message to the target data fairground;
The data synchronization request message includes encryption version information;
Step 602, disk is write, source data that will be after the encryption is write in the disk of the data warehouse;
Step 603, target data fairground sends data sync response message to data warehouse, i.e.,:Data Mart is according to Fig. 5
Shown flow generation data sync response message;
Step 604, data warehouse is pre-processed to data, contrasted, and solves colliding data or generation Conflict solving file,
And it is sent to target data set city;
Step 605, target data fairground solves inconsistent data, or inconsistent according to the solution of Conflict solving file
Data;
Here, step 604 and step 605, those skilled in the art can carry out conventional place according to various prior arts
Reason, is repeated no more here.
In embodiments of the present invention, by synchronous crypto-operation, to solve, data warehouse and data in Data Mart are inconsistent to ask
Topic.Data warehouse can trigger above-mentioned synchronous crypto-operation when newly-increased data source, digital source content change and data eliminate
Process.
Embodiment six
In step 301 in the step 201 and step 202 and embodiment three of the embodiment of the present invention two, in source data
Information can be divided into numerical value and nonumeric information, wherein, nonumeric information includes letter, controlling symbols and graphical symbol etc.,
Nonumeric information is that computer is stored in a manner of binary-coded character code and is handled, the character code commonly used in computer
There are ASCII character and extended binary coded decimal interchange code, and it is also relatively more for the standard of Chinese character, and such as GB2312, BIG-5, GBK, GB18030 are adopted
Encoded with multibyte code.
In the embodiment of the present invention two, AES is at least one of following AES in crypto engine:Character type is encrypted
Algorithm, visible single byte AES and numerical value AES, table 1 is for the encryption scope of these three AESs and on model
The description enclosed, wherein the hexadecimal describing mode that encryption scope uses.
AES title | Encrypt scope | Scope describes |
Character type AES | 0X21 to 0XFF | Numeral, letter, spcial character, Chinese character |
It can be seen that single byte AES | 0X21 to 0X7F | Numeral, letter, spcial character |
Numerical value AES | 0X30 to 0X39 | Numeral |
Table 1
For ensure data encryption and reduction stability, encrypt the character beyond scope be former state export, be not involved in into
The process of row enciphering and deciphering algorithm, i.e. these three AESs all avoid the spcial characters such as space, tab, so as to not to special
Character is encrypted.The encrypted characters that character type AES includes are most, it is seen that the encryption of single byte AES is keyboard
Upper all character visibles, what numerical value AES was encrypted is 0~9 numeral.With encryption " university of Inner Mongol Huhehaote City of province south
Exemplified by the Room of road illusion mansion Building A 501 ", the effect of above-mentioned three kinds of AESs is as shown in table 2:
AES title | Cipher round results |
Character type AES | D{a}sd@wer^0wfdsa[,./,/i]asj+k- |
It can be seen that single byte AES | University of Inner Mongol Huhehaote City of province South Road illusion mansion h seat Ab1 rooms |
Numerical value AES | The Room of university of Inner Mongol Huhehaote City of province South Road illusion mansion Building A 695 |
Table 2
Embodiment seven
A kind of distributed collaboration encryption method provided in an embodiment of the present invention, the ETL applied to isomeric data plateform system
Server, the isomeric data plateform system include ETL servers and data warehouse;Fig. 7 is that the embodiment of the present invention seven is distributed
The implementation process schematic diagram of encryption method is cooperateed with, as shown in fig. 7, this method includes:
Step 701, the private attribute of the private data in source data is configured;
Here, the private attribute comprises at least encryption policy;
Step 702, during being loaded to source data, determine what is loaded according to the scope of the private data
When source data includes private data, the encryption policy of loaded source data configuration is read as;
Here, execution input parameter of the encryption policy as function in dynamic link library;
Step 703, the private data is encrypted according to the encryption policy, the source data after being encrypted;
Step 704, the source data after the encryption is sent to the data warehouse.
Embodiment eight
A kind of distributed collaboration encryption method that the embodiment of the present invention eight provides, the number applied to isomeric data plateform system
According to warehouse, the isomeric data plateform system includes ETL servers, data warehouse and more than one Data Mart, the number
It is respectively present according between each second data object in the first data object in warehouse and one Data Mart above
Mapping relations;Fig. 8 is the implementation process schematic diagram of the distributed collaboration encryption method of the embodiment of the present invention eight, as shown in figure 8, the party
The method distributed collaboration encryption method includes:
Step 801, the source data after the encryption that ETL servers are sent is received, and the source data after the encryption is deposited
In the caching of the data warehouse;
Step 802, according to the mapping relations determine in Data Mart more than one target data fairground with
Corresponding target data objects in the target data fairground;
Here, the target data fairground is the more than one Data Mart, and the target data objects are encryption
The part in source data afterwards;
By step 803, the source data after the encryption is write in the disk of the data warehouse;
Step 804, after receiving the data sync response message that the target data fairground is sent, according to the data syn-chronization
Response message is handled the source data after the encryption that is deposited in the caching.
In the embodiment of the present invention, methods described also includes step G1 to G4,:Configure the model of the private data in source data
Enclose;
Step G1, when confirming that the scope of private data expands, the data warehouse is according to the encryption policy to newly-increased
Data object is encrypted;
Step G2, target data fairground in Data Mart more than one is determined according to the mapping relations;
Step G3, data synchronization request message is sent to the target data fairground, in the data synchronization request message
Including encryption version information;
Step G4, after receiving the data sync response message that the target data fairground is sent, according to the data syn-chronization
Response message is handled the source data after the encryption that is deposited in the caching.
Step 804 of the embodiment of the present invention and step G4, it is described according to the data sync response message to described in being deposited at
The source data after the encryption in caching is handled, including:
, will when determining that the target data fairground need not receive the data object according to the data sync response message
It is deposited at the removing of the source data after the encryption in the caching.
Step 804 of the embodiment of the present invention and step G4, it is described according to the data sync response message to described in being deposited at
The source data after the encryption in caching is handled, including:
Determine that the target data fairground needs to receive the target data objects according to the data sync response message
When, the target data objects being deposited in the caching of the data warehouse are sent to the target data fairground;
The first confirmation message that the target data fairground is sent is received, first confirmation message is used to show to receive number
According to success, the source data after the encryption that will be deposited in the caching is removed.
Embodiment nine
A kind of distributed collaboration encryption method provided in an embodiment of the present invention, it is described applied to isomeric data plateform system
Isomeric data plateform system includes data warehouse and more than one Data Mart, the first data object in the data warehouse
Mapping relations are respectively present between each second data object in one Data Mart above;Fig. 9 is real for the present invention
The implementation process schematic diagram of the distributed collaboration encryption method of example nine is applied, as shown in figure 9, this method also includes:
Step 901, synchronization policy is configured for one Data Mart above, the synchronization policy includes synchronous week
Phase;
Step 902, the target data fairground receives the data synchronization request message that the data warehouse is sent, institute
Stating data synchronization request message includes the identity of sender and the first version information of currently used AES;
Step 903, according to communication of the identity of sender in the data synchronization request message to described sender
Identity is confirmed, when confirming successfully, continues to determine the reception time of the data synchronization request message and the synchronizing cycle
When consistent, data sync response message is sent to the data warehouse, the data sync response message includes first and confirmed
Information, first confirmation show to make the data warehouse when receiving the data sync response message, to described
Synchronous target data objects needed for the transmission of target data fairground.
In the embodiment of the present invention, the first edition of currently used AES is also included in the data synchronization request message
This information;Methods described also includes:
When determining that the second edition information of AES of the first version information with being stored is inconsistent, to the number
Data sync response message is sent according to warehouse, the data sync response message includes the first confirmation, and described first confirms
Information shows to make the data warehouse when receiving the data sync response message, and mesh is sent to the target data fairground
Mark data object.
In the embodiment of the present invention, the first version information and the second edition information one of the AES stored are determined
When causing, and determining the reception time and the mutually inconsistent synchronizing cycle of the data synchronization request message, to the data bins
Storehouse sends data sync response message, and the data sync response message includes the second confirmation, second confirmation
Show to make the data warehouse when receiving the data sync response message, not described sent to target data fairground needs
Synchronous target data objects.
Embodiment ten
A kind of distributed collaboration encryption device provided in an embodiment of the present invention, it is described applied to isomeric data plateform system
Isomeric data plateform system includes ETL servers, data warehouse and more than one Data Mart, and in the data warehouse
Mapping relations are respectively present between each second data object in one data object and one Data Mart above;Figure 10
For the composition structural representation of the distributed collaboration encryption device of the embodiment of the present invention ten, as shown in Figure 10, described device includes matching somebody with somebody
Put unit 1001, ciphering unit 1002, memory cell 1003, determining unit 1004, writing unit 1005 and first processing units
1006, wherein:
The dispensing unit 1001, for configuring the private attribute of the private data in source data, the private attribute is extremely
Include encryption version information and encryption policy less;
The ciphering unit 1002, for during being loaded to source data, according to the model of the private data
When enclosing the loaded source data of determination includes private data, the encryption policy configured is read, according to the encryption policy to institute
State private data to be encrypted, the source data after being encrypted;Source data after the encryption is sent to the data warehouse;
The memory cell 1003, for the source data after the encryption to be deposited in the caching of the data warehouse;
The determining unit 1004, for determining mesh in the Data Mart more than one according to the mapping relations
Mark corresponding target data objects in Data Mart and the target data fairground;
Said write unit 1005, for sending data synchronization request message, the data to the target data fairground
Synchronization request message includes encryption version information, the source data after the encryption is write in the disk of the data warehouse;
The first processing units 1006, the data sync response message sent for receiving the target data fairground
Afterwards, the source data after the encryption that is deposited in the caching is handled according to the data sync response message.
In the embodiment of the present invention, the dispensing unit 1001 and ciphering unit 1002 are arranged on the processor of ETL servers
On, and memory cell 1003, determining unit 1004, writing unit 1005 and first processing units 1006 are arranged on data warehouse
On processor;
In the embodiment of the present invention, the first processing units 1006 include determining module and remove module, wherein:
The determining module, for according to the data sync response message determine the target data fairground whether needs
The data object is received, when no, module is removed in triggering;Accordingly, the removing module, for that will be deposited in the caching
The encryption after source data remove.
The determining module, for according to the data sync response message determine the target data fairground whether needs
The target data objects are received, when being, the target data objects being deposited in the caching of the data warehouse are sent
To the target data fairground;Accordingly, the removing module, confirm for receiving the target data fairground is sent first
Message, first confirmation message are used to show to receive data success, the source after the encryption that will be deposited in the caching
Data dump.
In the embodiment of the present invention, described device also includes receiving unit, the second determining unit and the 3rd determining unit, its
In:
The receiving unit, the data synchronization request message is received for the target data fairground, the data are same
Step request message includes the identity of sender and the first version information of currently used AES;
Second determining unit, for according to the identity of sender in the data synchronization request message to described
The communications identities of sender are confirmed, when confirming successfully, trigger the 3rd determining unit;
3rd determining unit, for continue to determine reception time of the data synchronization request message with it is described synchronous
When cycle is consistent, data sync response message is sent to the data warehouse, the data sync response message includes first
Confirmation, first confirmation show to make the data warehouse when receiving the data sync response message, to
Synchronous target data objects needed for the target data fairground transmission.
Embodiment 11
A kind of distributed collaboration encryption device provided in an embodiment of the present invention, the data applied to isomeric data plateform system
ETL servers, the isomeric data plateform system include ETL servers and data warehouse;Figure 11 is the embodiment of the present invention 11
The composition structural representation of distributed collaboration encryption device, as shown in figure 11, the device includes dispensing unit 1101 and encryption is single
Member, wherein the ciphering unit includes reading unit 1102, the first ciphering unit 1103 and the first transmitting element 1104, wherein:
The dispensing unit 1101, for configuring the private attribute of the private data in source data, the private attribute is extremely
Include encryption policy less;
The reading unit 1102, for during being loaded to source data, according to the model of the private data
When enclosing the loaded source data of determination includes private data, the encryption policy configured is read, the encryption policy is as dynamic
The execution input parameter of function in chained library;
First ciphering unit 1103, for the private data to be encrypted according to the encryption policy, obtain
Source data after encryption;
First transmitting element 1104, for the source data after the encryption to be sent into the data warehouse.
Embodiment 12
A kind of distributed collaboration encryption device provided in an embodiment of the present invention, it is described applied to isomeric data plateform system
Isomeric data plateform system includes data warehouse and more than one Data Mart, the first data object in the data warehouse
Mapping relations are respectively present between each second data object in one Data Mart above;Figure 12 is real for the present invention
The composition structural representation of the distributed collaboration encryption device of example 12 is applied, as shown in figure 12, the device includes the second dispensing unit
1201st, receiving unit 1202, the second determining unit 1203 and the 3rd determining unit 1204, wherein:
Second dispensing unit 1201, it is described same for configuring synchronization policy for one Data Mart above
Step strategy includes synchronizing cycle;
The receiving unit 1202, the data synchronization request message sent for receiving the data warehouse are described
Data synchronization request message includes the identity of sender and the first version information of currently used AES;
Second determining unit 1203, for the identity pair according to sender in the data synchronization request message
The communications identities of described sender are confirmed, when confirming successfully, trigger the 3rd determining unit;
3rd determining unit 1204, for continue to determine reception time of the data synchronization request message with it is described
When synchronizing cycle is consistent, data sync response message is sent to the data warehouse, the data sync response message includes
First confirmation, first confirmation show to make the data warehouse receive the data sync response message
When, to target data objects synchronous needed for the transmission of the target data fairground.
In the embodiment of the present invention, the data synchronization request message includes the first version of currently used AES
Information, the 3rd determining unit 1204, be additionally operable to determine the first version information and the AES that prestores the
When two version informations are inconsistent, data sync response message, the data sync response message package are sent to the data warehouse
The first confirmation is included, first confirmation shows to make the data warehouse receive the data sync response message
When, send target data objects to the target data fairground.
3rd determining unit 1204, is additionally operable to determine the first version information and the AES that prestores
Second edition information is consistent, and determines that the reception time of the data synchronization request message and the synchronizing cycle are mutually inconsistent
When, data sync response message is sent to the data warehouse, the data sync response message includes the second confirmation, institute
State the second confirmation and show the data warehouse when receiving the data sync response message, not to the target data
Fairground, which is sent, needs synchronous target data objects.
In the embodiment of the present invention, the device also includes second processing unit, is finished for receiving the target data objects
Afterwards, the target data fairground according to the data synchronization request message to received data object carry out integrality with it is consistent
Property confirm;The data synchronization request message includes the original position and size of target data objects;When confirming successfully, to institute
Data warehouse is stated to send for showing to receive successful first confirmation message of data object;When confirming failure, to the data bins
Storehouse sends the second confirmation message for representing reception data object failure.
Embodiment 13
A kind of distributed collaboration encryption device provided in an embodiment of the present invention, applied to the number in isomeric data plateform system
According to warehouse;Figure 13 is the composition structural representation of the distributed collaboration encryption device of the embodiment of the present invention 13, as shown in figure 13, should
Device includes memory cell 1301, the first determining unit 1302, writing unit 1303 and first processing units 1304, wherein:
The memory cell 1301, the source data after encryption for receiving the transmission of ETL servers, and by after the encryption
Source data be deposited in the caching of the data warehouse;
First determining unit 1302, for determining the Data Mart more than one according to the mapping relations
Corresponding target data objects in middle target data fairground and the target data fairground;
Said write unit 1303, for the source data after the encryption to be write in the disk of the data warehouse;
The first processing units 1304, the data sync response message sent for receiving the target data fairground
Afterwards, the source data after the encryption that is deposited in the caching is handled according to the data sync response message.
In the embodiment of the present invention, described device also includes the 4th determining unit, the 5th determining unit and the second transmitting element,
Wherein:
4th determining unit, when the scope for confirming private data expands, according to the encryption policy to newly-increased
Data object be encrypted;
5th determining unit, for determining mesh according in Data Mart of the mapping relations more than one
Data Mart is marked, triggers second transmitting element;
Second transmitting element, for sending data synchronization request message, the data to the target data fairground
Synchronization request message includes encryption version information, triggers the processing unit;The processing unit, for the data warehouse
After receiving the data sync response message that the target data fairground is sent, according to the data sync response message to being deposited at
The source data after the encryption in the caching is handled.
In the embodiment of the present invention, the first processing units 1304 include determining module and remove module, wherein:
The determining module, for according to the data sync response message determine the target data fairground whether needs
The data object is received, when no, module is removed in triggering;Accordingly, the removing module, for that will be deposited in the caching
The encryption after source data remove.
The determining module, for according to the data sync response message determine the target data fairground whether needs
The target data objects are received, when being, the target data objects being deposited in the caching of the data warehouse are sent
To the target data fairground;Accordingly, the removing module, confirm for receiving the target data fairground is sent first
Message, first confirmation message are used to show to receive data success, the source after the encryption that will be deposited in the caching
Data dump.
The embodiment of the present invention also provides a kind of distributed collaboration management system, including described in above-described embodiment 11 to 13
Distributed collaboration encryption device.
Embodiments of the invention described above solve in data warehouse and system cooperateing with for private data between other databases
Problem of management, by the synchronous crypto-operation technology of data warehouse and Data Mart, ensured private data from import, transmission, load,
Using with safety and consistency problem in the overall process such as offline, be truly realized one-time pad encryption, synchronous transfer, point of dynamic renewal
Cloth coordinated management target.
It will be understood by those of skill in the art that the described distributed collaboration encryption device of embodiment ten to 13 is everywhere
Reason unit and the module in each unit realize that function can refer to the correlation of foregoing described distributed collaboration encryption method and retouch
State and understand.If the above-mentioned integrated unit of the present invention is realized in the form of software function module and is used as independent production marketing
Or in use, it can also be stored in a computer read/write memory medium.Based on such understanding, the embodiment of the present invention
The part that technical scheme substantially contributes to prior art in other words can be embodied in the form of software product, the meter
Calculation machine software product is stored in a storage medium, including some instructions are causing a computer equipment(Can be
People's computer, server or network equipment etc.)Perform all or part of each embodiment methods described of the present invention.It is and preceding
The storage medium stated includes:Movable storage device, read-only storage(ROM, Read-Only Memory), magnetic disc or CD etc.
It is various can be with the medium of store program codes.The foregoing is only a preferred embodiment of the present invention, is not intended to limit this
The protection domain of invention.
Claims (15)
1. a kind of distributed collaboration encryption method, it is characterised in that applied to isomeric data plateform system, the isomeric data is put down
Platform system includes data extraction, conversion and loading ETL servers, data warehouse and more than one Data Mart, the data
It is respectively present and reflects between each second data object in the first data object and one Data Mart above in warehouse
Penetrate relation;The scope and private attribute of the private data in source data are configured, the private attribute comprises at least the first of encryption
Version information and encryption policy, methods described include:
The ETL servers determine to be loaded during source data is loaded, according to the scope of the private data
Source data when including private data, the encryption policy of loaded source data configuration is read as, according to the encryption policy pair
The private data is encrypted, the source data after being encrypted;
Source data after the encryption is sent to the data warehouse by the ETL servers;
Source data after the encryption is deposited in caching by the data warehouse;
The data warehouse according to determined in Data Mart of the mapping relations more than one target data fairground and
Corresponding target data objects in the target data fairground;
The data warehouse sends data synchronization request message to the target data fairground, and by the source data after the encryption
Write in the disk of the data warehouse;Wherein, the data synchronization request message includes the first version letter of the encryption
Breath.
2. according to the method for claim 1, it is characterised in that methods described also includes:
After the data warehouse receives the data sync response message that the target data fairground is sent, according to the data syn-chronization
Response message is handled the source data after the encryption that is deposited in the caching.
3. according to the method for claim 1, it is characterised in that methods described also includes:Configure the privacy number in source data
According to scope;
When the data warehouse confirms that the scope of private data expands, newly-increased data object is carried out according to the encryption policy
Encryption;
The data warehouse determines target data fairground according in Data Mart of the mapping relations more than one;
The data warehouse sends data synchronization request message to the target data fairground, in the data synchronization request message
Including encryption version information;
After the data warehouse receives the data sync response message that the target data fairground is sent, according to the data syn-chronization
Response message is handled the source data after the encryption that is deposited in the caching.
4. according to the method in claim 2 or 3, it is characterised in that the data warehouse is according to the data sync response
Message is handled the source data after the encryption that is deposited in the caching, including:
The data warehouse determines that the target data fairground need not receive the encryption according to the data sync response message
During rear source data, the source data after the encryption that will be deposited in the caching is removed.
5. according to the method in claim 2 or 3, it is characterised in that the data warehouse is according to the data sync response
Message is handled the source data after the encryption that is deposited in the caching, including:
After the data warehouse determines that the target data fairground need to receive the encryption according to the data sync response message
Source data when, the source data after the encryption that will be deposited in the caching of the data warehouse is sent to the target data
Fairground;
The data warehouse receives the first confirmation message that the target data fairground is sent, and first confirmation message is used for table
Bright reception data success, the source data after the encryption that will be deposited in the caching are removed.
6. according to the method for claim 4, it is characterised in that methods described also includes:
The target data fairground receives the data synchronization request message, and the data synchronization request message includes currently making
The first version information of AES;
The target data fairground determine the first version information and the second edition information of AES that prestores not
When consistent, data sync response message is sent to the data warehouse, the data sync response message includes the first confirmation letter
Breath, first confirmation shows to make the data warehouse when receiving the data sync response message, to the mesh
Mark Data Mart and send the source data after encryption.
7. according to the method for claim 5, it is characterised in that methods described also includes:
The target data fairground receives the data synchronization request message, when determining the reception of the data synchronization request message
Between it is consistent with the synchronizing cycle of configuration when, send the data sync response message to the data warehouse, the data are same
Step response message includes the first confirmation, and first confirmation shows to make the data warehouse receive the data
During synchronous response message, the source data to after encryption synchronous needed for the transmission of target data fairground.
8. according to the method for claim 5, it is characterised in that methods described also includes:The target data fairground receives
The data synchronization request message, the data synchronization request message include the first version letter of currently used AES
Breath;
The target data fairground determines the first version information and the second edition information one of the AES prestored
Cause, and determine when receiving time and the mutually inconsistent synchronizing cycle of configuration of the data synchronization request message, to the data
Warehouse sends data sync response message, and the data sync response message includes the second confirmation, second confirmation letter
Breath shows the data warehouse when receiving the data sync response message, and not sent to the target data fairground needs together
Source data after the encryption of step.
9. according to the method described in claim 1 or 2 or 3 or 6 or 7 or 8, it is characterised in that methods described also includes:
The target data fairground receives the data synchronization request message, and the data synchronization request message includes sender
Identity;
The target data fairground is according to the identity of sender in the data synchronization request message to described sender
Communications identities are confirmed, when confirming successfully, the data sync response message is sent to the data warehouse.
10. according to the method for claim 9, it is characterised in that methods described also includes:
After the target data objects receive the source data after the encryption, the target data fairground is according to the data
Synchronization request message carries out integrality to received data object and uniformity confirms;Wrapped in the data synchronization request message
Include the original position and size of the source data after encryption;
When the target data objects are confirmed successfully, sent to the data warehouse for showing to receive data object successful the
One confirmation message;
When the target data objects confirm failure, to the data warehouse send for represent to receive data object failure the
Two confirmation messages.
A kind of 11. distributed collaboration encryption method, it is characterised in that applied to isomeric data plateform system, the isomeric data
Plateform system includes data extraction, conversion and loading ETL servers, data warehouse and more than one Data Mart, the number
It is respectively present according between each second data object in the first data object in warehouse and one Data Mart above
Mapping relations;
Methods described includes:
The data warehouse receives the source data after the encryption that ETL servers are sent, and the source data after the encryption is deposited
In the caching of the data warehouse;
The data warehouse determined according to the mapping relations in Data Mart more than one target data fairground with
Corresponding target data objects in the target data fairground;
The data warehouse sends data synchronization request message to the target data fairground;
The data warehouse writes the source data after the encryption in the disk of the data warehouse;
After the data warehouse receives the data sync response message that the target data fairground is sent, according to the data syn-chronization
Response message is handled the source data after the encryption that is deposited in the caching.
12. according to the method for claim 11, it is characterised in that methods described also includes:Configure the privacy in source data
The scope of data;
When the data warehouse confirms that the scope of private data expands, the data warehouse is according to the encryption policy to newly-increased
Data object is encrypted;
The data warehouse determines target data fairground in Data Mart more than one according to the mapping relations;
The data warehouse sends data synchronization request message to the target data fairground, in the data synchronization request message
Including encryption version information;
After the data warehouse receives the data sync response message that the target data fairground is sent, according to the data syn-chronization
Response message is handled the source data after the encryption that is deposited in the caching.
13. the method according to claim 11 or 12, it is characterised in that the data warehouse rings according to the data syn-chronization
Message is answered to handle the source data after the encryption that is deposited in the caching, including:
The data warehouse determines that the target data fairground needs to receive the encryption according to the data sync response message
During rear source data, the source data after the encryption that will be deposited in the caching of the data warehouse is sent to the number of targets
According to fairground;
The data warehouse receives the first confirmation message that the target data fairground is sent, and first confirmation message is used for table
Bright reception data success, the source data after the encryption that will be deposited in the caching are removed.
A kind of 14. distributed collaboration encryption device, it is characterised in that applied to isomeric data plateform system, the isomeric data
Plateform system includes data extraction, conversion and loading ETL servers, data warehouse and more than one Data Mart, the number
It is respectively present according between each second data object in the first data object in warehouse and one Data Mart above
Mapping relations;Described device include dispensing unit, ciphering unit, memory cell, the first determining unit, at writing unit and first
Unit is managed, wherein:
The dispensing unit, for configuring the private attribute of the private data in source data, the private attribute, which comprises at least, to be added
Close version information and encryption policy;
The ciphering unit, for the extraction to source data, conversion and load ETL loading procedure in, when according to described hidden
When the scope of private data determines that loaded source data includes private data, the encryption policy configured is read, is added according to described
The private data is encrypted close strategy, the source data after being encrypted, and the source data after the encryption is sent to
The data warehouse;
The memory cell, for the source data after the encryption to be deposited in the caching of the data warehouse;
First determining unit, for determining number of targets in the Data Mart more than one according to the mapping relations
According to corresponding target data objects in fairground and the target data fairground;
Said write unit, for sending data synchronization request message, the data synchronization request to the target data fairground
Message includes encryption version information, the source data after the encryption is write on the disk of the data warehouse;
The first processing units, after the data sync response message sent for receiving the target data fairground, according to institute
Data sync response message is stated to handle the source data after the encryption that is deposited in the caching.
A kind of 15. distributed collaboration encryption device, it is characterised in that applied to isomeric data plateform system, the isomeric data
Plateform system includes ETL servers, data warehouse and more than one Data Mart, the first data pair in the data warehouse
As being respectively present mapping relations between each second data object in one Data Mart above;Described device includes
Memory cell, the first determining unit, writing unit and first processing units, wherein:
The memory cell, the source data after encryption for receiving the transmission of ETL servers, and by the source data after the encryption
It is deposited in the caching of the data warehouse;
First determining unit, for determining number of targets in the Data Mart more than one according to the mapping relations
According to corresponding target data objects in fairground and the target data fairground;
Said write unit, for sending data synchronization request message to the target data fairground, by the source after the encryption
Data are write in the disk of the data warehouse;
The first processing units, after the data sync response message sent for receiving the target data fairground, according to institute
Data sync response message is stated to handle the source data after the encryption that is deposited in the caching.
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN201410017811.2A CN104780038B (en) | 2014-01-15 | 2014-01-15 | A kind of distributed collaboration encryption method and device |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN201410017811.2A CN104780038B (en) | 2014-01-15 | 2014-01-15 | A kind of distributed collaboration encryption method and device |
Publications (2)
Publication Number | Publication Date |
---|---|
CN104780038A CN104780038A (en) | 2015-07-15 |
CN104780038B true CN104780038B (en) | 2018-02-23 |
Family
ID=53621297
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
CN201410017811.2A Active CN104780038B (en) | 2014-01-15 | 2014-01-15 | A kind of distributed collaboration encryption method and device |
Country Status (1)
Country | Link |
---|---|
CN (1) | CN104780038B (en) |
Families Citing this family (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN110019462B (en) * | 2017-11-14 | 2021-09-03 | 南方电网科学研究院有限责任公司 | Electric power scientific research production data analysis method, device, system and storage medium |
US11240266B1 (en) * | 2021-07-16 | 2022-02-01 | Social Safeguard, Inc. | System, device and method for detecting social engineering attacks in digital communications |
CN113590719B (en) * | 2021-09-27 | 2022-03-22 | 北京奇虎科技有限公司 | Data synchronization method, device, equipment and storage medium |
Citations (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN101123494A (en) * | 2007-06-28 | 2008-02-13 | 深圳市中科新业信息科技发展有限公司 | A network access behavior data encryption system and method |
CN102023979A (en) * | 2009-09-09 | 2011-04-20 | 中国工商银行股份有限公司 | Meta-data management method and system |
-
2014
- 2014-01-15 CN CN201410017811.2A patent/CN104780038B/en active Active
Patent Citations (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN101123494A (en) * | 2007-06-28 | 2008-02-13 | 深圳市中科新业信息科技发展有限公司 | A network access behavior data encryption system and method |
CN102023979A (en) * | 2009-09-09 | 2011-04-20 | 中国工商银行股份有限公司 | Meta-data management method and system |
Non-Patent Citations (1)
Title |
---|
一种可靠的数据仓库中ETL策略与架构设计;尤玉林、等;《计算机工程与应用》;20051031;全文 * |
Also Published As
Publication number | Publication date |
---|---|
CN104780038A (en) | 2015-07-15 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
TWI701561B (en) | Data backup method and device, storage medium and server | |
US8345876B1 (en) | Encryption/decryption system and method | |
WO2021197105A1 (en) | Blockchain-based information release | |
CN110266682B (en) | Data encryption method and device, mobile terminal and decryption method | |
CN107667515A (en) | Synchronization group and validation group in relevant device | |
CN108520183A (en) | A kind of date storage method and device | |
CN104184733B (en) | A kind of RFID lightweight mutual authentication methods encoded based on CRC | |
CN105320613A (en) | Systems and methods for dynamic data storage | |
CN103107995A (en) | Cloud computing environmental data secure storage system and method | |
CN107370606A (en) | A kind of microblogging multi-signature method based on block chain | |
CN107070660A (en) | A kind of design Storage method of block chain encrypted radio-frequency chip | |
CN112804133B (en) | Encryption group chat method and system based on blockchain technology | |
CN105101183A (en) | Method and system for protecting private contents at mobile terminal | |
CN105025102B (en) | The network storage call method and storage system of a kind of 3D printing model file | |
CN104780038B (en) | A kind of distributed collaboration encryption method and device | |
CN107786331A (en) | Data processing method, device, system and computer-readable recording medium | |
CN104243149A (en) | Encrypting and decrypting method, device and server | |
CN113609508A (en) | Block chain-based federal learning method, device, equipment and storage medium | |
CN109521956A (en) | A kind of cloud storage method, apparatus, equipment and storage medium based on block chain | |
CN105279198A (en) | Data table storage method, data table modification method, data table query method and data table statistical method | |
US11438156B2 (en) | Method and system for securing data | |
CN105224262A (en) | Data processing method | |
CN103139324A (en) | Address book system with contact person feature description and contact information | |
CN114547204A (en) | Data synchronization method and device, computer equipment and storage medium | |
JP2003163662A (en) | Paperless record in aircraft maintenance |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
C06 | Publication | ||
PB01 | Publication | ||
EXSB | Decision made by sipo to initiate substantive examination | ||
SE01 | Entry into force of request for substantive examination | ||
GR01 | Patent grant | ||
GR01 | Patent grant |