CN104780038B - A kind of distributed collaboration encryption method and device - Google Patents

A kind of distributed collaboration encryption method and device Download PDF

Info

Publication number
CN104780038B
CN104780038B CN201410017811.2A CN201410017811A CN104780038B CN 104780038 B CN104780038 B CN 104780038B CN 201410017811 A CN201410017811 A CN 201410017811A CN 104780038 B CN104780038 B CN 104780038B
Authority
CN
China
Prior art keywords
data
encryption
warehouse
fairground
source
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN201410017811.2A
Other languages
Chinese (zh)
Other versions
CN104780038A (en
Inventor
尤新霞
庞哲翀
乔栋
郭翔宇
张大亮
郭向红
孙颖飞
王波
魏国华
白晶晶
岑春祥
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
China Mobile Group Inner Mongolia Co Ltd
Original Assignee
China Mobile Group Inner Mongolia Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by China Mobile Group Inner Mongolia Co Ltd filed Critical China Mobile Group Inner Mongolia Co Ltd
Priority to CN201410017811.2A priority Critical patent/CN104780038B/en
Publication of CN104780038A publication Critical patent/CN104780038A/en
Application granted granted Critical
Publication of CN104780038B publication Critical patent/CN104780038B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Abstract

The invention discloses a kind of distributed collaboration encryption method and device, methods described to include:In the loading procedure to the ETL of source data, when determining that loaded source data includes private data according to the scope of the private data, the encryption policy configured is read, the private data is encrypted according to the encryption policy;Source data after encryption is sent to the data warehouse, the source data after the encryption is deposited in the caching of the data warehouse;Corresponding target data objects in target data fairground and the target data fairground are determined in Data Mart more than one according to the mapping relations;Data synchronization request message is sent to the target data fairground;Source data after the encryption is write in the disk of the data warehouse;After receiving the data sync response message that the target data fairground is sent, the source data after the encryption that is deposited in the caching is handled according to the data sync response message.

Description

A kind of distributed collaboration encryption method and device
Technical field
The present invention relates to the encryption technology in big data field, more particularly to a kind of distributed collaboration encryption method and device.
Background technology
Available data cipher mode includes single system cipher mode and mirror-image copies mode, and both is suitable for forms data and put down Platform or the less situation of data scale;Wherein, because data are stored in a manner of table, mirror image cipher mode is substantially table level The mapping of data object is synchronous, is not suitable for multi-platform isomerous environment model.Encryption for mass data, prior art are general In the presence of two kinds of solutions:
A solution is to use the isomeric data plateform system being made up of Distributed Data Warehouse and other databases, In the isomeric data plateform system, the private data guard between synchronous each platform is using secondary cipher mode, i.e. data Warehouse is encrypted and the encryption of other databases is carried out at twice:First, data warehouse is according to the security strategy being set Private data in storehouse is encrypted;Afterwards, other database roots are encrypted again according to the security strategy of itself;When using hidden Include AES with respective security strategy during private data and key is reduced.
Another solution is in data extraction, conversion and loading(ETL, Extraction Transformation Loading)During use one-time pad encryption, then asynchronous transmission is to data warehouse and Data Mart.The problem of this scheme It is that can not ensure data warehouse due to when other reasonses change, the consistency synchronization of data, such as after key changes, Encryption data will update again, and ETL can not be responsible for increasing simultaneously ETL management load, so as to reduce the speed of data loading.
Secondary cipher mode causes same data source information to be encrypted twice used by prior art, and consumption is a large amount of Computing resource, while secondary encryption can cause the algorithm of separated encryption, key may be inconsistent, customer privacy data may produce Raw encrypted result difference;Asynchronous Transfer Mode can realize one-time pad encryption used by prior art, but can not ensure different Walk in transmitting procedure due to network failure, loss of data, incorrect order caused by Buffer Overflow, the data quality problem brought.
Both the above solution can not all solve isomery big data plateform system during operation, master data warehouse by After key version updating etc. causes customer privacy data ciphertext to change, data warehouse and client in other databases are hidden The associated synchronisation and consistency problem of private data.
The content of the invention
In view of this, the embodiment of the present invention provides a kind of distributed collaboration for solution problems of the prior art and added Decryption method and device, while one-time pad encryption is realized, it can solve the problem that privacy between data warehouse and other interior databases of system The coordinated management problem of data.
What the technical scheme of the embodiment of the present invention was realized in:
A kind of distributed collaboration encryption method, applied to isomeric data plateform system, the isomeric data plateform system bag Include data extraction, conversion and load ETL servers, data warehouse and more than one Data Mart, in the data warehouse Mapping relations are respectively present between each second data object in first data object and one Data Mart above;Match somebody with somebody Put the scope and private attribute of the private data in source data, the private attribute comprise at least encryption first version information and Encryption policy, methods described include:
The ETL servers determine institute during being loaded to source data, according to the scope of the private data When the source data of loading includes private data, the encryption policy of loaded source data configuration is read as, according to the encryption plan Slightly the private data is encrypted, the source data after being encrypted;
Source data after the encryption is sent to the data warehouse by the ETL servers;
Source data after the encryption is deposited in caching by the data warehouse;
The data warehouse determines target data set according in Data Mart of the mapping relations more than one Corresponding target data objects in city and the target data fairground;
The data warehouse sends data synchronization request message to the target data fairground, and by the source after the encryption Data are write in the disk of the data warehouse;Wherein, the data synchronization request message includes the first edition of the encryption This information.
A kind of distributed collaboration encryption method, data extraction, conversion and loading ETL applied to isomeric data plateform system Server, the isomeric data plateform system include ETL servers and data warehouse;
The private attribute of private data in the ETL servers configuration source data, the private attribute, which comprises at least, to be added Close strategy, methods described include:
During being loaded to source data, the ETL servers determine institute according to the scope of the private data When the source data of loading includes private data, the encryption policy of loaded source data configuration is read as, the encryption policy is made For the execution input parameter of function in dynamic link library;
The private data is encrypted according to the encryption policy for the ETL servers, the source number after being encrypted According to;
Source data after the encryption is sent to the data warehouse by the ETL servers.
A kind of distributed collaboration encryption method, applied to isomeric data plateform system, the isomeric data plateform system bag Include data extraction, conversion and load ETL servers, data warehouse and more than one Data Mart, in the data warehouse Mapping relations are respectively present between each second data object in first data object and one Data Mart above;
Methods described includes:
The data warehouse receives the source data after the encryption that ETL servers are sent, and by the source data after the encryption It is deposited in the caching of the data warehouse;
The data warehouse determines target data set in Data Mart more than one according to the mapping relations Corresponding target data objects in city and the target data fairground;
The data warehouse writes the source data after the encryption in the disk of the data warehouse;
After the data warehouse receives the data sync response message that the target data fairground is sent, according to the data Synchronous response message is handled the source data after the encryption that is deposited in the caching.
A kind of distributed collaboration encryption device, applied to isomeric data plateform system, the isomeric data plateform system bag Include data extraction, conversion and load ETL servers, data warehouse and more than one Data Mart, in the data warehouse Mapping relations are respectively present between each second data object in first data object and one Data Mart above;Institute Stating device includes dispensing unit, ciphering unit, memory cell, the first determining unit, writing unit and first processing units, its In:
The dispensing unit, for configuring the private attribute of the private data in source data, the private attribute is at least wrapped Include encryption version information and encryption policy;
The ciphering unit, for the extraction to source data, conversion and load ETL loading procedure in, when according to institute When stating the scope of private data and determining that loaded source data includes private data, encryption policy that reading has configured, according to institute State encryption policy the private data is encrypted, the source data after being encrypted, and the source data after the encryption is sent out Give the data warehouse;
The memory cell, for the source data after the encryption to be deposited in the caching of the data warehouse;
First determining unit, for determining mesh in the Data Mart more than one according to the mapping relations Mark corresponding target data objects in Data Mart and the target data fairground;
Said write unit, for sending data synchronization request message, the data syn-chronization to the target data fairground Request message includes encryption version information, the source data after the encryption is write on the disk of the data warehouse;
The first processing units, after the data sync response message sent for receiving the target data fairground, root The source data after the encryption that is deposited in the caching is handled according to the data sync response message.
A kind of distributed collaboration encryption device, data extraction, conversion and loading ETL applied to isomeric data plateform system Server, the isomeric data plateform system include ETL servers and data warehouse;
Described device includes dispensing unit, reading unit, the first ciphering unit and the first transmitting element, wherein:
The dispensing unit, for configuring the private attribute of the private data in source data, the private attribute is at least wrapped Include encryption policy;
The reading unit, for the extraction to source data, conversion and load ETL loading procedure in, when it is determined that institute When the source data of loading includes private data, the encryption policy configured is read, the encryption policy is as in dynamic link library The execution input parameter of function;
First ciphering unit, for the private data to be encrypted according to the encryption policy, encrypted Source data afterwards;
First transmitting element, for the source data after the encryption to be sent into the data warehouse.
A kind of distributed collaboration encryption device, applied to isomeric data plateform system, described device includes memory cell, the One determining unit, writing unit and processing unit, wherein:
The memory cell, the source data after encryption for receiving the transmission of ETL servers, and by the source after the encryption Data register is in the caching of the data warehouse;
First determining unit, for determining mesh in the Data Mart more than one according to the mapping relations Mark corresponding target data objects in Data Mart and the target data fairground;
Said write unit, for the source data after the encryption to be write in the disk of the data warehouse;
The processing unit, after the data sync response message sent for receiving the target data fairground, according to institute Data sync response message is stated to handle the source data after the encryption that is deposited in the caching.
In the embodiment of the present invention, in the loading procedure to the ETL of source data, when the scope according to the private data is true When fixed loaded source data includes private data, the encryption policy configured is read, according to the encryption policy to described hidden Private data are encrypted;Source data after encryption is sent to the data warehouse, the source data after the encryption is deposited at institute In the caching for stating data warehouse;Target data fairground in Data Mart more than one is determined according to the mapping relations With corresponding target data objects in the target data fairground;Data synchronization request is sent to the target data fairground to disappear Breath;Source data after the encryption is write in the disk of the data warehouse;Receive the number that the target data fairground is sent After synchronous response message, according to the data sync response message to the source number after the encryption that is deposited in the caching According to being handled;In this way, while one-time pad encryption is realized, can solve the problem that hidden between other databases in data warehouse and system The coordinated management problem of private data.
Brief description of the drawings
Fig. 1 is the composition structural representation of the isomeric data plateform system of the embodiment of the present invention one;
Fig. 2 is the implementation process schematic diagram of the distributed collaboration encryption method of the embodiment of the present invention two;
Fig. 3 is the implementation process signal that the distributed collaboration encryption method of the embodiment of the present invention three is deployed on data warehouse Figure;
Fig. 4 is the implementation process schematic diagram that the embodiment of the present invention cooperates with encryption method;
Fig. 5 is the implementation process schematic diagram that the target data fairground of the embodiment of the present invention four determines synchronous response message;
Fig. 6 implementation processs of data synchronization process between the data warehouse of the embodiment of the present invention five and target data fairground are shown It is intended to;
Fig. 7 is the implementation process schematic diagram of the distributed collaboration encryption method of the embodiment of the present invention seven;
Fig. 8 is the implementation process schematic diagram of the distributed collaboration encryption method of the embodiment of the present invention eight;
Fig. 9 is the implementation process schematic diagram of the distributed collaboration encryption method of the embodiment of the present invention nine;
Figure 10 is the composition structural representation of the distributed collaboration encryption device of the embodiment of the present invention ten;
Figure 11 is the composition structural representation of the distributed collaboration encryption device of the embodiment of the present invention 11;
Figure 12 is the composition structural representation of the distributed collaboration encryption device of the embodiment of the present invention 12;
Figure 13 is the composition structural representation of the distributed collaboration encryption device of the embodiment of the present invention 13.
Embodiment
Embodiment one
A kind of distributed collaboration encryption method and device that the embodiment of the present invention one provides, applied to isomeric data platform system System, Fig. 1 is the composition structural representation of the isomeric data plateform system of the embodiment of the present invention one, as shown in figure 1, the isomeric data is put down Platform system includes privacy information protection layer equipment 11, obtains layer equipment 12, data Layer equipment 13 and application layer equipment 14, wherein:
The privacy information protection layer equipment 11 includes removing privacyization processing engine and privacy reduction engine, goes at privacy It can be decipher to manage engine, for the decryption to privacy information;Correspondingly, privacy reduction engine can be encryption equipment, be used for Encryption to privacy information.
The layer equipment 12 that obtains is used to pass through ETL to the source data of acquisition and goes privacyization to handle, and the acquisition layer is set Standby 12 can be ETL servers.
The data Layer equipment 13 includes data warehouse(DW or DWH, Data Warehouse)With each Data Mart;By It is described to obtain the data input after layer equipment 12 is handled to the DW, then it is synchronized to each Data Mart by judgement.
Application layer equipment 14 by the data input that the data Layer equipment 13 stores into corresponding application, such as some numbers Reduced according to without privacy, be then directly inputted to and be not required in the application of reduction;Some data need privacy to reduce, then by privacy also It is input to after original in the application for needing privacy to reduce;Application layer equipment 14 also includes the function of key management.
Here, the ETL servers are used to extract source data, changed and be loaded, to be stored in data warehouse In.Data warehouse is also known as data master depot, the first data object in the data warehouse with it is one more than data Mapping relations are respectively present between each second data object in fairground;The mapping relations are reflected including first, second, and third Relation is penetrated, first mapping relations are used to show the corresponding relation between data warehouse and Data Mart, second mapping Correspondence in the first table and Data Mart of the relation for showing data storage in data warehouse between the second table of data storage Relation, the second number that the 3rd mapping relations are used to show in the first data object and second table in first table According to the corresponding relation between object, first data object and second data object are all indicated with behavior unit. This mapping relations in the embodiment of the present invention between data warehouse and Data Mart can navigate to capable level, can be more accurate Ground synchronizes to data, synchronous so as to breach the mapping of the table level of traditional encryption.
Embodiment two
The embodiment of the present invention provides a kind of distributed collaboration encryption method, described different applied to isomeric data plateform system Structure data platform system includes ETL servers, data warehouse and more than one Data Mart, and first in the data warehouse Mapping relations are respectively present between each second data object in data object and one Data Mart above;Fig. 2 is The implementation process schematic diagram of the distributed collaboration encryption method of the embodiment of the present invention two, as shown in Fig. 2 the distributed collaboration encryption side Method comprises the following steps:
Step 201, the scope and private attribute of the private data in source data are configured;
Here, the source data can come from business intelligence(BI, Business Intelligence)Deng source database, When isomeric data plateform system is used for the communications field, the source data may be customer data and single etc. in detail, customer data and in detail The privacy information of client may be included in list, i.e. the source data includes private data.
Here, the private attribute comprises at least private attribute type, the first version information of encryption and encryption policy;Its In, the private attribute type includes phone number, opposite-terminal number, name, address and passport NO.;The private attribute is set Put:The operation increased to private attribute, delete, change, looked into.The encryption policy includes AES, the initial value of key inputs, Start encrypted location and encryption length;The AES is at least one of following AES:It is character type AES, visible Single byte AES and numerical value AES.
Step 202, in loading procedure is carried out to source data, when what is loaded according to the determination of the scope of the private data When source data includes private data, the encryption policy that loaded source data has configured is read as, according to the encryption policy pair The private data is encrypted, the source data after being encrypted;Source data after the encryption is sent to the data bins Storehouse;
Here, the step 201 and step 202 can be completed on ETL servers, and the encryption policy is as dynamic The execution input parameter of function in chained library;The first crypto engine is provided with ETL servers, first crypto engine being capable of root The source data converted is encrypted according to the encryption policy configured.It should be noted that privacy in the embodiment of the present invention Data scope and private attribute configuration can also be completed on other servers, ETL servers can needs when Wait the scope and private attribute for reading the private data configured in the server.In the embodiment of the present invention, the encryption behaviour of source data Work is completed on ETL servers, and consumption is to consume ETL server resources, the cryptographic operation and data warehouse of source data It is unrelated with Data Mart.
Step 203, the source data after the encryption is deposited in the caching of the data warehouse;Closed according to the mapping System determines in Data Mart more than one corresponding number of targets in target data fairground and the target data fairground According to object, data synchronization request message is sent to the target data fairground;Source data after the encryption is write into the number According in the disk in warehouse;
Here, the target data fairground is the more than one Data Mart, and the target data objects are encryption The part in source data afterwards;The data synchronization request message includes the first version information of the encryption;
Step 204, after receiving the data sync response message that the target data fairground is sent, according to the data syn-chronization Response message is handled the source data after the encryption that is deposited in the caching.
Here, the step 203 and step 204 can be completed on data warehouse, and data warehouse receives to be serviced from ETL Source data after the encryption of device, first it can typically be stored on caching;It is then determined that whether the source data after the encryption needs together Walk on Data Mart, when needing to be synchronized on target data fairground, the source data after will this be encrypted is sent to target Data Mart;It is that is, synchronous using data between data warehouse and target data fairground in the embodiment of the present invention The mode do not landed, so, save the cpu resource of data warehouse.
In the embodiment of the present invention, methods described also includes step B1, step B2 and step B3, wherein:
Step B1, the target data fairground receive the data synchronization request message, the data synchronization request message Include the identity of sender;
Step B2, according to communication body of the identity of sender in the data synchronization request message to described sender Part is confirmed, when confirming successfully, data sync response message is sent to the data warehouse;
Step B3, according to communication body of the identity of sender in the data synchronization request message to described sender Part is confirmed;When confirming failure, data sync response message is not sent to the data warehouse.
Above-mentioned steps B1 to B3 is completed on target data fairground, is provided by above-mentioned steps in transmitting procedure Communications identities confirm, and the data syn-chronization between follow-up data warehouse and target data fairground provides guaranteed reliability.
In the embodiment of the present invention, methods described also includes step C1, step C2 and step C3, wherein:
Step C1, after the target data objects receive the target data objects, target data fairground root Integrality is carried out to received data object according to the data synchronization request message and uniformity confirms;The data syn-chronization please Message is asked to include the original position and size of target data objects;
Step C2, when confirming successfully, sent to the data warehouse for showing that reception data object successful first is true Recognize message;
Step C3, when confirming failure, sent to the data warehouse for representing that receive data object failure second is true Recognize message.
Above-mentioned steps B1 to B3 pages is completed on target data fairground, is provided and is being transmitted by above-mentioned steps Rear verification and check.
Embodiment three
Step 201 and step 202 in the embodiment of the present invention two are that the first crypto engine is provided with ETL servers, Can also be so as to which newly-increased source data be encrypted according to the scope of private data and encryption policy, in the embodiment of the present invention Second crypto engine is set on data warehouse, second crypto engine is identical with the first crypto engine, except that, the One crypto engine is encrypted for newly-increased source data, and the second crypto engine be then in the scope expansion of private data, The first data object to be stored on data warehouse is encrypted, and specifically, Fig. 3 is the distributed association of the embodiment of the present invention three The implementation process schematic diagram being deployed in encryption method on data warehouse, as shown in figure 3, the distributed collaboration encryption method includes Following steps:
Step 301, when confirming that the scope of private data expands, the data warehouse is according to the encryption policy to newly-increased First data object is encrypted, and the first data object after encryption is deposited in caching;
Step 302, target data fairground in Data Mart more than one is determined according to the mapping relations;
Step 303, data synchronization request message is sent to the target data fairground, in the data synchronization request message Including encryption version information, the source data after the encryption is write in the disk of the data warehouse;
Step 304, after receiving the data sync response message that the target data fairground is sent, according to the data syn-chronization Response message is handled the source data after the encryption that is deposited in the caching.
The step 304 in step 204 and the embodiment of the present invention three in the embodiment of the present invention two, it is described according to the number The source data after the encryption that is deposited in the caching is handled according to synchronous response message, including step F1:
Step F1, determine that the target data fairground need not receive the data pair according to the data sync response message As when, source data after the encryption that will be deposited in the caching is removed.
Step 204 in the embodiment of the present invention, it is described according to the data sync response message to being deposited at the caching In the encryption after source data handled, including step F2 and step F3, wherein:
Step F2, determine that the target data fairground needs to receive the number of targets according to the data sync response message During according to object, the target data objects being deposited in the caching of the data warehouse are sent to the target data set City;
Step F3, receives the first confirmation message that the target data fairground is sent, and first confirmation message is used for table Bright reception data success, the source data after the encryption that will be deposited in the caching are removed.
Fig. 4 is the implementation process schematic diagram that the embodiment of the present invention cooperates with encryption method, as shown in figure 4, wherein, step 401 It is the first ciphering process on ETL servers to 403, step 404 to step 406 is encrypted for second on data warehouse Journey, wherein the first ciphering process is step 201 is to as described in step 203 in the embodiment of the present invention two, the second ciphering process such as this hair Bright embodiment step 301 is to described in 304.Wherein, privacy identification encryption identification module can be carried out according to the scope of private data Set, during specific implementation, above-mentioned encryption function can be by increasing User-Defined Functions in data warehouse(UDF, User Define Function)To realize.
Example IV
It is described to be disappeared according to the data sync response in the step 204 of the embodiment of the present invention two and embodiment three in step 304 The data sync response message in handling the source data after the encryption that is deposited in the caching is ceased, is by target Data Mart is sent to data warehouse, and target data fairground can carry out a series of sentencing when receiving data synchronization request message It is disconnected, when judging to need synchronous target data objects, the first confirmation will be carried in data sync response message;When sentencing It is disconnected when not needing synchronous target data objects, the second confirmation can be just carried in data sync response message.Fig. 5 is this hair Bright example IV target data fairground determines the implementation process schematic diagram of synchronous response message, as shown in figure 5, target data fairground Determine that synchronous response message comprises the following steps:
Step 501, synchronization policy is configured for one Data Mart above;
Here, the synchronization policy includes synchronizing cycle;It is described to configure synchronous plan for one Data Mart above Slightly can be that user configuration or data warehouse or Data Mart are automatically configured according to the setting of user, Those skilled in the art can realize according to various prior arts, repeat no more here.
Step 502, the data synchronization request message that the data warehouse is sent is received;
Here, the data synchronization request message includes the first version information of currently used AES;
Step 503, judge whether the second edition information of AES of the first version information with being stored is consistent, When consistent, step 504 is performed, when inconsistent, performs step 505;
Step 504, the data sync response message for carrying the first confirmation is sent to the data warehouse;
Here, the data sync response message includes the first confirmation, and first confirmation shows to make described Data warehouse sends target data objects when receiving the data sync response message, to the target data fairground;
Step 505, continue to judge whether the reception time of the data synchronization request message and the synchronizing cycle are consistent, When consistent, step 504 is performed, when inconsistent, performs step 506;
Step 506, the data sync response message for carrying the second confirmation is sent to the data warehouse;
Here, the data sync response message includes the second confirmation, and second confirmation shows to make described For data warehouse when receiving the data sync response message, not sent to the target data fairground needs synchronous target Data object.
Embodiment five
In the embodiment of the present invention two and three, when target data fairground judges to need synchronous target data objects, target Data Mart sends the data sync response message for carrying the first confirmation to data warehouse, and data warehouse is receiving carrying When having the data sync response message of the first confirmation message, target data objects can be sent to target data fairground, start data Transmitting procedure;In data transmission procedure is implemented, application programming can be established between data warehouse and Data Mart and connect Mouthful(API, Application Programming Interface).
The data syn-chronization support full dose method of synchronization and increment synchronization mode two between data warehouse and target data fairground The kind method of synchronization, wherein, the full dose method of synchronization is a kind of method of synchronization of cover type, and increment synchronization mode is the same of additional formula Step mode;When target data fairground receives target data objects, the second mapping relations inquiry pair in mapping relations The second table answered, then the 3rd mapping relations in mapping relations inquire the associated row in the second table, then according to same Step mode synchronizes to the associated row in the second table.It should be noted that after the completion of ETL encryptions, can also be to the encryption Source data afterwards creates encrypted indexes, and the encrypted indexes are with behavior unit;In this manner it is possible to it will be added according to the 3rd mapping relations Source data after close is synchronized in target data fairground from data warehouse, can also by the newly-increased data after private data scope from Data warehouse is synchronized in target data fairground.
Target data objects can also be deposited in caching by target data fairground, when complete data integrality with it is consistent Property examine after, the associated row that is written to target data objects according to mapping relations in the second table.Can also be when the number of caching When reaching certain threshold value according to amount, the associated row that is written to target data objects according to mapping relations in the second table.Target data Fairground is when carrying out data syn-chronization, it is also necessary to which record data changes situation to form change daily record, and change daily record includes encryption Strategy;If failure of data synchronization, data recovery is carried out according to change daily record.
Above-mentioned data synchronization process can be by increasing User-Defined Functions(UDF, User Define Function)To realize;The reality of Fig. 6 data synchronization process between the data warehouse of the embodiment of the present invention five and target data fairground Existing schematic flow sheet, as shown in fig. 6, the data synchronization process comprises the following steps:
Step 601, data warehouse sends data synchronization request message to the target data fairground;
The data synchronization request message includes encryption version information;
Step 602, disk is write, source data that will be after the encryption is write in the disk of the data warehouse;
Step 603, target data fairground sends data sync response message to data warehouse, i.e.,:Data Mart is according to Fig. 5 Shown flow generation data sync response message;
Step 604, data warehouse is pre-processed to data, contrasted, and solves colliding data or generation Conflict solving file, And it is sent to target data set city;
Step 605, target data fairground solves inconsistent data, or inconsistent according to the solution of Conflict solving file Data;
Here, step 604 and step 605, those skilled in the art can carry out conventional place according to various prior arts Reason, is repeated no more here.
In embodiments of the present invention, by synchronous crypto-operation, to solve, data warehouse and data in Data Mart are inconsistent to ask Topic.Data warehouse can trigger above-mentioned synchronous crypto-operation when newly-increased data source, digital source content change and data eliminate Process.
Embodiment six
In step 301 in the step 201 and step 202 and embodiment three of the embodiment of the present invention two, in source data Information can be divided into numerical value and nonumeric information, wherein, nonumeric information includes letter, controlling symbols and graphical symbol etc., Nonumeric information is that computer is stored in a manner of binary-coded character code and is handled, the character code commonly used in computer There are ASCII character and extended binary coded decimal interchange code, and it is also relatively more for the standard of Chinese character, and such as GB2312, BIG-5, GBK, GB18030 are adopted Encoded with multibyte code.
In the embodiment of the present invention two, AES is at least one of following AES in crypto engine:Character type is encrypted Algorithm, visible single byte AES and numerical value AES, table 1 is for the encryption scope of these three AESs and on model The description enclosed, wherein the hexadecimal describing mode that encryption scope uses.
AES title Encrypt scope Scope describes
Character type AES 0X21 to 0XFF Numeral, letter, spcial character, Chinese character
It can be seen that single byte AES 0X21 to 0X7F Numeral, letter, spcial character
Numerical value AES 0X30 to 0X39 Numeral
Table 1
For ensure data encryption and reduction stability, encrypt the character beyond scope be former state export, be not involved in into The process of row enciphering and deciphering algorithm, i.e. these three AESs all avoid the spcial characters such as space, tab, so as to not to special Character is encrypted.The encrypted characters that character type AES includes are most, it is seen that the encryption of single byte AES is keyboard Upper all character visibles, what numerical value AES was encrypted is 0~9 numeral.With encryption " university of Inner Mongol Huhehaote City of province south Exemplified by the Room of road illusion mansion Building A 501 ", the effect of above-mentioned three kinds of AESs is as shown in table 2:
AES title Cipher round results
Character type AES D{a}sd@wer^0wfdsa[,./,/i]asj+k-
It can be seen that single byte AES University of Inner Mongol Huhehaote City of province South Road illusion mansion h seat Ab1 rooms
Numerical value AES The Room of university of Inner Mongol Huhehaote City of province South Road illusion mansion Building A 695
Table 2
Embodiment seven
A kind of distributed collaboration encryption method provided in an embodiment of the present invention, the ETL applied to isomeric data plateform system Server, the isomeric data plateform system include ETL servers and data warehouse;Fig. 7 is that the embodiment of the present invention seven is distributed The implementation process schematic diagram of encryption method is cooperateed with, as shown in fig. 7, this method includes:
Step 701, the private attribute of the private data in source data is configured;
Here, the private attribute comprises at least encryption policy;
Step 702, during being loaded to source data, determine what is loaded according to the scope of the private data When source data includes private data, the encryption policy of loaded source data configuration is read as;
Here, execution input parameter of the encryption policy as function in dynamic link library;
Step 703, the private data is encrypted according to the encryption policy, the source data after being encrypted;
Step 704, the source data after the encryption is sent to the data warehouse.
Embodiment eight
A kind of distributed collaboration encryption method that the embodiment of the present invention eight provides, the number applied to isomeric data plateform system According to warehouse, the isomeric data plateform system includes ETL servers, data warehouse and more than one Data Mart, the number It is respectively present according between each second data object in the first data object in warehouse and one Data Mart above Mapping relations;Fig. 8 is the implementation process schematic diagram of the distributed collaboration encryption method of the embodiment of the present invention eight, as shown in figure 8, the party The method distributed collaboration encryption method includes:
Step 801, the source data after the encryption that ETL servers are sent is received, and the source data after the encryption is deposited In the caching of the data warehouse;
Step 802, according to the mapping relations determine in Data Mart more than one target data fairground with Corresponding target data objects in the target data fairground;
Here, the target data fairground is the more than one Data Mart, and the target data objects are encryption The part in source data afterwards;
By step 803, the source data after the encryption is write in the disk of the data warehouse;
Step 804, after receiving the data sync response message that the target data fairground is sent, according to the data syn-chronization Response message is handled the source data after the encryption that is deposited in the caching.
In the embodiment of the present invention, methods described also includes step G1 to G4,:Configure the model of the private data in source data Enclose;
Step G1, when confirming that the scope of private data expands, the data warehouse is according to the encryption policy to newly-increased Data object is encrypted;
Step G2, target data fairground in Data Mart more than one is determined according to the mapping relations;
Step G3, data synchronization request message is sent to the target data fairground, in the data synchronization request message Including encryption version information;
Step G4, after receiving the data sync response message that the target data fairground is sent, according to the data syn-chronization Response message is handled the source data after the encryption that is deposited in the caching.
Step 804 of the embodiment of the present invention and step G4, it is described according to the data sync response message to described in being deposited at The source data after the encryption in caching is handled, including:
, will when determining that the target data fairground need not receive the data object according to the data sync response message It is deposited at the removing of the source data after the encryption in the caching.
Step 804 of the embodiment of the present invention and step G4, it is described according to the data sync response message to described in being deposited at The source data after the encryption in caching is handled, including:
Determine that the target data fairground needs to receive the target data objects according to the data sync response message When, the target data objects being deposited in the caching of the data warehouse are sent to the target data fairground;
The first confirmation message that the target data fairground is sent is received, first confirmation message is used to show to receive number According to success, the source data after the encryption that will be deposited in the caching is removed.
Embodiment nine
A kind of distributed collaboration encryption method provided in an embodiment of the present invention, it is described applied to isomeric data plateform system Isomeric data plateform system includes data warehouse and more than one Data Mart, the first data object in the data warehouse Mapping relations are respectively present between each second data object in one Data Mart above;Fig. 9 is real for the present invention The implementation process schematic diagram of the distributed collaboration encryption method of example nine is applied, as shown in figure 9, this method also includes:
Step 901, synchronization policy is configured for one Data Mart above, the synchronization policy includes synchronous week Phase;
Step 902, the target data fairground receives the data synchronization request message that the data warehouse is sent, institute Stating data synchronization request message includes the identity of sender and the first version information of currently used AES;
Step 903, according to communication of the identity of sender in the data synchronization request message to described sender Identity is confirmed, when confirming successfully, continues to determine the reception time of the data synchronization request message and the synchronizing cycle When consistent, data sync response message is sent to the data warehouse, the data sync response message includes first and confirmed Information, first confirmation show to make the data warehouse when receiving the data sync response message, to described Synchronous target data objects needed for the transmission of target data fairground.
In the embodiment of the present invention, the first edition of currently used AES is also included in the data synchronization request message This information;Methods described also includes:
When determining that the second edition information of AES of the first version information with being stored is inconsistent, to the number Data sync response message is sent according to warehouse, the data sync response message includes the first confirmation, and described first confirms Information shows to make the data warehouse when receiving the data sync response message, and mesh is sent to the target data fairground Mark data object.
In the embodiment of the present invention, the first version information and the second edition information one of the AES stored are determined When causing, and determining the reception time and the mutually inconsistent synchronizing cycle of the data synchronization request message, to the data bins Storehouse sends data sync response message, and the data sync response message includes the second confirmation, second confirmation Show to make the data warehouse when receiving the data sync response message, not described sent to target data fairground needs Synchronous target data objects.
Embodiment ten
A kind of distributed collaboration encryption device provided in an embodiment of the present invention, it is described applied to isomeric data plateform system Isomeric data plateform system includes ETL servers, data warehouse and more than one Data Mart, and in the data warehouse Mapping relations are respectively present between each second data object in one data object and one Data Mart above;Figure 10 For the composition structural representation of the distributed collaboration encryption device of the embodiment of the present invention ten, as shown in Figure 10, described device includes matching somebody with somebody Put unit 1001, ciphering unit 1002, memory cell 1003, determining unit 1004, writing unit 1005 and first processing units 1006, wherein:
The dispensing unit 1001, for configuring the private attribute of the private data in source data, the private attribute is extremely Include encryption version information and encryption policy less;
The ciphering unit 1002, for during being loaded to source data, according to the model of the private data When enclosing the loaded source data of determination includes private data, the encryption policy configured is read, according to the encryption policy to institute State private data to be encrypted, the source data after being encrypted;Source data after the encryption is sent to the data warehouse;
The memory cell 1003, for the source data after the encryption to be deposited in the caching of the data warehouse;
The determining unit 1004, for determining mesh in the Data Mart more than one according to the mapping relations Mark corresponding target data objects in Data Mart and the target data fairground;
Said write unit 1005, for sending data synchronization request message, the data to the target data fairground Synchronization request message includes encryption version information, the source data after the encryption is write in the disk of the data warehouse;
The first processing units 1006, the data sync response message sent for receiving the target data fairground Afterwards, the source data after the encryption that is deposited in the caching is handled according to the data sync response message.
In the embodiment of the present invention, the dispensing unit 1001 and ciphering unit 1002 are arranged on the processor of ETL servers On, and memory cell 1003, determining unit 1004, writing unit 1005 and first processing units 1006 are arranged on data warehouse On processor;
In the embodiment of the present invention, the first processing units 1006 include determining module and remove module, wherein:
The determining module, for according to the data sync response message determine the target data fairground whether needs The data object is received, when no, module is removed in triggering;Accordingly, the removing module, for that will be deposited in the caching The encryption after source data remove.
The determining module, for according to the data sync response message determine the target data fairground whether needs The target data objects are received, when being, the target data objects being deposited in the caching of the data warehouse are sent To the target data fairground;Accordingly, the removing module, confirm for receiving the target data fairground is sent first Message, first confirmation message are used to show to receive data success, the source after the encryption that will be deposited in the caching Data dump.
In the embodiment of the present invention, described device also includes receiving unit, the second determining unit and the 3rd determining unit, its In:
The receiving unit, the data synchronization request message is received for the target data fairground, the data are same Step request message includes the identity of sender and the first version information of currently used AES;
Second determining unit, for according to the identity of sender in the data synchronization request message to described The communications identities of sender are confirmed, when confirming successfully, trigger the 3rd determining unit;
3rd determining unit, for continue to determine reception time of the data synchronization request message with it is described synchronous When cycle is consistent, data sync response message is sent to the data warehouse, the data sync response message includes first Confirmation, first confirmation show to make the data warehouse when receiving the data sync response message, to Synchronous target data objects needed for the target data fairground transmission.
Embodiment 11
A kind of distributed collaboration encryption device provided in an embodiment of the present invention, the data applied to isomeric data plateform system ETL servers, the isomeric data plateform system include ETL servers and data warehouse;Figure 11 is the embodiment of the present invention 11 The composition structural representation of distributed collaboration encryption device, as shown in figure 11, the device includes dispensing unit 1101 and encryption is single Member, wherein the ciphering unit includes reading unit 1102, the first ciphering unit 1103 and the first transmitting element 1104, wherein:
The dispensing unit 1101, for configuring the private attribute of the private data in source data, the private attribute is extremely Include encryption policy less;
The reading unit 1102, for during being loaded to source data, according to the model of the private data When enclosing the loaded source data of determination includes private data, the encryption policy configured is read, the encryption policy is as dynamic The execution input parameter of function in chained library;
First ciphering unit 1103, for the private data to be encrypted according to the encryption policy, obtain Source data after encryption;
First transmitting element 1104, for the source data after the encryption to be sent into the data warehouse.
Embodiment 12
A kind of distributed collaboration encryption device provided in an embodiment of the present invention, it is described applied to isomeric data plateform system Isomeric data plateform system includes data warehouse and more than one Data Mart, the first data object in the data warehouse Mapping relations are respectively present between each second data object in one Data Mart above;Figure 12 is real for the present invention The composition structural representation of the distributed collaboration encryption device of example 12 is applied, as shown in figure 12, the device includes the second dispensing unit 1201st, receiving unit 1202, the second determining unit 1203 and the 3rd determining unit 1204, wherein:
Second dispensing unit 1201, it is described same for configuring synchronization policy for one Data Mart above Step strategy includes synchronizing cycle;
The receiving unit 1202, the data synchronization request message sent for receiving the data warehouse are described Data synchronization request message includes the identity of sender and the first version information of currently used AES;
Second determining unit 1203, for the identity pair according to sender in the data synchronization request message The communications identities of described sender are confirmed, when confirming successfully, trigger the 3rd determining unit;
3rd determining unit 1204, for continue to determine reception time of the data synchronization request message with it is described When synchronizing cycle is consistent, data sync response message is sent to the data warehouse, the data sync response message includes First confirmation, first confirmation show to make the data warehouse receive the data sync response message When, to target data objects synchronous needed for the transmission of the target data fairground.
In the embodiment of the present invention, the data synchronization request message includes the first version of currently used AES Information, the 3rd determining unit 1204, be additionally operable to determine the first version information and the AES that prestores the When two version informations are inconsistent, data sync response message, the data sync response message package are sent to the data warehouse The first confirmation is included, first confirmation shows to make the data warehouse receive the data sync response message When, send target data objects to the target data fairground.
3rd determining unit 1204, is additionally operable to determine the first version information and the AES that prestores Second edition information is consistent, and determines that the reception time of the data synchronization request message and the synchronizing cycle are mutually inconsistent When, data sync response message is sent to the data warehouse, the data sync response message includes the second confirmation, institute State the second confirmation and show the data warehouse when receiving the data sync response message, not to the target data Fairground, which is sent, needs synchronous target data objects.
In the embodiment of the present invention, the device also includes second processing unit, is finished for receiving the target data objects Afterwards, the target data fairground according to the data synchronization request message to received data object carry out integrality with it is consistent Property confirm;The data synchronization request message includes the original position and size of target data objects;When confirming successfully, to institute Data warehouse is stated to send for showing to receive successful first confirmation message of data object;When confirming failure, to the data bins Storehouse sends the second confirmation message for representing reception data object failure.
Embodiment 13
A kind of distributed collaboration encryption device provided in an embodiment of the present invention, applied to the number in isomeric data plateform system According to warehouse;Figure 13 is the composition structural representation of the distributed collaboration encryption device of the embodiment of the present invention 13, as shown in figure 13, should Device includes memory cell 1301, the first determining unit 1302, writing unit 1303 and first processing units 1304, wherein:
The memory cell 1301, the source data after encryption for receiving the transmission of ETL servers, and by after the encryption Source data be deposited in the caching of the data warehouse;
First determining unit 1302, for determining the Data Mart more than one according to the mapping relations Corresponding target data objects in middle target data fairground and the target data fairground;
Said write unit 1303, for the source data after the encryption to be write in the disk of the data warehouse;
The first processing units 1304, the data sync response message sent for receiving the target data fairground Afterwards, the source data after the encryption that is deposited in the caching is handled according to the data sync response message.
In the embodiment of the present invention, described device also includes the 4th determining unit, the 5th determining unit and the second transmitting element, Wherein:
4th determining unit, when the scope for confirming private data expands, according to the encryption policy to newly-increased Data object be encrypted;
5th determining unit, for determining mesh according in Data Mart of the mapping relations more than one Data Mart is marked, triggers second transmitting element;
Second transmitting element, for sending data synchronization request message, the data to the target data fairground Synchronization request message includes encryption version information, triggers the processing unit;The processing unit, for the data warehouse After receiving the data sync response message that the target data fairground is sent, according to the data sync response message to being deposited at The source data after the encryption in the caching is handled.
In the embodiment of the present invention, the first processing units 1304 include determining module and remove module, wherein:
The determining module, for according to the data sync response message determine the target data fairground whether needs The data object is received, when no, module is removed in triggering;Accordingly, the removing module, for that will be deposited in the caching The encryption after source data remove.
The determining module, for according to the data sync response message determine the target data fairground whether needs The target data objects are received, when being, the target data objects being deposited in the caching of the data warehouse are sent To the target data fairground;Accordingly, the removing module, confirm for receiving the target data fairground is sent first Message, first confirmation message are used to show to receive data success, the source after the encryption that will be deposited in the caching Data dump.
The embodiment of the present invention also provides a kind of distributed collaboration management system, including described in above-described embodiment 11 to 13 Distributed collaboration encryption device.
Embodiments of the invention described above solve in data warehouse and system cooperateing with for private data between other databases Problem of management, by the synchronous crypto-operation technology of data warehouse and Data Mart, ensured private data from import, transmission, load, Using with safety and consistency problem in the overall process such as offline, be truly realized one-time pad encryption, synchronous transfer, point of dynamic renewal Cloth coordinated management target.
It will be understood by those of skill in the art that the described distributed collaboration encryption device of embodiment ten to 13 is everywhere Reason unit and the module in each unit realize that function can refer to the correlation of foregoing described distributed collaboration encryption method and retouch State and understand.If the above-mentioned integrated unit of the present invention is realized in the form of software function module and is used as independent production marketing Or in use, it can also be stored in a computer read/write memory medium.Based on such understanding, the embodiment of the present invention The part that technical scheme substantially contributes to prior art in other words can be embodied in the form of software product, the meter Calculation machine software product is stored in a storage medium, including some instructions are causing a computer equipment(Can be People's computer, server or network equipment etc.)Perform all or part of each embodiment methods described of the present invention.It is and preceding The storage medium stated includes:Movable storage device, read-only storage(ROM, Read-Only Memory), magnetic disc or CD etc. It is various can be with the medium of store program codes.The foregoing is only a preferred embodiment of the present invention, is not intended to limit this The protection domain of invention.

Claims (15)

1. a kind of distributed collaboration encryption method, it is characterised in that applied to isomeric data plateform system, the isomeric data is put down Platform system includes data extraction, conversion and loading ETL servers, data warehouse and more than one Data Mart, the data It is respectively present and reflects between each second data object in the first data object and one Data Mart above in warehouse Penetrate relation;The scope and private attribute of the private data in source data are configured, the private attribute comprises at least the first of encryption Version information and encryption policy, methods described include:
The ETL servers determine to be loaded during source data is loaded, according to the scope of the private data Source data when including private data, the encryption policy of loaded source data configuration is read as, according to the encryption policy pair The private data is encrypted, the source data after being encrypted;
Source data after the encryption is sent to the data warehouse by the ETL servers;
Source data after the encryption is deposited in caching by the data warehouse;
The data warehouse according to determined in Data Mart of the mapping relations more than one target data fairground and Corresponding target data objects in the target data fairground;
The data warehouse sends data synchronization request message to the target data fairground, and by the source data after the encryption Write in the disk of the data warehouse;Wherein, the data synchronization request message includes the first version letter of the encryption Breath.
2. according to the method for claim 1, it is characterised in that methods described also includes:
After the data warehouse receives the data sync response message that the target data fairground is sent, according to the data syn-chronization Response message is handled the source data after the encryption that is deposited in the caching.
3. according to the method for claim 1, it is characterised in that methods described also includes:Configure the privacy number in source data According to scope;
When the data warehouse confirms that the scope of private data expands, newly-increased data object is carried out according to the encryption policy Encryption;
The data warehouse determines target data fairground according in Data Mart of the mapping relations more than one;
The data warehouse sends data synchronization request message to the target data fairground, in the data synchronization request message Including encryption version information;
After the data warehouse receives the data sync response message that the target data fairground is sent, according to the data syn-chronization Response message is handled the source data after the encryption that is deposited in the caching.
4. according to the method in claim 2 or 3, it is characterised in that the data warehouse is according to the data sync response Message is handled the source data after the encryption that is deposited in the caching, including:
The data warehouse determines that the target data fairground need not receive the encryption according to the data sync response message During rear source data, the source data after the encryption that will be deposited in the caching is removed.
5. according to the method in claim 2 or 3, it is characterised in that the data warehouse is according to the data sync response Message is handled the source data after the encryption that is deposited in the caching, including:
After the data warehouse determines that the target data fairground need to receive the encryption according to the data sync response message Source data when, the source data after the encryption that will be deposited in the caching of the data warehouse is sent to the target data Fairground;
The data warehouse receives the first confirmation message that the target data fairground is sent, and first confirmation message is used for table Bright reception data success, the source data after the encryption that will be deposited in the caching are removed.
6. according to the method for claim 4, it is characterised in that methods described also includes:
The target data fairground receives the data synchronization request message, and the data synchronization request message includes currently making The first version information of AES;
The target data fairground determine the first version information and the second edition information of AES that prestores not When consistent, data sync response message is sent to the data warehouse, the data sync response message includes the first confirmation letter Breath, first confirmation shows to make the data warehouse when receiving the data sync response message, to the mesh Mark Data Mart and send the source data after encryption.
7. according to the method for claim 5, it is characterised in that methods described also includes:
The target data fairground receives the data synchronization request message, when determining the reception of the data synchronization request message Between it is consistent with the synchronizing cycle of configuration when, send the data sync response message to the data warehouse, the data are same Step response message includes the first confirmation, and first confirmation shows to make the data warehouse receive the data During synchronous response message, the source data to after encryption synchronous needed for the transmission of target data fairground.
8. according to the method for claim 5, it is characterised in that methods described also includes:The target data fairground receives The data synchronization request message, the data synchronization request message include the first version letter of currently used AES Breath;
The target data fairground determines the first version information and the second edition information one of the AES prestored Cause, and determine when receiving time and the mutually inconsistent synchronizing cycle of configuration of the data synchronization request message, to the data Warehouse sends data sync response message, and the data sync response message includes the second confirmation, second confirmation letter Breath shows the data warehouse when receiving the data sync response message, and not sent to the target data fairground needs together Source data after the encryption of step.
9. according to the method described in claim 1 or 2 or 3 or 6 or 7 or 8, it is characterised in that methods described also includes:
The target data fairground receives the data synchronization request message, and the data synchronization request message includes sender Identity;
The target data fairground is according to the identity of sender in the data synchronization request message to described sender Communications identities are confirmed, when confirming successfully, the data sync response message is sent to the data warehouse.
10. according to the method for claim 9, it is characterised in that methods described also includes:
After the target data objects receive the source data after the encryption, the target data fairground is according to the data Synchronization request message carries out integrality to received data object and uniformity confirms;Wrapped in the data synchronization request message Include the original position and size of the source data after encryption;
When the target data objects are confirmed successfully, sent to the data warehouse for showing to receive data object successful the One confirmation message;
When the target data objects confirm failure, to the data warehouse send for represent to receive data object failure the Two confirmation messages.
A kind of 11. distributed collaboration encryption method, it is characterised in that applied to isomeric data plateform system, the isomeric data Plateform system includes data extraction, conversion and loading ETL servers, data warehouse and more than one Data Mart, the number It is respectively present according between each second data object in the first data object in warehouse and one Data Mart above Mapping relations;
Methods described includes:
The data warehouse receives the source data after the encryption that ETL servers are sent, and the source data after the encryption is deposited In the caching of the data warehouse;
The data warehouse determined according to the mapping relations in Data Mart more than one target data fairground with Corresponding target data objects in the target data fairground;
The data warehouse sends data synchronization request message to the target data fairground;
The data warehouse writes the source data after the encryption in the disk of the data warehouse;
After the data warehouse receives the data sync response message that the target data fairground is sent, according to the data syn-chronization Response message is handled the source data after the encryption that is deposited in the caching.
12. according to the method for claim 11, it is characterised in that methods described also includes:Configure the privacy in source data The scope of data;
When the data warehouse confirms that the scope of private data expands, the data warehouse is according to the encryption policy to newly-increased Data object is encrypted;
The data warehouse determines target data fairground in Data Mart more than one according to the mapping relations;
The data warehouse sends data synchronization request message to the target data fairground, in the data synchronization request message Including encryption version information;
After the data warehouse receives the data sync response message that the target data fairground is sent, according to the data syn-chronization Response message is handled the source data after the encryption that is deposited in the caching.
13. the method according to claim 11 or 12, it is characterised in that the data warehouse rings according to the data syn-chronization Message is answered to handle the source data after the encryption that is deposited in the caching, including:
The data warehouse determines that the target data fairground needs to receive the encryption according to the data sync response message During rear source data, the source data after the encryption that will be deposited in the caching of the data warehouse is sent to the number of targets According to fairground;
The data warehouse receives the first confirmation message that the target data fairground is sent, and first confirmation message is used for table Bright reception data success, the source data after the encryption that will be deposited in the caching are removed.
A kind of 14. distributed collaboration encryption device, it is characterised in that applied to isomeric data plateform system, the isomeric data Plateform system includes data extraction, conversion and loading ETL servers, data warehouse and more than one Data Mart, the number It is respectively present according between each second data object in the first data object in warehouse and one Data Mart above Mapping relations;Described device include dispensing unit, ciphering unit, memory cell, the first determining unit, at writing unit and first Unit is managed, wherein:
The dispensing unit, for configuring the private attribute of the private data in source data, the private attribute, which comprises at least, to be added Close version information and encryption policy;
The ciphering unit, for the extraction to source data, conversion and load ETL loading procedure in, when according to described hidden When the scope of private data determines that loaded source data includes private data, the encryption policy configured is read, is added according to described The private data is encrypted close strategy, the source data after being encrypted, and the source data after the encryption is sent to The data warehouse;
The memory cell, for the source data after the encryption to be deposited in the caching of the data warehouse;
First determining unit, for determining number of targets in the Data Mart more than one according to the mapping relations According to corresponding target data objects in fairground and the target data fairground;
Said write unit, for sending data synchronization request message, the data synchronization request to the target data fairground Message includes encryption version information, the source data after the encryption is write on the disk of the data warehouse;
The first processing units, after the data sync response message sent for receiving the target data fairground, according to institute Data sync response message is stated to handle the source data after the encryption that is deposited in the caching.
A kind of 15. distributed collaboration encryption device, it is characterised in that applied to isomeric data plateform system, the isomeric data Plateform system includes ETL servers, data warehouse and more than one Data Mart, the first data pair in the data warehouse As being respectively present mapping relations between each second data object in one Data Mart above;Described device includes Memory cell, the first determining unit, writing unit and first processing units, wherein:
The memory cell, the source data after encryption for receiving the transmission of ETL servers, and by the source data after the encryption It is deposited in the caching of the data warehouse;
First determining unit, for determining number of targets in the Data Mart more than one according to the mapping relations According to corresponding target data objects in fairground and the target data fairground;
Said write unit, for sending data synchronization request message to the target data fairground, by the source after the encryption Data are write in the disk of the data warehouse;
The first processing units, after the data sync response message sent for receiving the target data fairground, according to institute Data sync response message is stated to handle the source data after the encryption that is deposited in the caching.
CN201410017811.2A 2014-01-15 2014-01-15 A kind of distributed collaboration encryption method and device Active CN104780038B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201410017811.2A CN104780038B (en) 2014-01-15 2014-01-15 A kind of distributed collaboration encryption method and device

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201410017811.2A CN104780038B (en) 2014-01-15 2014-01-15 A kind of distributed collaboration encryption method and device

Publications (2)

Publication Number Publication Date
CN104780038A CN104780038A (en) 2015-07-15
CN104780038B true CN104780038B (en) 2018-02-23

Family

ID=53621297

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201410017811.2A Active CN104780038B (en) 2014-01-15 2014-01-15 A kind of distributed collaboration encryption method and device

Country Status (1)

Country Link
CN (1) CN104780038B (en)

Families Citing this family (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110019462B (en) * 2017-11-14 2021-09-03 南方电网科学研究院有限责任公司 Electric power scientific research production data analysis method, device, system and storage medium
US11240266B1 (en) * 2021-07-16 2022-02-01 Social Safeguard, Inc. System, device and method for detecting social engineering attacks in digital communications
CN113590719B (en) * 2021-09-27 2022-03-22 北京奇虎科技有限公司 Data synchronization method, device, equipment and storage medium

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101123494A (en) * 2007-06-28 2008-02-13 深圳市中科新业信息科技发展有限公司 A network access behavior data encryption system and method
CN102023979A (en) * 2009-09-09 2011-04-20 中国工商银行股份有限公司 Meta-data management method and system

Patent Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101123494A (en) * 2007-06-28 2008-02-13 深圳市中科新业信息科技发展有限公司 A network access behavior data encryption system and method
CN102023979A (en) * 2009-09-09 2011-04-20 中国工商银行股份有限公司 Meta-data management method and system

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
一种可靠的数据仓库中ETL策略与架构设计;尤玉林、等;《计算机工程与应用》;20051031;全文 *

Also Published As

Publication number Publication date
CN104780038A (en) 2015-07-15

Similar Documents

Publication Publication Date Title
TWI701561B (en) Data backup method and device, storage medium and server
US8345876B1 (en) Encryption/decryption system and method
WO2021197105A1 (en) Blockchain-based information release
CN110266682B (en) Data encryption method and device, mobile terminal and decryption method
CN107667515A (en) Synchronization group and validation group in relevant device
CN108520183A (en) A kind of date storage method and device
CN104184733B (en) A kind of RFID lightweight mutual authentication methods encoded based on CRC
CN105320613A (en) Systems and methods for dynamic data storage
CN103107995A (en) Cloud computing environmental data secure storage system and method
CN107370606A (en) A kind of microblogging multi-signature method based on block chain
CN107070660A (en) A kind of design Storage method of block chain encrypted radio-frequency chip
CN112804133B (en) Encryption group chat method and system based on blockchain technology
CN105101183A (en) Method and system for protecting private contents at mobile terminal
CN105025102B (en) The network storage call method and storage system of a kind of 3D printing model file
CN104780038B (en) A kind of distributed collaboration encryption method and device
CN107786331A (en) Data processing method, device, system and computer-readable recording medium
CN104243149A (en) Encrypting and decrypting method, device and server
CN113609508A (en) Block chain-based federal learning method, device, equipment and storage medium
CN109521956A (en) A kind of cloud storage method, apparatus, equipment and storage medium based on block chain
CN105279198A (en) Data table storage method, data table modification method, data table query method and data table statistical method
US11438156B2 (en) Method and system for securing data
CN105224262A (en) Data processing method
CN103139324A (en) Address book system with contact person feature description and contact information
CN114547204A (en) Data synchronization method and device, computer equipment and storage medium
JP2003163662A (en) Paperless record in aircraft maintenance

Legal Events

Date Code Title Description
C06 Publication
PB01 Publication
EXSB Decision made by sipo to initiate substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant