CN104780038A

CN104780038A - A distributed collaborative encryption method and device

Info

Publication number: CN104780038A
Application number: CN201410017811.2A
Authority: CN
Inventors: 尤新霞; 庞哲翀; 乔栋; 郭翔宇; 张大亮; 郭向红; 孙颖飞; 王波; 魏国华; 白晶晶; 岑春祥
Original assignee: China Mobile Group Inner Mongolia Co Ltd
Current assignee: China Mobile Group Inner Mongolia Co Ltd
Priority date: 2014-01-15
Filing date: 2014-01-15
Publication date: 2015-07-15
Anticipated expiration: 2034-01-15
Also published as: CN104780038B

Abstract

The present invention discloses a distributed collaborative encryption method and device. The method includes: when determining loaded metadata includes privacy data according to the scope of the privacy data during the loading process of ETL of the metadata, reading configured an encryption strategy; encrypting the privacy data according to the encryption strategy; sending the encrypted metadata to the data warehouse, the encrypted metadata being registered in a cache of the data warehouse; determining a target data market in the more than one of the data markets and a corresponding target data object in the target data market according to the mapping relation; sending a data synchronous request message to the target data market; writing the encrypted metadata into a disk of the data warehouse; and processing the encrypted metadata registered in the cache according to a data synchronous responding message after receiving the data synchronous responding message sent by the target data market.

Description

A kind of distributed collaboration encryption method and device

Technical field

The present invention relates to the encryption technology of large data fields, particularly relate to a kind of distributed collaboration encryption method and device.

Background technology

Available data cipher mode comprises single system cipher mode and mirror-image copies mode, and both is suitable for forms data platform or the less situation of data scale; Wherein, because data store to show mode, mirror image cipher mode is that the mapping of table DBMS object is synchronous substantially, is not suitable for multi-platform isomerous environment model.For the encryption of mass data, generally there are two kinds of solutions in prior art:

A solution adopts the isomeric data plateform system be made up of Distributed Data Warehouse and other databases, in this isomeric data plateform system, what the private data guard between synchronous each platform adopted is superencipher mode, namely the encryption of data warehouse encryption and other databases is carried out at twice: first, data warehouse is encrypted the private data in storehouse according to the security strategy set; Afterwards, other database roots are encrypted according to the security strategy of self again; Cryptographic algorithm is comprised by respective security strategy and key reduces when using private data.

Another kind of solution be extract in data, conversion and load (ETL, Extraction TransformationLoading) process in adopt and once encrypt, then asynchronous transmission is to data warehouse and Data Mart.The problem of this scheme is when cannot ensure data warehouse due to other reasons generation change, the consistency synchronization of data, after such as key changes, enciphered data will upgrade again, ETL cannot be responsible for the load management simultaneously increasing ETL, thus reduces the speed of data loading.

The superencipher mode that prior art adopts causes same data source information to carry out twice encryption, consume a large amount of computational resource, superencipher can cause the algorithm of separately encryption, key possibility inconsistent simultaneously, and customer privacy data may produce encrypted result difference; The Asynchronous Transfer Mode that prior art adopts can realize once encrypting, but cannot ensure due to network failure in asynchronous transmission process, the loss of data that Buffer Overflow causes, incorrect order, the data quality problem brought.

Above two kinds of solutions all cannot solve the large data platform system of isomery in operation process, master data warehouse is due to key version updating etc. and after causing customer privacy data ciphertext to change, the associated synchronisation of customer privacy data and consistency problem in data warehouse and other databases.

Summary of the invention

In view of this, the embodiment of the present invention provides a kind of distributed collaboration encryption method and device for solving problems of the prior art, while realization is once encrypted, the coordinated management problem of private data between other databases in data warehouse and system can be solved.

The technical scheme of the embodiment of the present invention is achieved in that

A kind of distributed collaboration encryption method, be applied to isomeric data plateform system, described isomeric data plateform system comprises that data are extracted, conversion and load ETL server, data warehouse and more than one Data Mart, there are mapping relations respectively between each second data object in the first data object in described data warehouse and described more than one Data Mart; The scope of the private data in source of configuration data and private attribute, described private attribute at least comprises first version information and the encryption policy of encryption, and described method comprises:

Described ETL server is in the process loaded source data, when determining that loaded source data comprises private data according to the scope of described private data, be read as the encryption policy of loaded source data configuration, according to described encryption policy, described private data is encrypted, obtains the source data after encrypting;

Source data after described encryption is sent to described data warehouse by described ETL server;

Source data after described encryption is deposited in buffer memory by described data warehouse;

Described data warehouse determines target data objects corresponding in target data fairground and described target data fairground according to described mapping relations in described more than one Data Mart;

Described data warehouse sends data synchronization request message to described target data fairground, and the source data after described encryption is write in the disk of described data warehouse; Wherein, described data synchronization request message comprises the first version information of described encryption.

A kind of distributed collaboration encryption method, be applied to the data extraction of isomeric data plateform system, conversion and load ETL server, described isomeric data plateform system comprises ETL server and data warehouse;

The private attribute of the private data in described ETL server source of configuration data, described private attribute at least comprises encryption policy, and described method comprises:

In the process that source data is loaded, when described ETL server determines that loaded source data comprises private data according to the scope of described private data, be read as the encryption policy of loaded source data configuration, described encryption policy is as the execution input parameter of function in dynamic link library;

Described ETL server is encrypted described private data according to described encryption policy, obtains the source data after encrypting;

Source data after described encryption is sent to described data warehouse by described ETL server.

A kind of distributed collaboration encryption method, be applied to isomeric data plateform system, described isomeric data plateform system comprises that data are extracted, conversion and load ETL server, data warehouse and more than one Data Mart, there are mapping relations respectively between each second data object in the first data object in described data warehouse and described more than one Data Mart;

Described method comprises:

Described data warehouse receives the source data after the encryption of ETL server transmission, and the source data after described encryption is deposited in the buffer memory of described data warehouse;

Described data warehouse determines target data objects corresponding in target data fairground and described target data fairground in described more than one Data Mart according to described mapping relations;

Source data after described encryption writes in the disk of described data warehouse by described data warehouse;

Described data warehouse processes the source data after the described encryption be deposited in described buffer memory according to described data sync response message after receiving the data sync response message of described target data fairground transmission.

A kind of distributed collaboration encryption device, be applied to isomeric data plateform system, described isomeric data plateform system comprises that data are extracted, conversion and load ETL server, data warehouse and more than one Data Mart, there are mapping relations respectively between each second data object in the first data object in described data warehouse and described more than one Data Mart; Described device comprises dispensing unit, ciphering unit, memory cell, the first determining unit, writing unit and the first processing unit, wherein:

Described dispensing unit, for the private attribute of the private data in source of configuration data, described private attribute at least comprises encryption version information and encryption policy;

Described ciphering unit, for in the loading procedure of the extraction to source data, conversion and loading ETL, when determining that loaded source data comprises private data according to the scope of described private data, read the encryption policy configured, according to described encryption policy, described private data is encrypted, obtain the source data after encrypting, and the source data after described encryption is sent to described data warehouse;

Described memory cell, for being deposited in the buffer memory of described data warehouse by the source data after described encryption;

Described first determining unit, for determining target data objects corresponding in target data fairground and described target data fairground in described more than one Data Mart according to described mapping relations;

Said write unit, for sending data synchronization request message to described target data fairground, described data synchronization request message comprises encryption version information, the source data after described encryption is write on the disk of described data warehouse;

Described first processing unit, for receive described target data fairground send data sync response message after, according to described data sync response message, the source data after the described encryption be deposited in described buffer memory is processed.

A kind of distributed collaboration encryption device, be applied to the data extraction of isomeric data plateform system, conversion and load ETL server, described isomeric data plateform system comprises ETL server and data warehouse;

Described device comprises dispensing unit, reading unit, the first ciphering unit and the first transmitting element, wherein:

Described dispensing unit, for the private attribute of the private data in source of configuration data, described private attribute at least comprises encryption policy;

Described reading unit, for in the loading procedure of the extraction to source data, conversion and loading ETL, when determining that loaded source data comprises private data, read the encryption policy configured, described encryption policy is as the execution input parameter of function in dynamic link library;

Described first ciphering unit, for being encrypted described private data according to described encryption policy, obtains the source data after encrypting;

Described first transmitting element, for sending to described data warehouse by the source data after described encryption.

A kind of distributed collaboration encryption device, be applied to isomeric data plateform system, described device comprises memory cell, the first determining unit, writing unit and processing unit, wherein:

Described memory cell, for receiving the source data after encryption that ETL server sends, and is deposited in the buffer memory of described data warehouse by the source data after described encryption;

Said write unit, for writing in the disk of described data warehouse by the source data after described encryption;

Described processing unit, for receive described target data fairground send data sync response message after, according to described data sync response message, the source data after the described encryption be deposited in described buffer memory is processed.

In the embodiment of the present invention, in the loading procedure of the ETL to source data, when determining that loaded source data comprises private data according to the scope of described private data, reading the encryption policy configured, according to described encryption policy, described private data being encrypted; Source data after encryption is sent to described data warehouse, and the source data after described encryption is deposited in the buffer memory of described data warehouse; Target data objects corresponding in target data fairground and described target data fairground in described more than one Data Mart is determined according to described mapping relations; Data synchronization request message is sent to described target data fairground; Source data after described encryption is write in the disk of described data warehouse; After receiving the data sync response message of described target data fairground transmission, according to described data sync response message, the source data after the described encryption be deposited in described buffer memory is processed; So, while realization is once encrypted, the coordinated management problem of private data between other databases in data warehouse and system can be solved.

Accompanying drawing explanation

Fig. 1 is the composition structural representation of the embodiment of the present invention one isomeric data plateform system;

Fig. 2 is the realization flow schematic diagram of the embodiment of the present invention two distributed collaboration encryption method;

Fig. 3 is the realization flow schematic diagram that the embodiment of the present invention three distributed collaboration encryption method is deployed on data warehouse;

Fig. 4 is the realization flow schematic diagram that the embodiment of the present invention works in coordination with encryption method;

Fig. 5 is the realization flow schematic diagram that synchronous response message is determined in the embodiment of the present invention four target data fairground;

Fig. 6 is the realization flow schematic diagram of data synchronization process between the embodiment of the present invention five data warehouse and target data fairground;

Fig. 7 is the realization flow schematic diagram of the embodiment of the present invention seven distributed collaboration encryption method;

Fig. 8 is the realization flow schematic diagram of the embodiment of the present invention eight distributed collaboration encryption method;

Fig. 9 is the realization flow schematic diagram of the embodiment of the present invention nine distributed collaboration encryption method;

Figure 10 is the composition structural representation of the embodiment of the present invention ten distributed collaboration encryption device;

Figure 11 is the composition structural representation of the embodiment of the present invention 11 distributed collaboration encryption device;

Figure 12 is the composition structural representation of the embodiment of the present invention 12 distributed collaboration encryption device;

Figure 13 is the composition structural representation of the embodiment of the present invention 13 distributed collaboration encryption device.

Embodiment

Embodiment one

A kind of distributed collaboration encryption method that the embodiment of the present invention one provides and device; be applied to isomeric data plateform system; Fig. 1 is the composition structural representation of the embodiment of the present invention one isomeric data plateform system; as shown in Figure 1; this isomeric data plateform system comprises privacy information protection layer equipment 11, securing layer equipment 12, data Layer equipment 13 and application layer equipment 14, wherein:

Described privacy information protection layer equipment 11 comprises privacy processing engine and privacy reduction engine, goes privacy processing engine to be decipher, for the deciphering to privacy information; Correspondingly, privacy reduction engine can be encryption equipment, for the encryption to privacy information.

Described securing layer equipment 12 for the source data obtained through ETL and go privacy process, described securing layer equipment 12 can be ETL server.

Described data Layer equipment 13 comprises data warehouse (DW or DWH, Data Warehouse) and each Data Mart; Data after described securing layer equipment 12 processes are input to described DW, are then synchronized to each Data Mart through judgement.

The data that described data Layer equipment 13 stores are input in corresponding application by application layer equipment 14, and such as some data is reduced without the need to privacy, be then directly inputted in the application not needing to reduce; Some data needs privacy to reduce, be then input in the application needing privacy to reduce after privacy reduction; Application layer equipment 14 also comprises the function of key management.

Here, described ETL server is used for extracting source data, change and loading, to be stored in data warehouse.Data warehouse is also called data master depot, there are mapping relations respectively between each second data object in the first data object in described data warehouse and described more than one Data Mart; Described mapping relations comprise first, second, and third mapping relations, described first mapping relations are for showing the corresponding relation between data warehouse and Data Mart, described second mapping relations are for showing to store in data warehouse the corresponding relation between the second table storing data in the first table of data and Data Mart, described 3rd mapping relations are for showing the corresponding relation between the first data object in described first table and the second data object in described second table, and described first data object and described second data object all represent with behavior unit.This mapping relations in the embodiment of the present invention between data warehouse and Data Mart can navigate to capable level, can more accurately carry out synchronously data, thus the table level breaching traditional encryption maps synchronous.

Embodiment two

The embodiment of the present invention provides a kind of distributed collaboration encryption method, be applied to isomeric data plateform system, described isomeric data plateform system comprises ETL server, data warehouse and more than one Data Mart, there are mapping relations respectively between each second data object in the first data object in described data warehouse and described more than one Data Mart; Fig. 2 is the realization flow schematic diagram of the embodiment of the present invention two distributed collaboration encryption method, and as shown in Figure 2, this distributed collaboration encryption method comprises the following steps:

Step 201, the scope of the private data in source of configuration data and private attribute;

Here, described source data can from business intelligence (BI, Business Intelligence) etc. source database, when isomeric data plateform system is used for the communications field, this source data may be customer data and detailed list etc., may comprise the privacy information of client in customer data and in detail list, namely this source data comprises private data.

Here, described private attribute at least comprises private attribute type, the first version information of encryption and encryption policy; Wherein, described private attribute type comprises phone number, opposite-terminal number, name, address and passport NO.; Described private attribute is arranged: the operation of increase private attribute, delete, change, looking into.Described encryption policy comprises cryptographic algorithm, the initial value of key inputs, start encrypted location and encryption length; Described cryptographic algorithm is at least one of following cryptographic algorithm: character type cryptographic algorithm, visible byte cryptographic algorithm and numerical value cryptographic algorithm.

Step 202, carrying out in loading procedure to source data, when determining that loaded source data comprises private data according to the scope of described private data, be read as the encryption policy that loaded source data configures, according to described encryption policy, described private data is encrypted, obtains the source data after encrypting; Source data after described encryption is sent to described data warehouse;

Here, described step 201 and step 202 can complete on ETL server, and described encryption policy is as the execution input parameter of function in dynamic link library; ETL server is provided with the first crypto engine, this first crypto engine can be encrypted the source data converted according to the encryption policy configured.It should be noted that, in the embodiment of the present invention, the scope of private data and private attribute configuration also can come on other server, and ETL server can read scope and the private attribute of the private data configured in this server when needing.In the embodiment of the present invention, the cryptographic operation of source data completes on ETL server, consumption be consume ETL server resource, the cryptographic operation of source data and data warehouse and Data Mart irrelevant.

Step 203, is deposited in the buffer memory of described data warehouse by the source data after described encryption; Determine target data objects corresponding in target data fairground and described target data fairground in described more than one Data Mart according to described mapping relations, send data synchronization request message to described target data fairground; Source data after described encryption is write in the disk of described data warehouse;

Here, described target data fairground is more than one described Data Mart, and described target data objects is the part in the source data after encryption; Described data synchronization request message comprises the first version information of described encryption;

Step 204, after receiving the data sync response message of described target data fairground transmission, processes the source data after the described encryption be deposited in described buffer memory according to described data sync response message.

Here, described step 203 and step 204 can complete on data warehouse, and data warehouse receives from the source data after the encryption of ETL server, generally can first be kept on buffer memory; Then determining that the source data after this encryption is the need of being synchronized on Data Mart, when needs are synchronized on target data fairground, the source data after this encryption will be sent to target data fairground; That is, the synchronous employing in the embodiment of the present invention between data warehouse with target data fairground be the mode that data are not landed, like this, save the cpu resource of data warehouse.

In the embodiment of the present invention, described method also comprises step B1, step B2 and step B3, wherein:

Step B1, described target data fairground receives described data synchronization request message, and described data synchronization request message comprises the identify label of transmit leg;

Step B2, confirms according to the communications identities of identify label to described transmit leg of transmit leg in described data synchronization request message, when confirming successfully, sends data sync response message to described data warehouse;

Step B3, confirms according to the communications identities of identify label to described transmit leg of transmit leg in described data synchronization request message; When confirming unsuccessfully, do not send data sync response message to described data warehouse.

Above-mentioned steps B1 to B3 completes on target data fairground, provides communications identities in transmitting procedure confirm by above-mentioned steps, and provides guaranteed reliability for the data syn-chronization between follow-up data warehouse and target data fairground.

In the embodiment of the present invention, described method also comprises step C1, step C2 and step C3, wherein:

Step C1, after described target data objects receives described target data objects, described target data fairground carries out integrality and consistency confirmation according to described data synchronization request message to received data object; Described data synchronization request message comprises original position and the size of target data objects;

Step C2, when confirming successfully, sends to described data warehouse and is used for showing to receive successful first acknowledge message of data object;

Step C3, when confirming unsuccessfully, sends for representing the second acknowledge message receiving data object failure to described data warehouse.

Above-mentioned steps B1 to B3 page completes on target data fairground, and the School Affairs provided upon completion of the transmission by above-mentioned steps is checked.

Embodiment three

Step 201 in the embodiment of the present invention two and step 202 are provided with the first crypto engine on ETL server, thus according to the scope of private data and encryption policy, newly-increased source data is encrypted, in the embodiment of the present invention, can also the second crypto engine be set on data warehouse, this second crypto engine is identical with the first crypto engine, difference is, first crypto engine is for newly-increased source data is encrypted, second crypto engine is then when the expanded range of private data, for the first data object be stored on data warehouse is encrypted, specifically, Fig. 3 is the realization flow schematic diagram that the embodiment of the present invention three distributed collaboration encryption method is deployed on data warehouse, as shown in Figure 3, this distributed collaboration encryption method comprises the following steps:

Step 301, when confirming the expanded range of private data, described data warehouse is encrypted the first newly-increased data object according to described encryption policy, is deposited in buffer memory by the first data object after encryption;

Step 302, determines target data fairground in described more than one Data Mart according to described mapping relations;

Step 303, send data synchronization request message to described target data fairground, described data synchronization request message comprises encryption version information, the source data after described encryption is write in the disk of described data warehouse;

Step 304, after receiving the data sync response message of described target data fairground transmission, processes the source data after the described encryption be deposited in described buffer memory according to described data sync response message.

Step 304 in step 204 in the embodiment of the present invention two and the embodiment of the present invention three, describedly processes the source data after the described encryption be deposited in described buffer memory according to described data sync response message, comprises step F 1:

Step F 1, when determining described target data fairground without the need to receiving described data object according to described data sync response message, removes the source data after the described encryption be deposited in described buffer memory.

Step 204 in the embodiment of the present invention, describedly processes the source data after the described encryption be deposited in described buffer memory according to described data sync response message, comprises step F 2 and step F 3, wherein:

Step F 2, when determining that described target data fairground needs to receive described target data objects according to described data sync response message, sends to described target data fairground by the described target data objects be deposited in the buffer memory of described data warehouse;

Step F 3, receives the first acknowledge message that described target data fairground sends, and described first acknowledge message is used for showing to receive data success, the source data after the described encryption be deposited in described buffer memory is removed.

Fig. 4 is the realization flow schematic diagram that the embodiment of the present invention works in coordination with encryption method, as shown in Figure 4, wherein, step 401 is the first ciphering process on ETL server to 403, step 404 to step 406 is the second ciphering process on data warehouse, wherein the first ciphering process is as in the embodiment of the present invention two as described in step 201 to step 203, and the second ciphering process is as described in embodiment of the present invention step 301 to 304.Wherein, privacy identification encryption identification module can be arranged according to the scope of private data, and in the concrete process implemented, in data warehouse, above-mentioned encryption function can be passed through adding users SQL (UDF, User DefineFunction) and realizes.

Embodiment four

In the embodiment of the present invention two step 204 and embodiment three in step 304, described according to described data sync response message, the source data after the described encryption be deposited in described buffer memory is processed in data sync response message, send to data warehouse by target data fairground, target data fairground is when receiving data synchronization request message, a series of judgement can be carried out, when judging to need synchronous target data objects, the first confirmation will be carried in data sync response message; When judging not need synchronous target data objects, just the second confirmation can be carried in data sync response message.Fig. 5 is the realization flow schematic diagram that synchronous response message is determined in the embodiment of the present invention four target data fairground, and as shown in Figure 5, target data fairground determines that synchronous response message comprises the following steps:

Step 501 is described more than one Data Mart configuration synchronization strategy;

Here, described synchronization policy comprises synchronizing cycle; Described for described more than one Data Mart configuration synchronization strategy can be user configured, also can be that data warehouse or Data Mart carry out configuration automatically according to the setting of user, those skilled in the art can realize according to various prior art, repeats no more here.

Step 502, receives the described data synchronization request message that described data warehouse sends;

Here, described data synchronization request message comprises the first version information of the cryptographic algorithm of current use;

Step 503, judges that whether described first version information is consistent with the second edition information of stored cryptographic algorithm, time consistent, performs step 504, time inconsistent, performs step 505;

Step 504, sends the data sync response message carrying the first confirmation to described data warehouse;

Here, described data sync response message comprises the first confirmation, and described first confirmation shows to make described data warehouse when receiving described data sync response message, sends target data objects to described target data fairground;

Step 505, continues to judge that whether the time of reception of described data synchronization request message is consistent with described synchronizing cycle, time consistent, performs step 504, time inconsistent, performs step 506;

Step 506, sends the data sync response message carrying the second confirmation to described data warehouse;

Here, described data sync response message comprises the second confirmation, and described second confirmation shows to make described data warehouse when receiving described data sync response message, and not sending to described target data fairground needs synchronous target data objects.

Embodiment five

In the embodiment of the present invention two and three, when target data fairground judges to need synchronous target data objects, target data fairground sends the data sync response message carrying the first confirmation to data warehouse, data warehouse is when receiving the data sync response message carrying the first acknowledge message, target data objects can be sent to target data fairground, start data transmission procedure; In implementation data transmitting procedure, application programming interface (API, Application ProgrammingInterface) can be set up between data warehouse and Data Mart.

Data syn-chronization between data warehouse and target data fairground supports the full dose method of synchronization and increment synchronization mode two kinds of methods of synchronization, and wherein, the full dose method of synchronization is a kind of method of synchronization of cover type, and increment synchronization mode is the method for synchronization of additional formula; When target data fairground receives target data objects, second table corresponding according to the second mapping relations inquiry in mapping relations, then inquire the associated row in the second table according to the 3rd mapping relations in mapping relations, then according to the method for synchronization, the associated row in the second table is carried out synchronously.It should be noted that, after ETL has encrypted, also can create encrypted indexes to the source data after described encryption, described encrypted indexes is with behavior unit; Like this, just the source data after encryption can be synchronized to target data fairground from data warehouse according to the 3rd mapping relations, also the newly-increased data after private data scope can be synchronized to target data fairground from data warehouse.

Target data objects also can be deposited in buffer memory by target data fairground, after the integrality completing data and consistency check, according to mapping relations target data objects is written to the associated row in the second table.Also, when the data volume can working as buffer memory reaches certain threshold value, according to mapping relations target data objects is written to the associated row in the second table.Target data fairground, when carrying out data syn-chronization, also needs record data to change situation to form change daily record, changes daily record and comprise encryption policy; If failure of data synchronization, then carry out date restoring according to change daily record.

Above-mentioned data synchronization process can be passed through adding users SQL (UDF, User DefineFunction) and realize; Fig. 6 is the realization flow schematic diagram of data synchronization process between the embodiment of the present invention five data warehouse and target data fairground, and as shown in Figure 6, this data synchronization process comprises the following steps:

Step 601, data warehouse sends data synchronization request message to described target data fairground;

Described data synchronization request message comprises encryption version information;

Step 602, write disk, writes in the disk of described data warehouse by the source data after described encryption;

Step 603, target data fairground sends data sync response message to data warehouse, that is: the flow process of Data Mart according to Fig. 5 generates data sync response message;

Step 604, data warehouse carries out preliminary treatment, contrast to data, the data that manage conflict or generation Conflict solving file, and sends to target data set city;

Step 605, target data fairground solves inconsistent data, or solves inconsistent data according to Conflict solving file;

Here, step 604 and step 605, those skilled in the art can carry out conventional treatment according to various prior art, repeats no more here.

In embodiments of the present invention, solve by synchronous crypto-operation the problem that in data warehouse and Data Mart, data are inconsistent.Data warehouse, in the situations such as newly-increased data source, digital source content change and data elimination, can trigger above-mentioned synchronous crypto-operation process.

Embodiment six

In step 301 in the step 201 of the embodiment of the present invention two and step 202 and embodiment three, information in source data can be divided into numerical value and nonumeric information, wherein, nonumeric information comprises letter, control character and graphic symbol etc., nonumeric information is processed stored in computer in the mode of binary-coded character code, character code conventional in computer has ASCII character and EBCDIC, and also many for the standard of Chinese character, as GB2312, BIG-5, GBK, GB18030 etc. adopt multibyte code to encode.

In the embodiment of the present invention two, in crypto engine, cryptographic algorithm is at least one of following cryptographic algorithm: character type cryptographic algorithm, visible byte cryptographic algorithm and numerical value cryptographic algorithm, table 1 is the encryption scope of these three kinds of cryptographic algorithm and the description about scope, the hexadecimal describing mode that scope of wherein encrypting adopts.

Cryptographic algorithm title	Encryption scope	Scope describes
			Character type cryptographic algorithm	0X21 to 0XFF	Numeral, letter, spcial character, Chinese character
Visible byte cryptographic algorithm	0X21 to 0X7F	Numeral, letter, spcial character
			Numerical value cryptographic algorithm	0X30 to 0X39	Numeral

Table 1

For ensureing the stability of data encryption and reduction, character beyond encryption scope is that former state exports, do not participate in the process of carrying out enciphering and deciphering algorithm, namely these three kinds of cryptographic algorithm all avoid the spcial character such as space, tab, thus are not encrypted spcial character.The encrypted characters that character type cryptographic algorithm comprises is maximum, visible byte cryptographic algorithm encryption be all character visible on keyboard, the encryption of numerical value cryptographic algorithm be 0 ~ 9 numeral.For encryption " illusion Room, Building A, mansion 501, university of Huhehaote City of province, Inner Mongol South Road ", the effect of above-mentioned three kinds of cryptographic algorithm is as shown in table 2:

Cryptographic algorithm title	Cipher round results
		Character type cryptographic algorithm	D{a}sd@wer^0wfdsa[,./,/i]asj+k-
Visible byte cryptographic algorithm	Illusion h seat Ab1 room, mansion, university of Huhehaote City of province, Inner Mongol South Road
		Numerical value cryptographic algorithm	Illusion Room, Building A, mansion 695, university of Huhehaote City of province, Inner Mongol South Road

Table 2

Embodiment seven

A kind of distributed collaboration encryption method that the embodiment of the present invention provides, be applied to the ETL server of isomeric data plateform system, described isomeric data plateform system comprises ETL server and data warehouse; Fig. 7 is the realization flow schematic diagram of the embodiment of the present invention seven distributed collaboration encryption method, and as shown in Figure 7, the method comprises:

Step 701, the private attribute of the private data in source of configuration data;

Here, described private attribute at least comprises encryption policy;

Step 702, in the process loaded source data, when determining that loaded source data comprises private data according to the scope of described private data, is read as the encryption policy of loaded source data configuration;

Here, described encryption policy is as the execution input parameter of function in dynamic link library;

Step 703, is encrypted described private data according to described encryption policy, obtains the source data after encrypting;

Step 704, sends to described data warehouse by the source data after described encryption.

Embodiment eight

A kind of distributed collaboration encryption method that the embodiment of the present invention eight provides, be applied to the data warehouse of isomeric data plateform system, described isomeric data plateform system comprises ETL server, data warehouse and more than one Data Mart, there are mapping relations respectively between each second data object in the first data object in described data warehouse and described more than one Data Mart; Fig. 8 is the realization flow schematic diagram of the embodiment of the present invention eight distributed collaboration encryption method, and as shown in Figure 8, this distributed collaboration encryption method of the method comprises:

Step 801, receives the source data after the encryption of ETL server transmission, and the source data after described encryption is deposited in the buffer memory of described data warehouse;

Step 802, determines target data objects corresponding in target data fairground and described target data fairground in described more than one Data Mart according to described mapping relations;

Here, described target data fairground is more than one described Data Mart, and described target data objects is the part in the source data after encryption;

By step 803, the source data after described encryption writes in the disk of described data warehouse;

Step 804, after receiving the data sync response message of described target data fairground transmission, processes the source data after the described encryption be deposited in described buffer memory according to described data sync response message.

In the embodiment of the present invention, described method also comprises step G1 to G4: the scope of the private data in source of configuration data;

Step G1, when confirming the expanded range of private data, described data warehouse is encrypted newly-increased data object according to described encryption policy;

Step G2, determines target data fairground in described more than one Data Mart according to described mapping relations;

Step G3, send data synchronization request message to described target data fairground, described data synchronization request message comprises encryption version information;

Step G4, after receiving the data sync response message of described target data fairground transmission, processes the source data after the described encryption be deposited in described buffer memory according to described data sync response message.

Embodiment of the present invention step 804 and step G4, describedly process the source data after the described encryption be deposited in described buffer memory according to described data sync response message, comprising:

When determining described target data fairground without the need to receiving described data object according to described data sync response message, the source data after the described encryption be deposited in described buffer memory is removed.

When determining that described target data fairground needs to receive described target data objects according to described data sync response message, the described target data objects be deposited in the buffer memory of described data warehouse is sent to described target data fairground;

Receive the first acknowledge message that described target data fairground sends, described first acknowledge message is used for showing to receive data success, the source data after the described encryption be deposited in described buffer memory is removed.

Embodiment nine

A kind of distributed collaboration encryption method that the embodiment of the present invention provides, be applied to isomeric data plateform system, described isomeric data plateform system comprises data warehouse and more than one Data Mart, there are mapping relations respectively between each second data object in the first data object in described data warehouse and described more than one Data Mart; Fig. 9 is the realization flow schematic diagram of the embodiment of the present invention nine distributed collaboration encryption method, and as shown in Figure 9, the method also comprises:

Step 901, be described more than one Data Mart configuration synchronization strategy, described synchronization policy comprises synchronizing cycle;

Step 902, described target data fairground receives the described data synchronization request message that described data warehouse sends, and described data synchronization request message comprises the first version information of the identify label of transmit leg and the cryptographic algorithm of current use;

Step 903, confirm according to the communications identities of identify label to described transmit leg of transmit leg in described data synchronization request message, when confirming successfully, when continuing to determine that the time of reception of described data synchronization request message is consistent with described synchronizing cycle, data sync response message is sent to described data warehouse, described data sync response message comprises the first confirmation, described first confirmation shows to make described data warehouse when receiving described data sync response message, sends required synchronous target data objects to described target data fairground.

In the embodiment of the present invention, in described data synchronization request message, also comprise the first version information of the cryptographic algorithm of current use; Described method also comprises:

When determining that described first version information and the second edition information of the cryptographic algorithm stored are inconsistent, data sync response message is sent to described data warehouse, described data sync response message comprises the first confirmation, described first confirmation shows to make described data warehouse when receiving described data sync response message, sends target data objects to described target data fairground.

In the embodiment of the present invention, determine that described first version information is consistent with the second edition information of stored cryptographic algorithm, and when determining that the time of reception of described data synchronization request message and described synchronizing cycle are mutually inconsistent, data sync response message is sent to described data warehouse, described data sync response message comprises the second confirmation, described second confirmation shows to make described data warehouse when receiving described data sync response message, and not described transmission to target data fairground needs synchronous target data objects.

Embodiment ten

A kind of distributed collaboration encryption device that the embodiment of the present invention provides, be applied to isomeric data plateform system, described isomeric data plateform system comprises ETL server, data warehouse and more than one Data Mart, there are mapping relations respectively between each second data object in the first data object in described data warehouse and described more than one Data Mart; Figure 10 is the composition structural representation of the embodiment of the present invention ten distributed collaboration encryption device, as shown in Figure 10, described device comprises dispensing unit 1001, ciphering unit 1002, memory cell 1003, determining unit 1004, writing unit 1005 and the first processing unit 1006, wherein:

Described dispensing unit 1001, for the private attribute of the private data in source of configuration data, described private attribute at least comprises encryption version information and encryption policy;

Described ciphering unit 1002, for in the process loaded source data, when determining that loaded source data comprises private data according to the scope of described private data, read the encryption policy configured, according to described encryption policy, described private data is encrypted, obtains the source data after encrypting; Source data after described encryption is sent to described data warehouse;

Described memory cell 1003, for being deposited in the buffer memory of described data warehouse by the source data after described encryption;

Described determining unit 1004, for determining target data objects corresponding in target data fairground and described target data fairground in described more than one Data Mart according to described mapping relations;

Said write unit 1005, for sending data synchronization request message to described target data fairground, described data synchronization request message comprises encryption version information, the source data after described encryption is write in the disk of described data warehouse;

Described first processing unit 1006, for receive described target data fairground send data sync response message after, according to described data sync response message, the source data after the described encryption be deposited in described buffer memory is processed.

In the embodiment of the present invention, described dispensing unit 1001 and ciphering unit 1002 are arranged on the processor of ETL server, and memory cell 1003, determining unit 1004, writing unit 1005 and the first processing unit 1006 are arranged on the processor of data warehouse;

In the embodiment of the present invention, described first processing unit 1006 comprises determination module and removes module, wherein:

Described determination module, for determining that according to described data sync response message described target data fairground is the need of the described data object of reception, time no, trigger and removes module; Accordingly, described removing module, removes for the source data after the described encryption that will be deposited in described buffer memory.

Described determination module, for determining that according to described data sync response message described target data fairground is the need of the described target data objects of reception, when being, the described target data objects be deposited in the buffer memory of described data warehouse is sent to described target data fairground; Accordingly, described removing module, for receiving the first acknowledge message that described target data fairground sends, described first acknowledge message is used for showing to receive data success, the source data after the described encryption be deposited in described buffer memory is removed.

In the embodiment of the present invention, described device also comprises receiving element, the second determining unit and the 3rd determining unit, wherein:

Described receiving element, receives described data synchronization request message for described target data fairground, and described data synchronization request message comprises the first version information of the identify label of transmit leg and the cryptographic algorithm of current use;

Described second determining unit, for confirming according to the communications identities of identify label to described transmit leg of transmit leg in described data synchronization request message, when confirming successfully, triggers described 3rd determining unit;

Described 3rd determining unit, during for continuing to determine that the time of reception of described data synchronization request message is consistent with described synchronizing cycle, data sync response message is sent to described data warehouse, described data sync response message comprises the first confirmation, described first confirmation shows to make described data warehouse when receiving described data sync response message, sends required synchronous target data objects to described target data fairground.

Embodiment 11

A kind of distributed collaboration encryption device that the embodiment of the present invention provides, be applied to the ETL process server of isomeric data plateform system, described isomeric data plateform system comprises ETL server and data warehouse; Figure 11 is the composition structural representation of the embodiment of the present invention 11 distributed collaboration encryption device, as shown in figure 11, this device comprises dispensing unit 1101 and ciphering unit, and wherein said ciphering unit comprises reading unit 1102, first ciphering unit 1103 and the first transmitting element 1104, wherein:

Described dispensing unit 1101, for the private attribute of the private data in source of configuration data, described private attribute at least comprises encryption policy;

Described reading unit 1102, for in the process loaded source data, when determining that loaded source data comprises private data according to the scope of described private data, read the encryption policy configured, described encryption policy is as the execution input parameter of function in dynamic link library;

Described first ciphering unit 1103, for being encrypted described private data according to described encryption policy, obtains the source data after encrypting;

Described first transmitting element 1104, for sending to described data warehouse by the source data after described encryption.

Embodiment 12

A kind of distributed collaboration encryption device that the embodiment of the present invention provides, be applied to isomeric data plateform system, described isomeric data plateform system comprises data warehouse and more than one Data Mart, there are mapping relations respectively between each second data object in the first data object in described data warehouse and described more than one Data Mart; Figure 12 is the composition structural representation of the embodiment of the present invention 12 distributed collaboration encryption device, and as shown in figure 12, this device comprises the second dispensing unit 1201, receiving element 1202, second determining unit 1203 and the 3rd determining unit 1204, wherein:

Described second dispensing unit 1201, for being described more than one Data Mart configuration synchronization strategy, described synchronization policy comprises synchronizing cycle;

Described receiving element 1202, for receiving the described data synchronization request message that described data warehouse sends, described data synchronization request message comprises the first version information of the identify label of transmit leg and the cryptographic algorithm of current use;

Described second determining unit 1203, for confirming according to the communications identities of identify label to described transmit leg of transmit leg in described data synchronization request message, when confirming successfully, triggers described 3rd determining unit;

Described 3rd determining unit 1204, during for continuing to determine that the time of reception of described data synchronization request message is consistent with described synchronizing cycle, data sync response message is sent to described data warehouse, described data sync response message comprises the first confirmation, described first confirmation shows to make described data warehouse when receiving described data sync response message, sends required synchronous target data objects to described target data fairground.

In the embodiment of the present invention, described data synchronization request message comprises the first version information of the cryptographic algorithm of current use, described 3rd determining unit 1204, time also for determining that described first version information and the second edition information of the cryptographic algorithm prestored are inconsistent, data sync response message is sent to described data warehouse, described data sync response message comprises the first confirmation, described first confirmation shows to make described data warehouse when receiving described data sync response message, sends target data objects to described target data fairground.

Described 3rd determining unit 1204, also for determining that described first version information is consistent with the second edition information of the cryptographic algorithm prestored, and when determining that the time of reception of described data synchronization request message and described synchronizing cycle are mutually inconsistent, data sync response message is sent to described data warehouse, described data sync response message comprises the second confirmation, described second confirmation shows that described data warehouse is when receiving described data sync response message, and not sending to described target data fairground needs synchronous target data objects.

In the embodiment of the present invention, this device also comprises the second processing unit, and after receiving described target data objects, described target data fairground carries out integrality and consistency confirmation according to described data synchronization request message to received data object; Described data synchronization request message comprises original position and the size of target data objects; When confirming successfully, send to described data warehouse and be used for showing to receive successful first acknowledge message of data object; When confirming unsuccessfully, send for representing the second acknowledge message receiving data object failure to described data warehouse.

Embodiment 13

A kind of distributed collaboration encryption device that the embodiment of the present invention provides, is applied to the data warehouse in isomeric data plateform system; Figure 13 is the composition structural representation of the embodiment of the present invention 13 distributed collaboration encryption device, and as shown in figure 13, this device comprises memory cell 1301, first determining unit 1302, writing unit 1303 and the first processing unit 1304, wherein:

Described memory cell 1301, for receiving the source data after encryption that ETL server sends, and is deposited in the buffer memory of described data warehouse by the source data after described encryption;

Described first determining unit 1302, for determining target data objects corresponding in target data fairground and described target data fairground in described more than one Data Mart according to described mapping relations;

Said write unit 1303, for writing in the disk of described data warehouse by the source data after described encryption;

Described first processing unit 1304, for receive described target data fairground send data sync response message after, according to described data sync response message, the source data after the described encryption be deposited in described buffer memory is processed.

In the embodiment of the present invention, described device also comprises the 4th determining unit, the 5th determining unit and the second transmitting element, wherein:

Described 4th determining unit, during for confirming the expanded range of private data, is encrypted newly-increased data object according to described encryption policy;

Described 5th determining unit, for determining target data fairground according to described mapping relations in described more than one Data Mart, triggers described second transmitting element;

Described second transmitting element, for sending data synchronization request message to described target data fairground, described data synchronization request message comprises encryption version information, triggers described processing unit; Described processing unit, after receiving the data sync response message of described target data fairground transmission, processes the source data after the described encryption be deposited in described buffer memory according to described data sync response message for described data warehouse.

In the embodiment of the present invention, described first processing unit 1304 comprises determination module and removes module, wherein:

The embodiment of the present invention also provides a kind of distributed collaboration management system, comprises the distributed collaboration encryption device described in above-described embodiment ten one to ten three.

The embodiment of the invention described above solves the coordinated management problem of private data between other databases in data warehouse and system, by the synchronous crypto-operation technology of data warehouse and Data Mart, private data safety and consistency problem from overall processes such as importing, transmit, load, use and roll off the production line are ensured, really achieve and once encrypt, synchronous transmission, the distributed collaboration management objectives dynamically updated.

It will be understood by those of skill in the art that the practical function of the distributed collaboration encryption device each processing unit described in embodiment ten to ten three and the module in each unit can refer to the associated description of aforementioned described distributed collaboration encryption method and understands.If the above-mentioned integrated unit of the present invention using the form of software function module realize and as independently production marketing or use time, also can be stored in a computer read/write memory medium.Based on such understanding, the technical scheme of the embodiment of the present invention can embody with the form of software product the part that prior art contributes in essence in other words, this computer software product is stored in a storage medium, comprises some instructions and performs all or part of of method described in each embodiment of the present invention in order to make a computer equipment (can be personal computer, server or the network equipment etc.).And aforesaid storage medium comprises: movable storage device, read-only memory (ROM, Read-Only Memory), magnetic disc or CD etc. various can be program code stored medium.The above, be only preferred embodiment of the present invention, be not intended to limit protection scope of the present invention.

Claims

1. a distributed collaboration encryption method, it is characterized in that, be applied to isomeric data plateform system, described isomeric data plateform system comprises that data are extracted, conversion and load ETL server, data warehouse and more than one Data Mart, there are mapping relations respectively between each second data object in the first data object in described data warehouse and described more than one Data Mart; The scope of the private data in source of configuration data and private attribute, described private attribute at least comprises first version information and the encryption policy of encryption, and described method comprises:

2. method according to claim 1, is characterized in that, described method also comprises:

3. method according to claim 1, is characterized in that, described method also comprises: the scope of the private data in source of configuration data;

When described data warehouse confirms the expanded range of private data, according to described encryption policy, newly-increased data object is encrypted;

Described data warehouse determines target data fairground according to described mapping relations in described more than one Data Mart;

Described data warehouse sends data synchronization request message to described target data fairground, and described data synchronization request message comprises encryption version information;

4. according to the method in claim 2 or 3, it is characterized in that, described data warehouse processes the source data after the described encryption be deposited in described buffer memory according to described data sync response message, comprising:

When described data warehouse determines described target data fairground without the need to receiving described data object according to described data sync response message, the source data after the described encryption be deposited in described buffer memory is removed.

5. according to the method in claim 2 or 3, it is characterized in that, described data warehouse processes the source data after the described encryption be deposited in described buffer memory according to described data sync response message, comprising:

When described data warehouse determines that described target data fairground need receive described target data objects according to described data sync response message, the described target data objects be deposited in the buffer memory of described data warehouse is sent to described target data fairground;

Described data warehouse receives the first acknowledge message that described target data fairground sends, and described first acknowledge message is used for showing to receive data success, the source data after the described encryption be deposited in described buffer memory is removed.

6. method according to claim 4, is characterized in that, described method also comprises:

Described target data fairground receives described data synchronization request message, and described data synchronization request message comprises the first version information of the cryptographic algorithm of current use;

When described target data fairground determines that described first version information and the second edition information of the cryptographic algorithm prestored are inconsistent, data sync response message is sent to described data warehouse, described data sync response message comprises the first confirmation, described first confirmation shows to make described data warehouse when receiving described data sync response message, sends target data objects to described target data fairground.

7. method according to claim 5, is characterized in that, described method also comprises:

Described target data fairground receives described data synchronization request message, when determining that the time of reception of described data synchronization request message is consistent with the synchronizing cycle of configuration, described data sync response message is sent to described data warehouse, described data sync response message comprises the first confirmation, described first confirmation shows to make described data warehouse when receiving described data sync response message, sends required synchronous target data objects to target data fairground.

8. method according to claim 5, is characterized in that, described method also comprises: described target data fairground receives described data synchronization request message, and described data synchronization request message comprises the first version information of the cryptographic algorithm of current use;

Described target data fairground determines that described first version information is consistent with the second edition information of the cryptographic algorithm prestored, and when determining that the time of reception of described data synchronization request message and the synchronizing cycle of configuration are mutually inconsistent, data sync response message is sent to described data warehouse, described data sync response message comprises the second confirmation, described second confirmation shows that described data warehouse is when receiving described data sync response message, and not sending to described target data fairground needs synchronous target data objects.

9. the method according to claim 1 or 2 or 3 or 6 or 7 or 8, it is characterized in that, described method also comprises:

Described target data fairground receives described data synchronization request message, and described data synchronization request message comprises the identify label of transmit leg;

Described target data fairground confirms according to the communications identities of identify label to described transmit leg of transmit leg in described data synchronization request message, when confirming successfully, sends described data sync response message to described data warehouse.

10. method according to claim 9, is characterized in that, described method also comprises:

After described target data objects receives described target data objects, described target data fairground carries out integrality and consistency confirmation according to described data synchronization request message to received data object; Described data synchronization request message comprises original position and the size of target data objects;

When described target data objects confirms successfully, send to described data warehouse and be used for showing to receive successful first acknowledge message of data object;

When described target data objects confirms unsuccessfully, send for representing the second acknowledge message receiving data object failure to described data warehouse.

11. 1 kinds of distributed collaboration encryption methods, is characterized in that, be applied to the data extraction of isomeric data plateform system, conversion and load ETL server, described isomeric data plateform system comprises ETL server and data warehouse;

In the process that source data is loaded, when described ETL server determines that loaded source data comprises private data according to the scope of described private data, be read as the encryption policy of loaded source data configuration;

12. methods according to claim 11, is characterized in that, described method also comprises: described encryption policy is as the execution input parameter of function in dynamic link library.

13. 1 kinds of distributed collaboration encryption methods, it is characterized in that, be applied to isomeric data plateform system, described isomeric data plateform system comprises that data are extracted, conversion and load ETL server, data warehouse and more than one Data Mart, there are mapping relations respectively between each second data object in the first data object in described data warehouse and described more than one Data Mart;

Described method comprises:

14. methods according to claim 13, is characterized in that, described method also comprises: the scope of the private data in source of configuration data;

When described data warehouse confirms the expanded range of private data, described data warehouse is encrypted newly-increased data object according to described encryption policy;

Described data warehouse determines target data fairground in described more than one Data Mart according to described mapping relations;

15. methods according to claim 13 or 14, it is characterized in that, described data warehouse processes the source data after the described encryption be deposited in described buffer memory according to described data sync response message, comprising:

When described data warehouse determines that described target data fairground needs to receive described target data objects according to described data sync response message, the described target data objects be deposited in the buffer memory of described data warehouse is sent to described target data fairground;

16. 1 kinds of distributed collaboration encryption devices, it is characterized in that, be applied to isomeric data plateform system, described isomeric data plateform system comprises that data are extracted, conversion and load ETL server, data warehouse and more than one Data Mart, there are mapping relations respectively between each second data object in the first data object in described data warehouse and described more than one Data Mart; Described device comprises dispensing unit, ciphering unit, memory cell, the first determining unit, writing unit and the first processing unit, wherein:

17. 1 kinds of distributed collaboration encryption devices, is characterized in that, be applied to the data extraction of isomeric data plateform system, conversion and load ETL server, described isomeric data plateform system comprises ETL server and data warehouse;

18. 1 kinds of distributed collaboration encryption devices, is characterized in that, are applied to isomeric data plateform system, and described device comprises memory cell, the first determining unit, writing unit and processing unit, wherein: