CN117910003A - Data processing method, apparatus, device, medium, and program product - Google Patents

Data processing method, apparatus, device, medium, and program product Download PDF

Info

Publication number
CN117910003A
CN117910003A CN202311816926.3A CN202311816926A CN117910003A CN 117910003 A CN117910003 A CN 117910003A CN 202311816926 A CN202311816926 A CN 202311816926A CN 117910003 A CN117910003 A CN 117910003A
Authority
CN
China
Prior art keywords
data
deformed
file
deformation
key
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202311816926.3A
Other languages
Chinese (zh)
Inventor
罗海波
林晖
蔡婷玉
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Industrial and Commercial Bank of China Ltd ICBC
Original Assignee
Industrial and Commercial Bank of China Ltd ICBC
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Industrial and Commercial Bank of China Ltd ICBC filed Critical Industrial and Commercial Bank of China Ltd ICBC
Priority to CN202311816926.3A priority Critical patent/CN117910003A/en
Publication of CN117910003A publication Critical patent/CN117910003A/en
Pending legal-status Critical Current

Links

Landscapes

  • Storage Device Security (AREA)

Abstract

The disclosure provides a data processing method, which can be applied to the fields of information security and financial technology. The method comprises the following steps: backing up batch data to obtain backup data in response to the end of the daily batch; in response to generating the public key and the private key, encrypting the deformed file by adopting the public key to obtain a ciphertext file; transmitting the ciphertext file among different departments; decrypting the ciphertext file by adopting a private key to obtain a deformed file; deforming the backup data by adopting the deformed file to obtain deformed data; and transmitting the deformation data to a test environment for simulation production test. The present disclosure also provides a data processing apparatus, device, storage medium, and program product.

Description

Data processing method, apparatus, device, medium, and program product
Technical Field
The present disclosure relates to the field of information security and finance, and in particular, to a data processing method, apparatus, device, medium, and program product.
Background
Sensitive data leakage such as customer information in the financial field is becoming an information security problem that damages public interests, threatens banking health development and has a great influence on bank reputation. Authoritative data indicates that 70% of information security events such as data leakage come from within the enterprise.
The test environment needs to simulate the production environment for various tests, and because a large amount of sensitive data such as bank customer information and the like are stored in the database, the production sensitive data needs to be extracted and deformed, so that the test environment can not inquire the production real data. Therefore, a set of host data deformation processing method and optimization device of the system are needed to protect the security of the client data.
Disclosure of Invention
In view of the foregoing, the present disclosure provides a data processing method, apparatus, device, medium and program product for improving the security of production data, for at least partially solving the above technical problems.
According to a first aspect of the present disclosure, there is provided a data processing method comprising: backing up batch data to obtain backup data in response to the end of the daily batch; in response to generating the public key and the private key, encrypting the deformed file by adopting the public key to obtain a ciphertext file; transmitting the ciphertext file among different departments; decrypting the ciphertext file by adopting a private key to obtain a deformed file; deforming the backup data by adopting the deformed file to obtain deformed data; and transmitting the deformation data to a test environment for simulation production test.
According to an embodiment of the present disclosure, the deformed file includes a kanji comparison table and a deformed key, and in response to generating the public key and the private key, encrypting the deformed file with the public key to obtain the ciphertext file includes: encrypting the Chinese character comparison table by adopting a public key to obtain an encrypted Chinese character comparison table; encrypting the deformed key by adopting the public key to obtain an encrypted deformed key; the deformed key is used for storing the deformed strategy of the non-Chinese characters.
According to an embodiment of the present disclosure, before encrypting the deformed file, the method further includes: calculating a first hash value of the deformed file by using a hash function; transmitting the ciphertext file and the hash value among different departments; and after decrypting the ciphertext file, the method further comprises: re-calculating a second hash value of the deformed file; comparing the first hash value with the second hash value; and under the condition that the first hash value is the same as the second hash value, determining that the deformed file is successfully transmitted.
According to an embodiment of the present disclosure, deforming backup data using a deformed file includes: constructing a host deformation tool according to the Chinese character comparison table and the deformation key; acquiring a deformation configuration table, wherein the deformation configuration table comprises backup data to be deformed; and deforming the backup data to be deformed by adopting a host deformation tool to obtain deformed data.
According to an embodiment of the present disclosure, deforming backup data to be deformed using a host deforming tool, the obtaining deformed data includes: respectively determining storage spaces of a plurality of deformed volumes; determining the CPU occupancy rate; respectively judging threshold values of the storage space and the CPU occupancy rate of the deformed volume; deforming the backup data to be deformed under the condition that the storage space of the deformed volume is larger than a first preset threshold and the CPU occupancy rate is smaller than a second preset threshold to obtain deformed data; wherein the morphing data is stored concurrently in a plurality of morphing volumes.
According to an embodiment of the present disclosure, the backup data to be deformed includes chinese field data, communication address data, mailbox address data, and password data, deforming the backup data to be deformed, and obtaining deformed data includes: according to the Chinese character comparison table, chinese character replacement is carried out on the Chinese character segment data, and a replaced Chinese character field is obtained; and/or at least partially character shielding the communication address data according to the deformed key to obtain a shielded communication address; and/or according to the deformed key, unifying the format of the mailbox address data to obtain a unified mailbox address; and/or according to the deformed key, unifying the formats of the password data to obtain unified password data.
According to an embodiment of the present disclosure, the data processing method further includes: and performing spot check on one or more of the replaced Chinese field, the shielding communication address, the unified mailbox address and the unified password data in a test environment.
According to an embodiment of the present disclosure, after obtaining the deformation data, the data processing method further includes: restoring the deformed data according to the Chinese character comparison table and the deformed key to obtain restored data; comparing the restored data with the backup data to be deformed, and determining the deformation error rate; threshold judgment is carried out on the deformation error rate; and under the condition that the deformation error rate is larger than a third preset threshold value, the backup data to be deformed is deformed again.
According to an embodiment of the present disclosure, transmitting deformation data to a test environment for simulated production testing includes: determining the network bandwidth occupancy rate; threshold value judgment is carried out on the network bandwidth occupancy rate; transmitting the deformed data by adopting a disk PPRC replication technology under the condition that the network bandwidth occupancy rate is smaller than a fourth preset threshold value; wherein the network bandwidth occupancy comprises a network bandwidth occupancy of the deformed volume.
A second aspect of the present disclosure provides a data processing apparatus comprising: the backup module is used for backing up batch data to obtain backup data in response to the end of the daily batch; the encryption module is used for encrypting the deformed file by adopting the public key to obtain a ciphertext file in response to the generation of the public key and the private key; the transmission module is used for transmitting the ciphertext files among different departments; the decryption module is used for decrypting the ciphertext file by adopting the private key to obtain a deformed file; the deformation module is used for deforming the backup data by adopting the deformation file to obtain deformed data; and the test module is used for transmitting the deformation data to the test environment to perform simulated production test.
A third aspect of the present disclosure provides an electronic device, comprising: one or more processors; and a memory for storing one or more programs, wherein the one or more programs, when executed by the one or more processors, cause the one or more processors to perform the method of any of the embodiments described above.
A fourth aspect of the present disclosure also provides a computer readable storage medium having stored thereon executable instructions which, when executed by a processor, cause the processor to perform the method of any of the embodiments described above.
A fifth aspect of the present disclosure also provides a computer program product comprising a computer program which, when executed by a processor, implements the method of any of the embodiments described above.
Compared with the prior art, the data processing method, the device, the electronic equipment, the storage medium and the program product provided by the disclosure have at least the following beneficial effects:
(1) According to the data processing method disclosed by the invention, after the daily final batch is finished, batch data are automatically backed up, and after the batch data are deformed and desensitized, the batch data are transferred into a testing environment to carry out a testing task, so that the leakage of production data is avoided, and the safety of sensitive data is improved. The configuration file (i.e. the deformed file) for deforming the batch data is encrypted and decrypted in the transmission process, so that the deformed logic of the data is prevented from being cracked, and the safety of data processing is further improved.
(2) The data processing method disclosed by the invention specifically adopts the Chinese character comparison table and the deformed key as the deformed file, and can realize the desensitization processing of sensitive information such as names, numbers (passwords), addresses and the like in batch data. And the encryption and decryption of the deformed file are carried out by adopting the public key and the private key, so that the security department is convenient to encrypt and the reference authority of the security department to related sensitive information is limited.
(3) According to the data processing method disclosed by the invention, on the basis of encrypting and decrypting the deformed file, hash value matching can be performed on the deformed file before and after transmission, so that the integrity (unchanged) of the deformed file is ensured, and the safety and reliability of data processing are further improved.
Drawings
The foregoing and other objects, features and advantages of the disclosure will be more apparent from the following description of embodiments of the disclosure with reference to the accompanying drawings, in which:
FIG. 1 schematically illustrates an application scenario diagram of a data processing method, apparatus, device, medium and program product according to an embodiment of the present disclosure;
FIG. 2 schematically illustrates a flow chart of a data processing method according to an embodiment of the disclosure;
FIG. 3 schematically illustrates a flow chart of a method of encrypting a variant file according to an embodiment of the disclosure;
FIG. 4 schematically illustrates a flow chart of a method of validating a deformed document according to an embodiment of the present disclosure;
FIG. 5A schematically illustrates a flow chart of a method of morphing backup data according to an embodiment of the disclosure; FIG. 5B schematically illustrates a flow diagram of a method of building a host morphing tool according to an embodiment of the disclosure;
FIG. 6 schematically illustrates a flow chart of a method of morphing backup data according to another embodiment of the disclosure;
FIG. 7 schematically illustrates a flow chart of a method of morphing backup data according to yet another embodiment of the disclosure;
FIG. 8 schematically illustrates a flow chart of a data processing method according to another embodiment of the present disclosure;
FIG. 9 schematically illustrates a flow chart of a method of error monitoring deformation data according to an embodiment of the disclosure;
FIG. 10 schematically illustrates a flow chart of a method of transmitting deformed data in accordance with an embodiment of the present disclosure;
FIG. 11 schematically illustrates a block diagram of a data processing apparatus according to an embodiment of the present disclosure; and
Fig. 12 schematically illustrates a block diagram of an electronic device adapted to implement a data processing method according to an embodiment of the disclosure.
Detailed Description
Hereinafter, embodiments of the present disclosure will be described with reference to the accompanying drawings. It should be understood that the description is only exemplary and is not intended to limit the scope of the present disclosure. In the following detailed description, for purposes of explanation, numerous specific details are set forth in order to provide a thorough understanding of the embodiments of the present disclosure. It may be evident, however, that one or more embodiments may be practiced without these specific details. In addition, in the following description, descriptions of well-known structures and techniques are omitted so as not to unnecessarily obscure the concepts of the present disclosure.
The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the disclosure. The terms "comprises," "comprising," and/or the like, as used herein, specify the presence of stated features, steps, operations, and/or components, but do not preclude the presence or addition of one or more other features, steps, operations, or components.
All terms (including technical and scientific terms) used herein have the same meaning as commonly understood by one of ordinary skill in the art unless otherwise defined. It should be noted that the terms used herein should be construed to have meanings consistent with the context of the present specification and should not be construed in an idealized or overly formal manner.
Where a convention analogous to "at least one of A, B and C, etc." is used, in general such a convention should be interpreted in accordance with the meaning of one of skill in the art having generally understood the convention (e.g., "a system having at least one of A, B and C" would include, but not be limited to, systems having a alone, B alone, C alone, a and B together, a and C together, B and C together, and/or A, B, C together, etc.).
Embodiments of the present disclosure provide a data processing method, apparatus, device, medium, and program product, which may be used in the financial field or other fields. It should be noted that the data processing method, apparatus, device, medium and program product of the present disclosure may be used in the financial field, and may also be used in any field other than the financial field, and the application fields of the data processing method, apparatus, device, medium and program product of the present disclosure are not limited.
In the technical scheme of the invention, the related user information (including but not limited to user personal information, user image information, user equipment information, such as position information and the like) and data (including but not limited to data for analysis, stored data, displayed data and the like) are information and data authorized by a user or fully authorized by all parties, and the processing of the related data such as collection, storage, use, processing, transmission, provision, disclosure, application and the like are all conducted according to the related laws and regulations and standards of related countries and regions, necessary security measures are adopted, no prejudice to the public welfare is provided, and corresponding operation inlets are provided for the user to select authorization or rejection.
The embodiment of the disclosure provides a data processing method, which comprises the following steps: backing up batch data to obtain backup data in response to the end of the daily batch; in response to generating the public key and the private key, encrypting the deformed file by adopting the public key to obtain a ciphertext file; transmitting the ciphertext file among different departments; decrypting the ciphertext file by adopting a private key to obtain a deformed file; deforming the backup data by adopting the deformed file to obtain deformed data; and transmitting the deformation data to a test environment for simulation production test. The batch data is automatically backed up, deformed and desensitized, and then is transferred into a testing environment for testing tasks, so that the leakage of the production data is avoided, and the safety of the sensitive data is improved.
Fig. l schematically illustrates application scenario diagrams of data processing methods, apparatuses, devices, media and program products according to embodiments of the present disclosure.
As shown in fig. 1, an application scenario 100 according to this embodiment may include terminal devices 101, 102, 103, a network 104, and a server 105. The network 104 is used as a medium to provide communication links between the terminal devices 101, 102, 103 and the server 105. The network 104 may include various connection types, such as wired, wireless communication links, or fiber optic cables, among others.
The user may interact with the server 105 via the network 104 using the terminal devices 101, 102, 103 to receive or send messages or the like. Various communication client applications, such as shopping class applications, web browser applications, search class applications, instant messaging tools, mailbox clients, social platform software, etc. (by way of example only) may be installed on the terminal devices 101, 102, 103.
The terminal devices 101, 102, 103 may be a variety of electronic devices having a display screen and supporting web browsing, including but not limited to smartphones, tablets, laptop and desktop computers, and the like.
The server 105 may be a server providing various services, such as a background management server (by way of example only) providing support for websites browsed by users using the terminal devices 101, 102, 103. The background management server may analyze and process the received data such as the user request, and feed back the processing result (e.g., the web page, information, or data obtained or generated according to the user request) to the terminal device. In particular, the server 105 may be a database server, and is provided with a host morphing tool for obtaining ciphertext files from the terminal devices 101, 102, 103, morphing mass production data, and outputting the morphed data to the test environment.
It should be noted that the data processing method provided in the embodiments of the present disclosure may be generally performed by the server 105. Accordingly, the data processing apparatus provided by the embodiments of the present disclosure may be generally provided in the server 105. The data processing method provided by the embodiments of the present disclosure may also be performed by a server or a server cluster that is different from the server 105 and is capable of communicating with the terminal devices 101, 102, 103 and/or the server 105. Accordingly, the data processing apparatus provided by the embodiments of the present disclosure may also be provided in a server or a server cluster different from the server 105 and capable of communicating with the terminal devices 101, 102, 103 and/or the server 105.
It should be understood that the number of terminal devices, networks and servers in fig. 1 is merely illustrative. There may be any number of terminal devices, networks, and servers, as desired for implementation.
First, the related terms of the embodiments of the present disclosure are explained as follows:
Batch size: in databases, a batch generally refers to operations that execute multiple SQL statements at once. The operation can effectively reduce the interaction times with the database and improve the data processing efficiency. Common batch operations include insert, update, delete, etc.
Application table: is a table in the database for storing user data, which is used in practical applications to record and organize information. Application tables are typically created and managed by an application program or software system for storing and managing data associated therewith. An application table typically contains a series of columns and rows, each row representing a record, each record containing some relevant data. The data is organized in columns, each column having its own data type and constraints.
Public and private keys: is a key pair (i.e., a public key and a private key) obtained by an algorithm. The public key is the public part of the key pair and the private key is the non-public part. The public key is typically used to encrypt a session key, verify a digital signature, or encrypt data that may be decrypted with a corresponding private key. The key pairs obtained using this algorithm can be guaranteed to be unique worldwide. When using this key pair, if a piece of data is encrypted with one of the keys, it must be decrypted with the other key. For example, encrypting data with a public key must be decrypted with a private key, and if encrypted with a private key must also be decrypted with a public key, otherwise decryption will not succeed.
Deforming the coil: and storing the deformation result of the magnetic disk.
Deformation tool: desensitization processing tools for abstracting or generalizing bulk data.
Deformed file: a file of the configuration of the morphing tool.
Disk PPRC replication technique: is a Remote Copy technology of data based on storage, which is based on a high-end storage platform in DS6000 and DS8000 of IBM and is a collective name of Peer-to-Peer Remote Copy (PPRC). It contains various components such as Metro Mirror and Global Copy, etc. The Metro Mirror is a synchronous data copying mode, can ensure data consistency under any condition, and is suitable for copying data between two stations which are close in distance and have enough link bandwidth. Global Copy is an asynchronous data replication mode without consistency group function, which cannot guarantee data consistency. The PPRC can realize a disaster recovery architecture with two places and three centers. For example, in the same metropolitan distance range, a Metro Mirror can be used for synchronous data replication between the site A and the site B; and the asynchronous data replication with consistency group can be performed between the site B and the site C located in a remote place range by using the Global Mirror. In addition, in order to improve the efficiency of the PPRC data backup scheme, it may be considered to implement PPRC data backup in an asynchronous manner in combination with FlashCopy function software of IBM corporation. In an asynchronous mode of operation, the PPRC can return a write success signal to the host whenever a local update is successful, with the remote update not being complete.
The data processing method of the disclosed embodiment will be described in detail below with reference to fig. 2 to 10 based on the scenario described in fig. 1.
Fig. 2 schematically illustrates a flow chart of a data processing method according to an embodiment of the present disclosure.
As shown in fig. 2, the data processing method of this embodiment includes, for example, operations S210 to S260, and the method may be executed by a computer program on corresponding computer hardware.
In operation S210, in response to the end of the daily lot, the lot data is backed up to obtain backup data.
In operation S220, in response to generating the public key and the private key, the deformed file is encrypted with the public key to obtain a ciphertext file.
In operation S230, the ciphertext file is transmitted between different departments.
In operation S240, the ciphertext file is decrypted using the private key, thereby obtaining a deformed file.
In operation S250, the backup data is deformed by using the deformed file to obtain deformed data.
In operation S260, the deformation data is transmitted to the test environment for the simulated production test.
For example, a data processing method for backing up, encrypting, decrypting and simulating production testing of batch data at the end of a daily end batch. And when the business is finished every day, the system automatically triggers a script to backup the batch data to obtain backup data. Such backup data is stored, for example, on an encrypted storage device that is transferred between the secure portion and the system portion. The system automatically generates a pair of public and private keys when encryption of data (deformed files) is required. The public key is used to encrypt data and the private key is used to decrypt data. When the deformed file is required to be transmitted from the security part to the system part, the system encrypts the deformed file by using the public key to obtain the ciphertext file. Thus, the content of the deformed file can be protected during the transmission process, and unauthorized personnel can be prevented from accessing the deformed file. The encrypted ciphertext file is transmitted to the system part through a secure channel. This process can be performed between different departments, ensuring that only authorized personnel can receive and access the ciphertext file. After the system part receives the ciphertext file, the ciphertext file is decrypted by using the private key, and the deformed file is obtained. The private key can be obtained and used only by authorized personnel, so that only legal users can decrypt the ciphertext file. And deforming the backup data by adopting the deformed file to obtain deformed data. The deformation process can be set according to actual requirements, for example, the data can be processed according to a certain rule or algorithm, so that the data becomes unrecognizable or difficult to analyze. And transmitting the deformed data to a test environment for simulating production test. These tests may include performance tests, load tests, fault recovery tests, etc. to verify the stability and reliability of the system. The tester can use the deformed data to perform the test without directly accessing the production data, thereby protecting confidentiality and integrity of the production data.
Through the steps, the whole process automation processing of backup, encryption, decryption and simulated production test of batch data at the end of daily final batch can be realized. The method can effectively protect confidentiality and integrity of data and improve safety and stability of a system. Meanwhile, through simulation production test, the performance and reliability of the system can be better known, and powerful support is provided for decision making of enterprises.
In some embodiments, triggering a routine backup of the application table to the host database after the end of the daily host routine day batch corresponds to retaining the application table daily data. Taking database table name TABLEA as an example, a backup image file may be generated by a host database self-contained backup tool, each named pb0aimgl.
Acquiring the end-of-day batch data may involve acquiring user information in operation S210.
In embodiments of the present disclosure, the user's consent or authorization may be obtained prior to obtaining the user's information. For example, before operation S210, a request to acquire user information may be issued to the user. In case the user agrees or authorizes that the user information can be acquired, the operation S210 is performed.
Fig. 3 schematically illustrates a flow chart of a method of encrypting a variant file according to an embodiment of the disclosure.
According to an embodiment of the present disclosure, the deformed file includes a kanji comparison table and a deformed key, and in response to generating the public key and the private key, the deformed file is encrypted, for example, through operations S321 to S322, as shown in fig. 3, to obtain a ciphertext file.
In operation S321, the chinese character comparison table is encrypted using the public key to obtain an encrypted chinese character comparison table.
In operation S322, the deformed key is encrypted with the public key to obtain an encrypted deformed key. The deformed key is used for storing the deformed strategy of the non-Chinese characters.
For example, a "morph file" is a special file that contains a specific morph policy and a look-up table for converting data in one form to another. For example, it may be used to obfuscate text to increase the security of the data. The system generates a pair of public keys for encrypting data and private keys for decrypting data. These keys may be used to encrypt and decrypt the morphed file. And encrypting the deformed file by adopting the public key. In this embodiment, the morph file includes a chinese character map and a morph key. The Chinese character comparison table comprises the corresponding relation between the original Chinese characters and the deformed Chinese characters. And encrypting the Chinese character by using the public key to obtain an encrypted Chinese character comparison table. The morph key is used to store a morph policy for non-Chinese characters. And encrypting the data by using the public key to obtain an encryption deformation key. The encrypted deformed file comprises two parts, namely an encrypted Chinese character comparison table and an encrypted deformed key. Together, the two parts constitute a ciphertext file. And decrypting the ciphertext file by adopting the private key. And decrypting the encrypted Chinese character comparison table by using the private key to obtain an original Chinese character comparison table. And decrypting the encrypted deformed key by using the private key to obtain an original deformed key. Then, with the original Chinese character comparison table and the deformation key, the data can be deformed by using the Chinese character comparison table and the deformation key to obtain deformed data.
It is noted that this example provides a specific implementation, and the actual implementation may vary depending on the specific needs and circumstances. Furthermore, this example does not relate to other aspects of error handling and security, such as how to ensure secure storage and transmission of keys, how to handle errors in encryption and decryption processes, etc. In practical applications, these are important issues to be considered.
Fig. 4 schematically illustrates a flow chart of a method of validating a deformed document according to an embodiment of the present disclosure.
According to an embodiment of the present disclosure, before encrypting the deformed file, and after decrypting the ciphertext file, the deformed file is verified, for example, through operations S421 to S425, as shown in fig. 4.
In operation S421, a first hash value of the deformed file is calculated using a hash function.
In operation S422, different departments transmit the ciphertext file and the hash value. And
In operation S423, the second hash value of the deformed file is recalculated.
In operation S424, the first hash value and the second hash value are compared.
In operation S425, it is determined that the deformed file transmission is successful in the case where the first hash value is the same as the second hash value.
For example, the following is a more complete data processing method, including the processes of calculating hash values, transmitting ciphertext files and hash values, re-calculating hash values, and comparing: and when the business is finished every day, the system automatically triggers a script to backup the batch data to obtain backup data. And simultaneously, acquiring a deformed file from the safety part, wherein the deformed file comprises a Chinese character comparison table and a deformed key. A first hash value of the morphed file is calculated using a hash function (e.g., SHA-256). This hash value may be used to verify the integrity of the deformed file. And encrypting the deformed file by adopting the public key to obtain the ciphertext file. In this process, the ciphertext file is stored with the first hash value on the encrypted storage device between the secure portion and the system portion. The ciphertext file and the first hash value are transmitted to the system part through a secure channel. This process is performed between the various departments, ensuring that only authorized personnel can receive and access the data. After the system part receives the ciphertext file, the ciphertext file is decrypted by using the private key, and the deformed file is obtained. At the same time, a second hash value of the deformed file is recalculated. The first hash value and the second hash value are compared. If the two hash values are the same, the deformed file is not modified in the transmission process, and the successful transmission of the deformed file can be determined. And deforming the backup data by adopting the deformed file to obtain deformed data. And transmitting the deformed data to a test environment for simulating production test.
This example provides a specific data processing method that includes the steps of calculating a hash value, transmitting a ciphertext file and the hash value, recalculating the hash value, and comparing. The method can effectively protect confidentiality and integrity of data, ensure integrity of deformed files in the transmission process, and improve safety and stability of the system.
FIG. 5A schematically illustrates a flow chart of a method of morphing backup data according to an embodiment of the disclosure. FIG. 5B schematically illustrates a flow diagram of a method of building a host morphing tool according to an embodiment of the disclosure.
According to an embodiment of the present disclosure, as shown in fig. 5A, the backup data is deformed with the deformed file, for example, through operations S551 to S553, to obtain deformed data.
In operation S551, a host morphing tool is constructed according to the chinese character map and the morphing key.
In operation S552, a morphing configuration table including backup data to be morphed is acquired. And
In operation S553, the host morphing tool is used to morph the backup data to be morphed to obtain morphed data.
For example, the deformed file is used to deform the backup data to obtain deformed data. According to the Chinese character comparison table and the deformation key, a host deformation tool is constructed: in this step, a host morphing tool may be constructed based on the chinese character lookup table and morphing key. The tool can be a program or a script for deforming the backup data to be deformed according to the Chinese character comparison table and the deformation key. After the host morphing tool is built, a morphing configuration table needs to be obtained. This variant configuration table includes information about the data to be deformed back-up, such as the type, format, range, etc. of the data. The configuration table may be a text file or a database record for guiding the host morphing tool to perform the correct morphing operation on the backup data to be morphed. And according to the obtained deformation configuration table, the host deformation tool carries out corresponding deformation operation on the backup data to be deformed. This process may be set according to actual requirements, for example, the data may be processed according to certain rules or algorithms, such that the data becomes unrecognizable or difficult to analyze. The deformed data can be transmitted to a test environment for simulated production test.
This example provides a more complete data processing method, including the specific process of transforming backup data using a transformed file, and the steps of constructing a host transformation tool, obtaining a transformation configuration table, and transforming the backup data to be transformed using the host transformation tool. The execution logic of the host deformation tool is controlled through the Chinese character comparison table and the deformation key, and the deformation range of the backup data is limited through the deformation configuration table, so that the data deformation can be automatically and accurately executed, and meanwhile, the related configuration file can be modified, and the deformation process can be flexibly adjusted.
In some embodiments, as shown in FIG. 5B, constructing a host morphing tool from the Chinese character lookup table and morphing keys, for example, includes: the host operation generates a public key and a private key pair, the public key is given to the security part, and the private key is reserved to the host. The safety part encrypts the Chinese character comparison table and the deformed key by using the public key, generates a ciphertext file (comprising the encrypted Chinese character comparison table and the encrypted deformed key) and a check code of the HASH value, and delivers the ciphertext file and the check code to the host. The host FTP (FILE TRANSFER Protocol, FTP)) uploads the ciphertext file to the host and checks successfully. The host computer uses the private key to decrypt the ciphertext file delivered by the safety part to obtain a Chinese character comparison table and a deformed key. Then, the host configures the constructed host deforming tool according to the Chinese character comparison table and the deforming key.
In some embodiments, the list of deformation configuration tables is scanned in real time according to the list of deformation configuration tables for handover, confirming the list of the deformation required at this time. And producing deformation operation item by item according to the list in the deformation configuration list and the host deformation tool for subsequent execution.
FIG. 6 schematically illustrates a flow chart of a method of morphing backup data according to another embodiment of the disclosure.
According to an embodiment of the present disclosure, as shown in fig. 6, the backup data is deformed using the deformed file, for example, through operations S651 to S654, to obtain deformed data.
In operation S651, storage spaces of the plurality of deformed volumes are determined, respectively.
In operation S652, the CPU occupancy rate is determined.
In operation S653, threshold determination is made for the storage space and CPU occupancy of the deformed volume, respectively.
In operation S654, the backup data to be deformed is deformed to obtain deformed data when the storage space of the deformed volume is greater than the first preset threshold and the CPU occupancy rate is less than the second preset threshold. Wherein the morphing data is stored concurrently in a plurality of morphing volumes.
For example, a host morphing tool is constructed from the chinese character lookup table and morphing keys. The tool can be a program or a script for deforming the backup data to be deformed according to the Chinese character comparison table and the deformation key.
In the host morphing tool, the storage space of a plurality of morphs may be determined separately. These morph volumes may be used to store morphed data to ensure the integrity and availability of the data. In the host deformation tool, the CPU occupancy rate can be determined, and the index can reflect the load condition of the system, thereby affecting the speed and efficiency of data deformation. The threshold determination may be made separately for the memory space and CPU occupancy of each deformed volume. These thresholds may be some preset value for determining whether the memory space and CPU occupancy are within acceptable ranges. Under the condition that the threshold condition is met, namely that the storage space of the deformed volume is larger than a first preset threshold and the CPU occupancy rate is smaller than a second preset threshold, the host deformation tool can deform the backup data to be deformed to obtain deformed data. This process may be set according to actual requirements, for example, the data may be processed according to certain rules or algorithms, such that the data becomes unrecognizable or difficult to analyze. The deformed data can be stored in a plurality of deformed volumes simultaneously, so that the reliability and usability of the data can be improved, and the risk of data storage can be dispersed.
This example provides a more specific data processing method, including the detailed process of transforming the backup data to be transformed using a host transforming tool, and the steps of determining storage space for a plurality of transformed volumes, determining CPU occupancy, making a threshold decision, and transforming the backup data to be transformed if a threshold condition is met. By judging the storage spaces of the deformed volumes and judging the occupancy rate of the CPU, the system resources are fully utilized and the data deformation tasks are processed simultaneously on the premise of ensuring the normal execution of the main service, so that the utilization rate of the system resources and the data deformation efficiency are improved.
In some embodiments, the last deformation result of the deformed roll may be cleaned to ensure that the deformed roll is sufficiently space. To prevent the CPU from being flushed due to the operation of the deformed job, the deformed job uses a unified job CLASS (device for controlling concurrency), and dynamically adjusts the job CLASS concurrency, wherein the job concurrency is 60 in the peak period of business, and 120 in the low peak period of business. Triggering the deformation content which is implemented on demand, continuously paying attention to whether the CPU exceeds a threshold value, and if so, automatically limiting the priority of the deformation operation by the system.
FIG. 7 schematically illustrates a flow chart of a method of morphing backup data according to yet another embodiment of the disclosure.
According to an embodiment of the present disclosure, the backup data to be deformed includes chinese field data, communication address data, mailbox address data, and password data, as shown in fig. 7, for example, the backup data to be deformed is deformed through operations S751 to S754 to obtain deformed data.
In operation S751, chinese character substitution is performed on the Chinese character segment data according to the Chinese character comparison table, thereby obtaining a substituted Chinese character field. And/or
In operation S752, at least a portion of the communication address data is character masked according to the deformed key to obtain a masked communication address. And/or
In operation S753, the mailbox address data is unified according to the deformed key, and a unified mailbox address is obtained. And/or
In operation S754, the format of the password data is unified according to the deformed key, thereby obtaining unified password data.
For example, backup data to be deformed including chinese field data, communication address data, mailbox address data, and password data is acquired. The data may be obtained from a database, file, or other data source. And performing Chinese character replacement on the Chinese character segment data according to the Chinese character comparison table to obtain a replaced Chinese field. According to the mapping relation in the comparison table, the process can replace the characters in the original Chinese field data with other characters, so that confusion and protection of the data are realized. And at least partially character shielding the communication address data according to the deformed key to obtain a shielded communication address. The process can replace, hide or encrypt part of characters in the communication address according to the deformed key so as to protect the privacy and safety of the original communication address. And according to the deformed key, unifying the format of the mailbox address data to obtain the unified mailbox address. The process can perform standardized processing on the mailbox addresses with different formats according to the deformed key, for example, the mailbox addresses with different domain names are unified into the same format, so that subsequent data processing and analysis are convenient. And according to the deformed key, unifying the formats of the password data to obtain unified password data. The process can standardize the password data with different formats according to the deformed key, for example, the password data of different websites are unified into the same format, so that the subsequent data analysis and management are convenient. Then, the data after the deformation processing is stored in an encrypted storage device between the security part and the system part. Such data may be used for subsequent data analysis and mining, or for testing and verification purposes, among other purposes.
This example provides a more specific data processing method, including the detailed process of transforming the backup data to be transformed, and the steps of replacing the Chinese character with the Chinese character segment data according to the Chinese character comparison table, masking the communication address data partially according to the transformation key, formatting the mailbox address data according to the transformation key, and formatting the password data according to the transformation key. According to the Chinese character comparison table and the deformation key, the name, address, mailbox, digital password and the like can be deformed respectively, the scene application of the financial industry is satisfied, and the security of sensitive data is improved.
Fig. 8 schematically illustrates a flow chart of a data processing method according to another embodiment of the present disclosure.
As shown in fig. 8, the data processing method of this embodiment further includes, for example, operation S810.
In operation S810, spot check is performed on one or more of the replacement chinese field, the mask communication address, the unified mailbox address, and the unified password data in the test environment.
For example, in the data processing method, the item to be spot checked may be determined according to actual requirements, such as replacing a chinese field, masking a communication address, unifying a mailbox address, unifying password data, and the like. These items may be selected and combined according to actual needs. For each spot check item, the corresponding data may be selected for spot check by sampling, randomly or according to a certain rule. For example, some of the replacement chinese fields, the masked communication address, the unified mailbox address, and the unified password data may be randomly selected for spot check. For each spot check item, verification may be performed to ensure accuracy and integrity of the deformation data. For example, for replacing Chinese fields, some of the Chinese characters may be replaced with mapped characters in a lookup table and then compared with the original data to verify the correctness of the replacement. The accuracy and reliability of the data processing method can be evaluated through spot check and analysis of deformation data in the test environment. For example, indexes such as error rate, missing rate and the like of the spot check result can be counted to evaluate the performance and quality of the data processing method. According to the analysis result in the test environment, the data processing method can be adjusted and processed to improve the accuracy and reliability of the data processing method. For example, the mapping relation of the Chinese character comparison table can be adjusted, and measures such as optimizing the deformation key can be adopted to realize more accurate deformation processing. The data after spot check and processing is stored in an encrypted storage device between the security part and the system part. Such data may be used for subsequent data analysis and mining, or for testing and verification purposes, among other purposes.
This example provides a more specific data processing method that includes the steps of spot checking one or more of replacement chinese fields, masked communication addresses, unified mailbox addresses, and unified password data in a test environment, and validating spot check results, test environment analysis, adjustment and processing, and storage and processing of results. By conducting spot check on the deformation result in the test environment, the test data received by the test environment are all deformed data, and the safety of sensitive data is further improved.
In some embodiments, after the testing environment receives the deformation result, the deformed data can be subjected to spot check comparison through the verification function of the deformation tool, and whether the deformation result meets the expectations or not is verified.
According to the category of the current sensitive information, the sample of the deformation result spot check can be:
for data in Chinese fields such as family names, whether the deformed data has no practical meaning is checked, for example:
The delivery roller is driven;
post cure history;
correcting the bare by hand;
The prison leisure altar is scared;
Each of them;
A car Shi Su;
gangster and other side effects;
Towel dream bank;
a gangway discusses;
hunger and scar reduction;
and (5) epitaxial growth in the sedan period.
A field of the communication address class, checking whether the deformed data has undergone a corresponding masking process, for example:
rabdosia rubescens X1X 03;
rabdosia rubescens X1X 03;
Century X-way X0X-ball X melting X heart X0X new X benefit;
Emerging X3X-X01;
hangzhou X city X West X big X2X2X clear X quotient X4F;
rabdosia rubescens X1X 03;
rabdosia rubescens X1X 03;
pu Ming X9X/X Single X7X1.
The EMAIL address field checks whether the deformed data has been uniformly set to "BIANXING@BIAN.XING".
The fields of the password and key class check if the deformed data has been set uniformly to "888888".
Fig. 9 schematically illustrates a flow chart of a method of error monitoring of deformation data according to an embodiment of the disclosure.
According to an embodiment of the present disclosure, after obtaining the deformation data, as shown in fig. 9, the deformation data is error-monitored, for example, through operations S951 to S954.
In operation S951, the deformed data is restored according to the chinese character comparison table and the deformed key, so as to obtain restored data.
In operation S952, the restoration data and the backup data to be deformed are compared to determine a deformation error rate.
In operation S953, a threshold determination is made on the deformation error rate. And
In operation S954, in case that the deformation error rate is greater than the third preset threshold, the backup data to be deformed is deformed again.
For example, according to the Chinese character comparison table and the deformed key, the deformed data is restored to obtain restored data. This can be achieved by looking up the chinese character look-up table in reverse and decrypting based on the morphed key. And comparing the restored data with the backup data to be deformed so as to determine the deformation error rate. This process may calculate the difference between the restored data and the backup data to be morphed, for example, comparing whether the values of each field are the same or similar. For the deformation error rate, a threshold may be set for determining the accuracy of the deformation data. If the deformation error rate is larger than a third preset threshold value, a larger error or inaccuracy place exists in the deformation data. And under the condition that the deformation error rate is larger than a third preset threshold value, the backup data to be deformed needs to be deformed again. The process can re-execute the previous deformation steps, such as re-performing the operations of Chinese character replacement, communication address shielding, mailbox address format unification, password data format unification and the like, so as to ensure the accuracy and the reliability of deformed data. The reshaped data is stored in an encrypted storage device between the secure portion and the system portion. Such data may be used for subsequent data analysis and mining, or for testing and verification purposes, among other purposes.
This example provides a more specific data processing method comprising the steps of how to perform data restoration after obtaining deformed data, comparing the restored data with the backup data to be deformed, determining a deformation error rate, thresholding the deformation error rate, and how to deform the backup data to be deformed again if the deformation error rate is greater than a third preset threshold. After the deformation is completed, the deformation result is checked, errors in the deformation are found, the deformation task with the error rate exceeding the threshold value is regulated and then re-executed, and the stability of the deformation operation is improved.
In some embodiments, to monitor whether errors occur during the deformation process, the deformed data may also be restored to the original data using the deformation key, or the deformed Chinese characters may be converted back to the original Chinese characters using the Chinese character comparison table. The error condition of the deformation operation is dynamically concerned, and the automatic re-lifting of the deformation operation can be realized.
Fig. 10 schematically illustrates a flow chart of a method of transmitting deformed data according to an embodiment of the present disclosure.
According to an embodiment of the present disclosure, as shown in fig. 10, the simulated production test is performed by transmitting the deformation data to the test environment, for example, through operations S1061 to S1063.
In operation S1061, a network bandwidth occupancy rate is determined.
In operation S1062, a threshold determination is made for the network bandwidth occupancy. And
In operation S1063, if the network bandwidth occupancy rate is less than the fourth preset threshold, the deformed data is transmitted using the disk PPRC replication technique. Wherein the network bandwidth occupancy comprises a network bandwidth occupancy of the deformed volume.
For example, the network bandwidth occupancy may be determined first before the network transmits the deformation data. This index may reflect the current network conditions and available bandwidth, thereby affecting the speed and quality of the data transmission. A threshold value of the network bandwidth occupancy rate can be set according to actual conditions and used for judging whether the current network state is suitable for data transmission. If the network bandwidth occupancy is greater than the fourth preset threshold, indicating that the current network state may be already congested or approaching saturation, careful handling or waiting for better timing for transmission is required. And under the condition that the network bandwidth occupancy rate is smaller than a fourth preset threshold value, transmitting the deformed data by adopting a disk PPRC replication technology. The technology can utilize a cache and an optimization algorithm to furthest improve the data transmission speed and efficiency, and ensure the integrity and consistency of data. In the data transmission process, the transmission progress and the network state can be monitored in real time so as to discover problems in time and adjust. After the data transmission is completed, the deformed data can be verified and tested in a test environment so as to ensure the accuracy and the integrity of the data and the normal operation of the system functions.
This example provides a more specific data transmission and testing procedure including determining network bandwidth occupancy, thresholding the network bandwidth occupancy, and transmitting the morphed data using disk PPRC replication techniques. The method can effectively improve the efficiency and accuracy of data transmission, and simultaneously ensure the stability and reliability of the system in a simulated production environment.
In some embodiments, the data is transferred from the production To the testing environment by a disk PPRC (Peer-To-Peer Remote Copy) replication technique. Because of the morph state, disk PPRC setup fails, so before replication is started, the test environment needs to morph the volume. Disk PPRC replication affects network bandwidth because the deformed volume and core master share network bandwidth in bulk. In order not to influence the core main batch, an automation device is used, the deformed disk PPRC is automatically stopped before the core main batch is started, and the deformed disk PPRC is restarted after the main batch is solved. Confirm whether the full deformed magnetic disk (i.e. the full deformed volume) is transferred. The examination file is viewed 7:00 a next time earlier. Confirming whether transmission is finished: for example, 180 deformed rolls, the date is the next day.
Based on the data processing method, the disclosure also provides a data processing device. The data processing apparatus will be described in detail below with reference to fig. 11.
Fig. 11 schematically shows a block diagram of a data processing apparatus according to an embodiment of the present disclosure.
As shown in fig. 11, the data processing apparatus 1100 of this embodiment includes, for example: backup module 1110, encryption module 1120, transmission module 1130, decryption module 1140, morphing module 1150, and test module 1160.
The backup module 1110 is configured to backup batch data to obtain backup data in response to the end of the daily batch. In an embodiment, the backup module 1110 may be used to perform the operation S210 described above, which is not described herein.
The encryption module 1120 is configured to encrypt the deformed file with the public key to obtain the ciphertext file in response to generating the public key and the private key. In an embodiment, the encryption module 1120 may be used to perform the operation S220 described above, which is not described herein.
The transmission module 1130 is used for transmitting the ciphertext file between different departments. In an embodiment, the transmission module 1130 may be configured to perform the operation S230 described above, which is not described herein.
The decryption module 1140 is configured to decrypt the ciphertext file using the private key to obtain a deformed file. In an embodiment, the decryption module 1140 may be used to perform the operation S240 described above, which is not described herein.
The morphing module 1150 is configured to morph the backup data with the morphed file to obtain morphed data. In an embodiment, the deforming module 1150 may be configured to perform the operation S250 described above, which is not described herein.
The test module 1160 is used for transmitting the deformation data to the test environment for the simulated production test. In an embodiment, the test module 1160 may be used to perform the operation S260 described above, which is not described herein.
Any of the backup module 1110, the encryption module 1120, the transmission module 1130, the decryption module 1140, the morphing module 1150, and the test module 1160 may be combined in one module to be implemented, or any of the modules may be split into a plurality of modules according to embodiments of the present disclosure. Or at least some of the functionality of one or more of the modules may be combined with, and implemented in, at least some of the functionality of other modules. According to embodiments of the present disclosure, at least one of backup module 1110, encryption module 1120, transmission module 1130, decryption module 1140, morphing module 1150, and test module 1160 may be implemented, at least in part, as hardware circuitry, such as a Field Programmable Gate Array (FPGA), programmable Logic Array (PLA), system-on-chip, system-on-substrate, system-on-package, application Specific Integrated Circuit (ASIC), or in hardware or firmware, such as any other reasonable manner of integrating or packaging the circuitry, or in any one of or a suitable combination of three of software, hardware, and firmware. Or at least one of the backup module 1110, the encryption module 1120, the transmission module 1130, the decryption module 1140, the morphing module 1150, and the test module 1160 may be at least partially implemented as a computer program module, which when executed, may perform the corresponding functions.
Fig. 12 schematically illustrates a block diagram of an electronic device adapted to implement a data processing method according to an embodiment of the disclosure.
As shown in fig. 12, an electronic device 1200 according to an embodiment of the present disclosure includes a processor 1201, which can perform various appropriate actions and processes according to a program stored in a Read Only Memory (ROM) 1202 or a program loaded from a storage section 1208 into a Random Access Memory (RAM) 1203. The processor 1201 may include, for example, a general purpose microprocessor (e.g., a CPU), an instruction set processor and/or an associated chipset and/or a special purpose microprocessor (e.g., an Application Specific Integrated Circuit (ASIC)), or the like. Processor 1201 may also include on-board memory for caching purposes. The processor 1201 may include a single processing unit or multiple processing units for performing the different actions of the method flows according to embodiments of the disclosure.
In the RAM 1203, various programs and data required for the operation of the electronic apparatus 1200 are stored. The processor 1201, the ROM 1202, and the RAM 1203 are connected to each other through a bus 1204. The processor 1201 performs various operations of the method flow according to the embodiments of the present disclosure by executing programs in the ROM 1202 and/or RAM 1203. Note that the program may be stored in one or more memories other than the ROM 1202 and the RAM 1203. The processor 1201 may also perform various operations of the method flow according to embodiments of the present disclosure by executing programs stored in the one or more memories.
According to an embodiment of the disclosure, the electronic device 1200 may also include an input/output (I/O) interface 1205, the input/output (I/O) interface 1205 also being connected to the bus 1204. The electronic device 1200 may also include one or more of the following components connected to the I/O interface 1205: an input section 1206 including a keyboard, a mouse, and the like; an output portion 1207 including a Cathode Ray Tube (CRT), a Liquid Crystal Display (LCD), and the like, a speaker, and the like; a storage section 1208 including a hard disk or the like; and a communication section 1209 including a network interface card such as a LAN card, a modem, or the like. The communication section 1209 performs communication processing via a network such as the internet. The drive 1210 is also connected to the I/O interface 1205 as needed. A removable medium 1211 such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, or the like is installed as needed on the drive 1210 so that a computer program read out therefrom is installed into the storage section 1208 as needed.
The present disclosure also provides a computer-readable storage medium that may be embodied in the apparatus/device/system described in the above embodiments; or may exist alone without being assembled into the apparatus/device/system. The computer-readable storage medium described above carries one or more programs, which when executed, implement a data processing method according to an embodiment of the present disclosure.
According to embodiments of the present disclosure, the computer-readable storage medium may be a non-volatile computer-readable storage medium, which may include, for example, but is not limited to: a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the context of this disclosure, a computer-readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. For example, according to embodiments of the present disclosure, the computer-readable storage medium may include the ROM 1202 and/or the RAM 1203 and/or one or more memories other than the ROM 1202 and the RAM 1203 described above.
Embodiments of the present disclosure also include a computer program product comprising a computer program containing program code for performing the methods shown in the flowcharts. The program code means for causing a computer system to carry out the data processing methods provided by the embodiments of the present disclosure when the computer program product is run on the computer system.
The above-described functions defined in the system/apparatus of the embodiments of the present disclosure are performed when the computer program is executed by the processor 1201. The systems, apparatus, modules, units, etc. described above may be implemented by computer program modules according to embodiments of the disclosure.
In one embodiment, the computer program may be based on a tangible storage medium such as an optical storage device, a magnetic storage device, or the like. In another embodiment, the computer program can also be transmitted, distributed over a network medium in the form of signals, and downloaded and installed via a communication portion 1209, and/or from a removable medium 1211. The computer program may include program code that may be transmitted using any appropriate network medium, including but not limited to: wireless, wired, etc., or any suitable combination of the foregoing.
In such an embodiment, the computer program can be downloaded and installed from a network via the communication portion 1209, and/or installed from the removable media 1211. The above-described functions defined in the system of the embodiments of the present disclosure are performed when the computer program is executed by the processor 1201. The systems, devices, apparatus, modules, units, etc. described above may be implemented by computer program modules according to embodiments of the disclosure.
According to embodiments of the present disclosure, program code for performing computer programs provided by embodiments of the present disclosure may be written in any combination of one or more programming languages, and in particular, such computer programs may be implemented in high-level procedural and/or object-oriented programming languages, and/or assembly/machine languages. Programming languages include, but are not limited to, such as Java, c++, python, "C" or similar programming languages. The program code may execute entirely on the user's computing device, partly on the user's device, partly on a remote computing device, or entirely on the remote computing device or server. In the case of remote computing devices, the remote computing device may be connected to the user computing device through any kind of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or may be connected to an external computing device (e.g., connected via the Internet using an Internet service provider).
The flowcharts and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams or flowchart illustration, and combinations of blocks in the block diagrams or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.
Those skilled in the art will appreciate that the features recited in the various embodiments of the disclosure and/or in the claims may be provided in a variety of combinations and/or combinations, even if such combinations or combinations are not explicitly recited in the disclosure. In particular, the features recited in the various embodiments of the present disclosure and/or the claims may be variously combined and/or combined without departing from the spirit and teachings of the present disclosure. All such combinations and/or combinations fall within the scope of the present disclosure.
The embodiments of the present disclosure are described above. These examples are for illustrative purposes only and are not intended to limit the scope of the present disclosure. Although the embodiments are described above separately, this does not mean that the measures in the embodiments cannot be used advantageously in combination. The scope of the disclosure is defined by the appended claims and equivalents thereof. Various alternatives and modifications can be made by those skilled in the art without departing from the scope of the disclosure, and such alternatives and modifications are intended to fall within the scope of the disclosure.

Claims (13)

1. A method of data processing, comprising:
backing up batch data to obtain backup data in response to the end of the daily batch;
in response to generating the public key and the private key, encrypting the deformed file by adopting the public key to obtain a ciphertext file;
transmitting the ciphertext file among different departments;
Decrypting the ciphertext file by adopting the private key to obtain the deformed file;
deforming the backup data by adopting the deformed file to obtain deformed data;
And transmitting the deformation data to a test environment for simulating production test.
2. The method of claim 1, wherein the morphed file includes a chinese character lookup table and a morphed key, wherein the encrypting the morphed file using the public key in response to generating the public key and the private key to obtain the ciphertext file comprises:
encrypting the Chinese character comparison table by adopting the public key to obtain an encrypted Chinese character comparison table;
encrypting the deformed key by adopting the public key to obtain an encrypted deformed key;
the deformation key is used for storing a deformation strategy of the non-Chinese characters.
3. The method of claim 2, wherein prior to encrypting the deformed file, the method further comprises:
Calculating a first hash value of the deformed file by using a hash function;
transmitting the ciphertext file and the hash value among different departments; and
After decrypting the ciphertext file, the method further comprises:
Re-calculating a second hash value of the deformed file;
comparing the first hash value with the second hash value;
And under the condition that the first hash value is the same as the second hash value, determining that the deformed file is successfully transmitted.
4. The method of claim 2, wherein the transforming the backup data using the transformed file comprises:
constructing a host deformation tool according to the Chinese character comparison table and the deformation key;
Obtaining a deformation configuration table, wherein the deformation configuration table comprises backup data to be deformed; and
And adopting the host deformation tool to deform the backup data to be deformed to obtain the deformed data.
5. The method of claim 4, wherein the transforming the backup data to be transformed using the host transformation tool to obtain the transformed data comprises:
Respectively determining storage spaces of a plurality of deformed volumes;
determining the CPU occupancy rate; and
Respectively carrying out threshold judgment on the storage space of the deformed volume and the CPU occupancy rate;
Deforming the backup data to be deformed under the condition that the storage space of the deformed volume is larger than a first preset threshold and the CPU occupancy rate is smaller than a second preset threshold to obtain deformed data;
The deformation data are stored in the plurality of deformation volumes concurrently.
6. The method according to any one of claims 2 to 5, wherein the backup data to be deformed includes chinese field data, communication address data, mailbox address data, and password data, and the deforming the backup data to be deformed includes:
according to the Chinese character comparison table, carrying out Chinese character replacement on the Chinese field data to obtain a replaced Chinese field; and/or
According to the deformed key, at least partial character shielding is carried out on the communication address data, and a shielding communication address is obtained; and/or
According to the deformed key, unifying the format of the mailbox address data to obtain a unified mailbox address; and/or
And according to the deformed key, unifying the formats of the password data to obtain unified password data.
7. The method as recited in claim 6, further comprising:
And performing spot check on one or more of the replacement Chinese field, the shielding communication address, the unified mailbox address and the unified password data in a test environment.
8. The method of claim 4, wherein after obtaining the deformation data, the method further comprises:
Restoring the deformed data according to the Chinese character comparison table and the deformed key to obtain restored data;
comparing the restored data with the backup data to be deformed, and determining a deformation error rate;
threshold judgment is carried out on the deformation error rate; and
And under the condition that the deformation error rate is larger than a third preset threshold value, the backup data to be deformed is deformed again.
9. The method of claim 5, wherein said transmitting said deformation data to a test environment for simulated production testing comprises:
determining the network bandwidth occupancy rate;
threshold value judgment is carried out on the network bandwidth occupancy rate; and
Transmitting the deformed data by adopting a disk PPRC replication technology under the condition that the network bandwidth occupancy rate is smaller than a fourth preset threshold value;
wherein the network bandwidth occupancy rate includes a network bandwidth occupancy rate of the deformed volume.
10. A data processing apparatus, comprising:
The backup module is used for backing up batch data to obtain backup data in response to the end of the daily batch;
the encryption module is used for encrypting the deformed file by adopting the public key to obtain a ciphertext file in response to the generation of the public key and the private key;
The transmission module is used for transmitting the ciphertext file among different departments;
the decryption module is used for decrypting the ciphertext file by adopting the private key to obtain the deformed file;
The deformation module is used for deforming the backup data by adopting the deformation file to obtain deformed data; and
And the testing module is used for transmitting the deformation data to a testing environment to perform simulated production testing.
11. An electronic device, comprising:
one or more processors;
storage means for storing one or more programs,
Wherein the one or more programs, when executed by the one or more processors, cause the one or more processors to perform the method of any of claims 1-9.
12. A computer readable storage medium having stored thereon executable instructions which, when executed by a processor, cause the processor to perform the method according to any of claims 1-9.
13. A computer program product comprising a computer program which, when executed by a processor, implements the method according to any one of claims 1 to 9.
CN202311816926.3A 2023-12-27 2023-12-27 Data processing method, apparatus, device, medium, and program product Pending CN117910003A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202311816926.3A CN117910003A (en) 2023-12-27 2023-12-27 Data processing method, apparatus, device, medium, and program product

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202311816926.3A CN117910003A (en) 2023-12-27 2023-12-27 Data processing method, apparatus, device, medium, and program product

Publications (1)

Publication Number Publication Date
CN117910003A true CN117910003A (en) 2024-04-19

Family

ID=90683187

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202311816926.3A Pending CN117910003A (en) 2023-12-27 2023-12-27 Data processing method, apparatus, device, medium, and program product

Country Status (1)

Country Link
CN (1) CN117910003A (en)

Similar Documents

Publication Publication Date Title
CN110414268B (en) Access control method, device, equipment and storage medium
US9081978B1 (en) Storing tokenized information in untrusted environments
US9652512B2 (en) Secure matching supporting fuzzy data
US20220075900A1 (en) Tracing objects across different parties
Talha et al. Analysis of research on amazon AWS cloud computing seller data security
US10127401B2 (en) Redacting restricted content in files
CN109657492B (en) Database management method, medium, and electronic device
WO2019072293A2 (en) Data isolation in a blockchain network
US10878126B1 (en) Batch tokenization service
US20190377896A1 (en) Systems and methods for securing data in electronic communications
US11966488B2 (en) De-tokenization patterns and solutions
CN106899563A (en) Method for authenticating and device, authentication code generating method and device, right discriminating system
CN111756684B (en) Method, system and non-transitory computer-readable storage medium for transmitting critical data
CN113904821A (en) Identity authentication method and device and readable storage medium
US9853817B2 (en) Generating enhanced digital signatures for artifacts
CN110830428A (en) Block chain financial big data processing method and system
CN113609531B (en) Information interaction method, device, equipment, medium and product based on block chain
CN117910003A (en) Data processing method, apparatus, device, medium, and program product
US11645650B1 (en) Systems and methods for blockchain-based transaction break prevention
EP3716124A1 (en) System and method of transmitting confidential data
CN112528330B (en) Log scanning method, device and equipment
US20240184919A1 (en) Batch tokenization service
US20240111856A1 (en) Providing password security in non-federated computing arrangements
CN117556453B (en) In-field processing method and system for data elements based on original data without domain
US11354227B2 (en) Conducting software testing using dynamically masked data

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination