WO2019233118A1 - Data processing device and method - Google Patents

Data processing device and method Download PDF

Info

Publication number
WO2019233118A1
WO2019233118A1 PCT/CN2019/075037 CN2019075037W WO2019233118A1 WO 2019233118 A1 WO2019233118 A1 WO 2019233118A1 CN 2019075037 W CN2019075037 W CN 2019075037W WO 2019233118 A1 WO2019233118 A1 WO 2019233118A1
Authority
WO
WIPO (PCT)
Prior art keywords
data
processed
component
logic
storage module
Prior art date
Application number
PCT/CN2019/075037
Other languages
French (fr)
Chinese (zh)
Inventor
陈国涛
孙忠岩
Original Assignee
华为技术有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 华为技术有限公司 filed Critical 华为技术有限公司
Publication of WO2019233118A1 publication Critical patent/WO2019233118A1/en

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F21/00—Security arrangements for protecting computers, components thereof, programs or data against unauthorised activity
    • G06F21/60—Protecting data
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F21/00—Security arrangements for protecting computers, components thereof, programs or data against unauthorised activity
    • G06F21/60—Protecting data
    • G06F21/602—Providing cryptographic facilities or services
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601—Interfaces specially adapted for storage systems
    • G06F3/0602—Interfaces specially adapted for storage systems specifically adapted to achieve a particular effect
    • G06F3/061—Improving I/O performance
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601—Interfaces specially adapted for storage systems
    • G06F3/0602—Interfaces specially adapted for storage systems specifically adapted to achieve a particular effect
    • G06F3/0625—Power saving in storage systems
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601—Interfaces specially adapted for storage systems
    • G06F3/0628—Interfaces specially adapted for storage systems making use of a particular technique
    • G06F3/0629—Configuration or reconfiguration of storage systems
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601—Interfaces specially adapted for storage systems
    • G06F3/0668—Interfaces specially adapted for storage systems adopting a particular infrastructure
    • G06F3/067—Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]
    • H—ELECTRICITY
    • H03—ELECTRONIC CIRCUITRY
    • H03M—CODING; DECODING; CODE CONVERSION IN GENERAL
    • H03M7/00—Conversion of a code where information is represented by a given sequence or number of digits to a code where the same, similar or subset of information is represented by a different sequence or number of digits
    • H03M7/30—Compression; Expansion; Suppression of unnecessary data, e.g. redundancy reduction

Definitions

  • the present application relates to the field of data processing, and in particular, to a data processing device and method.
  • the embodiments of the present application provide a data processing device and method, which improve the efficiency of compression and decryption or decryption and decompression of data, and reduce the system bandwidth.
  • a first aspect of the present application provides a data processing device, which includes: data input logic, control logic, compression component, encryption component, and data output logic; the control logic is separate from the data input logic, compression component, encryption component, and data output logic Connection, the data input logic is connected with the compression component, the compression component is connected with the encryption component, and the encryption component is connected with the data output logic.
  • the control logic is used to respond to the compression and encryption instruction.
  • the compression and encryption instruction is an instruction for instructing compression and encryption of the data to be processed stored in the storage module.
  • the control logic can control the data input logic to read the data to be processed from the storage module.
  • the storage module may include a memory and / or a buffer pool, where the data to be processed and program code are stored in the memory, and the data to be processed can be stored in the buffer pool, compressed. Data and encrypted data.
  • the control logic can control the data input module to input the read pending data to the compression component.
  • the control component can control the compression component to compress the processed data to obtain compressed data;
  • the control module can control the encryption component to encrypt the compressed data to obtain encrypted data;
  • the control module can control the data output logic to write the encrypted data to the storage module.
  • the compressed encryption instruction carries the first address and data length of the data to be processed
  • Reading the data to be processed from the storage module includes:
  • the embodiments of the present application provide a way to read the data to be processed from the storage module. According to the first address and data length of the data to be processed from the storage module, the data to be processed can be more targeted and accurate.
  • controlling the encryption component to encrypt compressed data includes:
  • the encryption component is controlled to encrypt the compressed data.
  • the formed compressed data is encrypted by the encryption component, so as to realize the parallel processing of the compression operation of the compression component and the encryption operation of the encryption component without having to wait for all
  • the compressed data is encrypted, which further improves the efficiency of compression and encryption.
  • the device further includes key input logic, which is connected to the encryption component and control logic;
  • Controlling the encryption component to encrypt compressed data includes:
  • the control key input logic reads the encryption key from the storage module, and controls the encryption component to use the encryption key to encrypt the compressed data.
  • the encryption key can be read through the input logic, so that the encryption component uses the encryption key to encrypt the compressed data, thereby making the encrypted data more secure.
  • the device further includes a first format conversion logic, and the first format conversion logic is connected between the data input logic and the compression component;
  • the control logic is also used to control the first format conversion logic to convert the format of the data to be processed from the original format to a preset format, and send the data to be processed in the preset format to the compression component, and the preset format is a format matched by the compression component.
  • the first format conversion logic is used to perform format processing on the data to be processed, and convert the format of the data to be processed from the original format to a preset format.
  • data can be protected by adding protection information to each of the data to be processed. Integrity makes data processing more secure.
  • the device further includes a second format conversion logic, and the second format conversion logic is connected between the data output logic and the encryption component;
  • control logic is also used to control the second format conversion logic to convert the format of the encrypted data into the original format, and send the encrypted data in the original format to the data output logic.
  • the second format conversion logic is used to perform format processing on the encrypted data, and convert the format of the encrypted data into the original format, so that it can be conveniently used when the encrypted data is needed.
  • the storage module includes:
  • to-be-processed data and program code are stored in the memory, and the to-be-processed data, compressed data, and encrypted data can be stored in the cache pool to improve data read speed and write speed.
  • the second aspect of the present application provides another data processing device, which includes: data input logic, control logic, decryption component, decompression component, and data output logic; the control logic is separately from the data input logic, decryption component, decompression component, and data output Logical connection, data input logic is connected with the decryption component, decryption component is connected with the decompression component, and decompression component is connected with the data output logic.
  • the control logic is used to respond to the decryption and decompression instruction, and the decryption and decompression instruction is used to instruct the storage module to store data in the storage module. Instructions to decrypt and decompress the pending data.
  • the control logic can control the data input logic to read the data to be processed from the storage module.
  • the storage module may include a memory and / or a buffer pool, where the data to be processed and program code are stored in the memory, and the data to be processed can be stored in the buffer pool, compressed. Data and encrypted data.
  • the control logic can control the data input module to input the data to be processed to the decryption component; the control logic can control the decryption component to decrypt the processed data to obtain the decrypted data; the control logic can control the decompression component to decompress the decrypted data to obtain the decompressed data; control logic
  • the data output logic can be controlled to write the decompressed data to the storage module.
  • the decrypted data does not need to be written into the storage module, and the decrypted data does not need to be read from the storage module before the decrypted data is decompressed.
  • the storage module only needs to perform one read and one write, that is, the processor only needs to interact with the storage module twice. Compared with the traditional technology, the number of interactions between the processor and the storage module is reduced by half, which improves the efficiency of decryption and decompression. Reduced system bandwidth by half.
  • decryption and decompression are independent processes, after the decrypted data reaches a certain amount of data, the decompression of the decrypted data can be started, so that the decryption and decompression can be processed in parallel, further improving the efficiency of decryption and decompression, and reducing the system bandwidth.
  • the first address and data length of the data to be processed are carried in the decryption and decompression instructions
  • Reading the data to be processed from the storage module includes:
  • the embodiments of the present application provide a way to read the data to be processed from the storage module. According to the first address and data length of the data to be processed from the storage module, the data to be processed can be more targeted and accurate.
  • the device further includes key input logic, which is connected to the decryption component and control logic;
  • Decrypting the data to be processed includes:
  • the control key input logic reads the decryption key from the storage module, and controls the decryption component to decrypt the processed data using the decryption key.
  • the formed decrypted data is decompressed by the decompression component, so that the decryption operation of the decryption component and the decompression operation of the decompression component can be processed in parallel without waiting for all Decompress the decrypted data after completing the decryption process for all the data to be processed, so as to further improve the efficiency of decryption and decompression.
  • the device further includes a first format conversion logic, and the first format conversion logic is connected between the data input logic and the decryption component;
  • the control logic is also used to control the first format conversion logic to convert the format of the data to be processed from the original format to a preset format, and send the data to be processed in the preset format to the decryption component.
  • the preset format is a format that matches the decryption component. .
  • the first format conversion logic is used to convert the format of the data to be processed from the original format to a preset format.
  • protection information may be added to each of the data to be processed to protect data integrity.
  • the device further includes a second format conversion logic, and the second format conversion logic is connected between the data output logic and the decompression component;
  • control logic is also used to control the second format conversion logic to convert the format of the decompressed data into the original format, and send the decompressed data in the original format to the data output logic.
  • the second format conversion logic is used to convert the format of the decompressed data into the original format, so that the decompressed data can be conveniently used, compressed, or encrypted.
  • the storage module includes:
  • to-be-processed data and program code are stored in the memory, and the to-be-processed data, compressed data, and encrypted data can be stored in the cache pool to improve data read speed and write speed.
  • a third aspect of the present application provides a data processing method, which includes:
  • the compressed encryption instruction carries the first address and data length of the data to be processed
  • Reading the data to be processed from the storage module includes:
  • encrypting compressed data includes:
  • the compressed data is encrypted.
  • encrypting compressed data includes:
  • the method also includes:
  • the format of the data to be processed is converted from the original format into a preset format, and the preset format is a format matched by the compression component.
  • the method also includes:
  • the storage module includes:
  • the fourth aspect of the present application provides another data processing method, which includes:
  • the first address and data length of the data to be processed are carried in the decryption and decompression instructions
  • Reading the data to be processed from the storage module includes:
  • decrypting the data to be processed includes:
  • the decryption key is read from the storage module, and the decrypted key is used to decrypt the processed data.
  • the method also includes:
  • the format of the data to be processed is changed from the original format to a preset format, and the preset format is a format that matches the decryption component.
  • the method also includes:
  • the storage module includes:
  • a fifth aspect of the present application provides a data processing device, which includes: a processor and a memory;
  • Memory for storing instructions
  • the processor is configured to execute an instruction in the memory and execute the data processing method provided by the third aspect of the present application.
  • a sixth aspect of the present application provides another data processing device, which includes: a processor and a memory;
  • Memory for storing instructions
  • the processor is configured to execute instructions in the memory and execute the data processing method provided by the fourth aspect of the present application.
  • a seventh aspect of the present application provides a computer-readable storage medium including instructions that, when run on a computer, cause the computer to execute the data processing method provided by the third aspect of the present application.
  • An eighth aspect of the present application provides another computer-readable storage medium, including instructions, which, when run on a computer, cause the computer to execute the data processing method provided by the fourth aspect of the present application.
  • a ninth aspect of the present application provides a computer program product containing instructions, which when executed on a computer, causes the computer to execute the data processing method provided by the third aspect of the present application.
  • the tenth aspect of the present application provides another computer program product containing instructions that, when run on a computer, causes the computer to execute the data processing method provided by the fourth aspect of the present application.
  • the data processing device includes: data input logic, control logic, compression component, encryption component, and data output logic, control logic and data input logic, compression component, encryption component, and The data output logic is connected, the data input logic is connected with the compression component, the compression component is connected with the encryption component, the encryption component is connected with the data output logic, and the control logic responds to the compression encryption instruction, and the control data input logic reads the data to be processed from the storage module.
  • the data to be processed is input to the compression component, the compression component is controlled to compress the processed data to obtain compressed data, and the encryption component is controlled to encrypt the compressed data to obtain encrypted data, and the data output logic is controlled to write the encrypted data to the storage module.
  • the data processing device includes: data input logic, control logic, decryption component, decompression component, and data output logic; the control logic is separately from the data input logic, decryption component, and decompression The component is connected to the data output logic, the data input logic is connected to the decryption component, the decryption component is connected to the decompression component, and the decompression component is connected to the data output logic; among them, the control logic is used to respond to the decryption and decompression instruction and control the data input logic from the storage module Read the data to be processed and input the data to be decrypted; control the decryption component to decrypt the processed data to obtain the decrypted data; control the decompression component to decompress the decrypted data to obtain the decompressed data; and control the data output logic to decompress the data Write to memory module.
  • the decrypted data does not need to be written into the storage module, and the decrypted data does not need to be read from the storage module before the decrypted data is decompressed.
  • the entire process only needs to perform one read and one write to the storage module, without having to perform multiple reads and writes. Compared with traditional technologies, it improves the efficiency of decryption and decompression and reduces the system bandwidth.
  • FIG. 1 is a schematic diagram of a data processing process provided by the prior art
  • FIG. 2 is a schematic diagram of a data processing process according to an embodiment of the present application.
  • FIG. 3 is a structural block diagram of a data processing apparatus according to an embodiment of the present application.
  • FIG. 4 is a schematic diagram of data processing provided by an embodiment of the present application.
  • FIG. 5 is a schematic diagram of a data processing scenario provided by an embodiment of the present application.
  • FIG. 6 is a structural block diagram of another data processing apparatus according to an embodiment of the present application.
  • FIG. 7 is a schematic diagram of another data processing scenario according to an embodiment of the present application.
  • FIG. 8 is a flowchart of a data processing method according to an embodiment of the present application.
  • FIG. 9 is a flowchart of another data processing method according to an embodiment of the present application.
  • FIG. 10 is a structural block diagram of a data processing device according to an embodiment of the present application.
  • the embodiments of the present application provide a data processing device and method, which are used to improve the efficiency of compression encryption, decryption and decompression, and reduce system bandwidth.
  • the processor generally compresses and encrypts the data first, and then stores the compressed and encrypted data to the storage module.
  • the processing of the data is usually read by the processor from the storage module. Take the compressed and encrypted data, and decrypt and decompress the compressed and encrypted data to obtain the original data.
  • the storage module is a device for storing data.
  • the storage module may be used to store data to be processed and / or processed data.
  • the memory module may include volatile memory, such as nonvolatile dynamic random access memory (NVRAM), phase change random access memory (PRAM), magnetoresistive random access memory (magetoresistive random access memory (MRAM)), such as double-rate synchronous dynamic random access memory (DDR) synchronous random random access memory (DDR SDRAM), may also include non-volatile memory, such as at least one disk storage device, flash memory device For example, a hard disk drive (HDD) or a solid state drive (SSD), a NOR flash memory, or a NAND flash memory.
  • HDD hard disk drive
  • SSD solid state drive
  • NOR flash memory NOR flash memory
  • NAND flash memory a NAND flash memory
  • the processor is a device having functions such as processing instructions, performing operations, and processing data.
  • the processor is used to read data from a storage module and process the data.
  • the data can be compressed, decompressed, and encrypted. And decryption operations.
  • the processor may be a central processing unit (CPU), a general purpose processor, a digital signal processor (DSP), an application specific integrated circuit (ASIC), or a field programmable gate array (field) programmable gate array (FPGA) or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof, or a combination that implements computing functions, such as a combination of one or more microprocessors, a DSP and a microprocessor Combination, etc.
  • CPU central processing unit
  • DSP digital signal processor
  • ASIC application specific integrated circuit
  • FPGA field programmable gate array
  • a processor that currently processes data may include a compression component and an encryption component.
  • the compression component implements data compression
  • the encryption component implements data encryption.
  • the compression component and the encryption component are independent of each other and cannot transmit data to each other. Therefore, for the two processes of data compression and data encryption, the processor needs to read data from the storage module for each process separately, and separately Write the data processed by the two processes to the storage module.
  • FIG. 1 (a) is a schematic diagram of interaction between a processor and a storage module in a current compression and encryption process.
  • the processor needs to perform the following steps in the compression and encryption process: 1) read the data to be processed from the storage module, and compress the data to be processed; 2) write the compressed data to the storage module; 3) read from the storage module The compressed data is encrypted; 4) the encrypted data is written into the storage module.
  • the processor needs to read and write the storage module twice, that is, the processor needs to interact with the storage module four times, so the efficiency of data compression and encryption is low, and the system bandwidth is increased.
  • the current processor for processing data may further include a decryption component and a decompression component.
  • the decryption component is used to decrypt the data
  • the decompression component is used to decompress the data.
  • the decryption component and the decompression component are independent of each other and cannot transmit data to each other. Therefore, for the two processes of data decryption and data decompression, the processor also needs to read data from the storage module for each process separately, and separately Write the data processed by the two processes to the storage module.
  • FIG. 1 (b) is a schematic diagram of the interaction between the processor and the storage module in the current process of decryption and decompression.
  • the processor needs to perform the following steps during the decryption and decompression process: 1) read the data to be processed from the storage module and decrypt the data to be processed; 2) write the decrypted data to the storage module; 3) read from the storage module Decrypt the data, and decompress the decrypted data; 4) Write the decompressed data to the storage module.
  • the processor needs to read and write the storage module twice, that is, the processor needs to interact with the storage module four times, so the efficiency of decrypting and decompressing the data is low, and the system bandwidth is increased.
  • an embodiment of the present application provides a data processing device.
  • a compression component and an encryption component are bridged, so that data transmission can be performed between the compression component and the encryption component, and the compression component can
  • the compressed data is transmitted to the encryption component, and the encrypted data is encrypted by the encryption component. Therefore, during the compression and encryption process, the processor only needs to read and write the storage module once.
  • FIG. 2 (a) is a schematic diagram of interaction between a processor and a storage module when implementing compression and encryption in a data processing method provided by an embodiment of the present application.
  • the processor responds to the compression and encryption instruction and performs the following steps: 1) reads the data to be processed from the storage module, compresses the data to be processed to obtain compressed data, and then encrypts the compressed data to obtain encrypted data; 2) writes the encrypted data Into the storage module.
  • the processor when implementing compression and encryption, the processor only needs to read and write the storage module once, that is, the processor only needs to interact with the storage module twice. Compared with the traditional technology, the processor and The number of interactions of the storage module is reduced by half, thereby improving the efficiency of compression and encryption and reducing the system bandwidth by half.
  • FIG. 2 (b) is a schematic diagram of an interaction between a processor and a storage module when decryption and decompression are implemented in a data processing method provided by an embodiment of the present application.
  • the processor responds to the decryption and decompression instruction and performs the following steps: 1) reads the data to be processed from the storage module, decrypts the data to be processed, obtains the decrypted data, and decompresses the decrypted data to obtain the decompressed data; 2) writes the decompressed data Into the storage module.
  • the processor when the decryption and decompression are implemented, the processor only needs to read and write once, that is, the processor only needs to interact with the storage module twice. Compared with the conventional technology, the processor and the storage module The number of interactions is reduced by half, which improves the efficiency of decryption and decompression, and reduces the system bandwidth by half.
  • the apparatus includes: a data input logic 120, a control logic 110, a compression component 130, an encryption component 140, and a data output logic 150.
  • the control logic 110 is connected to the data input logic 120, the compression component 130, the encryption component 140, and the data output logic 150, the data input logic 120 is connected to the compression component 130, the compression component 130 is connected to the encryption component 140, and the encryption component 140 is connected to the data.
  • Output logic 150 is connected.
  • the control logic 110 is used to control the data input logic 120, the compression component 130, the encryption component 140, and the data output logic 150 in response to the compression encryption instruction.
  • the compression and encryption instruction is an instruction for instructing compression and encryption of the data to be processed stored in the storage module 100.
  • compression and encryption instructions can be sent through the APP and call related components to execute compression and encryption instructions, such as the kernel (Kernel) of the driver layer.
  • control logic 110 may control the data input logic 120 to read the data to be processed from the storage module 100.
  • the Zlib APP interface may be called to read the data to be processed from the storage module 100.
  • the storage module 100 may include a memory and / or a cache pool, in which the pending data and program code are stored in the memory, and the cache pool may store the to-be-processed data, compressed data, encrypted data, etc. to improve data reading speed and writing speed. .
  • the data to be processed may be uncompressed and encrypted raw data stored in the storage module 100.
  • the data to be processed may be a single file, multiple files, or a folder including one or more files. Wait.
  • the file type of the data to be processed may be at least one of text, form, email, database data, and application types.
  • the file format of the data to be processed corresponds to the file type of the data to be processed.
  • the format of text data can be documents (doc), text files (txt), etc.
  • the format of table data can be xls
  • the format of mail data can be eml, msg, etc.
  • the format of database data can be db (database), etc.
  • the format of application data can be executable program (executable program, EXE).
  • the data to be processed may also be compressed and / or encrypted data stored in the storage module 100 and further compressed and encrypted.
  • a standard audio layer 3 moving picture
  • Experts group audio layer III, mp3 and other compressed data formats the data to be processed may also be other compressed or encrypted data, and have other data formats.
  • the compressed encryption instruction may carry the first address and data length of the data to be processed.
  • the data input logic 120 may read the data to be processed from the storage module 100 according to the first address and data length of the data to be processed.
  • the compression encryption instruction may also carry multiple first address and multiple data lengths corresponding to the first address, so that the data input logic 120 reads the data to be processed.
  • the data processing device may further include fragmentation logic to perform fragmentation processing on the data to be processed, that is, to divide the data to be processed into multiple data.
  • fragmentation logic to perform fragmentation processing on the data to be processed, that is, to divide the data to be processed into multiple data.
  • Data segments each of which is a piece of fragmented data. Referring to FIG. 4, after sharding the data to be processed, multiple shard data can be obtained, and the length of each shard data is small, so that multiple shard data can be compressed and encrypted at the same time to improve compression. Encryption efficiency.
  • control logic 110 may control the data input logic 120 to input the read data to be processed to the compression component 130 so that the compression component 130 performs compression processing on the data to be processed.
  • the data processing apparatus may further include a first format conversion logic.
  • the first format conversion logic is connected between the data input logic 120 and the compression component 130, and is used for processing the data output logic 120 output to be processed.
  • the data is formatted, and the format of the data to be processed is converted from the original format into a preset format, and the preset format is a format that matches the compression component 130. For example, data integrity can be protected by adding protection information to each pending data.
  • the control logic 110 may control the first format conversion logic to perform format conversion on the data to be processed, and after the format conversion, send the processed data in a preset format to the compression component 130.
  • the control logic 110 controls the compression component 130 to perform compression processing on the data to be processed according to a compression algorithm.
  • the data compression is to process the data according to the compression algorithm, using less than the uncompressed data. Bits to represent the process of information.
  • the compression encryption instruction can carry the identification of the compression algorithm, and the compression component 130 selects the corresponding decompression algorithm according to the identification of the compression algorithm.
  • the data to be processed is compressed to form compressed data, and the format of the compressed data may be, for example, ZIP, ZLIB, and the like.
  • compressing data such as text, tables, emails, database data, and application programs can obtain compressed data in ZIP format.
  • the compression component 130 may further add a compression header to the compressed data to form compressed data.
  • the compression header may include a verification result of the compressed data verification in order to pass the compression.
  • the verification result of the header realizes the error detection of the compressed data.
  • the compressed header may also include a description of the data.
  • control logic 110 may control the compression component 130 to transmit the compressed compressed data to the encryption component 140 so that the encryption component 140 encrypts the compressed data by using an encryption algorithm.
  • the control logic 110 may control the encryption component 140 to encrypt the compressed data by using an encryption algorithm.
  • the data encryption is to process the original data in plain text according to some encryption algorithm to make it unreadable cipher text. It is difficult for someone or a device other than a legitimate receiver to obtain the data in plain text before encryption. The security of transmission and storage is increased.
  • DES data encryption standard
  • triple data encryption algorithm triple data encryption algorithm
  • MD5 message-digest algorithm 5
  • RSA Raster-Shamir-Adleman Algorithms
  • the compressed data can be encrypted by using an encryption key.
  • the key can be a symmetric key, that is, the same key can be used for encryption and decryption.
  • the key can also be an asymmetric key, which can be obtained by
  • the public key is encrypted and decrypted by the private key.
  • the encryption key may be stored in the storage module 100, and the storage location of the encryption key may also be carried in the compressed encryption instruction.
  • the data processing device further includes a key input logic.
  • the key input logic is connected to the encryption component 140 and the control logic 110, and is configured to read the encryption key from the storage module 100 according to the storage location of the encryption key. key.
  • the control logic 110 controls the key input logic to read the encryption key from the storage module 100, so that the encryption component 140 uses the encryption key read by the key input logic to encrypt the compressed data.
  • the compressed data that has been compressed may not be written into the storage module 100, and the compression component 130 may send the compressed data to the encryption component 140, so that the encryption component 140 encrypts the compressed data.
  • the control logic 110 can also determine whether the data amount of the compressed data that has been compressed is greater than or equal to The first threshold value is the lowest amount of data that can be encrypted, and may be, for example, 16 bytes. If the data amount of the compressed data is greater than or equal to the first threshold, the control logic 110 may control the encryption component 140 to encrypt the compressed data that has been compressed to obtain encrypted data.
  • the formed compressed data is encrypted by the encryption component 140, so that the compression operation of the compression component 130 and the encryption operation of the encryption component 140 are performed in parallel. Processing without having to wait until all the pending data has completed the compression process before encrypting the compressed data. For example, if the first part of the data to be processed has been compressed to form the first part of the compressed data, if the amount of data in the first part of the compressed data is 16 Bytes and reaches the first threshold of 16 Bytes, the first part of the compression can be compressed. The data is encrypted. At the same time, the second part of the data to be processed is being compressed. Encryption of the first part of the compressed data and compression of the second part of the data to be processed can be performed simultaneously to achieve the efficiency of compression and encryption. Further improvement.
  • the control logic 110 controls the data output logic 150 to write the formed encrypted data to the storage module 100, and the writing position of the encrypted data may be the storage position of the original data to be processed or other unused storage positions.
  • the encrypted data can be written into the storage module 100 by calling an Open Secure Sockets Layer Protocol (Open Secure Sockets Layer) (OpenSLL) interface.
  • OpenSLL Open Secure Sockets Layer Protocol
  • the data processing apparatus may further include a second format conversion logic, where the second format conversion logic is connected between the data output logic 150 and the encryption component 140, and after the compressed data is encrypted, the control The logic 110 can also control the encryption component 140 to transmit the encrypted data to the second format conversion logic.
  • the control logic 110 controls the second format conversion logic to convert the format of the encrypted data into the original format, so that when the encrypted data is needed, it can be easily decrypted and Decompress and transmit the transformed encrypted data to the data output module, so that the data output module writes the encrypted data in the original format into the storage module 100.
  • the data processing device provided in the embodiment of the present application will be described with reference to FIG. 5 in combination with a specific scenario. Specifically, the modules in the data processing device can be executed according to the following steps:
  • the APP sends a compression and encryption instruction to the control logic 110.
  • control data input module 120 reads the data to be processed from the storage module, and sends the read data to be processed to the first format conversion logic.
  • the control logic 110 controls the first format conversion logic to perform format conversion on the data to be processed to form a processing format corresponding to the compression component 130, and transmits the formatted to-be-processed data to the compression component 130.
  • the control logic 110 controls the compression component 130 to compress the data to be processed according to a compression algorithm to form compressed data, and sends the compressed data to the encryption component 140.
  • the control logic 110 controls the key input module to read the encryption key from the storage module 100, and sends the read encryption key to the encryption component 140.
  • the control logic 110 controls the encryption component 140 to encrypt the compressed data according to the encryption algorithm, or the encryption algorithm and the encryption key to form encrypted data, and sends the encrypted data to the second format exchange logic.
  • the control logic 110 controls the second format conversion logic to perform format conversion on the encrypted data, and sends the encrypted data after the format conversion to the data output module 150.
  • the control logic 110 controls the data output module 150 to write the encrypted data to the storage module 100.
  • the data processing device includes: data input logic, control logic, compression component, encryption component, and data output logic, control logic and data input logic, compression component, encryption component, and data output Logical connection.
  • Data input logic is connected to the compression component.
  • Compression component is connected to the encryption component.
  • Encryption component is connected to the data output logic.
  • the control logic responds to the compression encryption instruction.
  • the control data input logic reads the data to be processed from the storage module.
  • the data to be processed is input to the compression component, and the compression component is controlled to compress the processed data to obtain compressed data, and the encryption component is controlled to encrypt the compressed data to obtain encrypted data, and the data output logic is controlled to write the encrypted data to the storage module.
  • the APP sends a compression instruction, calls the driver's Zlib APP interface to read the data to be processed from the storage module, compresses the data to be processed, and then writes the compressed data to The storage module; then the APP sends an encryption instruction, calls the driver's OpenSLL interface to read the compressed data from the storage module, encrypts the compressed data, and writes the encrypted data to the storage module.
  • the APP needs to send two instructions, namely a compression instruction and an encryption instruction, respectively calling the Zlib APP interface and the OpenSLL interface. In this process, the storage module needs to be read twice and written twice.
  • the APP only needs to send an instruction to the driver layer, that is, a compression and encryption instruction, and the driver layer can implement the compression and encryption process of the data to be processed.
  • the driver layer first reads the data to be processed from the storage module and compresses it, and then directly encrypts the compressed data. After encryption, the compressed and encrypted data is written into the storage module. In this process, The driver layer only needs to read and write to the storage module once. Compared with traditional technologies, it reduces the number of interactions with the storage module, improves the efficiency of compression and encryption, and reduces system bandwidth.
  • the apparatus includes: a data input logic 220, a control logic 210, a decryption component 230, a decompression component 240, and a data output logic 250.
  • the control logic 210 is connected to the data input logic 220, the decryption component 230, the decompression component 240, and the data output logic 250, the data input logic 220 is connected to the decryption component 230, the decryption component 230 is connected to the decompression component 240, and the decompression component 240 is connected to the data.
  • Output logic 250 is connected.
  • the control logic 210 is used to control the data input logic 220, the decryption component 230, the decompression component 240, and the data output logic 250 in response to the decryption and decompression instruction.
  • the decryption and decompression instruction is an instruction for instructing decryption and decompression of the compressed and encrypted to-be-processed data stored in the storage module 200. Analogous to compressing and encrypting instructions, the decryption and decompression instructions can be sent through the APP, and the relevant components are called to execute the decryption and decompression instructions, such as the kernel of the driver layer (Kernel).
  • control logic 210 can control the data input logic 220 to read the data to be processed from the storage module 200.
  • the OpenSSL interface can be called to read the data to be processed from the storage module 200.
  • the storage module 200 may include a memory and / or a buffer pool, in which the data to be processed and program code are stored in the memory, and the buffer pool may store the data to be processed, compressed data, encrypted data, etc., to improve data read speed and write speed .
  • the data to be processed is compressed and encrypted data stored in the storage module 200.
  • the data to be processed may be a single compressed and encrypted file, or multiple compressed and encrypted files, and may include one or more files.
  • the file format of the data to be processed may be, for example, ZIP, ZLIB, and the like.
  • the first address and data length of the data to be processed may be carried, so that the data input logic 220 reads the data to be processed from the storage module 200 according to the first address and data length of the data to be processed.
  • the decryption and decompression instruction may also carry multiple first address and multiple data lengths corresponding to the first address.
  • control logic 210 controls the data input logic 220 to transmit the read data to be processed to the decryption component 230 so that the decryption component 230 performs decryption processing on the data to be processed.
  • the data processing device provided in the embodiment of the present application may further include a first format conversion logic.
  • the first format conversion logic is connected between the data input logic 220 and the decryption component 230, and is used for converting the format of the data to be processed from the original format. It is converted into a preset format, and the preset format is a format matching the decryption component 230. Specifically, protection information may be added to each to-be-processed data to protect data integrity.
  • the control logic 210 controls the data input logic 220 to transmit the read data to be processed to the first format conversion logic, controls the first format conversion logic to perform format conversion on the data to be processed, and controls the first format conversion logic to send a preset format to the decryption component 230 Of pending data.
  • the control logic 210 may control the decryption component 230 to decrypt the data to be processed according to a decryption algorithm to form decrypted data.
  • data decryption is the process of processing encrypted data through a decryption algorithm to make it display the original content.
  • the decryption algorithm is an inverse algorithm corresponding to the encryption algorithm.
  • the decryption algorithm corresponds to an encryption algorithm, such as the DES, 3DES, MD5, and RSA algorithms.
  • the compressed encryption instruction may carry the identifier of the decryption algorithm, and the decryption component 230 selects the corresponding decryption algorithm according to the identifier of the decryption algorithm.
  • the data to be processed may be decrypted by using a decryption key corresponding to the encryption key of the data to be processed to obtain decrypted data.
  • the decryption key is the same as the encryption key
  • the decryption key is a private key corresponding to the public key.
  • the decryption key may be stored in the storage module 200, and the storage location of the decryption key may also be carried in the decryption and decompression instruction.
  • the data control device may further include a key input logic, which is connected to the decryption component 230 and the control logic 210, and is configured to read and decrypt from the storage module 200 according to the storage location of the decryption key. Key.
  • the control logic 210 may control the key input logic to read the decryption key from the storage module 200 so that the decryption component 230 uses the decryption key read by the key input logic to decrypt the processed data.
  • control logic 210 controls the decryption component 230 to transmit the decrypted decrypted data to the decompression component 240 so that the decompression component 240 decompresses the decrypted data by using a decompression algorithm.
  • the control logic 210 may control the decompression component 240 to decompress the decrypted data by using a decompression algorithm.
  • Data decompression is the inverse operation of data compression, which can recover compressed data. For example, you can obtain compressed data in ZIP format by compressing data such as text, tables, videos, pictures, audio, and applications. Data decompression is to restore the data in ZIP format to the original format.
  • the format of decompressed text data can be It is doc, txt, etc.
  • the format of the decompressed form data can be xls, etc.
  • the format of the decompressed mail data can be eml, msg, etc.
  • the format of the decompressed database data can be db, etc.
  • the format can be EXE, etc.
  • the decompression algorithm may be, for example, the LZ77 algorithm or the Huffman algorithm.
  • the identifier of the decompression algorithm may be carried in the decryption and decompression instruction, and the decompression component 240 may select a corresponding decompression algorithm according to the identifier of the decompression algorithm.
  • the decrypted data that has been decrypted may not be written into the storage module 200, and the decryption component 230 may send the decrypted data to the decompression component 240, so that the decompression component 240 decompresses the decrypted data.
  • the control logic 210 can also determine whether the data amount of the decrypted data that has been decrypted is greater than or equal to The second threshold, the second threshold is the lowest amount of data that can be decompressed, and can be, for example, 16 Bytes. If the data amount of the decrypted data is greater than or equal to the second threshold, the control logic 210 may control the decompression component 240 to decompress the decrypted data that has been decrypted to obtain the decompressed data.
  • the formed decrypted data can be decompressed by the decompression component 240, so that the decryption operation of the decryption component 230 and the decompression operation of the decompression component 240 can be implemented.
  • Parallel processing instead of waiting for all the data to be processed to complete the decryption process before decompressing the decrypted data.
  • the third part of the data to be processed has been decrypted to form the decrypted data of the third part
  • the data amount of the decrypted data of the third part is 16 Byte and reaches the second threshold 16 Byte
  • the first The decryption data of the three parts is decompressed.
  • the data to be processed in the fourth part is being decrypted. Decompressing the decrypted data in the third part and decrypting the data to be processed in the fourth part can be performed simultaneously. The efficiency of decryption and decompression is further improved.
  • the control logic 210 can control other components to process and use the formed decompressed data. It can also control the decompression component 240 to transmit the decompressed data to the data output logic 250, and the control logic 210 controls the data output.
  • the logic 250 writes the formed decompressed data into the storage module 200, and the write location of the decompressed data may be a storage location of the original to-be-processed data, or may be another unused storage location.
  • the decompressed data may also be combined, and the combined data may be written into the storage module 200.
  • the data processing apparatus may further include a second format conversion logic, where the second format conversion logic is connected between the data output logic 250 and the decompression component 240, and after the decompression of the decrypted data is completed, the control The logic 210 can also control the decompression component 240 to transmit the decompressed data to the second format conversion logic.
  • the control logic 210 controls the second format conversion logic to convert the format of the decompressed data to the original format, so that the decompressed data can be conveniently used and compressed. Or encrypted.
  • the second format conversion logic transmits the decompressed data in the transformed format to the data output module, so that the data output module writes the decompressed data in the original format into the storage module 200.
  • the data processing device provided in the embodiment of the present application will be described with reference to FIG. 7 in combination with specific scenarios.
  • the modules in the data processing apparatus may be executed according to the following steps:
  • the App sends a decryption and decompression instruction to the control logic 210.
  • control data input module 220 reads the data to be processed from the storage module 200 and sends the read data to be processed to the first format conversion logic.
  • the control logic 210 controls the first format conversion logic to perform format conversion on the data to be processed to form a processing format corresponding to the decryption component 230, and transmits the formatted to-be-processed data to the decryption component 240.
  • the control logic 210 controls the key input module to read the decryption key from the storage module 200 and sends the read decryption key to the decryption component 230.
  • the control logic 210 controls the decryption component 230 to decrypt the data to be processed according to the decryption algorithm, or the decryption algorithm and the decryption key to form decrypted data, and sends the decrypted data to the decompression component 240.
  • the control logic 210 controls the decompression component 240 to decompress the decrypted data according to the decompression algorithm to form decompressed data, and sends the decompressed data to the second format conversion logic.
  • the control logic 210 controls the second format conversion logic to perform format conversion on the decompressed data, and sends the decompressed data after the format conversion to the data output module 250.
  • the control logic 210 controls the data output module 250 to write the decompressed data to the storage module 200.
  • the data processing device includes: data input logic, control logic, decryption component, decompression component, and data output logic; the control logic is separately from the data input logic, decryption component, and decompression The component is connected to the data output logic, the data input logic is connected to the decryption component, the decryption component is connected to the decompression component, and the decompression component is connected to the data output logic; among them, the control logic is used to respond to the decryption and decompression instruction and control the data input logic from the storage module Read the data to be processed and input the data to be decrypted; control the decryption component to decrypt the processed data to obtain the decrypted data; control the decompression component to decompress the decrypted data to obtain the decompressed data; and control the data output logic to decompress the data Write to memory module.
  • the decrypted data does not need to be written into the storage module, and the decrypted data does not need to be read from the storage module before the decrypted data is decompressed.
  • the storage module only needs to perform one read and one write, that is, the processor only needs to interact with the storage module twice. Compared with the traditional technology, the number of interactions between the processor and the storage module is reduced by half, which improves the efficiency of decryption and decompression. Reduced system bandwidth by half.
  • decryption and decompression are independent processes, after the decrypted data reaches a certain amount of data, the decompression of the decrypted data can be started, so that the decryption and decompression can be processed in parallel, further improving the efficiency of decryption and decompression, and reducing the system bandwidth.
  • the APP sends a decryption instruction, calls the driver's OpenSLL interface to read the data to be processed from the storage module, decrypts the data to be processed, and writes the decrypted data to the storage module;
  • the APP sends a decompression instruction, calls the driver's Zlib APP interface to read the decrypted data from the storage module, decompresses the decrypted data, and writes the decompressed data to the storage module.
  • the APP needs to send two instructions, namely a decryption instruction and a decompression instruction, respectively calling the OpenSLL interface and the Zlib APP interface. In this process, the memory module needs to be read twice and two writes.
  • the APP only needs to send an instruction to the driver layer, that is, the decryption and decompression instruction, and the driver layer can implement the decryption and decompression process of the data to be processed.
  • the driver layer first reads the data to be processed from the storage module and decrypts it, and then directly decompresses the decrypted data. After decompression, the decrypted and decompressed data is written into the storage module. In this process, The driver layer only needs to read and write the storage module once. Compared with traditional technologies, it reduces the number of interactions with the storage module, improves the efficiency of decryption and decompression, and reduces system bandwidth.
  • a flowchart of a data processing method according to an embodiment of the present application may be executed by a processor, and specifically includes the following steps:
  • the compression and encryption instruction is an instruction for instructing compression and encryption of the data to be processed stored in the storage module.
  • compression and encryption instructions can be sent through the APP and call related components to execute compression and encryption instructions, such as the kernel (Kernel) of the driver layer.
  • kernel Kernel
  • the storage module 100 may include a memory and / or a cache pool, in which the data to be processed and program code are stored in the memory, and the cache pool may store the data to be processed, compressed data, encrypted data, and the like.
  • the data to be processed may be uncompressed and encrypted raw data stored in the storage module 100, or may be compressed and / or encrypted data stored in the storage module 100 and need to be further compressed and encrypted.
  • the data to be processed can be a single file, multiple files, or a folder containing one or more files.
  • the first address and data length of the data to be processed may be carried, so that the data to be processed is read from the storage module 100 according to the first address and data length of the data to be processed.
  • the compression and encryption instruction may also carry multiple first address and multiple data lengths corresponding to the first address.
  • the data to be processed may be fragmented first, that is, the data to be processed is divided into multiple data segments, and each data segment Is a piece of data.
  • Data compression is the process of processing data according to a compression algorithm and representing information with fewer bits than uncompressed data.
  • Common compression algorithms such as LZ (Lempel-Ziv) 77 algorithm or Huffman algorithm, etc.
  • the compression encryption instruction can carry the identification of the compression algorithm, and the corresponding decompression algorithm can be selected according to the identification of the compression algorithm.
  • the data to be processed is compressed to form compressed data, and the format of the compressed data may be, for example, ZIP, ZLIB, and the like.
  • compressing data such as text, tables, emails, database data, and application programs can obtain compressed data in ZIP format.
  • the data to be processed may also be formatted, and the format of the data to be processed may be changed from the original format to a preset format, and the preset format is a format that matches the component to be compressed.
  • a compression header can be added to the compressed data.
  • Data encryption is to process the original data in plain text according to some encryption algorithm to make it unreadable cipher text. It is difficult for someone or a device other than a legitimate receiver to obtain the plain text data before encryption. Data transmission And storage security goes up.
  • DES data encryption standard
  • triple data encryption algorithm triple data encryption algorithm
  • MD5 message-digest algorithm 5
  • RSA Raster-Shamir-Adleman Algorithms
  • the compressed data can be encrypted by using an encryption key.
  • the key can be a symmetric key, that is, the same key can be used for encryption and decryption.
  • the key can also be an asymmetric key, which can be obtained by The public key is encrypted and decrypted by the private key.
  • the encryption key can be stored in the storage module, and the storage location of the encryption key can also be carried in the compressed encryption instruction.
  • the encryption key may be read from the storage module according to the storage location of the encryption key, so as to use the encryption key read by the key input logic to encrypt the compressed data.
  • the compressed data that has been compressed may not be written into the storage module, but the compressed data may be encrypted.
  • the encrypted data is formed, and the formed encrypted data is written into the storage module.
  • the written data may be written to the storage location of the original to-be-processed data or other unused storage locations.
  • the encrypted data can be written into the storage module by calling an Open Secure Sockets Layer Protocol (Open Secure Sockets Layer) (OpenSLL) interface.
  • OpenSLL Open Secure Sockets Layer Protocol
  • the format of the encrypted data can also be converted into the original format, so that when the encrypted data is needed, it can be easily decrypted and decompressed, and the converted encrypted data is written into the storage module.
  • FIG. 9 is a flowchart of another data processing method according to an embodiment of the present application.
  • the method may be executed by a processor, and specifically includes the following steps:
  • the decryption and decompression instruction is an instruction for instructing decryption and decompression of the compressed and encrypted to-be-processed data stored in the storage module.
  • the decryption and decompression instructions can be sent through the APP, and the relevant components are called to execute the decryption and decompression instructions, such as the kernel of the driver layer (Kernel).
  • the storage module 200 may include a memory and / or a cache pool, where the memory stores the data to be processed and program code, etc., and the cache pool may store the data to be processed, compressed data, encrypted data, and the like.
  • the data to be processed is compressed and encrypted data stored in the storage module.
  • the data to be processed may be a single compressed and encrypted file, or multiple compressed and encrypted files, or may include one or more files.
  • a folder of compressed encrypted files may be a single compressed and encrypted file, or multiple compressed and encrypted files, or may include one or more files.
  • the decryption and decompression instruction may carry a first address and a data length of the data to be processed, so as to read the data to be processed from the storage module according to the first address and the data length of the data to be processed.
  • the decryption and decompression instruction may also carry multiple first address and multiple data lengths corresponding to the first address.
  • S202 Decrypt the data to be processed to obtain decrypted data.
  • Data decryption is the process of processing encrypted data through a decryption algorithm to make it display its original content.
  • the decryption algorithm is the inverse algorithm corresponding to the encryption algorithm.
  • the data to be processed can be decrypted by a decryption algorithm.
  • the decryption algorithm corresponds to the encryption algorithm, such as DES, 3DES, MD5, and RSA algorithms.
  • the compressed encryption instruction can carry the identifier of the decryption algorithm, and the corresponding decryption algorithm is selected according to the identifier of the decryption algorithm.
  • the data to be processed may be decrypted by using a decryption key corresponding to the encryption key of the data to be processed to obtain decrypted data.
  • the decryption key is the same as the encryption key
  • the decryption key is a private key corresponding to the public key.
  • the decryption key can be stored in the storage module, and the storage location of the decryption key can also be carried in the decryption and decompression instructions.
  • the decryption key is read from the storage module, so that the key can be read logically by using the key input logic.
  • the obtained decryption key is used to decrypt the data to be processed.
  • the format of the data to be processed may also be processed, and the format of the data to be processed may be changed from the original format to a preset format, and the preset format is a format that matches the component to be decrypted. For example, data integrity is protected by adding protection information to each pending data.
  • S203 Decompress the decrypted data to obtain decompressed data.
  • Data decompression is the inverse operation of data compression, which can recover compressed data.
  • the decrypted data can be decompressed according to a decompression algorithm, such as the LZ77 algorithm or the Huffman algorithm.
  • a decompression algorithm such as the LZ77 algorithm or the Huffman algorithm.
  • the identifier of the decompression algorithm may be carried in the decryption and decompression instruction, and a corresponding decompression algorithm is selected according to the identifier of the decompression algorithm.
  • the decrypted data that has been decrypted is not written into the storage module, but is sent to a module that performs data decompression to decompress the decrypted data.
  • it can also be determined whether the amount of decrypted data that has been decrypted is greater than or equal to the second threshold . If the data amount of the decrypted data is greater than or equal to the second threshold, the decrypted data that has been decrypted can be decompressed to obtain the decompressed data.
  • the formed decrypted data can be decompressed, so that the parallel operation of the decryption operation and the decompression operation can be realized without waiting for all the pending data to be processed. Decompress the decrypted data after completing the decryption process to further improve the efficiency of decryption and decompression.
  • the formed decompressed data can be processed and used, and the formed decompressed data can be written to the storage module.
  • the write location of the decompressed data can be the original storage location of the data to be processed, or it can be other Unused storage location.
  • the decompressed data may also be combined, and the combined data may be written into the storage module.
  • the format of the decompressed data can also be adjusted. If the format of the decompressed data is a preset format, the format of the decompressed data can be converted to the original format, so that the decompressed data can be conveniently used. Use, compress, or encrypt.
  • the decrypted data in response to a decryption and decompression instruction, reading data to be processed from a storage module, first decrypting the data to be processed to obtain decrypted data, and then decompressing the decrypted data to obtain decompressed data, The decompressed data is written into the storage module.
  • the decrypted data after the data to be processed is decrypted, the decrypted data does not need to be written into the storage module, and the decrypted data does not need to be read from the storage module before the decrypted data is decompressed.
  • the storage module only needs to perform one read and one write, that is, the processor only needs to interact with the storage module twice.
  • the number of interactions between the processor and the storage module is reduced by half, which improves the efficiency of decryption and decompression. Reduced system bandwidth by half.
  • decryption and decompression are independent processes, after the decrypted data reaches a certain amount of data, the decompression of the decrypted data can be started, so that the decryption and decompression can be processed in parallel, further improving the efficiency of decryption and decompression, and reducing the system bandwidth.
  • the data processing device 1000 includes:
  • the receiver 1001, the transmitter 1002, the processor 1003, and the memory 1004 (the number of the processors 1003 in the data processing device 1000 may be one or more, and one processor is taken as an example in FIG. 10).
  • the receiver 1001, the transmitter 1002, the processor 1003, and the memory 1004 may be connected through a bus or other manners. In FIG. 10, the connection through the bus is taken as an example.
  • the memory 1004 may include a read-only memory and a random access memory, and provide instructions and data to the processor 1003. A part of the memory 1004 may further include NVRAM.
  • the memory 1004 stores an operating system and operation instructions, executable modules or data structures, or a subset thereof, or an extended set thereof.
  • the operation instructions may include various operation instructions for implementing various operations.
  • the operating system may include various system programs for implementing various basic services and processing hardware-based tasks.
  • the processor 1003 controls operations of the terminal device, and the processor 1003 may also be referred to as a CPU.
  • the various components of the data processing equipment are coupled together through a bus system.
  • the bus system may include a power bus, a control bus, and a status signal bus in addition to the data bus.
  • various buses are called bus systems in the figure.
  • the method disclosed in the foregoing embodiments of the present application may be applied to the processor 1003, or implemented by the processor 1003.
  • the processor 1003 may be an integrated circuit chip and has a signal processing capability. In the implementation process, each step of the above method may be completed by an integrated logic circuit of hardware in the processor 1003 or an instruction in the form of software.
  • the processor 1003 may be a general-purpose processor, a DSP, an ASIC, an FPGA, or other programmable logic devices, a discrete gate or transistor logic device, or a discrete hardware component.
  • a general-purpose processor may be a microprocessor or the processor may be any conventional processor or the like.
  • a software module may be located in a mature storage medium such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory, or an electrically erasable programmable memory, a register, and the like.
  • the storage medium is located in the memory 1004, and the processor 1003 reads the information in the memory 1004 and completes the steps of the foregoing method in combination with its hardware.
  • the receiver 1001 can be used to receive inputted digital or character information, and generate signal inputs related to the related settings and function control of the user plane device.
  • the transmitter 1002 can include display devices such as a display screen, and the transmitter 1002 can be used to output through an external interface. Numeric or character information.
  • the receiver 1001 and the transmitter 1002 are used to implement data transmission and reception.
  • the processor 1003 is configured to implement data transmission and reception through the receiver 1001 and the transmitter 1002, and complete a process of one data processing method and another process of a data processing method performed by the foregoing data processing device.
  • An embodiment of the present application further provides a computer-readable storage medium for storing program code, where the program code is used to execute any one of the data processing methods of the foregoing embodiments.
  • the embodiment of the present application further provides another computer-readable storage medium for storing program code, where the program code is used to execute any one of the other data processing methods of the foregoing embodiments.
  • the embodiment of the present application further provides a computer program product including instructions, which when executed on a computer, causes the computer to execute any one of the data processing methods of the foregoing embodiments.
  • the embodiment of the present application also provides another computer program product including instructions, which when executed on a computer, causes the computer to execute any one of the other data processing methods of the foregoing embodiments.
  • the disclosed systems, devices, and methods may be implemented in other ways.
  • the device embodiments described above are only schematic.
  • the division of the unit is only a logical function division.
  • multiple units or components may be combined or Can be integrated into another system, or some features can be ignored or not implemented.
  • the displayed or discussed mutual coupling or direct coupling or communication connection may be indirect coupling or communication connection through some interfaces, devices or units, which may be electrical, mechanical or other forms.
  • the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, may be located in one place, or may be distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the objective of the solution of this embodiment.
  • each functional unit in each embodiment of the present application may be integrated into one processing unit, or each of the units may exist separately physically, or two or more units may be integrated into one unit.
  • the above integrated unit may be implemented in the form of hardware or in the form of software functional unit.
  • the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it may be stored in a computer-readable storage medium.
  • the technical solution of the present application is essentially a part that contributes to the existing technology or all or part of the technical solution can be embodied in the form of a software product, which is stored in a storage medium , Including a number of instructions to enable a computer device (which may be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application.
  • the foregoing storage media include: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), magnetic disks or optical disks and other media that can store program codes .

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Human Computer Interaction (AREA)
  • Health & Medical Sciences (AREA)
  • Bioethics (AREA)
  • General Health & Medical Sciences (AREA)
  • Computer Hardware Design (AREA)
  • Computer Security & Cryptography (AREA)
  • Software Systems (AREA)
  • Storage Device Security (AREA)

Abstract

A data processing device and method, used to improve the efficiency of compression/encryption and decryption/decompression, and to reduce a system bandwidth. According to the data processing device, a compression assembly (130) is bridged to an encryption assembly (140), such that data transmission can be performed between the compression assembly (130) and the encryption assembly (140). The compression assembly (130) transmits compressed data to the encryption assembly (140), and the encryption assembly (140) achieves encryption of the compressed data, such that in a compression and encryption process, a processor needs only to read and to write a storage module (100) once. Correspondingly, a decryption assembly is bridged to a decompression assembly, such that data transmission can be performed between the decryption assembly and the decompression assembly. The decryption assembly transmits decrypted data to the decompression assembly, and the decompression assembly achieves decompression of the decrypted data, such that in a decryption and decompression process, the processor needs only to read and to write the storage module (100) once.

Description

一种数据处理装置及方法Data processing device and method
本申请要求于2018年6月6日提交中国专利局、申请号为201810576152.4、发明名称为“一种数据处理装置及方法”的中国专利申请的优先权，其全部内容通过引用结合在本申请中。This application claims priority from a Chinese patent application filed with the Chinese Patent Office on June 6, 2018, with an application number of 201810576152.4, and the invention name is "a data processing device and method", the entire contents of which are incorporated herein by reference. .
技术领域Technical field
本申请涉及数据处理领域，尤其涉及一种数据处理装置及方法。The present application relates to the field of data processing, and in particular, to a data processing device and method.
背景技术Background technique
随着云计算和大数据的不断发展，全球信息的数据量逐年猛增，对数据进行压缩处理，以节约存储成本的需求越来越强烈。同时，对数据进行加密以实现对数据的安全性保护也是非常重要的课题。With the continuous development of cloud computing and big data, the amount of global information data has increased year by year, and the need to compress the data to save storage costs has become stronger. At the same time, encrypting data to achieve data security protection is also a very important subject.
在传统的数据处理过程中，一般对数据进行先压缩后加密，当需要使用该数据的时候，对该数据先解密后解压。在先压缩后加密的过程中，压缩和加密是两个独立的流程；在先解密后解压的过程中，解密和解压也是两个独立的流程。因此，这两个过程都需要从存储数据的存储模块中读取和写入多次才能实现，从而使得对数据进行压缩加密或解密解压的效率降低，增加了系统带宽。In the traditional data processing process, data is generally compressed first and then encrypted. When the data is needed, the data is first decrypted and then decompressed. In the process of compression before encryption, compression and encryption are two independent processes; in the process of decryption after decompression, decryption and decompression are also two independent processes. Therefore, these two processes need to be read and written multiple times from the storage module that stores the data, so that the efficiency of compressing, encrypting, decrypting and decompressing the data is reduced, and the system bandwidth is increased.
发明内容Summary of the Invention
本申请实施例提供了一种数据处理装置及方法，提高了对数据进行压缩解密或解密解压的效率，降低了系统带宽。The embodiments of the present application provide a data processing device and method, which improve the efficiency of compression and decryption or decryption and decompression of data, and reduce the system bandwidth.
本申请第一方面提供一种数据处理装置，该装置包括：数据输入逻辑、控制逻辑、压缩组件、加密组件和数据输出逻辑；控制逻辑分别与数据输入逻辑、压缩组件、加密组件和数据输出逻辑连接，数据输入逻辑与压缩组件连接，压缩组件与加密组件连接，加密组件与数据输出逻辑连接。其中，控制逻辑，用于响应压缩加密指令，压缩加密指令是用于指示对存储模块中存储的待处理数据进行压缩加密的指令。控制逻辑可以控制数据输入逻辑从存储模块中读取待处理数据，存储模块可以包括存储器和/或缓存池，其中存储器中存储有待处理数据和程序代码等，缓存池中可以存储有待处理数据、压缩数据和加密数据等。控制逻辑可以控制数据输入模块将读取的待处理数据输入到压缩组件。控制组件可以控制压缩组件对待处理数据进行压缩，得到压缩数据；控制模块可以控制加密组件对压缩数据进行加密，得到加密数据；控制模块可以控制数据输出逻辑将加密数据写入存储模块。A first aspect of the present application provides a data processing device, which includes: data input logic, control logic, compression component, encryption component, and data output logic; the control logic is separate from the data input logic, compression component, encryption component, and data output logic Connection, the data input logic is connected with the compression component, the compression component is connected with the encryption component, and the encryption component is connected with the data output logic. The control logic is used to respond to the compression and encryption instruction. The compression and encryption instruction is an instruction for instructing compression and encryption of the data to be processed stored in the storage module. The control logic can control the data input logic to read the data to be processed from the storage module. The storage module may include a memory and / or a buffer pool, where the data to be processed and program code are stored in the memory, and the data to be processed can be stored in the buffer pool, compressed. Data and encrypted data. The control logic can control the data input module to input the read pending data to the compression component. The control component can control the compression component to compress the processed data to obtain compressed data; the control module can control the encryption component to encrypt the compressed data to obtain encrypted data; the control module can control the data output logic to write the encrypted data to the storage module.
在此压缩加密的过程中，在对待处理数据进行压缩后，无需将压缩数据存储至存储器中，在对压缩数据进行加密前，无需从存储器中读取压缩数据，整个过程中只需对存储器执行一次读取和一次写入即可，即处理器只要与存储模块进行两次交互，相比于传统技术，处理器与存储模块的交互次数减少了一半，提高了压缩加密的效率，降低了一半的系统带宽。此外，由于压缩组件和加密组件是独立的组件，可以在压缩数据达到一定数据量后， 开始进行压缩数据的加密，使压缩和加密可以并行处理，进一步提高压缩加密的效率，降低系统带宽。In this compression and encryption process, after the data to be processed is compressed, there is no need to store the compressed data in the memory. Before the compressed data is encrypted, there is no need to read the compressed data from the memory. The entire process only needs to be performed on the memory. One read and one write is sufficient, that is, the processor only needs to interact with the storage module twice. Compared with the traditional technology, the number of interactions between the processor and the storage module is reduced by half, the efficiency of compression and encryption is improved, and the half is reduced. System bandwidth. In addition, since the compression component and the encryption component are independent components, after the compressed data reaches a certain amount of data, the encryption of the compressed data can be started, so that the compression and encryption can be processed in parallel, which further improves the efficiency of compression encryption and reduces the system bandwidth.
在一些可能的设计中，压缩加密指令中携带有待处理数据的首位地址和数据长度；In some possible designs, the compressed encryption instruction carries the first address and data length of the data to be processed;
从存储模块中读取待处理数据包括：Reading the data to be processed from the storage module includes:
根据待处理数据的首位地址和数据长度从存储模块中读取待处理数据。Read the data to be processed from the storage module according to the first address and data length of the data to be processed.
本申请实施例提供了一种从存储模块中读取待处理数据的方式，根据待处理数据的首位地址和数据长度从存储模块中，能使待处理数据更加有针对性，更加准确。The embodiments of the present application provide a way to read the data to be processed from the storage module. According to the first address and data length of the data to be processed from the storage module, the data to be processed can be more targeted and accurate.
在一些可能的设计中，控制加密组件对压缩数据进行加密包括：In some possible designs, controlling the encryption component to encrypt compressed data includes:
若压缩数据的数据量大于或等于第一阈值，则控制加密组件对压缩数据进行加密。If the data volume of the compressed data is greater than or equal to the first threshold, the encryption component is controlled to encrypt the compressed data.
本申请实施例中，可以在有部分待处理数据完成压缩后，通过加密组件对形成的压缩数据进行加密，从而实现压缩组件的压缩操作和加密组件的加密操作的并行处理，而不必等到所有的待处理数据均完成压缩过程后再进行压缩数据的加密，进一步提高了压缩加密的效率。In the embodiment of the present application, after a part of the data to be processed is compressed, the formed compressed data is encrypted by the encryption component, so as to realize the parallel processing of the compression operation of the compression component and the encryption operation of the encryption component without having to wait for all After the data to be processed has been compressed, the compressed data is encrypted, which further improves the efficiency of compression and encryption.
在一些可能的设计中，该装置还包括密钥输入逻辑，与加密组件和控制逻辑连接；In some possible designs, the device further includes key input logic, which is connected to the encryption component and control logic;
控制加密组件对压缩数据进行加密包括：Controlling the encryption component to encrypt compressed data includes:
控制密钥输入逻辑从存储模块中读取加密密钥，并控制加密组件利用加密密钥对压缩数据进行加密。The control key input logic reads the encryption key from the storage module, and controls the encryption component to use the encryption key to encrypt the compressed data.
本申请实施例中，可以通过木有输入逻辑读取加密密钥，以便于加密组件利用加密密钥对压缩数据进行加密，从而使加密数据更加安全。In the embodiment of the present application, the encryption key can be read through the input logic, so that the encryption component uses the encryption key to encrypt the compressed data, thereby making the encrypted data more secure.
在一些可能的设计中，该装置还包括第一格式变换逻辑，第一格式变换逻辑连接在数据输入逻辑和压缩组件之间；In some possible designs, the device further includes a first format conversion logic, and the first format conversion logic is connected between the data input logic and the compression component;
控制逻辑，还用于控制第一格式变换逻辑将待处理数据的格式由原格式变换为预设格式，并向压缩组件发送预设格式的待处理数据，预设格式为压缩组件匹配的格式。The control logic is also used to control the first format conversion logic to convert the format of the data to be processed from the original format to a preset format, and send the data to be processed in the preset format to the compression component, and the preset format is a format matched by the compression component.
本申请实施例中，第一格式变换逻辑用于对待处理数据进行格式处理，将待处理数据的格式由原格式变换为预设格式，例如可以通过对每个待处理数据加入保护信息来保护数据完整性，使数据处理过程更加安全。In the embodiment of the present application, the first format conversion logic is used to perform format processing on the data to be processed, and convert the format of the data to be processed from the original format to a preset format. For example, data can be protected by adding protection information to each of the data to be processed. Integrity makes data processing more secure.
在一些可能的设计中，该装置还包括第二格式变换逻辑，第二格式变换逻辑连接在数据输出逻辑和加密组件之间；In some possible designs, the device further includes a second format conversion logic, and the second format conversion logic is connected between the data output logic and the encryption component;
控制逻辑，还用于控制第二格式变换逻辑将加密数据的格式变换为原格式，并向数据输出逻辑发送原格式的加密数据。The control logic is also used to control the second format conversion logic to convert the format of the encrypted data into the original format, and send the encrypted data in the original format to the data output logic.
本申请实施例中，第二格式变换逻辑用于对加密数据进行格式处理，将加密数据的格式变换为原格式，以便在需要该加密数据时能够方便使用。In the embodiment of the present application, the second format conversion logic is used to perform format processing on the encrypted data, and convert the format of the encrypted data into the original format, so that it can be conveniently used when the encrypted data is needed.
在一些可能的设计中，存储模块包括：In some possible designs, the storage module includes:
存储器和/或缓存池。Storage and / or cache pool.
本申请实施例中，存储器中存储有待处理数据和程序代码等，缓存池中可以存储有待处理数据、压缩数据和加密数据等，以提高数据读取速度和写入速度。In the embodiment of the present application, to-be-processed data and program code are stored in the memory, and the to-be-processed data, compressed data, and encrypted data can be stored in the cache pool to improve data read speed and write speed.
本申请第二方面提供另一种数据处理装置，该装置包括：数据输入逻辑、控制逻辑、解密组件、解压组件和数据输出逻辑；控制逻辑分别与数据输入逻辑、解密组件、解压组件和数据输出逻辑连接，数据输入逻辑与解密组件连接，解密组件与解压组件连接，解压组件与数据输出逻辑连接；其中，控制逻辑，用于响应解密解压指令，解密解压指令是用于指示对存储模块中存储的待处理数据进行解密解压的指令。控制逻辑可以控制数据输入逻辑从存储模块中读取待处理数据，存储模块可以包括存储器和/或缓存池，其中存储器中存储有待处理数据和程序代码等，缓存池中可以存储有待处理数据、压缩数据和加密数据等。控制逻辑可以控制数据输入模块将待处理数据输入到解密组件；控制逻辑可以控制解密组件对待处理数据进行解密，得到解密数据；控制逻辑可以控制解压组件对解密数据进行解压，得到解压数据；控制逻辑可以控制数据输出逻辑将解压数据写入存储模块。The second aspect of the present application provides another data processing device, which includes: data input logic, control logic, decryption component, decompression component, and data output logic; the control logic is separately from the data input logic, decryption component, decompression component, and data output Logical connection, data input logic is connected with the decryption component, decryption component is connected with the decompression component, and decompression component is connected with the data output logic. Among them, the control logic is used to respond to the decryption and decompression instruction, and the decryption and decompression instruction is used to instruct the storage module to store data in the storage module. Instructions to decrypt and decompress the pending data. The control logic can control the data input logic to read the data to be processed from the storage module. The storage module may include a memory and / or a buffer pool, where the data to be processed and program code are stored in the memory, and the data to be processed can be stored in the buffer pool, compressed. Data and encrypted data. The control logic can control the data input module to input the data to be processed to the decryption component; the control logic can control the decryption component to decrypt the processed data to obtain the decrypted data; the control logic can control the decompression component to decompress the decrypted data to obtain the decompressed data; control logic The data output logic can be controlled to write the decompressed data to the storage module.
在此解密解压的过程中，在对待处理数据进行解密后，无需将解密数据写入存储模块中，在对解密数据进行解压之前，也无需从存储模块中读取解密数据，整个过程只需对存储模块执行一次读取和一次写入即可，即处理器只要与存储模块进行两次交互，相比于传统技术，处理器与存储模块的交互次数减少了一半，提高了解密解压的效率，降低了一半的系统带宽。此外，由于解密和解压是独立的过程，可以在解密数据达到一定数据量后，开始进行解密数据的解压，使解密和解压可以并行处理，进一步提高解密解压的效率，降低系统带宽。In the process of decryption and decompression, after the data to be processed is decrypted, the decrypted data does not need to be written into the storage module, and the decrypted data does not need to be read from the storage module before the decrypted data is decompressed. The storage module only needs to perform one read and one write, that is, the processor only needs to interact with the storage module twice. Compared with the traditional technology, the number of interactions between the processor and the storage module is reduced by half, which improves the efficiency of decryption and decompression. Reduced system bandwidth by half. In addition, because decryption and decompression are independent processes, after the decrypted data reaches a certain amount of data, the decompression of the decrypted data can be started, so that the decryption and decompression can be processed in parallel, further improving the efficiency of decryption and decompression, and reducing the system bandwidth.
在一些可能的设计中，解密解压指令中携带有待处理数据的首位地址和数据长度；In some possible designs, the first address and data length of the data to be processed are carried in the decryption and decompression instructions;
从存储模块中读取待处理数据包括：Reading the data to be processed from the storage module includes:
根据待处理数据的首位地址和数据长度从存储模块中读取待处理数据。Read the data to be processed from the storage module according to the first address and data length of the data to be processed.
本申请实施例提供了一种从存储模块中读取待处理数据的方式，根据待处理数据的首位地址和数据长度从存储模块中，能使待处理数据更加有针对性，更加准确。The embodiments of the present application provide a way to read the data to be processed from the storage module. According to the first address and data length of the data to be processed from the storage module, the data to be processed can be more targeted and accurate.
在一些可能的设计中，该装置还包括密钥输入逻辑，与解密组件和控制逻辑连接；In some possible designs, the device further includes key input logic, which is connected to the decryption component and control logic;
对待处理数据进行解密包括：Decrypting the data to be processed includes:
控制密钥输入逻辑从存储模块中读取解密密钥，并控制解密组件利用解密密钥对待处理数据进行解密。The control key input logic reads the decryption key from the storage module, and controls the decryption component to decrypt the processed data using the decryption key.
本申请实施例中，可以在有部分待处理数据完成解密后，通过解压组件对形成的解密数据进行解压，从而可以实现解密组件的解密操作和解压组件的解压操作的并行处理，而不必等到所有的待处理数据均完成解密过程后再进行解密数据的解压，实现解密解压的效率的进一步提升。In the embodiment of the present application, after a part of the data to be processed is decrypted, the formed decrypted data is decompressed by the decompression component, so that the decryption operation of the decryption component and the decompression operation of the decompression component can be processed in parallel without waiting for all Decompress the decrypted data after completing the decryption process for all the data to be processed, so as to further improve the efficiency of decryption and decompression.
在一些可能的设计中，该装置还包括第一格式变换逻辑，第一格式变换逻辑连接在数据输入逻辑和解密组件之间；In some possible designs, the device further includes a first format conversion logic, and the first format conversion logic is connected between the data input logic and the decryption component;
控制逻辑，还用于控制第一格式变换逻辑将待处理数据的格式由原格式变换为预设格式，并向解密组件发送预设格式的待处理数据，预设格式为与解密组件匹配的格式。The control logic is also used to control the first format conversion logic to convert the format of the data to be processed from the original format to a preset format, and send the data to be processed in the preset format to the decryption component. The preset format is a format that matches the decryption component. .
本申请实施例中，第一格式变换逻辑用于将待处理数据的格式由原格式变换为预设格式，具体的，可以对每个待处理数据加入保护信息来保护数据完整性。In the embodiment of the present application, the first format conversion logic is used to convert the format of the data to be processed from the original format to a preset format. Specifically, protection information may be added to each of the data to be processed to protect data integrity.
在一些可能的设计中，该装置还包括第二格式变换逻辑，第二格式变换逻辑连接在数 据输出逻辑和解压组件之间；In some possible designs, the device further includes a second format conversion logic, and the second format conversion logic is connected between the data output logic and the decompression component;
控制逻辑，还用于控制第二格式变换逻辑将解压数据的格式变换为原格式，并向数据输出逻辑发送原格式的解压数据。The control logic is also used to control the second format conversion logic to convert the format of the decompressed data into the original format, and send the decompressed data in the original format to the data output logic.
本申请实施例中，第二格式变换逻辑用于将解压数据的格式变换为原格式，以便对该解压数据时能够方便使用、压缩或加密。In the embodiment of the present application, the second format conversion logic is used to convert the format of the decompressed data into the original format, so that the decompressed data can be conveniently used, compressed, or encrypted.
在一些可能的设计中，存储模块包括：In some possible designs, the storage module includes:
存储器和/或缓存池。Storage and / or cache pool.
本申请实施例中，存储器中存储有待处理数据和程序代码等，缓存池中可以存储有待处理数据、压缩数据和加密数据等，以提高数据读取速度和写入速度。In the embodiment of the present application, to-be-processed data and program code are stored in the memory, and the to-be-processed data, compressed data, and encrypted data can be stored in the cache pool to improve data read speed and write speed.
本申请第三方面提供一种数据处理方法，该方法包括：A third aspect of the present application provides a data processing method, which includes:
响应压缩加密指令，从存储模块中读取待处理数据；Respond to the compression and encryption instruction, and read the data to be processed from the storage module;
对待处理数据进行压缩，得到压缩数据；Compress the data to be processed to obtain compressed data;
对压缩数据进行加密，得到加密数据；Encrypt the compressed data to obtain encrypted data;
将加密数据写入存储模块中。Write encrypted data to the storage module.
在一些可能的设计中，压缩加密指令中携带有待处理数据的首位地址和数据长度；In some possible designs, the compressed encryption instruction carries the first address and data length of the data to be processed;
从存储模块中读取待处理数据包括：Reading the data to be processed from the storage module includes:
根据待处理数据的首位地址和数据长度从存储模块中读取待处理数据。Read the data to be processed from the storage module according to the first address and data length of the data to be processed.
在一些可能的设计中，对压缩数据进行加密包括：In some possible designs, encrypting compressed data includes:
若压缩数据的数据量大于或等于第一阈值，则对压缩数据进行加密。If the data amount of the compressed data is greater than or equal to the first threshold, the compressed data is encrypted.
在一些可能的设计中，对压缩数据进行加密包括：In some possible designs, encrypting compressed data includes:
从存储模块中读取加密密钥，并利用加密密钥对压缩数据进行加密。Read the encryption key from the storage module and use the encryption key to encrypt the compressed data.
在一些可能的设计中，该方法还包括：In some possible designs, the method also includes:
将待处理数据的格式由原格式变换为预设格式，预设格式为压缩组件匹配的格式。The format of the data to be processed is converted from the original format into a preset format, and the preset format is a format matched by the compression component.
在一些可能的设计中，该方法还包括：In some possible designs, the method also includes:
将加密数据的格式变换为原格式。Convert the format of the encrypted data to the original format.
在一些可能的设计中，存储模块包括：In some possible designs, the storage module includes:
存储器和/或缓存池。Storage and / or cache pool.
本申请第四方面提供另一种数据处理方法，该方法包括：The fourth aspect of the present application provides another data processing method, which includes:
响应解密解压指令，从存储模块中读取待处理数据；In response to the decryption and decompression instruction, read the data to be processed from the storage module;
对待处理数据进行解密，得到解密数据；Decrypt the data to be processed to obtain the decrypted data;
对解密数据进行解压，得到解压数据；Decompress the decrypted data to obtain the decompressed data;
将解压数据写入存储模块中。Write the decompressed data to the storage module.
在一些可能的设计中，解密解压指令中携带有待处理数据的首位地址和数据长度；In some possible designs, the first address and data length of the data to be processed are carried in the decryption and decompression instructions;
从存储模块中读取待处理数据包括：Reading the data to be processed from the storage module includes:
根据待处理数据的首位地址和数据长度从存储模块中读取待处理数据。Read the data to be processed from the storage module according to the first address and data length of the data to be processed.
在一些可能的设计中，对待处理数据进行解密包括：In some possible designs, decrypting the data to be processed includes:
从存储模块中读取解密密钥，利用解密密钥对待处理数据进行解密。The decryption key is read from the storage module, and the decrypted key is used to decrypt the processed data.
在一些可能的设计中，该方法还包括：In some possible designs, the method also includes:
将待处理数据的格式由原格式变换为预设格式，预设格式为与解密组件匹配的格式。The format of the data to be processed is changed from the original format to a preset format, and the preset format is a format that matches the decryption component.
在一些可能的设计中，该方法还包括：In some possible designs, the method also includes:
将解压数据的格式变换为原格式。Convert the format of the decompressed data to the original format.
在一些可能的设计中，存储模块包括：In some possible designs, the storage module includes:
存储器和/或缓存池。Storage and / or cache pool.
本申请第五方面提供一种数据处理设备，该设备包括：处理器和存储器；A fifth aspect of the present application provides a data processing device, which includes: a processor and a memory;
存储器，用于存储指令；Memory for storing instructions;
处理器，用于执行存储器中的指令，执行本申请第三方面提供的数据处理方法。The processor is configured to execute an instruction in the memory and execute the data processing method provided by the third aspect of the present application.
本申请第六方面提供另一种数据处理设备，该设备包括：处理器和存储器；A sixth aspect of the present application provides another data processing device, which includes: a processor and a memory;
存储器，用于存储指令；Memory for storing instructions;
处理器，用于执行存储器中的指令，执行本申请第四方面提供的数据处理方法。The processor is configured to execute instructions in the memory and execute the data processing method provided by the fourth aspect of the present application.
本申请第七方面提供一种计算机可读存储介质，包括指令，当其在计算机上运行时，使得计算机执行本申请第三方面提供的数据处理方法。A seventh aspect of the present application provides a computer-readable storage medium including instructions that, when run on a computer, cause the computer to execute the data processing method provided by the third aspect of the present application.
本申请第八方面提供另一种计算机可读存储介质，包括指令，当其在计算机上运行时，使得计算机执行本申请第四方面提供的数据处理方法。An eighth aspect of the present application provides another computer-readable storage medium, including instructions, which, when run on a computer, cause the computer to execute the data processing method provided by the fourth aspect of the present application.
本申请第九方面提供一种包含指令的计算机程序产品，当其在计算机上运行时，使得计算机执行本申请第三方面提供的数据处理方法。A ninth aspect of the present application provides a computer program product containing instructions, which when executed on a computer, causes the computer to execute the data processing method provided by the third aspect of the present application.
本申请第十方面提供另一种包含指令的计算机程序产品，当其在计算机上运行时，使得计算机执行本申请第四方面提供的数据处理方法。The tenth aspect of the present application provides another computer program product containing instructions that, when run on a computer, causes the computer to execute the data processing method provided by the fourth aspect of the present application.
从以上技术方案可以看出，本申请实施例具有以下优点：It can be seen from the above technical solutions that the embodiments of the present application have the following advantages:
本申请实施例提供的一种数据处理装置及方法中，数据处理装置包括：数据输入逻辑、控制逻辑、压缩组件、加密组件和数据输出逻辑，控制逻辑与数据输入逻辑、压缩组件、加密组件和数据输出逻辑连接，数据输入逻辑与压缩组件连接，压缩组件与加密组件连接，加密组件与数据输出逻辑连接，通过控制逻辑响应压缩加密指令，控制数据输入逻辑从存储模块中读取待处理数据，并将待处理数据输入到压缩组件，控制压缩组件对待处理数据进行压缩，得到压缩数据，控制加密组件对压缩数据进行加密，得到加密数据，控住数据输出逻辑将加密数据写入存储模块。基于以上数据处理装置，在压缩加密的过程中，在对 待处理数据进行压缩后，无需将压缩数据存储至存储器中，在对压缩数据进行加密前，无需从存储器中读取压缩数据，整个过程中只需对存储器执行一次读取和一次写入即可，而不必进行多次读取和写入，相比于传统技术，提高了压缩加密的效率，降低了系统带宽。In a data processing device and method provided in the embodiments of the present application, the data processing device includes: data input logic, control logic, compression component, encryption component, and data output logic, control logic and data input logic, compression component, encryption component, and The data output logic is connected, the data input logic is connected with the compression component, the compression component is connected with the encryption component, the encryption component is connected with the data output logic, and the control logic responds to the compression encryption instruction, and the control data input logic reads the data to be processed from the storage module. The data to be processed is input to the compression component, the compression component is controlled to compress the processed data to obtain compressed data, and the encryption component is controlled to encrypt the compressed data to obtain encrypted data, and the data output logic is controlled to write the encrypted data to the storage module. Based on the above data processing device, in the process of compression and encryption, after the data to be processed is compressed, there is no need to store the compressed data in the memory. Before the compressed data is encrypted, there is no need to read the compressed data from the memory. You only need to perform one read and one write to the memory, without having to perform multiple reads and writes. Compared with traditional technologies, it improves the efficiency of compression and encryption and reduces the system bandwidth.
本申请实施例提供的另一种数据处理装置及方法中，数据处理装置包括：数据输入逻辑、控制逻辑、解密组件、解压组件和数据输出逻辑；控制逻辑分别与数据输入逻辑、解密组件、解压组件和数据输出逻辑连接，数据输入逻辑与解密组件连接，解密组件与解压组件连接，解压组件与数据输出逻辑连接；其中，控制逻辑，用于响应解密解压指令，控制数据输入逻辑从存储模块中读取待处理数据，并将待处理数据输入到解密组件；控制解密组件对待处理数据进行解密，得到解密数据；控制解压组件对解密数据进行解压，得到解压数据；以及控制数据输出逻辑将解压数据写入存储模块。基于以上数据处理装置，在解密解压的过程中，在对待处理数据进行解密后，无需将解密数据写入存储模块中，在对解密数据进行解压之前，也无需从存储模块中读取解密数据，整个过程只需对存储模块执行一次读取和一次写入即可，而不必进行多次读取和写入，相比于传统技术，提高了解密解压的效率，降低了系统带宽。In another data processing device and method provided by the embodiments of the present application, the data processing device includes: data input logic, control logic, decryption component, decompression component, and data output logic; the control logic is separately from the data input logic, decryption component, and decompression The component is connected to the data output logic, the data input logic is connected to the decryption component, the decryption component is connected to the decompression component, and the decompression component is connected to the data output logic; among them, the control logic is used to respond to the decryption and decompression instruction and control the data input logic from the storage module Read the data to be processed and input the data to be decrypted; control the decryption component to decrypt the processed data to obtain the decrypted data; control the decompression component to decompress the decrypted data to obtain the decompressed data; and control the data output logic to decompress the data Write to memory module. Based on the above data processing device, in the process of decryption and decompression, after the data to be processed is decrypted, the decrypted data does not need to be written into the storage module, and the decrypted data does not need to be read from the storage module before the decrypted data is decompressed. The entire process only needs to perform one read and one write to the storage module, without having to perform multiple reads and writes. Compared with traditional technologies, it improves the efficiency of decryption and decompression and reduces the system bandwidth.
附图说明BRIEF DESCRIPTION OF THE DRAWINGS
图1为现有技术提供的一种数据处理流程示意图；FIG. 1 is a schematic diagram of a data processing process provided by the prior art; FIG.
图2为本申请实施例提供的一种数据处理流程示意图；FIG. 2 is a schematic diagram of a data processing process according to an embodiment of the present application; FIG.
图3为本申请实施例提供的一种数据处理装置的结构框图；3 is a structural block diagram of a data processing apparatus according to an embodiment of the present application;
图4为本申请实施例提供的一种数据处理示意图；4 is a schematic diagram of data processing provided by an embodiment of the present application;
图5为本申请实施例提供的一种数据处理场景示意图；5 is a schematic diagram of a data processing scenario provided by an embodiment of the present application;
图6为本申请实施例提供的另一种数据处理装置的结构框图；6 is a structural block diagram of another data processing apparatus according to an embodiment of the present application;
图7为本申请实施例提供的另一种数据处理场景示意图；7 is a schematic diagram of another data processing scenario according to an embodiment of the present application;
图8为本申请实施例提供的一种数据处理方法的流程图；8 is a flowchart of a data processing method according to an embodiment of the present application;
图9为本申请实施例提供的另一种数据处理方法的流程图；9 is a flowchart of another data processing method according to an embodiment of the present application;
图10为本申请实施例提供的一种数据处理设备的结构框图。FIG. 10 is a structural block diagram of a data processing device according to an embodiment of the present application.
具体实施方式Detailed ways
本申请实施例提供了一种数据处理装置及方法，用于提高压缩加密和解密解压的效率，降低系统带宽。The embodiments of the present application provide a data processing device and method, which are used to improve the efficiency of compression encryption, decryption and decompression, and reduce system bandwidth.
本申请的说明书和权利要求书及上述附图中的术语“第一”、“第二”、“第三”、“第四”等(如果存在)是用于区别类似的对象，而不必用于描述特定的顺序或先后次序。应该理解这样使用的数据在适当情况下可以互换，以便这里描述的实施例能够以除了在这里图示或描述的内容以外的顺序实施。此外，术语“包括”和“具有”以及他们的任何变形，意图在于覆盖不排他的包含，例如，包含了一系列步骤或单元的过程、方法、系统、产品或设备不必限于清楚地列出的那些步骤或单元，而是可包括没有清楚地列出的或对于这些过程、方法、产品或设备固有的其它步骤或单元。The terms "first", "second", "third", "fourth", etc. (if present) in the description and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and do not need to be used. Used to describe a specific order or sequence. It should be understood that the data used in this way are interchangeable where appropriate, so that the embodiments described herein can be implemented in an order other than what is illustrated or described herein. Furthermore, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product, or device that contains a series of steps or units need not be limited to those explicitly listed Those steps or units may instead include other steps or units not explicitly listed or inherent to these processes, methods, products or equipment.
当前对于数据的处理，一般通过处理器先对数据进行压缩加密，再将经过压缩加密的数据存储至存储模块；在需要使用数据时，对于数据的处理，通常是由处理器从存储模块中读取经过压缩加密的数据，并对经过压缩加密的数据进行解密解压，得到原数据。At present, for data processing, the processor generally compresses and encrypts the data first, and then stores the compressed and encrypted data to the storage module. When data is needed, the processing of the data is usually read by the processor from the storage module. Take the compressed and encrypted data, and decrypt and decompress the compressed and encrypted data to obtain the original data.
其中，存储模块是用来保存数据的设备，在本申请实施例中，可以用来存储待处理的数据和/或处理后的数据。存储模块可以包括易失性存储器，例如非挥发性动态随机存取内存(nonvolatile random access memory，NVRAM)、相变化随机存取内存(phase change random access memory，PRAM)、磁阻式随机存取内存(magetoresistive random access memory，MRAM)等，例如双倍速率同步动态随机存储器(double data rate synchronous dynamic random access memory，DDR SDRAM)，还可以包括非易失性存储器，例如至少一个磁盘存储器件、闪存器件，例如机械硬盘(hard disk drive，HDD)或固态硬盘(solid state driver，SSD)、反或闪存(NOR flash memory)或是反及闪存(NAND flash memory)。The storage module is a device for storing data. In the embodiment of the present application, the storage module may be used to store data to be processed and / or processed data. The memory module may include volatile memory, such as nonvolatile dynamic random access memory (NVRAM), phase change random access memory (PRAM), magnetoresistive random access memory (magetoresistive random access memory (MRAM)), such as double-rate synchronous dynamic random access memory (DDR) synchronous random random access memory (DDR SDRAM), may also include non-volatile memory, such as at least one disk storage device, flash memory device For example, a hard disk drive (HDD) or a solid state drive (SSD), a NOR flash memory, or a NAND flash memory.
处理器是具有处理指令、执行操作、处理数据等功能的设备，在本申请实施例中，用于从存储模块中读取数据，并对数据进行处理，例如可以对数据进行压缩、解压、加密和解密等操作中的至少一种操作。处理器可以是中央处理器(central processing unit，CPU)、通用处理器、数字信号处理器(digital signal processor，DSP)、专用集成电路(application specific integrated circuit，ASIC)，现场可编程门阵列(field programmable gate array，FPGA)或者其他可编程逻辑器件、晶体管逻辑器件，硬件部件或者其任意组合，也可以是实现计算功能的组合，例如包含一个或多个微处理器组合，DSP和微处理器的组合等。The processor is a device having functions such as processing instructions, performing operations, and processing data. In the embodiment of the present application, the processor is used to read data from a storage module and process the data. For example, the data can be compressed, decompressed, and encrypted. And decryption operations. The processor may be a central processing unit (CPU), a general purpose processor, a digital signal processor (DSP), an application specific integrated circuit (ASIC), or a field programmable gate array (field) programmable gate array (FPGA) or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof, or a combination that implements computing functions, such as a combination of one or more microprocessors, a DSP and a microprocessor Combination, etc.
当前对数据进行处理的处理器，可以包括压缩组件和加密组件，通过压缩组件实现对数据的压缩，通过加密组件实现对数据的加密。然而，压缩组件和加密组件相互独立，彼此之间不能进行数据传输，因此，对于数据压缩和数据加密的两个流程，处理器需要分别针对每个流程，从存储模块中读取数据，并分别将经过两个流程处理后的数据写入存储模块。在图1中，参考图1(a)所示，图1(a)为当前实现压缩加密过程中处理器和存储模块的交互示意图。处理器在压缩加密过程中需要进行如下步骤：1)从存储模块中读取待处理数据，对待处理数据进行压缩；2)将压缩后的数据写入存储模块；3)从存储模块中读取压缩后的数据，对压缩后的数据进行加密；4)将加密后的数据写入存储模块中。这样，处理器需要对存储模块进行两次读取和两次写入，即处理器需要与存储模块进行四次交互，所以数据进行压缩加密的效率较低，且增加了系统带宽。A processor that currently processes data may include a compression component and an encryption component. The compression component implements data compression, and the encryption component implements data encryption. However, the compression component and the encryption component are independent of each other and cannot transmit data to each other. Therefore, for the two processes of data compression and data encryption, the processor needs to read data from the storage module for each process separately, and separately Write the data processed by the two processes to the storage module. In FIG. 1, referring to FIG. 1 (a), FIG. 1 (a) is a schematic diagram of interaction between a processor and a storage module in a current compression and encryption process. The processor needs to perform the following steps in the compression and encryption process: 1) read the data to be processed from the storage module, and compress the data to be processed; 2) write the compressed data to the storage module; 3) read from the storage module The compressed data is encrypted; 4) the encrypted data is written into the storage module. In this way, the processor needs to read and write the storage module twice, that is, the processor needs to interact with the storage module four times, so the efficiency of data compression and encryption is low, and the system bandwidth is increased.
相应的，当前对数据进行处理的处理器，还可以包括解密组件和解压组件，通过解密组件实现对数据的解密，通过解压组件实现对数据的解压。然而，解密组件和解压组件相互独立，彼此之间不能进行数据传输，因此，对于数据解密和数据解压两个流程，处理器同样需要分别针对每个流程，从存储模块中读取数据，并分别将经过两个流程处理后的数据写入存储模块。在图1中，参考图1(b)所示，图1(b)为当前实现解密解压过程中处理器和存储模块的交互示意图。处理器在解密解压过程中需要进行如下步骤：1)从存储模块中读取待处理数据，对待处理数据进行解密；2)将解密后的数据写入存储模块；3)从存储模块中读取解密后的数据，对解密后的数据进行解压；4)将解压后的数据写入存储模块中。这样处理器需要对存储模块进行两次读取和两次写入，即处理器需要与存储模块进 行四次交互，所以数据进行解密解压的效率较低，且增加了系统带宽。Accordingly, the current processor for processing data may further include a decryption component and a decompression component. The decryption component is used to decrypt the data, and the decompression component is used to decompress the data. However, the decryption component and the decompression component are independent of each other and cannot transmit data to each other. Therefore, for the two processes of data decryption and data decompression, the processor also needs to read data from the storage module for each process separately, and separately Write the data processed by the two processes to the storage module. In FIG. 1, referring to FIG. 1 (b), FIG. 1 (b) is a schematic diagram of the interaction between the processor and the storage module in the current process of decryption and decompression. The processor needs to perform the following steps during the decryption and decompression process: 1) read the data to be processed from the storage module and decrypt the data to be processed; 2) write the decrypted data to the storage module; 3) read from the storage module Decrypt the data, and decompress the decrypted data; 4) Write the decompressed data to the storage module. In this way, the processor needs to read and write the storage module twice, that is, the processor needs to interact with the storage module four times, so the efficiency of decrypting and decompressing the data is low, and the system bandwidth is increased.
为了解决上述问题，本申请实施例提供了一种数据处理装置，在该装置中，将压缩组件和加密组件进行桥接，使压缩组件和加密组件之间可以进行数据传输，压缩组件可以将压缩后的压缩数据传输至加密组件，通过加密组件实现对压缩数据的加密，因此在压缩加密过程中，处理器只需对存储模块进行一次读取和一次写入即可。在图2中，参考图2(a)所示，图2(a)为本申请实施例提供的数据处理方法中实现压缩加密时处理器和存储模块的交互示意图。处理器响应压缩加密指令，进行如下步骤：1)从存储模块中读取待处理数据，对待处理数据进行压缩，得到压缩数据，再对压缩数据进行加密，得到加密数据；2)将加密数据写入存储模块。本申请实施例中，实现压缩加密时，处理器只需要对存储模块进行一次读取和一次写入，即处理器只要与存储模块进行两次交互，相比于传统技术而言，处理器与存储模块的交互次数减少了一半，从而提高了压缩加密的效率，降低了一半的系统带宽。In order to solve the above problems, an embodiment of the present application provides a data processing device. In the device, a compression component and an encryption component are bridged, so that data transmission can be performed between the compression component and the encryption component, and the compression component can The compressed data is transmitted to the encryption component, and the encrypted data is encrypted by the encryption component. Therefore, during the compression and encryption process, the processor only needs to read and write the storage module once. In FIG. 2, referring to FIG. 2 (a), FIG. 2 (a) is a schematic diagram of interaction between a processor and a storage module when implementing compression and encryption in a data processing method provided by an embodiment of the present application. The processor responds to the compression and encryption instruction and performs the following steps: 1) reads the data to be processed from the storage module, compresses the data to be processed to obtain compressed data, and then encrypts the compressed data to obtain encrypted data; 2) writes the encrypted data Into the storage module. In the embodiment of the present application, when implementing compression and encryption, the processor only needs to read and write the storage module once, that is, the processor only needs to interact with the storage module twice. Compared with the traditional technology, the processor and The number of interactions of the storage module is reduced by half, thereby improving the efficiency of compression and encryption and reducing the system bandwidth by half.
相应的，本申请实施例提供的另一种数据处理装置中，将解密组件和解压组件进行桥接，使解密组件和解压组件之间可以进行数据传输，解密组件可以将经过解密的解密数据传输至解压组件，通过解压组件实现对解密数据的解压，因此在解密解压过程中，处理器只需对存储模块进行一次读取和一次写入即可。在图2中，参考图2(b)所示，图2(b)为本申请实施例提供的数据处理方法中实现解密解压时处理器和存储模块的交互示意图。处理器响应解密解压指令，进行如下步骤：1)从存储模块中读取待处理数据，对待处理数据进行解密，得到解密数据，再对解密数据进行解压，得到解压数据；2)将解压数据写入存储模块。本申请实施例中，实现解密解压时，处理器只需要进行一次读取和一次写入，即处理器只要与存储模块进行两次交互，相比于传统技术而言，处理器与存储模块的交互次数减少了一半，从而提高了解密解压的效率，降低了一半的系统带宽。Correspondingly, in another data processing device provided by the embodiment of the present application, the decryption component and the decompression component are bridged, so that data transmission can be performed between the decryption component and the decompression component, and the decryption component can transmit the decrypted decrypted data to The decompression component realizes the decompression of the decrypted data through the decompression component. Therefore, during the decryption and decompression process, the processor only needs to read and write the storage module once. In FIG. 2, referring to FIG. 2 (b), FIG. 2 (b) is a schematic diagram of an interaction between a processor and a storage module when decryption and decompression are implemented in a data processing method provided by an embodiment of the present application. The processor responds to the decryption and decompression instruction and performs the following steps: 1) reads the data to be processed from the storage module, decrypts the data to be processed, obtains the decrypted data, and decompresses the decrypted data to obtain the decompressed data; 2) writes the decompressed data Into the storage module. In the embodiment of the present application, when the decryption and decompression are implemented, the processor only needs to read and write once, that is, the processor only needs to interact with the storage module twice. Compared with the conventional technology, the processor and the storage module The number of interactions is reduced by half, which improves the efficiency of decryption and decompression, and reduces the system bandwidth by half.
参考图3所示，为本申请实施例提供的一种数据处理装置的结构框图，该装置包括：数据输入逻辑120、控制逻辑110、压缩组件130、加密组件140和数据输出逻辑150。其中，控制逻辑110分别与数据输入逻辑120、压缩组件130、加密组件140和数据输出逻辑150连接，数据输入逻辑120与压缩组件130连接，压缩组件130与加密组件140连接，加密组件140与数据输出逻辑150连接。Referring to FIG. 3, a structural block diagram of a data processing apparatus according to an embodiment of the present application is provided. The apparatus includes: a data input logic 120, a control logic 110, a compression component 130, an encryption component 140, and a data output logic 150. The control logic 110 is connected to the data input logic 120, the compression component 130, the encryption component 140, and the data output logic 150, the data input logic 120 is connected to the compression component 130, the compression component 130 is connected to the encryption component 140, and the encryption component 140 is connected to the data. Output logic 150 is connected.
控制逻辑110用于响应压缩加密指令，对数据输入逻辑120、压缩组件130、加密组件140和数据输出逻辑150进行控制。压缩加密指令是用于指示对存储模块100中存储的待处理数据进行压缩加密的指令。在操作系统层面，压缩加密指令可以通过APP发送，调用相关组件来执行压缩加密指令，例如驱动层的内核(Kernel)。The control logic 110 is used to control the data input logic 120, the compression component 130, the encryption component 140, and the data output logic 150 in response to the compression encryption instruction. The compression and encryption instruction is an instruction for instructing compression and encryption of the data to be processed stored in the storage module 100. At the operating system level, compression and encryption instructions can be sent through the APP and call related components to execute compression and encryption instructions, such as the kernel (Kernel) of the driver layer.
响应于压缩加密指令，控制逻辑110可以控制数据输入逻辑120从存储模块100中读取待处理数据，实际操作中，可以调用Zlib APP接口从存储模块100中读取待处理数据。In response to the compression and encryption instruction, the control logic 110 may control the data input logic 120 to read the data to be processed from the storage module 100. In actual operation, the Zlib APP interface may be called to read the data to be processed from the storage module 100.
存储模块100可以包括存储器和/或缓存池，其中存储器中存储有待处理数据和程序代码等，缓存池中可以存储有待处理数据、压缩数据和加密数据等，以提高数据读取速度和写入速度。The storage module 100 may include a memory and / or a cache pool, in which the pending data and program code are stored in the memory, and the cache pool may store the to-be-processed data, compressed data, encrypted data, etc. to improve data reading speed and writing speed. .
该过程中，待处理数据可以是存储模块100中存储的未经压缩加密的原始数据，待处理数据可以是单个文件，也可以是多个文件，还可以是包括一个或多个文件的文件夹等。待处理数据的文件类型可以是文本、表格、邮件、数据库数据和应用程序等类型的至少一种。待处理数据的文件格式与待处理数据的文件类型相对应，例如文本数据的格式可以是文档(doc)、文本文件(txt)等，表格数据的格式可以是xls等，邮件数据的格式可以是eml、msg等，数据库数据的格式可以是db(database)等，应用程序数据的格式可以是可执行程序(executable program，EXE)等。In this process, the data to be processed may be uncompressed and encrypted raw data stored in the storage module 100. The data to be processed may be a single file, multiple files, or a folder including one or more files. Wait. The file type of the data to be processed may be at least one of text, form, email, database data, and application types. The file format of the data to be processed corresponds to the file type of the data to be processed. For example, the format of text data can be documents (doc), text files (txt), etc., the format of table data can be xls, and the format of mail data can be eml, msg, etc., the format of database data can be db (database), etc., and the format of application data can be executable program (executable program, EXE).
待处理数据也可以是存储模块100中存储的经过压缩和/或加密、且需要进一步压缩和加密的数据，例如待处理数据为音频文件时，可以为动态影像专家压缩标准音频层面3(moving picture experts group audio layer III，mp3)等经过压缩的数据的格式，待处理数据也可以是其他经过压缩或加密的数据，且具有其他数据格式。The data to be processed may also be compressed and / or encrypted data stored in the storage module 100 and further compressed and encrypted. For example, when the data to be processed is an audio file, a standard audio layer 3 (moving picture) may be compressed for a motion picture expert. Experts group audio layer III, mp3) and other compressed data formats, the data to be processed may also be other compressed or encrypted data, and have other data formats.
在压缩加密指令中，可以携带有待处理数据的首位地址和数据长度，此时，数据输入逻辑120可以根据待处理数据的首位地址和数据长度从存储模块100中读取待处理数据。在待处理数据为多个文件时，压缩加密指令中还可以携带有多个首位地址和与首位地址对应的多个数据长度，以便数据输入逻辑120读取待处理数据。The compressed encryption instruction may carry the first address and data length of the data to be processed. At this time, the data input logic 120 may read the data to be processed from the storage module 100 according to the first address and data length of the data to be processed. When the data to be processed is multiple files, the compression encryption instruction may also carry multiple first address and multiple data lengths corresponding to the first address, so that the data input logic 120 reads the data to be processed.
在本申请实施例中，若读取的待处理数据的长度较大时，可选的，数据处理装置还可以包括分片逻辑，对待处理数据进行分片处理，即将待处理数据切分为多个数据段，每个数据段为一个分片数据。参考图4所示，在对待处理数据进行分片之后，可以得到多个分片数据，每个分片数据的长度均较小，这样就可以同时对多个分片数据进行压缩加密，提高压缩加密的效率。In the embodiment of the present application, if the length of the data to be processed is large, optionally, the data processing device may further include fragmentation logic to perform fragmentation processing on the data to be processed, that is, to divide the data to be processed into multiple data. Data segments, each of which is a piece of fragmented data. Referring to FIG. 4, after sharding the data to be processed, multiple shard data can be obtained, and the length of each shard data is small, so that multiple shard data can be compressed and encrypted at the same time to improve compression. Encryption efficiency.
在数据输入逻辑120读取待处理数据后，控制逻辑110可以控制数据输入逻辑120将读取的待处理数据输入至压缩组件130，以便压缩组件130对待处理数据进行压缩处理。After the data input logic 120 reads the data to be processed, the control logic 110 may control the data input logic 120 to input the read data to be processed to the compression component 130 so that the compression component 130 performs compression processing on the data to be processed.
在本申请实施例提供的数据处理装置中，还可以包括第一格式变换逻辑，第一格式变换逻辑连接在数据输入逻辑120和压缩组件130之间，用于对数据输入逻辑120输出的待处理数据进行格式处理，将待处理数据的格式由原格式变换为预设格式，预设格式为与压缩组件130匹配的格式。例如可以通过对每个待处理数据加入保护信息来保护数据完整性。The data processing apparatus provided in the embodiment of the present application may further include a first format conversion logic. The first format conversion logic is connected between the data input logic 120 and the compression component 130, and is used for processing the data output logic 120 output to be processed. The data is formatted, and the format of the data to be processed is converted from the original format into a preset format, and the preset format is a format that matches the compression component 130. For example, data integrity can be protected by adding protection information to each pending data.
控制逻辑110可以控制第一格式变换逻辑对待处理数据进行格式变换，并在格式变换后，将处理后的预设格式的待处理数据发送至压缩组件130。The control logic 110 may control the first format conversion logic to perform format conversion on the data to be processed, and after the format conversion, send the processed data in a preset format to the compression component 130.
在压缩组件130获取到待处理数据后，控制逻辑110控制压缩组件130对待处理数据按照压缩算法进行压缩处理，其中，数据压缩是按照压缩算法对数据进行处理，用比未经压缩的数据更少的比特来表示信息的过程。After the compression component 130 obtains the data to be processed, the control logic 110 controls the compression component 130 to perform compression processing on the data to be processed according to a compression algorithm. Among them, the data compression is to process the data according to the compression algorithm, using less than the uncompressed data. Bits to represent the process of information.
常见的压缩算法例如LZ(Lempel-Ziv)77算法或哈尔曼(Huffman)算法等，具体的，压缩加密指令中可以携带压缩算法的标识，压缩组件130根据压缩算法的标识选择对应的解压算法。对待处理数据进行压缩处理，形成压缩数据，压缩数据的格式例如可以是ZIP、ZLIB等。例如对文本、表格、邮件、数据库数据、应用程序等数据进行压缩，可以得到ZIP格式的压缩数据。Common compression algorithms, such as LZ (Lempel-Ziv) 77 algorithm or Huffman algorithm, etc. Specifically, the compression encryption instruction can carry the identification of the compression algorithm, and the compression component 130 selects the corresponding decompression algorithm according to the identification of the compression algorithm. . The data to be processed is compressed to form compressed data, and the format of the compressed data may be, for example, ZIP, ZLIB, and the like. For example, compressing data such as text, tables, emails, database data, and application programs can obtain compressed data in ZIP format.
在对待处理数据进行压缩后，参考图4所示，压缩组件130还可以为压缩后的数据增 加压缩头，形成压缩数据，压缩头中可以包括对压缩数据校验的校验结果，以便通过压缩头的校验结果实现对压缩数据的错误检测，压缩头中还可以包括对数据的描述。After compressing the data to be processed, referring to FIG. 4, the compression component 130 may further add a compression header to the compressed data to form compressed data. The compression header may include a verification result of the compressed data verification in order to pass the compression. The verification result of the header realizes the error detection of the compressed data. The compressed header may also include a description of the data.
在对待处理数据进行压缩后，控制逻辑110可以控制压缩组件130将压缩后的压缩数据传输至加密组件140，以便加密组件140通过加密算法对压缩数据进行加密。After the data to be processed is compressed, the control logic 110 may control the compression component 130 to transmit the compressed compressed data to the encryption component 140 so that the encryption component 140 encrypts the compressed data by using an encryption algorithm.
加密组件140在获取到压缩数据后，控制逻辑110可以控制加密组件140通过加密算法对压缩数据进行加密。其中数据加密就是对原来为明文的数据按照某种加密算法进行处理，使其成为不可读的密文，合法接收者之外的其他人或设备较难获取到加密前的为明文的数据，数据传输和存储的安全性随之提高。After the encryption component 140 obtains the compressed data, the control logic 110 may control the encryption component 140 to encrypt the compressed data by using an encryption algorithm. The data encryption is to process the original data in plain text according to some encryption algorithm to make it unreadable cipher text. It is difficult for someone or a device other than a legitimate receiver to obtain the data in plain text before encryption. The security of transmission and storage is increased.
常见的加密算法例如数据加密标准(data encryption standard，DES)、三重数据加密算法(triple DES，3DES)、信息-摘要算法5(message-digest algorithm 5，MD5)、RSA(Rivest-Shamir-Adleman)算法等。在压缩加密指令中，可以携带的加密算法的标识，加密组件140可以根据加密算法的标识选择对应的加密算法。Common encryption algorithms such as data encryption standard (DES), triple data encryption algorithm (triple DES, 3DES), message-digest algorithm 5, MD5, and RSA (Rivest-Shamir-Adleman) Algorithms, etc. In the compressed encryption instruction, the identifier of the encryption algorithm that can be carried, and the encryption component 140 can select a corresponding encryption algorithm according to the identifier of the encryption algorithm.
实际操作中，可以通过加密密钥的方式来对压缩数据进行加密，密钥可以是对称密钥，即可以使用同一个密钥进行加密和解密，密钥也可以是非对称密钥，即可以通过公开密钥进行加密，通过私用密钥去解密。加密密钥可以存储于存储模块100中，在压缩加密指令中还可以携带有加密密钥的存储位置。在本申请实施例中，数据处理装置还包括密钥输入逻辑，密钥输入逻辑与加密组件140和控制逻辑110连接，用于根据加密密钥的存储位置，从存储模块100中读取加密密钥。具体的，控制逻辑110控制密钥输入逻辑从存储模块100中读取加密密钥，以便加密组件140利用密钥输入逻辑读取的加密密钥对压缩数据进行加密。In practice, the compressed data can be encrypted by using an encryption key. The key can be a symmetric key, that is, the same key can be used for encryption and decryption. The key can also be an asymmetric key, which can be obtained by The public key is encrypted and decrypted by the private key. The encryption key may be stored in the storage module 100, and the storage location of the encryption key may also be carried in the compressed encryption instruction. In the embodiment of the present application, the data processing device further includes a key input logic. The key input logic is connected to the encryption component 140 and the control logic 110, and is configured to read the encryption key from the storage module 100 according to the storage location of the encryption key. key. Specifically, the control logic 110 controls the key input logic to read the encryption key from the storage module 100, so that the encryption component 140 uses the encryption key read by the key input logic to encrypt the compressed data.
在本申请实施例中，已经完成压缩的压缩数据可以不写入存储模块100中，压缩组件130可以向加密组件140发送压缩数据，以便加密组件140对压缩数据进行加密。在此基础上，为了进一步提高压缩加密的效率，在对读取的待处理数据中的部分待处理数据进行压缩后，控制逻辑110还可以判断已经完成压缩的压缩数据的数据量是否大于或等于第一阈值，第一阈值是可以进行加密的最低的数据量，例如可以是16字节(Byte)。若压缩数据的数据量大于或等于第一阈值，则控制逻辑110可以控制加密组件140对已经完成压缩的压缩数据进行加密，得到加密数据。In the embodiment of the present application, the compressed data that has been compressed may not be written into the storage module 100, and the compression component 130 may send the compressed data to the encryption component 140, so that the encryption component 140 encrypts the compressed data. On this basis, in order to further improve the efficiency of compression and encryption, after compressing a part of the read data to be processed, the control logic 110 can also determine whether the data amount of the compressed data that has been compressed is greater than or equal to The first threshold value is the lowest amount of data that can be encrypted, and may be, for example, 16 bytes. If the data amount of the compressed data is greater than or equal to the first threshold, the control logic 110 may control the encryption component 140 to encrypt the compressed data that has been compressed to obtain encrypted data.
也就是说，本申请实施例中，可以在有部分待处理数据完成压缩后，通过加密组件140对形成的压缩数据进行加密，从而实现压缩组件130的压缩操作和加密组件140的加密操作的并行处理，而不必等到所有的待处理数据均完成压缩过程后再进行压缩数据的加密。举例来说，若当前已有第一部分的待处理数据完成压缩，形成第一部分的压缩数据，若第一部分的压缩数据的数据量为16Byte，达到了第一阈值16Byte，则可以对第一部分的压缩数据进行加密处理，在此同时，第二部分的待处理数据正在进行压缩处理，对第一部分的压缩数据进行加密和对第二部分的待处理数据进行压缩可以同步进行，实现压缩加密的效率的进一步提升。That is, in the embodiment of the present application, after compression of a part of the data to be processed is completed, the formed compressed data is encrypted by the encryption component 140, so that the compression operation of the compression component 130 and the encryption operation of the encryption component 140 are performed in parallel. Processing without having to wait until all the pending data has completed the compression process before encrypting the compressed data. For example, if the first part of the data to be processed has been compressed to form the first part of the compressed data, if the amount of data in the first part of the compressed data is 16 Bytes and reaches the first threshold of 16 Bytes, the first part of the compression can be compressed. The data is encrypted. At the same time, the second part of the data to be processed is being compressed. Encryption of the first part of the compressed data and compression of the second part of the data to be processed can be performed simultaneously to achieve the efficiency of compression and encryption. Further improvement.
在对压缩数据完成加密后，形成加密数据，加密组件140将加密数据传输至数据输出逻辑150。控制逻辑110控制数据输出逻辑150将形成的加密数据写入存储模块100，加密 数据的写入位置可以是原来待处理数据的存储位置，也可以是其他未被使用的存储位置。实际操作中，可以通过调用开放式安全套接层协议(Open Secure Sockets Layer，OpenSLL)接口将加密后的数据写入存储模块100。After the compressed data is encrypted, encrypted data is formed, and the encryption component 140 transmits the encrypted data to the data output logic 150. The control logic 110 controls the data output logic 150 to write the formed encrypted data to the storage module 100, and the writing position of the encrypted data may be the storage position of the original data to be processed or other unused storage positions. In actual operation, the encrypted data can be written into the storage module 100 by calling an Open Secure Sockets Layer Protocol (Open Secure Sockets Layer) (OpenSLL) interface.
在本申请实施例提供的数据处理装置中，还可以包括第二格式变换逻辑，其中，第二格式变换逻辑连接在数据输出逻辑150和加密组件140之间，在对压缩数据完成加密后，控制逻辑110还可以控制加密组件140将加密数据传输至第二格式变换逻辑，通过控制逻辑110控制第二格式变换逻辑将加密数据的格式变换为原格式，以便在需要该加密数据时能够方便解密和解压，并将变换后的加密数据传输至数据输出模块，以便数据输出模块将原格式的加密数据写入存储模块100。The data processing apparatus provided in the embodiment of the present application may further include a second format conversion logic, where the second format conversion logic is connected between the data output logic 150 and the encryption component 140, and after the compressed data is encrypted, the control The logic 110 can also control the encryption component 140 to transmit the encrypted data to the second format conversion logic. The control logic 110 controls the second format conversion logic to convert the format of the encrypted data into the original format, so that when the encrypted data is needed, it can be easily decrypted and Decompress and transmit the transformed encrypted data to the data output module, so that the data output module writes the encrypted data in the original format into the storage module 100.
为了便于理解，参考图5所示，结合具体场景，对本申请实施例提供的数据处理装置进行说明。具体的，数据处理装置中的模块可以按照如下步骤执行：For ease of understanding, the data processing device provided in the embodiment of the present application will be described with reference to FIG. 5 in combination with a specific scenario. Specifically, the modules in the data processing device can be executed according to the following steps:
501)APP向控制逻辑110下发压缩加密指令。501) The APP sends a compression and encryption instruction to the control logic 110.
502)控制逻辑110响应压缩加密指令后，控制数据输入模块120从存储模块中读取待处理数据，并将读取的待处理数据发送给第一格式变换逻辑。502) After the control logic 110 responds to the compression and encryption instruction, the control data input module 120 reads the data to be processed from the storage module, and sends the read data to be processed to the first format conversion logic.
503)控制逻辑110控制第一格式变换逻辑对待处理数据进行格式变换，形成与压缩组件130对应的处理格式，并将变换格式后的待处理数据传输给压缩组件130。503) The control logic 110 controls the first format conversion logic to perform format conversion on the data to be processed to form a processing format corresponding to the compression component 130, and transmits the formatted to-be-processed data to the compression component 130.
504)控制逻辑110控制压缩组件130根据压缩算法对待处理数据进行压缩，形成压缩数据，并将压缩数据发送给加密组件140。504) The control logic 110 controls the compression component 130 to compress the data to be processed according to a compression algorithm to form compressed data, and sends the compressed data to the encryption component 140.
505)控制逻辑110控制密钥输入模块从存储模块100中读取加密密钥，并将读取的加密密钥发送给加密组件140。505) The control logic 110 controls the key input module to read the encryption key from the storage module 100, and sends the read encryption key to the encryption component 140.
506)控制逻辑110控制加密组件140根据加密算法，或加密算法和加密密钥对压缩数据进行加密，形成加密数据，将加密数据发送给第二格式交换逻辑。506) The control logic 110 controls the encryption component 140 to encrypt the compressed data according to the encryption algorithm, or the encryption algorithm and the encryption key to form encrypted data, and sends the encrypted data to the second format exchange logic.
507)控制逻辑110控制第二格式变换逻辑对加密数据进行格式变换，将经过格式变换后的加密数据发送给数据输出模块150。507) The control logic 110 controls the second format conversion logic to perform format conversion on the encrypted data, and sends the encrypted data after the format conversion to the data output module 150.
508)控制逻辑110控制数据输出模块150将加密数据写入到存储模块100。508) The control logic 110 controls the data output module 150 to write the encrypted data to the storage module 100.
本申请实施例提供的一种数据处理装置中，数据处理装置包括：数据输入逻辑、控制逻辑、压缩组件、加密组件和数据输出逻辑，控制逻辑与数据输入逻辑、压缩组件、加密组件和数据输出逻辑连接，数据输入逻辑与压缩组件连接，压缩组件与加密组件连接，加密组件与数据输出逻辑连接，通过控制逻辑响应压缩加密指令，控制数据输入逻辑从存储模块中读取待处理数据，并将待处理数据输入到压缩组件，控制压缩组件对待处理数据进行压缩，得到压缩数据，控制加密组件对压缩数据进行加密，得到加密数据，控住数据输出逻辑将加密数据写入存储模块。在此压缩加密的过程中，在对待处理数据进行压缩后，无需将压缩数据存储至存储器中，在对压缩数据进行加密前，无需从存储器中读取压缩数据，整个过程中只需对存储器执行一次读取和一次写入即可，即处理器只要与存储模块进行两次交互，相比于传统技术，处理器与存储模块的交互次数减少了一半，提高了压缩加密的效率，降低了一半的系统带宽。此外，由于压缩组件和加密组件是独立的组件，可以在压缩数据达到一定数据量后，开始进行压缩数据的加密，使压缩和加密可以并行处理， 进一步提高压缩加密的效率，降低系统带宽。In a data processing device provided by an embodiment of the present application, the data processing device includes: data input logic, control logic, compression component, encryption component, and data output logic, control logic and data input logic, compression component, encryption component, and data output Logical connection. Data input logic is connected to the compression component. Compression component is connected to the encryption component. Encryption component is connected to the data output logic. The control logic responds to the compression encryption instruction. The control data input logic reads the data to be processed from the storage module. The data to be processed is input to the compression component, and the compression component is controlled to compress the processed data to obtain compressed data, and the encryption component is controlled to encrypt the compressed data to obtain encrypted data, and the data output logic is controlled to write the encrypted data to the storage module. In this compression and encryption process, after the data to be processed is compressed, there is no need to store the compressed data in the memory. Before the compressed data is encrypted, there is no need to read the compressed data from the memory. The entire process only needs to be performed on the memory. One read and one write is sufficient, that is, the processor only needs to interact with the storage module twice. Compared with the traditional technology, the number of interactions between the processor and the storage module is reduced by half, the efficiency of compression and encryption is improved, and the half is reduced. System bandwidth. In addition, since the compression component and the encryption component are independent components, after the compressed data reaches a certain amount of data, the encryption of the compressed data can be started, so that the compression and encryption can be processed in parallel, which further improves the efficiency of compression encryption and reduces the system bandwidth.
此外，在操作系统层面，传统的数据处理方式中，APP发送压缩指令，调用驱动层的Zlib APP接口从存储模块中读取待处理数据，对待处理数据进行压缩，再将压缩后的数据写入存储模块；然后APP发送加密指令，调用驱动层的OpenSLL接口从存储模块中读取压缩后的数据，对压缩后的数据进行加密，再将加密后的数据写入存储模块。也就是说，APP需要发送两次指令，即一次压缩指令和一次加密指令，分别调用Zlib APP接口和OpenSLL接口，在这个过程中需要对存储模块进行两次读取和两次写入。而本申请实施例提供的数据处理方法中，APP只需要向驱动层发送一次指令，即压缩加密指令，驱动层就可以实现对待处理数据的压缩加密过程。具体的，驱动层首先从存储模块中读取待处理数据并进行压缩，然后直接对压缩后的数据进行加密，在加密之后，将经过压缩加密后的数据写入存储模块中，在这个过程中驱动层只需要对存储模块进行一次读取和一次写入，相比于传统技术，减少了与存储模块的交互次数，提高了压缩加密的效率，降低了系统带宽。In addition, at the operating system level, in the traditional data processing method, the APP sends a compression instruction, calls the driver's Zlib APP interface to read the data to be processed from the storage module, compresses the data to be processed, and then writes the compressed data to The storage module; then the APP sends an encryption instruction, calls the driver's OpenSLL interface to read the compressed data from the storage module, encrypts the compressed data, and writes the encrypted data to the storage module. In other words, the APP needs to send two instructions, namely a compression instruction and an encryption instruction, respectively calling the Zlib APP interface and the OpenSLL interface. In this process, the storage module needs to be read twice and written twice. In the data processing method provided by the embodiment of the present application, the APP only needs to send an instruction to the driver layer, that is, a compression and encryption instruction, and the driver layer can implement the compression and encryption process of the data to be processed. Specifically, the driver layer first reads the data to be processed from the storage module and compresses it, and then directly encrypts the compressed data. After encryption, the compressed and encrypted data is written into the storage module. In this process, The driver layer only needs to read and write to the storage module once. Compared with traditional technologies, it reduces the number of interactions with the storage module, improves the efficiency of compression and encryption, and reduces system bandwidth.
参考图6所示，为本申请实施例提供的另一种数据处理装置的结构框图，该装置包括：数据输入逻辑220、控制逻辑210、解密组件230、解压组件240和数据输出逻辑250。其中，控制逻辑210分别与数据输入逻辑220、解密组件230、解压组件240和数据输出逻辑250连接，数据输入逻辑220与解密组件230连接，解密组件230与解压组件240连接，解压组件240与数据输出逻辑250连接。Referring to FIG. 6, a structural block diagram of another data processing apparatus according to an embodiment of the present application is provided. The apparatus includes: a data input logic 220, a control logic 210, a decryption component 230, a decompression component 240, and a data output logic 250. The control logic 210 is connected to the data input logic 220, the decryption component 230, the decompression component 240, and the data output logic 250, the data input logic 220 is connected to the decryption component 230, the decryption component 230 is connected to the decompression component 240, and the decompression component 240 is connected to the data. Output logic 250 is connected.
控制逻辑210用于响应解密解压指令，对数据输入逻辑220、解密组件230、解压组件240和数据输出逻辑250进行控制。解密解压指令是用于指示对存储模块200中存储的经过压缩加密的待处理数据进行解密解压的指令。类比于压缩加密指令，解密解压指令可以通过APP发送，调用相关组件来执行解密解压指令，例如驱动层的内核(Kernel)。The control logic 210 is used to control the data input logic 220, the decryption component 230, the decompression component 240, and the data output logic 250 in response to the decryption and decompression instruction. The decryption and decompression instruction is an instruction for instructing decryption and decompression of the compressed and encrypted to-be-processed data stored in the storage module 200. Analogous to compressing and encrypting instructions, the decryption and decompression instructions can be sent through the APP, and the relevant components are called to execute the decryption and decompression instructions, such as the kernel of the driver layer (Kernel).
控制逻辑210响应于解密解压指令，可以控制数据输入逻辑220从存储模块200中读取待处理数据，实际操作中，可以调用OpenSSL接口从存储模块200中读取待处理数据。In response to the decryption and decompression instruction, the control logic 210 can control the data input logic 220 to read the data to be processed from the storage module 200. In actual operation, the OpenSSL interface can be called to read the data to be processed from the storage module 200.
存储模块200可以包括存储器和/或缓存池，其中存储器中存储有待处理数据和程序代码等，缓存池中可以存储有待处理数据、压缩数据和加密数据等，以提高数据读取速度和写入速度。The storage module 200 may include a memory and / or a buffer pool, in which the data to be processed and program code are stored in the memory, and the buffer pool may store the data to be processed, compressed data, encrypted data, etc., to improve data read speed and write speed .
该过程中，待处理数据是存储模块200中存储的经过压缩加密的数据，待处理数据可以是单个经过压缩加密的文件，也可以是多个经过压缩加密的文件，还可以是包括一个或多个经过压缩加密的文件的文件夹。待处理数据的文件格式例如可以是ZIP、ZLIB等。In this process, the data to be processed is compressed and encrypted data stored in the storage module 200. The data to be processed may be a single compressed and encrypted file, or multiple compressed and encrypted files, and may include one or more files. A folder of compressed and encrypted files. The file format of the data to be processed may be, for example, ZIP, ZLIB, and the like.
在解密解压指令中，可以携带有待处理数据的首位地址和数据长度，以便数据输入逻辑220根据待处理数据的首位地址和数据长度从存储模块200中读取待处理数据。在待处理数据为多个经过压缩加密的文件时，解密解压指令中还可以携带有多个首位地址和与首位地址对应的多个数据长度。In the decryption and decompression instruction, the first address and data length of the data to be processed may be carried, so that the data input logic 220 reads the data to be processed from the storage module 200 according to the first address and data length of the data to be processed. When the data to be processed is multiple compressed and encrypted files, the decryption and decompression instruction may also carry multiple first address and multiple data lengths corresponding to the first address.
在数据输入逻辑220读取待处理数据后，控制逻辑210控制数据输入逻辑220将读取的待处理数据传输至解密组件230，以便解密组件230对待处理数据进行解密处理。After the data input logic 220 reads the data to be processed, the control logic 210 controls the data input logic 220 to transmit the read data to be processed to the decryption component 230 so that the decryption component 230 performs decryption processing on the data to be processed.
在本申请实施例提供的数据处理装置中，还可以包括第一格式变换逻辑，第一格式变 换逻辑连接在数据输入逻辑220和解密组件230之间，用于将待处理数据的格式由原格式变换为预设格式，预设格式为与解密组件230匹配的格式，具体的，可以对每个待处理数据加入保护信息来保护数据完整性。控制逻辑210控制数据输入逻辑220将读取的待处理数据传输至第一格式变换逻辑，控制第一格式变换逻辑对待处理数据进行格式变换，控制第一格式变换逻辑向解密组件230发送预设格式的待处理数据。The data processing device provided in the embodiment of the present application may further include a first format conversion logic. The first format conversion logic is connected between the data input logic 220 and the decryption component 230, and is used for converting the format of the data to be processed from the original format. It is converted into a preset format, and the preset format is a format matching the decryption component 230. Specifically, protection information may be added to each to-be-processed data to protect data integrity. The control logic 210 controls the data input logic 220 to transmit the read data to be processed to the first format conversion logic, controls the first format conversion logic to perform format conversion on the data to be processed, and controls the first format conversion logic to send a preset format to the decryption component 230 Of pending data.
控制逻辑210可以控制解密组件230对待处理数据按照解密算法进行解密，形成解密数据。其中，数据解密就是对经过加密的数据通过解密算法进行处理，使其显示出本来内容的过程，解密算法是与加密算法相对应的逆算法。解密算法是与加密算法对应的，例如DES、3DES、MD5、RSA算法等，具体的，压缩加密指令中可以携带解密算法的标识，解密组件230根据解密算法的标识选择对应的解密算法。The control logic 210 may control the decryption component 230 to decrypt the data to be processed according to a decryption algorithm to form decrypted data. Among them, data decryption is the process of processing encrypted data through a decryption algorithm to make it display the original content. The decryption algorithm is an inverse algorithm corresponding to the encryption algorithm. The decryption algorithm corresponds to an encryption algorithm, such as the DES, 3DES, MD5, and RSA algorithms. Specifically, the compressed encryption instruction may carry the identifier of the decryption algorithm, and the decryption component 230 selects the corresponding decryption algorithm according to the identifier of the decryption algorithm.
实际操作中，若待处理数据数通过加密密钥的方式进行加密的数据，则可以通过与待处理数据的加密密钥对应的解密密钥的方式，对待处理数据进行解密，得到解密数据。具体的，密钥为对称密钥时，解密密钥与加密密钥相同，密钥为非对称密钥时，解密密钥为与公开密钥对应的私用密钥。In actual operation, if the number of data to be processed is encrypted by means of an encryption key, the data to be processed may be decrypted by using a decryption key corresponding to the encryption key of the data to be processed to obtain decrypted data. Specifically, when the key is a symmetric key, the decryption key is the same as the encryption key, and when the key is an asymmetric key, the decryption key is a private key corresponding to the public key.
解密密钥可以存储在存储模块200中，在解密解压指令中还可以携带有解密密钥的存储位置。在本申请实施例中，数据控制装置还可以包括密钥输入逻辑，密钥输入逻辑与解密组件230和控制逻辑210连接，用于根据解密密钥的存储位置，从存储模块200中读取解密密钥。具体的，控制逻辑210可以控制密钥输入逻辑从存储模块200中读取解密密钥，以便解密组件230利用密钥输入逻辑读取的解密密钥对待处理数据进行解密。The decryption key may be stored in the storage module 200, and the storage location of the decryption key may also be carried in the decryption and decompression instruction. In the embodiment of the present application, the data control device may further include a key input logic, which is connected to the decryption component 230 and the control logic 210, and is configured to read and decrypt from the storage module 200 according to the storage location of the decryption key. Key. Specifically, the control logic 210 may control the key input logic to read the decryption key from the storage module 200 so that the decryption component 230 uses the decryption key read by the key input logic to decrypt the processed data.
在对待处理数据进行解密后，控制逻辑210控制解密组件230将解密后的解密数据传输至解压组件240，以便解压组件240通过解压算法对解密数据进行解压。After the data to be processed is decrypted, the control logic 210 controls the decryption component 230 to transmit the decrypted decrypted data to the decompression component 240 so that the decompression component 240 decompresses the decrypted data by using a decompression algorithm.
解压组件240在获取到解密数据后，控制逻辑210可以控制解压组件240通过解压算法对解密数据进行解压。其中数据解压是数据压缩的逆操作，可以恢复经过压缩的数据。例如通过对文本、表格、视频、图片、音频、应用程序等数据通过压缩可以得到ZIP格式的压缩数据，数据解压就是将ZIP格式的数据恢复到原来的格式，例如经过解压的文本数据的格式可以是doc、txt等，经过解压的表格数据的格式可以是xls等，经过解压的邮件数据格式可以是eml、msg等，经过解压的数据库数据的格式可以是db等，经过解压的应用程序数据的格式可以是EXE等。若待处理数据是对经过压缩的数据进行进一步压缩加密得到的，则相应的对待处理数据进行解密解压后的解压数据是经过压缩的数据，若待处理数据是对经过加密的数据进行进一步压缩加密得到的，则相应的对待处理数据进行解密解压后的解压数据是经过加密的数据。After the decompression component 240 obtains the decrypted data, the control logic 210 may control the decompression component 240 to decompress the decrypted data by using a decompression algorithm. Data decompression is the inverse operation of data compression, which can recover compressed data. For example, you can obtain compressed data in ZIP format by compressing data such as text, tables, videos, pictures, audio, and applications. Data decompression is to restore the data in ZIP format to the original format. For example, the format of decompressed text data can be It is doc, txt, etc. The format of the decompressed form data can be xls, etc., the format of the decompressed mail data can be eml, msg, etc., and the format of the decompressed database data can be db, etc. The format can be EXE, etc. If the data to be processed is obtained by further compressing and encrypting the compressed data, the corresponding data to be processed is decrypted and decompressed. The decompressed data is compressed data. If the data to be processed is further compressed and encrypted, the encrypted data is compressed. If it is obtained, the corresponding data to be processed is decrypted and decompressed. The decompressed data is encrypted data.
解压算法例如可以是LZ77算法或Huffman算法等。具体的，解密解压指令中可以携带解压算法的标识，解压组件240可以根据解压算法的标识选择对应的解压算法。The decompression algorithm may be, for example, the LZ77 algorithm or the Huffman algorithm. Specifically, the identifier of the decompression algorithm may be carried in the decryption and decompression instruction, and the decompression component 240 may select a corresponding decompression algorithm according to the identifier of the decompression algorithm.
在本申请实施例中，已经完成解密的解密数据可以不写入存储模块200中，解密组件230可以向解压组件240发送解密数据，以便解压组件240对解密数据进行解压。在此基础上，为了进一步提高解密解压的效率，在对读取的待处理数据中的部分待处理数据进行解密后，控制逻辑210还可以判断已经完成解密的解密数据的数据量是否大于或等于第二 阈值，第二阈值是可以进行解压的最低的数据量，例如可以是16Byte。若解密数据的数据量大于或等于第二阈值，则控制逻辑210可以控制解压组件240对已经完成解密的解密数据进行解压，得到解压数据。In the embodiment of the present application, the decrypted data that has been decrypted may not be written into the storage module 200, and the decryption component 230 may send the decrypted data to the decompression component 240, so that the decompression component 240 decompresses the decrypted data. On this basis, in order to further improve the efficiency of decryption and decompression, after decrypting a part of the read data to be processed, the control logic 210 can also determine whether the data amount of the decrypted data that has been decrypted is greater than or equal to The second threshold, the second threshold is the lowest amount of data that can be decompressed, and can be, for example, 16 Bytes. If the data amount of the decrypted data is greater than or equal to the second threshold, the control logic 210 may control the decompression component 240 to decompress the decrypted data that has been decrypted to obtain the decompressed data.
也就是说，本申请实施例中，可以在有部分待处理数据完成解密后，通过解压组件240对形成的解密数据进行解压，从而可以实现解密组件230的解密操作和解压组件240的解压操作的并行处理，而不必等到所有的待处理数据均完成解密过程后再进行解密数据的解压。举例来说，若当前已有第三部分的待处理数据完成解密，形成第三部分的解密数据，若第三部分的解密数据的数据量为16Byte，达到了第二阈值16Byte，则可以对第三部分的解密数据进行解压处理，在此同时，第四部分的待处理数据正在进行解密处理，对第三部分的解密数据进行解压和对第四部分的待处理数据进行解密可以同步进行，实现解密解压的效率的进一步提升。That is, in the embodiment of the present application, after decryption of a part of the data to be processed, the formed decrypted data can be decompressed by the decompression component 240, so that the decryption operation of the decryption component 230 and the decompression operation of the decompression component 240 can be implemented. Parallel processing, instead of waiting for all the data to be processed to complete the decryption process before decompressing the decrypted data. For example, if the third part of the data to be processed has been decrypted to form the decrypted data of the third part, if the data amount of the decrypted data of the third part is 16 Byte and reaches the second threshold 16 Byte, the first The decryption data of the three parts is decompressed. At the same time, the data to be processed in the fourth part is being decrypted. Decompressing the decrypted data in the third part and decrypting the data to be processed in the fourth part can be performed simultaneously. The efficiency of decryption and decompression is further improved.
在对解密数据完成解压后，形成解压数据，控制逻辑210可以控制其他组件对形成的解压数据进行处理使用，也可以控制解压组件240将解压数据传输至数据输出逻辑250，控制逻辑210控制数据输出逻辑250将形成的解压数据写入存储模块200，解压数据的写入位置可以是原来待处理数据的存储位置，也可以是其他未被使用的存储位置。After decompressing the decrypted data, the decompressed data is formed. The control logic 210 can control other components to process and use the formed decompressed data. It can also control the decompression component 240 to transmit the decompressed data to the data output logic 250, and the control logic 210 controls the data output. The logic 250 writes the formed decompressed data into the storage module 200, and the write location of the decompressed data may be a storage location of the original to-be-processed data, or may be another unused storage location.
若解压数据为分片后的多个数据片段，还可以将解压数据进行合并，将合并后的数据写入存储模块200中。If the decompressed data is a plurality of data fragments after slicing, the decompressed data may also be combined, and the combined data may be written into the storage module 200.
在本申请实施例提供的数据处理装置中，还可以包括第二格式变换逻辑，其中，第二格式变换逻辑连接在数据输出逻辑250和解压组件240之间，在对解密数据完成解压后，控制逻辑210还可以控制解压组件240经解压数据传输至第二格式变换逻辑，通过控制逻辑210控制第二格式变换逻辑将解压数据的格式变换为原格式，以便对该解压数据时能够方便使用、压缩或加密。第二格式变换逻辑将变换格式后的解压数据传输至数据输出模块，以便数据输出模块将原格式的解压数据写入存储模块200。The data processing apparatus provided in the embodiment of the present application may further include a second format conversion logic, where the second format conversion logic is connected between the data output logic 250 and the decompression component 240, and after the decompression of the decrypted data is completed, the control The logic 210 can also control the decompression component 240 to transmit the decompressed data to the second format conversion logic. The control logic 210 controls the second format conversion logic to convert the format of the decompressed data to the original format, so that the decompressed data can be conveniently used and compressed. Or encrypted. The second format conversion logic transmits the decompressed data in the transformed format to the data output module, so that the data output module writes the decompressed data in the original format into the storage module 200.
为了便于理解，参考图7所示，结合具体场景，对本申请实施例提供的数据处理装置进行说明。具体的，数据处理装置中的模块可以按照以下步骤执行：For ease of understanding, the data processing device provided in the embodiment of the present application will be described with reference to FIG. 7 in combination with specific scenarios. Specifically, the modules in the data processing apparatus may be executed according to the following steps:
701)App向控制逻辑210下发解密解压指令。701) The App sends a decryption and decompression instruction to the control logic 210.
702)控制逻辑210响应解密解压指令后，控制数据输入模块220从存储模块200中读取待处理数据，并将读取的待处理模块发送给第一格式变换逻辑。702) After the control logic 210 responds to the decryption and decompression instruction, the control data input module 220 reads the data to be processed from the storage module 200 and sends the read data to be processed to the first format conversion logic.
703)控制逻辑210控制第一格式变换逻辑对待处理数据进行格式变换，形成与解密组件230对应的处理格式，并将变换格式后的待处理数据传输给解密组件240。703) The control logic 210 controls the first format conversion logic to perform format conversion on the data to be processed to form a processing format corresponding to the decryption component 230, and transmits the formatted to-be-processed data to the decryption component 240.
704)控制逻辑210控制密钥输入模块从存储模块200中读取解密密钥，并将读取的解密密钥发送给解密组件230。704) The control logic 210 controls the key input module to read the decryption key from the storage module 200 and sends the read decryption key to the decryption component 230.
705)控制逻辑210控制解密组件230根据解密算法，或解密算法和解密密钥对待处理数据进行解密，形成解密数据，并将解密数据发送给解压组件240。705) The control logic 210 controls the decryption component 230 to decrypt the data to be processed according to the decryption algorithm, or the decryption algorithm and the decryption key to form decrypted data, and sends the decrypted data to the decompression component 240.
706)控制逻辑210控制解压组件240根据解压算法对解密数据进行解压，形成解压数据，并将解压数据发送给第二格式变换逻辑。706) The control logic 210 controls the decompression component 240 to decompress the decrypted data according to the decompression algorithm to form decompressed data, and sends the decompressed data to the second format conversion logic.
707)控制逻辑210控制第二格式变换逻辑对解压数据进行格式变换，并将经过格式变 换后的解压数据发给数据输出模块250。707) The control logic 210 controls the second format conversion logic to perform format conversion on the decompressed data, and sends the decompressed data after the format conversion to the data output module 250.
708)控制逻辑210控制数据输出模块250将解压数据写入到存储模块200。708) The control logic 210 controls the data output module 250 to write the decompressed data to the storage module 200.
本申请实施例提供的另一种数据处理装置及方法中，数据处理装置包括：数据输入逻辑、控制逻辑、解密组件、解压组件和数据输出逻辑；控制逻辑分别与数据输入逻辑、解密组件、解压组件和数据输出逻辑连接，数据输入逻辑与解密组件连接，解密组件与解压组件连接，解压组件与数据输出逻辑连接；其中，控制逻辑，用于响应解密解压指令，控制数据输入逻辑从存储模块中读取待处理数据，并将待处理数据输入到解密组件；控制解密组件对待处理数据进行解密，得到解密数据；控制解压组件对解密数据进行解压，得到解压数据；以及控制数据输出逻辑将解压数据写入存储模块。在此解密解压的过程中，在对待处理数据进行解密后，无需将解密数据写入存储模块中，在对解密数据进行解压之前，也无需从存储模块中读取解密数据，整个过程只需对存储模块执行一次读取和一次写入即可，即处理器只要与存储模块进行两次交互，相比于传统技术，处理器与存储模块的交互次数减少了一半，提高了解密解压的效率，降低了一半的系统带宽。此外，由于解密和解压是独立的过程，可以在解密数据达到一定数据量后，开始进行解密数据的解压，使解密和解压可以并行处理，进一步提高解密解压的效率，降低系统带宽。In another data processing device and method provided by the embodiments of the present application, the data processing device includes: data input logic, control logic, decryption component, decompression component, and data output logic; the control logic is separately from the data input logic, decryption component, and decompression The component is connected to the data output logic, the data input logic is connected to the decryption component, the decryption component is connected to the decompression component, and the decompression component is connected to the data output logic; among them, the control logic is used to respond to the decryption and decompression instruction and control the data input logic from the storage module Read the data to be processed and input the data to be decrypted; control the decryption component to decrypt the processed data to obtain the decrypted data; control the decompression component to decompress the decrypted data to obtain the decompressed data; and control the data output logic to decompress the data Write to memory module. In the process of decryption and decompression, after the data to be processed is decrypted, the decrypted data does not need to be written into the storage module, and the decrypted data does not need to be read from the storage module before the decrypted data is decompressed. The storage module only needs to perform one read and one write, that is, the processor only needs to interact with the storage module twice. Compared with the traditional technology, the number of interactions between the processor and the storage module is reduced by half, which improves the efficiency of decryption and decompression. Reduced system bandwidth by half. In addition, because decryption and decompression are independent processes, after the decrypted data reaches a certain amount of data, the decompression of the decrypted data can be started, so that the decryption and decompression can be processed in parallel, further improving the efficiency of decryption and decompression, and reducing the system bandwidth.
在操作系统层面，传统的数据处理方式中，APP发送解密指令，调用驱动层的OpenSLL接口从存储模块中读取待处理数据，对待处理数据进行解密，将解密后的数据写入存储模块；然后APP发送解压指令，调用驱动层的Zlib APP接口从存储模块中读取解密后的数据，对解密后的数据进行解压，再将解压后的数据写入存储模块。也就是说，APP需要发送两次指令，即一次解密指令和一次解压指令，分别调用OpenSLL接口和Zlib APP接口，在这个过程中需要对存储模块进行两次读取和两个写入。而本申请实施例提供的数据处理方法中，APP只需要向驱动层发送一次指令，即解密解压指令，驱动层就可以实现对待处理数据的解密解压过程。具体的，驱动层首先从存储模块中读取待处理数据并进行解密，然后直接对解密后的数据进行解压，在解压之后，将经过解密解压后的数据写入存储模块中，在这个过程中驱动层只需要对存储模块进行一次读取和一次写入，相比于传统技术，减少了与存储模块的交互次数，提高了解密解压的效率，降低了系统带宽。At the operating system level, in the traditional data processing method, the APP sends a decryption instruction, calls the driver's OpenSLL interface to read the data to be processed from the storage module, decrypts the data to be processed, and writes the decrypted data to the storage module; The APP sends a decompression instruction, calls the driver's Zlib APP interface to read the decrypted data from the storage module, decompresses the decrypted data, and writes the decompressed data to the storage module. In other words, the APP needs to send two instructions, namely a decryption instruction and a decompression instruction, respectively calling the OpenSLL interface and the Zlib APP interface. In this process, the memory module needs to be read twice and two writes. In the data processing method provided by the embodiment of the present application, the APP only needs to send an instruction to the driver layer, that is, the decryption and decompression instruction, and the driver layer can implement the decryption and decompression process of the data to be processed. Specifically, the driver layer first reads the data to be processed from the storage module and decrypts it, and then directly decompresses the decrypted data. After decompression, the decrypted and decompressed data is written into the storage module. In this process, The driver layer only needs to read and write the storage module once. Compared with traditional technologies, it reduces the number of interactions with the storage module, improves the efficiency of decryption and decompression, and reduces system bandwidth.
参考图8所示，为本申请实施例提供的一种数据处理方法的流程图，该方法可以由处理器执行，具体包括以下步骤：Referring to FIG. 8, a flowchart of a data processing method according to an embodiment of the present application may be executed by a processor, and specifically includes the following steps:
S101，响应压缩加密指令，从存储模块中读取待处理数据。S101. In response to the compression and encryption instruction, read the data to be processed from the storage module.
压缩加密指令是用于指示对存储模块中存储的待处理数据进行压缩加密的指令。在操作系统层面，压缩加密指令可以通过APP发送，调用相关组件来执行压缩加密指令，例如驱动层的内核(Kernel)。The compression and encryption instruction is an instruction for instructing compression and encryption of the data to be processed stored in the storage module. At the operating system level, compression and encryption instructions can be sent through the APP and call related components to execute compression and encryption instructions, such as the kernel (Kernel) of the driver layer.
存储模块100可以包括存储器和/或缓存池，其中存储器中存储有待处理数据和程序代码等，缓存池中可以存储有待处理数据、压缩数据和加密数据等。The storage module 100 may include a memory and / or a cache pool, in which the data to be processed and program code are stored in the memory, and the cache pool may store the data to be processed, compressed data, encrypted data, and the like.
该步骤中，待处理数据可以是存储模块100中存储的未经压缩加密的原始数据，也可以是存储模块100中存储的经过压缩和/或加密、且需要进一步压缩和加密的数据。待处理 数据可以是单个文件，也可以是多个文件，还可以是包括一个或多个文件的文件夹等。In this step, the data to be processed may be uncompressed and encrypted raw data stored in the storage module 100, or may be compressed and / or encrypted data stored in the storage module 100 and need to be further compressed and encrypted. The data to be processed can be a single file, multiple files, or a folder containing one or more files.
在压缩加密指令中，可以携带有待处理数据的首位地址和数据长度，以便根据待处理数据的首位地址和数据长度从存储模块100中读取待处理数据。在待处理数据为多个文件时，压缩加密指令中还可以携带有多个首位地址和与首位地址对应的多个数据长度。In the compression and encryption instruction, the first address and data length of the data to be processed may be carried, so that the data to be processed is read from the storage module 100 according to the first address and data length of the data to be processed. When the data to be processed is multiple files, the compression and encryption instruction may also carry multiple first address and multiple data lengths corresponding to the first address.
在本申请实施例中，若读取的待处理数据的长度较大时，可选的，可以先对待处理数据进行分片处理，即将待处理数据切分为多个数据段，每个数据段为一个分片数据。In the embodiment of the present application, if the length of the data to be processed is large, optionally, the data to be processed may be fragmented first, that is, the data to be processed is divided into multiple data segments, and each data segment Is a piece of data.
S102，对待处理数据进行压缩，得到压缩数据。S102. Compress the data to be processed to obtain compressed data.
数据压缩是按照压缩算法对数据进行处理，用比未经压缩的数据更少的比特来表示信息的过程。常见的压缩算法例如LZ(Lempel-Ziv)77算法或哈尔曼(Huffman)算法等，具体的，压缩加密指令中可以携带压缩算法的标识，可以根据压缩算法的标识选择对应的解压算法。对待处理数据进行压缩处理，形成压缩数据，压缩数据的格式例如可以是ZIP、ZLIB等。例如对文本、表格、邮件、数据库数据、应用程序等数据进行压缩，可以得到ZIP格式的压缩数据。Data compression is the process of processing data according to a compression algorithm and representing information with fewer bits than uncompressed data. Common compression algorithms such as LZ (Lempel-Ziv) 77 algorithm or Huffman algorithm, etc. Specifically, the compression encryption instruction can carry the identification of the compression algorithm, and the corresponding decompression algorithm can be selected according to the identification of the compression algorithm. The data to be processed is compressed to form compressed data, and the format of the compressed data may be, for example, ZIP, ZLIB, and the like. For example, compressing data such as text, tables, emails, database data, and application programs can obtain compressed data in ZIP format.
在本申请实施例中，还可以对待处理数据进行格式处理，将待处理数据的格式由原格式变换为预设格式，预设格式为与进行压缩的组件匹配的格式。在对待处理数据进行压缩后，还可以为压缩后的数据增加压缩头。In the embodiment of the present application, the data to be processed may also be formatted, and the format of the data to be processed may be changed from the original format to a preset format, and the preset format is a format that matches the component to be compressed. After the data to be processed is compressed, a compression header can be added to the compressed data.
S103，对压缩数据进行加密，得到加密数据。S103. Encrypt the compressed data to obtain encrypted data.
数据加密就是对原来为明文的数据按照某种加密算法进行处理，使其成为不可读的密文，合法接收者之外的其他人或设备较难获取到加密前的为明文的数据，数据传输和存储的安全性随之提高。Data encryption is to process the original data in plain text according to some encryption algorithm to make it unreadable cipher text. It is difficult for someone or a device other than a legitimate receiver to obtain the plain text data before encryption. Data transmission And storage security goes up.
常见的加密算法例如数据加密标准(data encryption standard，DES)、三重数据加密算法(triple DES，3DES)、信息-摘要算法5(message-digest algorithm 5，MD5)、RSA(Rivest-Shamir-Adleman)算法等。在压缩加密指令中，可以携带的加密算法的标识，以便根据加密算法的标识选择对应的加密算法。Common encryption algorithms such as data encryption standard (DES), triple data encryption algorithm (triple DES, 3DES), message-digest algorithm 5, MD5, and RSA (Rivest-Shamir-Adleman) Algorithms, etc. In the compressed encryption instruction, the identifier of the encryption algorithm may be carried, so that the corresponding encryption algorithm is selected according to the identifier of the encryption algorithm.
实际操作中，可以通过加密密钥的方式来对压缩数据进行加密，密钥可以是对称密钥，即可以使用同一个密钥进行加密和解密，密钥也可以是非对称密钥，即可以通过公开密钥进行加密，通过私用密钥去解密。In practice, the compressed data can be encrypted by using an encryption key. The key can be a symmetric key, that is, the same key can be used for encryption and decryption. The key can also be an asymmetric key, which can be obtained by The public key is encrypted and decrypted by the private key.
加密密钥可以存储于存储模块中，在压缩加密指令中还可以携带有加密密钥的存储位置。在本申请实施例中，可以根据加密密钥的存储位置，从存储模块中读取加密密钥，以便利用密钥输入逻辑读取的加密密钥对压缩数据进行加密。The encryption key can be stored in the storage module, and the storage location of the encryption key can also be carried in the compressed encryption instruction. In the embodiment of the present application, the encryption key may be read from the storage module according to the storage location of the encryption key, so as to use the encryption key read by the key input logic to encrypt the compressed data.
在本申请实施例中，已经完成压缩的压缩数据可以不写入存储模块中，而是对压缩数据进行加密。在此基础上，为了进一步提高压缩加密的效率，在对读取的待处理数据中的部分待处理数据进行压缩后，还可以判断已经完成压缩的压缩数据的数据量是否大于或等于第一阈值。若压缩数据的数据量大于或等于第一阈值，则对已经完成压缩的压缩数据进行加密，得到加密数据。也就是说，本申请实施例中，可以在有部分待处理数据完成压缩后，通过对形成的压缩数据进行加密，从而实现压缩操作和加密操作的并行处理，而不必等到所有的待处理数据均完成压缩过程后再进行压缩数据的加密，实现压缩加密的效率的 进一步提升。In the embodiment of the present application, the compressed data that has been compressed may not be written into the storage module, but the compressed data may be encrypted. On this basis, in order to further improve the efficiency of compression and encryption, after compressing a part of the data to be processed to be read, it can also be determined whether the amount of compressed data that has completed compression is greater than or equal to the first threshold . If the data amount of the compressed data is greater than or equal to the first threshold, the compressed data that has been compressed is encrypted to obtain encrypted data. That is to say, in the embodiment of the present application, after compression of a part of the pending data is completed, the formed compressed data can be encrypted, thereby achieving parallel processing of the compression operation and the encryption operation, without waiting for all the pending data to be processed. After the compression process is completed, the compressed data is encrypted to further improve the efficiency of compression and encryption.
S104，将加密数据写入存储模块中。S104. Write the encrypted data to the storage module.
在对压缩数据完成加密后，形成加密数据，将形成的加密数据写入存储模块，加密数据的写入位置可以是原来待处理数据的存储位置，也可以是其他未被使用的存储位置。实际操作中，可以通过调用开放式安全套接层协议(Open Secure Sockets Layer，OpenSLL)接口将加密后的数据写入存储模块。After the compressed data is encrypted, the encrypted data is formed, and the formed encrypted data is written into the storage module. The written data may be written to the storage location of the original to-be-processed data or other unused storage locations. In actual operation, the encrypted data can be written into the storage module by calling an Open Secure Sockets Layer Protocol (Open Secure Sockets Layer) (OpenSLL) interface.
在本申请实施例中，还可以将加密数据的格式变换为原格式，以便在需要该加密数据时能够方便解密和解压，并将变换后的加密数据写入存储模块。In the embodiment of the present application, the format of the encrypted data can also be converted into the original format, so that when the encrypted data is needed, it can be easily decrypted and decompressed, and the converted encrypted data is written into the storage module.
本申请实施例提供的数据处理方法中，响应于压缩加密指令，从存储模块中读取待处理数据，先对读取的待处理数据进行压缩，得到压缩数据，对压缩数据进行加密，得到加密数据，将加密数据写入存储模块中。在此压缩加密的过程中，在对待处理数据进行压缩后，无需将压缩数据存储至存储模块中，在对压缩数据进行加密前，无需从存储模块中读取压缩数据，整个过程中只需对存储模块执行一次读取和一次写入即可，即处理器只要与存储模块进行两次交互，相比于传统技术，处理器与存储模块的交互次数减少了一半，提高了压缩加密的效率，降低了一半的系统带宽。此外，由于压缩和加密是独立的过程，可以在压缩数据达到一定数据量后，开始进行压缩数据的加密，使压缩和加密可以并行处理，进一步提高压缩加密的效率，降低系统带宽。In the data processing method provided in the embodiment of the present application, in response to a compression and encryption instruction, reading data to be processed from a storage module, first compressing the read data to be processed to obtain compressed data, and encrypting the compressed data to obtain encryption. Data, write the encrypted data to the storage module. In this compression and encryption process, after the data to be processed is compressed, there is no need to store the compressed data in the storage module. Before the compressed data is encrypted, there is no need to read the compressed data from the storage module. The storage module only needs to perform one read and one write, that is, the processor only needs to interact with the storage module twice. Compared with the traditional technology, the number of interactions between the processor and the storage module is reduced by half, which improves the efficiency of compression and encryption. Reduced system bandwidth by half. In addition, since compression and encryption are independent processes, after the compressed data reaches a certain amount of data, compression of the compressed data can be started, so that compression and encryption can be processed in parallel, which further improves the efficiency of compression and encryption and reduces system bandwidth.
参考图9所示，为本申请实施例提供的另一种数据处理方法的流程图，该方法可以由处理器执行，具体包括以下步骤：Referring to FIG. 9, which is a flowchart of another data processing method according to an embodiment of the present application. The method may be executed by a processor, and specifically includes the following steps:
S201，响应解密解压指令，从存储模块中读取待处理数据。S201. In response to the decryption and decompression instruction, read the data to be processed from the storage module.
解密解压指令是用于指示对存储模块中存储的经过压缩加密的待处理数据进行解密解压的指令。类比于压缩加密指令，解密解压指令可以通过APP发送，调用相关组件来执行解密解压指令，例如驱动层的内核(Kernel)。The decryption and decompression instruction is an instruction for instructing decryption and decompression of the compressed and encrypted to-be-processed data stored in the storage module. Analogous to compressing and encrypting instructions, the decryption and decompression instructions can be sent through the APP, and the relevant components are called to execute the decryption and decompression instructions, such as the kernel of the driver layer (Kernel).
存储模块200可以包括存储器和/或缓存池，其中存储器中存储有待处理数据和程序代码等，缓存池中可以存储有待处理数据、压缩数据和加密数据等。The storage module 200 may include a memory and / or a cache pool, where the memory stores the data to be processed and program code, etc., and the cache pool may store the data to be processed, compressed data, encrypted data, and the like.
该步骤中，待处理数据是存储模块中存储的经过压缩加密的数据，待处理数据可以是单个经过压缩加密的文件，也可以是多个经过压缩加密的文件，还可以是包括一个或多个经过压缩加密的文件的文件夹。In this step, the data to be processed is compressed and encrypted data stored in the storage module. The data to be processed may be a single compressed and encrypted file, or multiple compressed and encrypted files, or may include one or more files. A folder of compressed encrypted files.
在解密解压指令中，可以携带有待处理数据的首位地址和数据长度，以便根据待处理数据的首位地址和数据长度从存储模块中读取待处理数据。在待处理数据为多个经过压缩加密的文件时，解密解压指令中还可以携带有多个首位地址和与首位地址对应的多个数据长度。The decryption and decompression instruction may carry a first address and a data length of the data to be processed, so as to read the data to be processed from the storage module according to the first address and the data length of the data to be processed. When the data to be processed is multiple compressed and encrypted files, the decryption and decompression instruction may also carry multiple first address and multiple data lengths corresponding to the first address.
S202，对待处理数据进行解密，得到解密数据。S202: Decrypt the data to be processed to obtain decrypted data.
数据解密就是对经过加密的数据通过解密算法进行处理，使其显示出本来内容的过程，解密算法是与加密算法相对应的逆算法。Data decryption is the process of processing encrypted data through a decryption algorithm to make it display its original content. The decryption algorithm is the inverse algorithm corresponding to the encryption algorithm.
在获取到待处理数据后，可以通过解密算法对待处理数据进行解密。解密算法是与加 密算法对应的，例如DES、3DES、MD5、RSA算法等，具体的，压缩加密指令中可以携带解密算法的标识，根据解密算法的标识选择对应的解密算法。After obtaining the data to be processed, the data to be processed can be decrypted by a decryption algorithm. The decryption algorithm corresponds to the encryption algorithm, such as DES, 3DES, MD5, and RSA algorithms. Specifically, the compressed encryption instruction can carry the identifier of the decryption algorithm, and the corresponding decryption algorithm is selected according to the identifier of the decryption algorithm.
实际操作中，若待处理数据数通过加密密钥的方式进行加密的数据，则可以通过与待处理数据的加密密钥对应的解密密钥的方式，对待处理数据进行解密，得到解密数据。具体的，密钥为对称密钥时，解密密钥与加密密钥相同，密钥为非对称密钥时，解密密钥为与公开密钥对应的私用密钥。In actual operation, if the number of data to be processed is encrypted by means of an encryption key, the data to be processed may be decrypted by using a decryption key corresponding to the encryption key of the data to be processed to obtain decrypted data. Specifically, when the key is a symmetric key, the decryption key is the same as the encryption key, and when the key is an asymmetric key, the decryption key is a private key corresponding to the public key.
解密密钥可以存储在存储模块中，在解密解压指令中还可以携带有解密密钥的存储位置，在本申请实施例中，从存储模块中读取解密密钥，以便利用密钥输入逻辑读取的解密密钥对待处理数据进行解密。The decryption key can be stored in the storage module, and the storage location of the decryption key can also be carried in the decryption and decompression instructions. In the embodiment of the present application, the decryption key is read from the storage module, so that the key can be read logically by using the key input logic. The obtained decryption key is used to decrypt the data to be processed.
在对待处理数据进行解密之前，还可以对待处理数据的格式进行处理，将待处理数据的格式由原格式变换为预设格式，预设格式为与进行解密的组件相匹配的格式。例如通过对每个待处理数据加入保护信息来保护数据完整性。Before decrypting the data to be processed, the format of the data to be processed may also be processed, and the format of the data to be processed may be changed from the original format to a preset format, and the preset format is a format that matches the component to be decrypted. For example, data integrity is protected by adding protection information to each pending data.
S203，对解密数据进行解压，得到解压数据。S203: Decompress the decrypted data to obtain decompressed data.
数据解压是数据压缩的逆操作，可以恢复经过压缩的数据。Data decompression is the inverse operation of data compression, which can recover compressed data.
在获取到解密数据后，可以对解密数据按照解压算法进行解压，例如可以是LZ77算法或Huffman算法等。具体的，解密解压指令中可以携带解压算法的标识，根据解压算法的标识选择对应的解压算法。After obtaining the decrypted data, the decrypted data can be decompressed according to a decompression algorithm, such as the LZ77 algorithm or the Huffman algorithm. Specifically, the identifier of the decompression algorithm may be carried in the decryption and decompression instruction, and a corresponding decompression algorithm is selected according to the identifier of the decompression algorithm.
在本申请实施例中，已经完成解密的解密数据不写入存储模块中，而是发送至执行数据解压的模块中，对解密数据进行解压。在此基础上，为了进一步提高解密解压的效率，在对读取的待处理数据中的部分待处理数据进行解密后，还可以判断已经完成解密的解密数据的数据量是否大于或等于第二阈值，。若解密数据的数据量大于或等于第二阈值，则可以对已经完成解密的解密数据进行解压，得到解压数据。也就是说，本申请实施例中，可以在有部分待处理数据完成解密后，对形成的解密数据进行解压，从而可以实现解密操作和解压操作的并行处理，而不必等到所有的待处理数据均完成解密过程后再进行解密数据的解压，实现解密解压的效率的进一步提升。In the embodiment of the present application, the decrypted data that has been decrypted is not written into the storage module, but is sent to a module that performs data decompression to decompress the decrypted data. On this basis, in order to further improve the efficiency of decryption and decompression, after decrypting a part of the read to-be-processed data, it can also be determined whether the amount of decrypted data that has been decrypted is greater than or equal to the second threshold . If the data amount of the decrypted data is greater than or equal to the second threshold, the decrypted data that has been decrypted can be decompressed to obtain the decompressed data. That is to say, in the embodiment of the present application, after the decryption of a part of the pending data is completed, the formed decrypted data can be decompressed, so that the parallel operation of the decryption operation and the decompression operation can be realized without waiting for all the pending data to be processed. Decompress the decrypted data after completing the decryption process to further improve the efficiency of decryption and decompression.
S204，将解压数据写入存储模块中。S204. Write the decompressed data to the storage module.
在对解密数据完成解压后，可以对形成的解压数据进行处理使用，也可以将形成的解压数据写入存储模块，解压数据的写入位置可以是原来待处理数据的存储位置，也可以是其他未被使用的存储位置。After the decompressed data is decompressed, the formed decompressed data can be processed and used, and the formed decompressed data can be written to the storage module. The write location of the decompressed data can be the original storage location of the data to be processed, or it can be other Unused storage location.
若解压数据为分片后的多个数据片段，还可以将解压数据进行合并，将合并后的数据写入存储模块中。在将解压数据写入存储模块之前，还可以对解压数据的格式进行调整，若解压数据的格式为预设格式，则可以将解压数据的格式变换为原格式，以便对该解压数据时能够方便使用、压缩或加密。If the decompressed data is multiple data fragments after slicing, the decompressed data may also be combined, and the combined data may be written into the storage module. Before writing the decompressed data to the storage module, the format of the decompressed data can also be adjusted. If the format of the decompressed data is a preset format, the format of the decompressed data can be converted to the original format, so that the decompressed data can be conveniently used. Use, compress, or encrypt.
本申请实施例提供的数据处理方法中，响应于解密解压指令，从存储模块中读取待处理数据，先对待处理数据进行解密，得到解密数据，再对解密数据进行解压，得到解压数据，将解压数据写入存储模块中。在此解密解压的过程中，在对待处理数据进行解密后，无需将解密数据写入存储模块中，在对解密数据进行解压之前，也无需从存储模块中读取 解密数据，整个过程只需对存储模块执行一次读取和一次写入即可，即处理器只要与存储模块进行两次交互，相比于传统技术，处理器与存储模块的交互次数减少了一半，提高了解密解压的效率，降低了一半的系统带宽。此外，由于解密和解压是独立的过程，可以在解密数据达到一定数据量后，开始进行解密数据的解压，使解密和解压可以并行处理，进一步提高解密解压的效率，降低系统带宽。In the data processing method provided in the embodiment of the present application, in response to a decryption and decompression instruction, reading data to be processed from a storage module, first decrypting the data to be processed to obtain decrypted data, and then decompressing the decrypted data to obtain decompressed data, The decompressed data is written into the storage module. In the process of decryption and decompression, after the data to be processed is decrypted, the decrypted data does not need to be written into the storage module, and the decrypted data does not need to be read from the storage module before the decrypted data is decompressed. The storage module only needs to perform one read and one write, that is, the processor only needs to interact with the storage module twice. Compared with the traditional technology, the number of interactions between the processor and the storage module is reduced by half, which improves the efficiency of decryption and decompression. Reduced system bandwidth by half. In addition, because decryption and decompression are independent processes, after the decrypted data reaches a certain amount of data, the decompression of the decrypted data can be started, so that the decryption and decompression can be processed in parallel, further improving the efficiency of decryption and decompression, and reducing the system bandwidth.
接下来介绍本申请实施例中的数据处理设备。请参阅图10所示，数据处理设备1000包括：The data processing device in the embodiment of the present application is described next. Referring to FIG. 10, the data processing device 1000 includes:
接收器1001、发射器1002、处理器1003和存储器1004(其中数据处理设备1000中的处理器1003的数量可以一个或多个，图10中以一个处理器为例)。在本申请的一些实施例中，接收器1001、发射器1002、处理器1003和存储器1004可通过总线或其它方式连接，其中，图10中以通过总线连接为例。The receiver 1001, the transmitter 1002, the processor 1003, and the memory 1004 (the number of the processors 1003 in the data processing device 1000 may be one or more, and one processor is taken as an example in FIG. 10). In some embodiments of the present application, the receiver 1001, the transmitter 1002, the processor 1003, and the memory 1004 may be connected through a bus or other manners. In FIG. 10, the connection through the bus is taken as an example.
存储器1004可以包括只读存储器和随机存取存储器，并向处理器1003提供指令和数据。存储器1004的一部分还可以包括NVRAM。存储器1004存储有操作系统和操作指令、可执行模块或者数据结构，或者它们的子集，或者它们的扩展集，其中，操作指令可包括各种操作指令，用于实现各种操作。操作系统可包括各种系统程序，用于实现各种基础业务以及处理基于硬件的任务。The memory 1004 may include a read-only memory and a random access memory, and provide instructions and data to the processor 1003. A part of the memory 1004 may further include NVRAM. The memory 1004 stores an operating system and operation instructions, executable modules or data structures, or a subset thereof, or an extended set thereof. The operation instructions may include various operation instructions for implementing various operations. The operating system may include various system programs for implementing various basic services and processing hardware-based tasks.
处理器1003控制终端设备的操作，处理器1003还可以称为CPU。具体的应用中，数据处理设备的各个组件通过总线系统耦合在一起，其中总线系统除包括数据总线之外，还可以包括电源总线、控制总线和状态信号总线等。但是为了清楚说明起见，在图中将各种总线都称为总线系统。The processor 1003 controls operations of the terminal device, and the processor 1003 may also be referred to as a CPU. In specific applications, the various components of the data processing equipment are coupled together through a bus system. The bus system may include a power bus, a control bus, and a status signal bus in addition to the data bus. However, for the sake of clarity, various buses are called bus systems in the figure.
上述本申请实施例揭示的方法可以应用于处理器1003中，或者由处理器1003实现。处理器1003可以是一种集成电路芯片，具有信号的处理能力。在实现过程中，上述方法的各步骤可以通过处理器1003中的硬件的集成逻辑电路或者软件形式的指令完成。上述的处理器1003可以是通用处理器、DSP、ASIC、FPGA或者其他可编程逻辑器件、分立门或者晶体管逻辑器件、分立硬件组件。可以实现或者执行本申请实施例中的公开的各方法、步骤及逻辑框图。通用处理器可以是微处理器或者该处理器也可以是任何常规的处理器等。结合本申请实施例所公开的方法的步骤可以直接体现为硬件译码处理器执行完成，或者用译码处理器中的硬件及软件模块组合执行完成。软件模块可以位于随机存储器，闪存、只读存储器，可编程只读存储器或者电可擦写可编程存储器、寄存器等本领域成熟的存储介质中。该存储介质位于存储器1004，处理器1003读取存储器1004中的信息，结合其硬件完成上述方法的步骤。The method disclosed in the foregoing embodiments of the present application may be applied to the processor 1003, or implemented by the processor 1003. The processor 1003 may be an integrated circuit chip and has a signal processing capability. In the implementation process, each step of the above method may be completed by an integrated logic circuit of hardware in the processor 1003 or an instruction in the form of software. The processor 1003 may be a general-purpose processor, a DSP, an ASIC, an FPGA, or other programmable logic devices, a discrete gate or transistor logic device, or a discrete hardware component. Various methods, steps, and logical block diagrams disclosed in the embodiments of the present application may be implemented or executed. A general-purpose processor may be a microprocessor or the processor may be any conventional processor or the like. The steps of the method disclosed in combination with the embodiments of the present application may be directly implemented by a hardware decoding processor, or may be performed by using a combination of hardware and software modules in the decoding processor. A software module may be located in a mature storage medium such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory, or an electrically erasable programmable memory, a register, and the like. The storage medium is located in the memory 1004, and the processor 1003 reads the information in the memory 1004 and completes the steps of the foregoing method in combination with its hardware.
接收器1001可用于接收输入的数字或字符信息，以及产生与用户面装置的相关设置以及功能控制有关的信号输入，发射器1002可包括显示屏等显示设备，发射器1002可用于通过外接接口输出数字或字符信息。The receiver 1001 can be used to receive inputted digital or character information, and generate signal inputs related to the related settings and function control of the user plane device. The transmitter 1002 can include display devices such as a display screen, and the transmitter 1002 can be used to output through an external interface. Numeric or character information.
本申请实施例中，接收器1001和发射器1002用于实现数据收发。处理器1003，用于通过接收器1001和发射器1002实现数据收发，完成前述数据处理设备所执行的一种数据 处理方法的过程和另一种数据处理方法的过程。In the embodiment of the present application, the receiver 1001 and the transmitter 1002 are used to implement data transmission and reception. The processor 1003 is configured to implement data transmission and reception through the receiver 1001 and the transmitter 1002, and complete a process of one data processing method and another process of a data processing method performed by the foregoing data processing device.
本申请实施例还提供一种计算机可读存储介质，用于存储程序代码，该程序代码用于执行前述各个实施例的一种数据处理方法中的任意一种实施方式。An embodiment of the present application further provides a computer-readable storage medium for storing program code, where the program code is used to execute any one of the data processing methods of the foregoing embodiments.
本申请实施例还提供另一种计算机可读存储介质，用于存储程序代码，该程序代码用于执行前述各个实施例的另一种数据处理方法中的任意一种实施方式。The embodiment of the present application further provides another computer-readable storage medium for storing program code, where the program code is used to execute any one of the other data processing methods of the foregoing embodiments.
本申请实施例还提供一种包括指令的计算机程序产品，当其在计算机上运行时，使得计算机执行前述各个实施例的一种数据处理方法中的任意一种实施方式。The embodiment of the present application further provides a computer program product including instructions, which when executed on a computer, causes the computer to execute any one of the data processing methods of the foregoing embodiments.
本申请实施例还提供另一种包括指令的计算机程序产品，当其在计算机上运行时，使得计算机执行前述各个实施例的另一种数据处理方法中的任意一种实施方式。The embodiment of the present application also provides another computer program product including instructions, which when executed on a computer, causes the computer to execute any one of the other data processing methods of the foregoing embodiments.
所属领域的技术人员可以清楚地了解到，为描述的方便和简洁，上述描述的系统，装置和单元的具体工作过程，可以参考前述方法实施例中的对应过程，在此不再赘述。Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices, and units described above can refer to the corresponding processes in the foregoing method embodiments, and are not repeated here.
在本申请所提供的几个实施例中，应该理解到，所揭露的系统，装置和方法，可以通过其它的方式实现。例如，以上所描述的装置实施例仅仅是示意性的，例如，所述单元的划分，仅仅为一种逻辑功能划分，实际实现时可以有另外的划分方式，例如多个单元或组件可以结合或者可以集成到另一个系统，或一些特征可以忽略，或不执行。另一点，所显示或讨论的相互之间的耦合或直接耦合或通信连接可以是通过一些接口，装置或单元的间接耦合或通信连接，可以是电性，机械或其它的形式。In the several embodiments provided in this application, it should be understood that the disclosed systems, devices, and methods may be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the unit is only a logical function division. In actual implementation, there may be another division manner. For example, multiple units or components may be combined or Can be integrated into another system, or some features can be ignored or not implemented. In addition, the displayed or discussed mutual coupling or direct coupling or communication connection may be indirect coupling or communication connection through some interfaces, devices or units, which may be electrical, mechanical or other forms.
所述作为分离部件说明的单元可以是或者也可以不是物理上分开的，作为单元显示的部件可以是或者也可以不是物理单元，即可以位于一个地方，或者也可以分布到多个网络单元上。可以根据实际的需要选择其中的部分或者全部单元来实现本实施例方案的目的。The units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, may be located in one place, or may be distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the objective of the solution of this embodiment.
另外，在本申请各个实施例中的各功能单元可以集成在一个处理单元中，也可以是各个单元单独物理存在，也可以两个或两个以上单元集成在一个单元中。上述集成的单元既可以采用硬件的形式实现，也可以采用软件功能单元的形式实现。In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each of the units may exist separately physically, or two or more units may be integrated into one unit. The above integrated unit may be implemented in the form of hardware or in the form of software functional unit.
所述集成的单元如果以软件功能单元的形式实现并作为独立的产品销售或使用时，可以存储在一个计算机可读取存储介质中。基于这样的理解，本申请的技术方案本质上或者说对现有技术做出贡献的部分或者该技术方案的全部或部分可以以软件产品的形式体现出来，该计算机软件产品存储在一个存储介质中，包括若干指令用以使得一台计算机设备(可以是个人计算机，服务器，或者网络设备等)执行本申请各个实施例所述方法的全部或部分步骤。而前述的存储介质包括：U盘、移动硬盘、只读存储器(ROM，Read-Only Memory)、随机存取存储器(RAM，Random Access Memory)、磁碟或者光盘等各种可以存储程序代码的介质。If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it may be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially a part that contributes to the existing technology or all or part of the technical solution can be embodied in the form of a software product, which is stored in a storage medium , Including a number of instructions to enable a computer device (which may be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application. The foregoing storage media include: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), magnetic disks or optical disks and other media that can store program codes .
以上实施例仅用以说明本申请的技术方案，而非对其限制；尽管参照前述实施例对本申请进行了详细的说明，本领域的普通技术人员应当理解：其依然可以对前述各实施例所记载的技术方案进行修改，或者对其中部分技术特征进行等同替换；而这些修改或者替换，并不使相应技术方案的本质脱离本申请各实施例技术方案的范围。The above embodiments are only used to describe the technical solutions of the present application, and are not limited thereto. Although the present application has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still apply the foregoing embodiments. The recorded technical solutions are modified, or some of the technical features are equivalently replaced; and these modifications or replacements do not make the essence of the corresponding technical solutions out of the scope of the technical solutions of the embodiments of the present application.

Claims (32)

  1. 一种数据处理装置，其特征在于，所述装置包括：数据输入逻辑、控制逻辑、压缩组件、加密组件和数据输出逻辑；所述控制逻辑分别与所述数据输入逻辑、所述压缩组件、所述加密组件和所述数据输出逻辑连接，所述数据输入逻辑与所述压缩组件连接，所述压缩组件与所述加密组件连接，所述加密组件与所述数据输出逻辑连接；A data processing device, characterized in that the device includes: data input logic, control logic, compression component, encryption component, and data output logic; the control logic is separate from the data input logic, the compression component, and The encryption component is connected to the data output logic, the data input logic is connected to the compression component, the compression component is connected to the encryption component, and the encryption component is connected to the data output logic;
    其中，所述控制逻辑，用于响应压缩加密指令，控制所述数据输入逻辑从存储模块中读取待处理数据，并将所述待处理数据输入到所述压缩组件；控制所述压缩组件对所述待处理数据进行压缩，得到压缩数据；控制所述加密组件对所述压缩数据进行加密，得到加密数据；以及控制所述数据输出逻辑将所述加密数据写入所述存储模块。The control logic is configured to respond to the compression and encryption instruction, control the data input logic to read data to be processed from a storage module, and input the data to be processed to the compression component; and control the compression component pair Compressing the data to be processed to obtain compressed data; controlling the encryption component to encrypt the compressed data to obtain encrypted data; and controlling the data output logic to write the encrypted data into the storage module.
  2. 根据权利要求1所述的装置，其特征在于，所述压缩加密指令中携带有所述待处理数据的首位地址和数据长度；The device according to claim 1, wherein the compressed encryption instruction carries a first address and a data length of the data to be processed;
    所述从存储模块中读取待处理数据包括：The reading the data to be processed from the storage module includes:
    根据所述待处理数据的首位地址和数据长度从所述存储模块中读取所述待处理数据。Reading the to-be-processed data from the storage module according to a first address and a data length of the to-be-processed data.
  3. 根据权利要求1所述的装置，其特征在于，所述控制所述加密组件对所述压缩数据进行加密包括：The device according to claim 1, wherein the controlling the encryption component to encrypt the compressed data comprises:
    若所述压缩数据的数据量大于或等于第一阈值，则控制所述加密组件对所述压缩数据进行加密。If the data amount of the compressed data is greater than or equal to a first threshold, controlling the encryption component to encrypt the compressed data.
  4. 根据权利要求1所述的装置，其特征在于，所述装置还包括密钥输入逻辑，与所述加密组件和所述控制逻辑连接；The device according to claim 1, further comprising a key input logic connected to the encryption component and the control logic;
    所述控制所述加密组件对所述压缩数据进行加密包括：The controlling the encryption component to encrypt the compressed data includes:
    控制所述密钥输入逻辑从所述存储模块中读取加密密钥，并控制所述加密组件利用所述加密密钥对所述压缩数据进行加密。Controlling the key input logic to read an encryption key from the storage module, and controlling the encryption component to use the encryption key to encrypt the compressed data.
  5. 根据权利要求1-4任意一项所述的装置，其特征在于，所述装置还包括第一格式变换逻辑，所述第一格式变换逻辑连接在所述数据输入逻辑和所述压缩组件之间；The device according to any one of claims 1-4, wherein the device further comprises a first format conversion logic, and the first format conversion logic is connected between the data input logic and the compression component. ;
    所述控制逻辑，还用于控制所述第一格式变换逻辑将所述待处理数据的格式由原格式变换为预设格式，并向所述压缩组件发送所述预设格式的待处理数据，所述预设格式为所述压缩组件匹配的格式。The control logic is further configured to control the first format conversion logic to convert a format of the data to be processed from an original format into a preset format, and send the to-be-processed data in the preset format to the compression component, The preset format is a format matched by the compression component.
  6. 根据权利要求5所述的装置，其特征在于，所述装置还包括第二格式变换逻辑，所述第二格式变换逻辑连接在所述数据输出逻辑和所述加密组件之间；The device according to claim 5, further comprising a second format conversion logic, the second format conversion logic being connected between the data output logic and the encryption component;
    所述控制逻辑，还用于控制所述第二格式变换逻辑将所述加密数据的格式变换为所述原格式，并向所述数据输出逻辑发送所述原格式的加密数据。The control logic is further configured to control the second format conversion logic to convert the format of the encrypted data into the original format, and send the encrypted data in the original format to the data output logic.
  7. 根据权利要求1所述的装置，其特征在于，所述存储模块包括：The apparatus according to claim 1, wherein the storage module comprises:
    存储器和/或缓存池。Storage and / or cache pool.
  8. 一种数据处理装置，其特征在于，所述装置包括：数据输入逻辑、控制逻辑、解密组件、解压组件和数据输出逻辑；所述控制逻辑分别与所述数据输入逻辑、所述解密组件、所述解压组件和所述数据输出逻辑连接，所述数据输入逻辑与所述解密组件连接，所述解密组件与所述解压组件连接，所述解压组件与所述数据输出逻辑连接；A data processing device, characterized in that the device includes: data input logic, control logic, decryption component, decompression component, and data output logic; the control logic is separately from the data input logic, the decryption component, and The decompression component is connected to the data output logic, the data input logic is connected to the decryption component, the decryption component is connected to the decompression component, and the decompression component is connected to the data output logic;
    其中，所述控制逻辑，用于响应解密解压指令，控制所述数据输入逻辑从存储模块中读取待处理数据，并将所述待处理数据输入到所述解密组件；控制所述解密组件对所述待处理数据进行解密，得到解密数据；控制所述解压组件对所述解密数据进行解压，得到解压数据；以及控制所述数据输出逻辑将所述解压数据写入所述存储模块。The control logic is configured to respond to a decryption and decompression instruction, control the data input logic to read data to be processed from a storage module, and input the data to be processed to the decryption component; and control the decryption component pair Decrypt the data to be processed to obtain decrypted data; control the decompression component to decompress the decrypted data to obtain decompressed data; and control the data output logic to write the decompressed data to the storage module.
  9. 根据权利要求8所述的装置，其特征在于，所述解密解压指令中携带有所述待处理数据的首位地址和数据长度；The apparatus according to claim 8, wherein the decryption and decompression instruction carries a first address and a data length of the data to be processed;
    所述从存储模块中读取待处理数据包括：The reading the data to be processed from the storage module includes:
    根据所述待处理数据的首位地址和数据长度从所述存储模块中读取所述待处理数据。Reading the to-be-processed data from the storage module according to a first address and a data length of the to-be-processed data.
  10. 根据权利要求8所述的装置，其特征在于，所述装置还包括密钥输入逻辑，与所述解密组件和所述控制逻辑连接；The device according to claim 8, further comprising a key input logic connected to the decryption component and the control logic;
    所述对所述待处理数据进行解密包括：The decrypting the to-be-processed data includes:
    控制所述密钥输入逻辑从所述存储模块中读取解密密钥，并控制所述解密组件利用所述解密密钥对所述待处理数据进行解密。Controlling the key input logic to read a decryption key from the storage module, and controlling the decryption component to use the decryption key to decrypt the data to be processed.
  11. 根据权利要求8-10任意一项所述的装置，其特征在于，所述装置还包括第一格式变换逻辑，所述第一格式变换逻辑连接在所述数据输入逻辑和所述解密组件之间；The device according to any one of claims 8 to 10, wherein the device further comprises a first format conversion logic, and the first format conversion logic is connected between the data input logic and the decryption component ;
    所述控制逻辑，还用于控制所述第一格式变换逻辑将所述待处理数据的格式由原格式变换为预设格式，并向所述解密组件发送所述预设格式的待处理数据，所述预设格式为与所述解密组件匹配的格式。The control logic is further configured to control the first format conversion logic to convert a format of the data to be processed from an original format into a preset format, and send the data to be processed in the preset format to the decryption component, The preset format is a format that matches the decryption component.
  12. 根据权利要求11所述的装置，其特征在于，所述装置还包括第二格式变换逻辑，所述第二格式变换逻辑连接在所述数据输出逻辑和所述解压组件之间；The device according to claim 11, further comprising a second format conversion logic, the second format conversion logic being connected between the data output logic and the decompression component;
    所述控制逻辑，还用于控制所述第二格式变换逻辑将所述解压数据的格式变换为所述原格式，并向所述数据输出逻辑发送所述原格式的解压数据。The control logic is further configured to control the second format conversion logic to convert the format of the decompressed data into the original format, and send the decompressed data in the original format to the data output logic.
  13. 根据权利要求8所述的装置，其特征在于，所述存储模块包括：The apparatus according to claim 8, wherein the storage module comprises:
    存储器和/或缓存池。Storage and / or cache pool.
  14. 一种数据处理方法，其特征在于，所述方法包括：A data processing method, characterized in that the method includes:
    响应压缩加密指令，从存储模块中读取待处理数据；Respond to the compression and encryption instruction, and read the data to be processed from the storage module;
    对所述待处理数据进行压缩，得到压缩数据；Compressing the data to be processed to obtain compressed data;
    对所述压缩数据进行加密，得到加密数据；Encrypting the compressed data to obtain encrypted data;
    将所述加密数据写入所述存储模块中。Writing the encrypted data into the storage module.
  15. 根据权利要求14所述的方法，其特征在于，所述压缩加密指令中携带有所述待处理数据的首位地址和数据长度；The method according to claim 14, wherein the compressed and encrypted instruction carries a first address and a data length of the data to be processed;
    所述从存储模块中读取待处理数据包括：The reading the data to be processed from the storage module includes:
    根据所述待处理数据的首位地址和数据长度从所述存储模块中读取所述待处理数据。Reading the to-be-processed data from the storage module according to a first address and a data length of the to-be-processed data.
  16. 根据权利要求14所述的方法，其特征在于，所述对所述压缩数据进行加密包括：The method according to claim 14, wherein the encrypting the compressed data comprises:
    若所述压缩数据的数据量大于或等于第一阈值，则对所述压缩数据进行加密。If the data amount of the compressed data is greater than or equal to a first threshold, the compressed data is encrypted.
  17. 根据权利要求14所述的方法，其特征在于，所述对所述压缩数据进行加密包括：The method according to claim 14, wherein the encrypting the compressed data comprises:
    从所述存储模块中读取加密密钥，并利用所述加密密钥对所述压缩数据进行加密。An encryption key is read from the storage module, and the compressed data is encrypted by using the encryption key.
  18. 根据权利要求14至17任意一项所述的方法，其特征在于，所述方法还包括：The method according to any one of claims 14 to 17, wherein the method further comprises:
    将所述待处理数据的格式由原格式变换为预设格式，所述预设格式为所述压缩组件匹配的格式。Converting the format of the data to be processed from an original format into a preset format, where the preset format is a format matched by the compression component.
  19. 根据权利要求18所述的方法，其特行在于，所述方法还包括：The method according to claim 18, wherein the method further comprises:
    将所述加密数据的格式变换为所述原格式。Converting the format of the encrypted data into the original format.
  20. 根据权利要求14所述的方法，其特征在于，所述存储模块包括：The method according to claim 14, wherein the storage module comprises:
    存储器和/或缓存池。Storage and / or cache pool.
  21. 一种数据处理方法，其特征在于，所述方法包括：A data processing method, characterized in that the method includes:
    响应解密解压指令，从存储模块中读取待处理数据；In response to the decryption and decompression instruction, read the data to be processed from the storage module;
    对所述待处理数据进行解密，得到解密数据；Decrypting the data to be processed to obtain decrypted data;
    对所述解密数据进行解压，得到解压数据；Decompressing the decrypted data to obtain decompressed data;
    将所述解压数据写入所述存储模块中。Write the decompressed data into the storage module.
  22. 根据权利要求21所述的方法，其特征在于，所述解密解压指令中携带有所述待处理数据的首位地址和数据长度；The method according to claim 21, wherein the decryption and decompression instruction carries a first address and a data length of the data to be processed;
    所述从存储模块中读取待处理数据包括：The reading the data to be processed from the storage module includes:
    根据所述待处理数据的首位地址和数据长度从所述存储模块中读取所述待处理数据。Reading the to-be-processed data from the storage module according to a first address and a data length of the to-be-processed data.
  23. 根据权利要求21所述的方法，其特征在于，所述对所述待处理数据进行解密包括：The method according to claim 21, wherein the decrypting the data to be processed comprises:
    从所述存储模块中读取解密密钥，利用所述解密密钥对所述待处理数据进行解密。Read the decryption key from the storage module, and use the decryption key to decrypt the data to be processed.
  24. 根据权利要求21-23任意一项所述的方法，其特征在于，所述方法还包括：The method according to any one of claims 21 to 23, wherein the method further comprises:
    将所述待处理数据的格式由原格式变换为预设格式，所述预设格式为与所述解密组件匹配的格式。Transforming the format of the data to be processed from an original format into a preset format, where the preset format is a format that matches the decryption component.
  25. 根据权利要求24所述的方法，其特征在于，所述方法还包括：The method according to claim 24, further comprising:
    将所述解压数据的格式变换为所述原格式。Converting the format of the decompressed data into the original format.
  26. 根据权利要求21所述的方法，其特征在于，所述存储模块包括：The method according to claim 21, wherein the storage module comprises:
    存储器和/或缓存池。Storage and / or cache pool.
  27. 一种数据处理设备，其特征在于，所述设备包括：处理器和存储器；A data processing device, characterized in that the device includes: a processor and a memory;
    所述存储器，用于存储指令；The memory is used for storing instructions;
    所述处理器，用于执行所述存储器中的所述指令，执行如权利要求14至20中任一项所述的方法。The processor is configured to execute the instructions in the memory and execute the method according to any one of claims 14 to 20.
  28. 一种数据处理设备，其特征在于，所述设备包括：处理器和存储器；A data processing device, characterized in that the device includes: a processor and a memory;
    所述存储器，用于存储指令；The memory is used for storing instructions;
    所述处理器，用于执行所述存储器中的所述指令，执行如权利要求21至26中任一项所述的方法。The processor is configured to execute the instructions in the memory and execute the method according to any one of claims 21 to 26.
  29. 一种计算机可读存储介质，包括指令，当其在计算机上运行时，使得计算机执行如权利要求14-20任意一项所述的方法。A computer-readable storage medium includes instructions that, when run on a computer, cause the computer to perform the method according to any one of claims 14-20.
  30. 一种计算机可读存储介质，包括指令，当其在计算机上运行时，使得计算机执行如权利要求21-26任意一项所述的方法。A computer-readable storage medium includes instructions that, when run on a computer, cause the computer to perform the method according to any one of claims 21-26.
  31. 一种包含指令的计算机程序产品，当其在计算机上运行时，使得计算机执行如权利要求14-20任意一项所述的方法。A computer program product containing instructions which, when run on a computer, causes the computer to perform the method according to any one of claims 14-20.
  32. 一种包含指令的计算机程序产品，当其在计算机上运行时，使得计算机执行如权利要求21-26任意一项所述的方法。A computer program product containing instructions which, when run on a computer, causes the computer to perform the method according to any one of claims 21-26.
PCT/CN2019/075037 2018-06-06 2019-02-14 Data processing device and method WO2019233118A1 (en)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201810576152.4 2018-06-06
CN201810576152.4A CN110568992A (en) 2018-06-06 2018-06-06 data processing device and method

Publications (1)

Publication Number Publication Date
WO2019233118A1 true WO2019233118A1 (en) 2019-12-12

Family

ID=68769503

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2019/075037 WO2019233118A1 (en) 2018-06-06 2019-02-14 Data processing device and method

Country Status (2)

Country Link
CN (1) CN110568992A (en)
WO (1) WO2019233118A1 (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20190342093A1 (en) * 2019-06-28 2019-11-07 Siddhartha Chhabra Converged cryptographic engine

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101646168A (en) * 2009-08-24 2010-02-10 深圳华为通信技术有限公司 Data encryption method, decryption method and mobile terminal
CN102306114A (en) * 2010-09-25 2012-01-04 广东电子工业研究院有限公司 Regular data backup and recovery method based on cloud storage
CN103902932A (en) * 2014-03-27 2014-07-02 杭州师范大学 Data encryption and decryption device and method for USB storage devices
CN106250783A (en) * 2016-08-31 2016-12-21 天津南大通用数据技术股份有限公司 A kind of database data encryption, decryption method and device
CN106506518A (en) * 2016-11-25 2017-03-15 天津津航计算技术研究所 The encryption/decryption module of big data safe transmission

Family Cites Families (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
DE10113829A1 (en) * 2001-03-21 2002-09-26 Infineon Technologies Ag Processor for security-relevant applications has memory storing coded and compressed data coupled to computing unit via decoding and de-compression device
CN101646994B (en) * 2006-12-06 2016-06-15 才智知识产权控股公司(2) Utilize memory bank to interlock and manage the device of order of solid-state memory, system and method
US20120102455A1 (en) * 2010-10-26 2012-04-26 Lsi Corporation System and apparatus for hosting applications on a storage array via an application integration framework
US9069703B2 (en) * 2011-04-29 2015-06-30 Seagate Technology Llc Encrypted-transport solid-state disk controller
US8751830B2 (en) * 2012-01-23 2014-06-10 International Business Machines Corporation Memory address translation-based data encryption/compression
US20150089245A1 (en) * 2013-09-26 2015-03-26 Asher M. Altman Data storage in persistent memory
CN104410683B (en) * 2014-11-21 2017-10-17 四川神琥科技有限公司 A kind of data back up method
CN106533652A (en) * 2015-09-15 2017-03-22 中兴通讯股份有限公司 XTS-SM4-based storage encryption and decryption method and device
CN108073353B (en) * 2016-11-15 2020-04-14 华为技术有限公司 Data processing method and device
CN106559427B (en) * 2016-11-22 2020-04-28 台州库得软件有限公司 Data storage and data transmission system

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101646168A (en) * 2009-08-24 2010-02-10 深圳华为通信技术有限公司 Data encryption method, decryption method and mobile terminal
CN102306114A (en) * 2010-09-25 2012-01-04 广东电子工业研究院有限公司 Regular data backup and recovery method based on cloud storage
CN103902932A (en) * 2014-03-27 2014-07-02 杭州师范大学 Data encryption and decryption device and method for USB storage devices
CN106250783A (en) * 2016-08-31 2016-12-21 天津南大通用数据技术股份有限公司 A kind of database data encryption, decryption method and device
CN106506518A (en) * 2016-11-25 2017-03-15 天津津航计算技术研究所 The encryption/decryption module of big data safe transmission

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20190342093A1 (en) * 2019-06-28 2019-11-07 Siddhartha Chhabra Converged cryptographic engine

Also Published As

Publication number Publication date
CN110568992A (en) 2019-12-13

Similar Documents

Publication Publication Date Title
US11558174B2 (en) Data storage method, device, related equipment and cloud system for hybrid cloud
US8645713B2 (en) Encrypting method, recording medium of encrypting program, decrypting method, and recording medium of decrypting program
US10958416B2 (en) Encrypted and compressed data transmission with padding
US9852025B2 (en) Protecting data stored on a solid state drive
US20090190760A1 (en) Encryption and compression of data for storage
US11387980B2 (en) Hardware multiple cipher engine
WO2021072878A1 (en) Audio/video data encryption and decryption method and apparatus employing rtmp, and readable storage medium
WO2022100498A1 (en) Method and device for file transmission
US8402282B2 (en) Method and device for encrypting and decrypting digital data
JP6647340B2 (en) Improved file compression and encryption
US8677123B1 (en) Method for accelerating security and management operations on data segments
EP3270322B1 (en) Encrypting system level data structures
CN108737353B (en) Data encryption method and device based on data analysis system
WO2019233118A1 (en) Data processing device and method
US20170288861A1 (en) Data encryption
CN113704206B (en) Metadata processing method and device, electronic equipment and storage medium
CN114174978B (en) Method and system for key compressible encryption
KR20230023586A (en) Systems, methods, and apparatus for dividing and encrypting data
KR20150105405A (en) Method and apparatus for a computable, large, variable and secure substitution box
US9734154B2 (en) Method and apparatus for storing a data file
CN114521260A (en) Method and system for data deduplication and compression in untrusted storage systems
CN116391185A (en) Method and system for differential deduplication in untrusted storage
WO2010146666A1 (en) Information processing device, data processing method and program
US10505713B2 (en) Compression and/or encryption of a file
CN113177213B (en) Encryption card and processing method of encrypted message thereof

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 19816179

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 19816179

Country of ref document: EP

Kind code of ref document: A1