CN111859426B - Universal encrypted database connector and setting method thereof - Google Patents
Universal encrypted database connector and setting method thereof Download PDFInfo
- Publication number
- CN111859426B CN111859426B CN202010707286.2A CN202010707286A CN111859426B CN 111859426 B CN111859426 B CN 111859426B CN 202010707286 A CN202010707286 A CN 202010707286A CN 111859426 B CN111859426 B CN 111859426B
- Authority
- CN
- China
- Prior art keywords
- database
- module
- encryption
- packet
- sql
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F21/00—Security arrangements for protecting computers, components thereof, programs or data against unauthorised activity
- G06F21/60—Protecting data
- G06F21/602—Providing cryptographic facilities or services
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/242—Query formulation
- G06F16/2433—Query languages
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F21/00—Security arrangements for protecting computers, components thereof, programs or data against unauthorised activity
- G06F21/60—Protecting data
- G06F21/62—Protecting access to data via a platform, e.g. using keys or access control rules
- G06F21/6218—Protecting access to data via a platform, e.g. using keys or access control rules to a system of files or objects, e.g. local or distributed file system or database
- G06F21/6227—Protecting access to data via a platform, e.g. using keys or access control rules to a system of files or objects, e.g. local or distributed file system or database where protection concerns the structure of data, e.g. records, types, queries
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D30/00—Reducing energy consumption in communication networks
- Y02D30/50—Reducing energy consumption in communication networks in wire-line communication networks, e.g. low power modes or reduced link rate
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Health & Medical Sciences (AREA)
- Computer Security & Cryptography (AREA)
- Software Systems (AREA)
- Computer Hardware Design (AREA)
- Bioethics (AREA)
- Health & Medical Sciences (AREA)
- Databases & Information Systems (AREA)
- Mathematical Physics (AREA)
- Computational Linguistics (AREA)
- Data Mining & Analysis (AREA)
- Storage Device Security (AREA)
- Data Exchanges In Wide-Area Networks (AREA)
Abstract
A universal encryption database connector and a setting method thereof are provided, wherein a communication connection module is arranged and used for communicating with an application layer, a database server and an encryption and decryption module; setting a protocol analysis module for analyzing important information in the protocol packet from various types of databases; the encryption and decryption module is arranged and used for encrypting or decrypting SQL statements or result sets which are submitted by a delivery layer in the protocol analysis module and need to be encrypted or decrypted, and transmitting the encrypted text sets or the decrypted plaintext sets to the protocol recombination module for recombination; setting a protocol recombination module for recombining the SQL sentences into SQL requests and recombining the result sets into SQL responses; and the service module is arranged and used for providing auxiliary functions. The invention provides uniform transparent database access for various types of database drivers of an application layer, stores user data in a ciphertext mode and facilitates the integration of the existing system and the migration of the database.
Description
Technical Field
The invention belongs to the field of information security, and more particularly to a universal encryption database connector and a method of setting the same.
Background
With the explosive growth of information in the current era, two closely-connected branches of big data and cloud computing are derived from the internet field, a profound revolution is brought to the storage mode of business data, the cost brought by the traditional method of storing business data by purchasing local physical storage equipment and managing and maintaining the distributed human resources is greatly increased, more and more enterprises and individuals select to store a large amount of data on the cloud, but the privacy problem brought by the service provided by a third party organization brings worry, and how to ensure the security of a database becomes a research hotspot in the current information security field.
Database encryption is a feasible solution proposed for such a requirement, and the scheme ensures the security of data by encrypting data in advance and then handing the data to a database server for storage. The access performance of the database is usually the bottleneck of the performance of an application layer, the encryption and decryption of data inevitably bring about the influence on the performance and the storage space, and in order to reduce the influence as much as possible and improve the flexibility of the application layer, fields in a table are selected as basic units of encryption, and a classification function encryption model is adopted to support different types of ciphertext operations. Cryptdb is proposed by MIT's CSAIL on the 11 year sp, which implements homomorphic encryption technology on databases from a functional point of view, but its platform support becomes its limitation, it can only support MySQL databases, while many businesses and individuals currently use other types of databases. In terms of performance, the cryptotdb core modules are relatively high in coupling degree, high performance requirements are difficult to expand and meet, and delay of application access to the database is increased. From the aspect of cost, the maintenance of the ready-made project can cost a great deal of cost, and the integration of a new module with as few modifications as possible or the improvement of the original module can greatly reduce the burden and reduce the probability of new problems.
Disclosure of Invention
The invention aims to provide a universal encryption database connector and a setting method thereof aiming at the problems that the encryption database technology cannot meet various types of database drives, is difficult to expand and has higher cost, so that uniform transparent database access can be provided for various types of database drives of an application layer, user data is stored in a ciphertext mode, the safety of the data is ensured, the coupling degree between core modules is reduced, the expansion is easy, and the integration of the existing system and the migration of the database are facilitated.
In order to achieve the purpose, the invention has the following technical scheme:
a universal encrypted database connector comprising:
the communication connection module is used for communicating with the application layer, the database server and the encryption and decryption module;
the protocol analysis module is used for analyzing important information in the protocol packet from various types of databases, wherein the important information comprises database driving information in a handshake stage, SQL sentences in a request packet and a result set in a response packet;
the encryption and decryption module is used for encrypting or decrypting SQL statements or result sets which are submitted by a delivery layer in the protocol analysis module and need to be encrypted or decrypted, and transmitting the encrypted or decrypted ciphertext sets or plaintext sets to the protocol recombination module for recombination;
the protocol recombination module is used for recombining the SQL sentences into SQL requests and recombining the result sets into SQL responses;
and the service module is used for providing an auxiliary function.
As a preferred scheme, the communication between the communication connection module and the application layer can adapt to various types of database drives in the aspect of functions, and the database types are obtained in the handshake stage according to the protocol packet format obtained by analysis of the preprocessing layer in the protocol analysis module; in the aspects of availability and performance, the connector cluster exists in the form of a reverse proxy, and is added with a Linux virtual service-dynamic routing (LVS-DR) dual-computer hot-standby layer and a Nginx load balancing layer; when the communication connection module communicates with the database server, the connector is connected with the database of the type corresponding to the application, each type of database cluster is deployed, and the database operation is balanced; when the communication connection module and the encryption and decryption module are communicated, a protocol for encapsulating SQL statements and result sets is designed, and the protocol is divided into two conditions of application requests and database responses: if the request is an application request, writing an unencrypted SQL statement packet, and reading an encrypted SQL statement packet; if the result is the database response, writing out the undecrypted result set packet, and reading in the decrypted result set packet.
As a preferred scheme, the protocol analysis module comprises a preprocessing layer, an analysis layer, a judgment layer and a delivery layer, wherein the preprocessing layer is used for analyzing the communication protocol format of the database; the analysis layer is used for analyzing the character strings in the communication protocol format of the database; the judgment layer is used for judging whether the extracted character string needs to be encrypted or decrypted, and comprises encryption judgment and decryption judgment, wherein the encryption judgment is used for matching according to the extracted SQL sentence and the pattern string, the decryption judgment is used for matching according to the extracted column metadata and the pattern string, if the white list matching is successful or the black list matching is failed, the encryption or the decryption is needed, otherwise, the encryption or the decryption is not needed; the delivery layer is used for sending SQL sentences or result sets which do not need to be encrypted or decrypted to the connection module for processing, and sending SQL sentences or result sets which need to be encrypted or decrypted to the encryption and decryption module for processing.
As a preferred scheme, the encryption and decryption module carries out serial processing on the SQL statement or the result set, wherein the serial processing comprises SQL statement recombination, key group query and encryption and decryption calculation; SQL sentence recombination is to replace the column attribute needing encryption or decryption with a special format which can be understood by a ciphertext database; in the stage of generating a syntax tree for the SQL statement, determining a corresponding encryption or decryption algorithm according to the data type of the column attribute to be encrypted or decrypted, and acquiring a key group corresponding to a database table from a key manager; according to the SQL statement recombination format, the data is encrypted by using an encryption algorithm and an encryption key group, the ciphertext data is returned to the delivery layer, the ciphertext data is decrypted by firstly obtaining a result set sent by the delivery layer, then the ciphertext data is decrypted by using a decryption algorithm and a decryption key group, and finally the plaintext data is returned to the delivery layer.
As a preferred scheme, the protocol recombination module obtains the protocol packet format analyzed in the preprocessing stage according to the type of the currently processed database, then analyzes the processed SQL statement or result set from the packet sent by the encryption and decryption module, completes the replacement and filling of the byte stream according to the format of the protocol packet, and finally hands the byte stream to the connection module to be sent to the receiver.
As a preferred scheme, the service module comprises a configuration analysis submodule, a keepalive submodule and a log submodule; when the connector service is started, the configuration analysis submodule reads the content in the configuration file into the memory and structurizes the content; the keepalive sub-module is used for judging whether the main LVS dispatcher survives from the LVS dispatcher and judging whether the database server survives from the main connector service, the main LVS dispatcher executes virtual address drifting operation if the main LVS dispatcher does not survive, and the database server is switched to the next database server if the database server does not survive; the log submodule records logs in a grading mode, the logs are embedded into all the modules, the slave servers of the database can be recovered, and the number of the database servers can be expanded dynamically.
The invention also provides a setting method of the universal encryption database connector, which comprises the following steps:
respectively establishing communication with the application layer, the database server and the encryption and decryption module;
analyzing database driving information of a handshake phase, SQL sentences in a request packet and result sets in a response packet for various types of databases, and analyzing whether the SQL sentences need to be encrypted and whether the result sets need to be decrypted;
the SQL statement or result set needing encryption or decryption is encrypted or decrypted, and the encrypted or decrypted secret text set or plaintext set is recombined;
recombining the SQL statements into SQL requests and the result sets into SQL responses;
and, auxiliary services in the above steps.
As a preferred scheme of the invention, when establishing communication with an application layer, receiving a data packet sent by the application layer by using a database driver, if the data packet is still in a handshake stage, obtaining a packet serial number according to a protocol packet format obtained by analysis of a preprocessing layer in a protocol analysis module, judging whether a current packet belongs to a request packet or a response packet according to the packet serial number, assigning the database client information analyzed from the request packet to a database type, judging whether the handshake is successful from the response packet and recording the state, and if the handshake is successful, skipping the handshake stage; when communication is established with a database server, the operation reading and writing of the database are separated, and the operation load of the same type of database is loaded to different database servers by using a hash load balancing algorithm based on an ip address; when the communication is established with the encryption and decryption module, a simple protocol for encapsulating SQL statements and result sets is designed, and the protocol is divided into two conditions of application requests and database responses: if the request is an application request, writing an unencrypted SQL statement packet, and reading an encrypted SQL statement packet; if the result is the database response, writing out the undecrypted result set packet, and reading in the decrypted result set packet.
As a preferred scheme of the invention, when analyzing whether the SQL statement needs to be encrypted, matching is carried out according to the extracted SQL statement and the pattern string, when analyzing whether the result set needs to be decrypted, matching is carried out according to the metadata of the extracted column and the pattern string, if the white list matching is successful or the black list matching is failed, encryption or decryption is required, otherwise, encryption or decryption is not required; the encryption or decryption process performs a serial process on the SQL statement or result set.
As a preferred solution of the present invention, the auxiliary service includes reading the content in the configuration file into the memory and structuring when the connector service is started; judging whether the master LVS scheduler is alive or not and judging whether the database server is alive or not from the master connector service by the slave LVS scheduler, executing virtual address drifting operation if the master LVS scheduler is not alive, and switching to the next database server if the database server is not alive; and (4) recording logs in a grading manner, embedding the logs into each module, recovering the slave servers of the database, and dynamically expanding the number of the servers of the database.
Compared with the prior art, the invention has the following beneficial effects:
aiming at the defects that the existing encryption database technology cannot meet the requirements of various types of database drives, is difficult to expand, has higher integration cost and the like, the invention provides uniform transparent database access for various types of database drives of an application layer in a reverse proxy mode on the premise that the application layer only needs to modify a database address (ip) and a port (port), simultaneously ensures that the corresponding type of database is connected, and data is stored in the database in a ciphertext mode. The invention can provide uniform transparent database access for various types of database drivers of an application layer, store user data in a ciphertext mode, ensure the safety of data, reduce the coupling degree between core modules, is easy to expand and is convenient for the integration of the existing system and the migration of the database.
Drawings
FIG. 1 is a block diagram of a universal cryptographic database connector of the present invention;
FIG. 2 is a system architecture diagram of the universal cryptographic database connector of the present invention;
fig. 3 is a flow chart of the universal encryption database connector setup method of the present invention.
Detailed Description
The present invention will be described in further detail with reference to the accompanying drawings and examples.
Referring to fig. 1, the universal encrypted database connector provided by the present invention includes a communication connection module, a protocol parsing module, an encryption/decryption module, a protocol reassembly module, and a service module. With reference to fig. 2 and 3, the setting method of the present invention includes:
1. a communication connection module:
the module management is connected with the socket among the application layer, the database server and the encryption and decryption module, wherein:
for the socket connection with the application layer, receiving a data packet sent by the application layer by using a database driver, if the data packet is still in a handshake stage, obtaining a packet serial number according to a protocol packet format obtained by analysis of a preprocessing layer in a protocol analysis module, judging whether a current packet belongs to a request packet or a response packet according to the packet serial number, analyzing database client information from the request packet, assigning the database client information to a DBType (database type), judging whether the handshake is successful from the response packet, and recording the state (isConnected). And if the handshake is successful, skipping a handshake stage, wherein the socket connection between corresponding databases needs to be established in the first interactive process after the handshake is successful, and finally, the SQL request packet and the DBType are delivered to the protocol analysis module for processing. In order to improve the availability of the system, an LVS-DR dual-computer hot-standby layer and an Nginx load balancing layer are added in front of the connection, and a polling mode can be used for distributing requests;
and for the socket connection with the database, the write-read protocol analysis module processes the communication packets of the database before and after the processing. The operation of the databases is separated from reading and writing, and the operation load of the same type of database can be loaded to different database servers by using a hash load balancing algorithm based on an ip address;
for the socket connection with the encryption and decryption module, a simple protocol for encapsulating SQL statements and result sets (ResultSet) is designed, and the protocol is divided into two cases of application requests and database responses: if the request is an application request, writing an unencrypted SQL statement packet, and reading an encrypted SQL statement packet; if the result is the database response, writing out the undecrypted result set packet, and reading in the decrypted result set packet. The following protocol may be used:
SQL statement package: the Payload is Payload partial byte number and occupies 4 bytes. Payload is a SQL plain text statement obtained by a 1-bit packet type, a 4-bit database type obtained by a communication connection module, a database name in a length coding format and a protocol analysis module or an SQL ciphertext statement encrypted by an encryption and decryption module.
ResultSet packet: the Header is Payload partial byte number, and occupies 4 bytes. Payload is a ResultSet ciphertext string analyzed by a 1-bit packet type + protocol analysis module or a ResultSet plaintext string decrypted by an encryption and decryption module, metadata of each column occupies one row, each record occupies one row, a # C # marks one datum, a # L # marks one row end, metadata of a # T # mark column ends, a type of a # R # mark column and a width of a # G # mark column.
2. A protocol analysis module:
(1) A hierarchy to achieve generic analytical properties;
the protocol analysis module is divided into 4 layers in total, and sequentially comprises a pretreatment layer, an analysis layer, a fault judgment layer and a delivery layer from top to bottom.
Wherein, the preprocessing layer is responsible for analyzing the communication protocol format of the database. If the application layer uses the open source database, directly skipping the stage; if the application layer uses a hybrid database (open source and closed source hybrid databases), important fields in the communication protocol format of the database can be obtained through the analysis of the stage. The analysis layer is responsible for analyzing the character strings in the communication protocol format of the database, and is divided into SQL request analysis and SQL response analysis, wherein the SQL request analysis is used for extracting character string information such as database names (Schema) and SQL sentences, and the SQL response analysis is used for mainly extracting result sets (including metadata and records of columns). The judgment layer is responsible for judging whether the extracted character string needs to be encrypted or decrypted, and comprises encryption judgment and decryption judgment, wherein the encryption judgment and the decryption judgment are carried out on the extracted character string according to the extracted SQL sentence and the extracted pattern string, the matching method comprises a white list and a black list, the two methods are generally used in a mixed mode, if the white list is successfully matched or the black list is failed to be matched, encryption and decryption are needed, and otherwise, encryption and decryption are not needed. The delivery layer is responsible for delivering the SQL statement/result set which does not need to be encrypted/decrypted to the connection module for processing, and delivering the SQL statement/result set which needs to be encrypted/decrypted to the encryption/decryption module for processing.
(2) The pretreatment layer realizes the flow;
the preprocessing layer is executed along with the start of the connector service and comprises the following three steps:
step one, if a user is configured with a closed source database in an application layer, executing step two, otherwise, directly executing step three;
and step two, traversing each configured closed source database, and obtaining the meaning represented by the important field in the corresponding database communication protocol format by using the conventional protocol reverse analysis technology. A certain fixed-version database can be used for analysis, a large number of repeated tests are carried out on the data replaced by the same type of operation control variable method, and a byte segmentation format corresponding to the minimum distance between the data and a characteristic vector (segmenting bytes) of the data is calculated by taking certain data operation as a reference. The byte stream format of a data operation is then presumed in that format and verified. Finally, the format and meaning of the protocol main field corresponding to the main operation are obtained;
and step three, jumping to a resolution layer.
(3) A resolution layer implementation process;
the analysis layer comprises two parts of SQL request analysis and SQL response analysis.
Step one, obtaining a database protocol format of a preprocessing layer according to a DBType (database type), obtaining a current packet serial number sid from a byte stream according to the format, if sid =0, executing step two, and if sid =1, executing step three;
step two, analyzing a database name (Schema) and an SQL statement in the SQL request, submitting the Schema, the SQL statement and the DBType to an encryption judgment part for processing, and ending;
step three, analyzing a result set (ResultSet) in the SQL response, and handing 'ResultSet and DBType' to a decryption judgment part for processing;
(4) Judging a fault implementation flow;
the judgment layer comprises an encryption judgment part and a decryption judgment part, wherein the encryption judgment part judges whether the SQL statement needs to be encrypted or not, and the decryption judgment part judges whether the analysis result set needs to be decrypted or not.
The encryption judgment comprises the following three steps:
step one, defining a template character string comprising a white list and a black list, and reducing a memory occupied by the list by using a wildcard or a regular expression. Different databases have different SQL grammars, and each database can be defined with a set of template character strings, but a great deal of redundancy is necessarily brought. On the other hand, the categories of different SQL sentences are fixed, the SQL sentences can be classified to form a classification tree, each node in the tree represents a category, the template character strings can summarize different databases, the root of the tree is the SQL sentences, the second layer of the tree is divided into three categories of SELECT, DML and DDL, the included template character strings are the first keywords of the SQL sentences, the third layer of the tree is divided into two categories of FUNC and NOT FUNC, the included template character strings are method mark strings, the fourth layer of the tree is divided into two categories of MARKED and NORMAL, and the included template character strings are database names (Schema) encrypted prefix and prefix mark strings;
matching nodes in the classification tree by using 'Schema and SQL statements', if the white list matching is successful or the black list matching is failed, assigning the needEnc (whether the identification needs to be encrypted) as True, otherwise, assigning the needEnc as default False;
and step three, transmitting the 'Schema, SQL statement, DBType and needEnc' to a delivery layer for processing.
The decryption judgment comprises the following three steps:
step one, if the packet is an OK packet or an ERROR packet, directly executing step three;
step two, if the metadata (generally column names) of the columns in the result set (ResultSet) contains encryption flag strings (such as oedet, opope), assigning value to needDec (identifying whether decryption is needed) as True, otherwise, assigning value to default False;
and step three, transmitting 'ResultSet, DBType and needDec' to a delivery layer for processing.
(5) A delivery layer implementation flow;
the delivery layer comprises two parts of SQL statement delivery and result set delivery.
And (3) SQL statement delivery: if the needEnc is True, packaging the Schema, the SQL statement and the DBType into an SQL statement packet, and sending the SQL statement packet to the encryption and decryption module for processing to obtain the encrypted SQL statement. Finally, the SQL statement and the DBType are handed to the protocol reorganization module.
Result set delivery: and if the needDec is True, encapsulating the result set into a ResultSet packet and sending the ResultSet packet to the encryption and decryption module for processing to obtain the decrypted ResultSet. Finally, the "ResultSet and DBType" are handed to the protocol reassembly module.
3. An encryption and decryption module:
(1) Serialization processing for realizing encryption and decryption;
the encryption and decryption module carries out serial processing on the SQL statement/result set, and comprises the following three steps: SQL statement recombination, key group query and encryption and decryption calculation. The SQL sentence recombination is responsible for converting the SQL sentences of the plaintext into a special format which can be understood by the ciphertext database; the key group inquiry is responsible for acquiring a key group corresponding to the column attribute needing encryption/decryption of the database table from the key manager; the encryption and decryption calculation is responsible for encrypting plaintext data or decrypting ciphertext data.
(2) SQL sentence recombination realization flow;
the SQL statement restructuring part comprises the determination of a database parser and the restructuring replacement of the SQL statement.
The method comprises the following steps: and analyzing the SQL statement by using a corresponding database analyzer according to the DBType (database type) in the SQL statement packet.
Step two: the database parser performs lexical analysis, syntactic analysis and semantic analysis on the SQL sentences, wherein in the grammar tree generation stage of the syntactic analysis, the encryption/decryption algorithm corresponding to the column attributes needing encryption/decryption is determined, and the column attributes encrypted/decrypted by the SQL sentences are replaced by the SQL sentences in a special format which can be understood by the ciphertext database according to the name of the algorithm.
(3) A key group query realization process;
the key group inquiry part comprises an encryption/decryption algorithm corresponding to the determined column attribute and a key group of the database table.
The method comprises the following steps: in the syntax tree generation phase of the SQL statement syntax analysis, an encryption/decryption algorithm corresponding to the column attribute is determined according to the data type of the column attribute needing encryption/decryption along the generation process of the syntax tree.
Step two: and acquiring a key group corresponding to the database table from the key manager according to the determined encryption/decryption algorithm, mapping and storing the key group in the memory, and supplying the key group to subsequent encryption/decryption operation call.
(4) Encryption and decryption calculation realization flow;
the method comprises the following steps: and performing encryption operation on the inserted data, encrypting the data to be encrypted by using an encryption algorithm and an encryption key group according to a special format which can be understood by the ciphertext database after the SQL statement is recombined, and returning the encrypted ciphertext data to the delivery layer.
Step two: and (3) carrying out decryption operation on the ciphertext data, firstly obtaining corresponding ciphertext data from a ciphertext database according to the SQL recombination statement, then carrying out decryption processing on the ciphertext data by using a decryption algorithm and a decryption key group, and finally returning the plaintext data to the delivery layer.
4. A protocol recombination module:
the inverse operation of the protocol analysis comprises two parts of the restructuring of SQL statements into SQL requests and the restructuring of result sets into SQL responses. The database protocol format of a preprocessing layer is obtained according to DBType (database type), SQL statement/ResultSet character strings are converted into corresponding byte streams according to the format, and finally the byte streams are delivered to a connecting module to be sent to a receiver. The process of restructuring the decrypted result set corresponding to the select simple query of the MySQL type database is as follows:
step one, creating a class structure A of a MySQL type response packet according to the value of the DBType;
receiving a byte array transmitted by the communication connection module, namely a ResultSet packet decrypted and packaged by the encryption and decryption module, analyzing field metadata (mainly field names of plaintext) and plaintext records according to the format of the packet, and assigning values to the fields in A;
and step three, assembling the fields in the A according to the MySQL response packet format, delivering the assembled fields to a communication connection module, and sending the assembled fields to an application.
5. A service module:
providing auxiliary functions for the whole system, wherein the auxiliary functions are divided into a configuration analysis submodule, a keepalive submodule and a log submodule, and the configuration analysis submodule comprises:
when the connector service is started, the configuration analysis submodule reads the content in the configuration file into a memory and structures the content, and can use Map as a data storage structure, key as the name of a configuration section, value as a Map structure, key as an option name, and value as an option value;
the keepalive submodule belongs to a kind of detection and activation mechanism, including judging whether the main LVS scheduler is alive (if not, executing virtual address drift operation) from the LVS scheduler and judging whether the database server is alive (if not, switching to the next database server) from the main connector service, wherein the maximum non-response time is determined by using two parameters of interval and maxCounts;
and the log submodule records logs in a grading way and is embedded into each module. Bin _ log for the protocol parsing module may be used to recover the database slave servers, dynamically expanding the database server count.
The universal encrypted database connector provided by the invention can provide uniform transparent database access for various database drivers of an application layer, stores user data in a ciphertext mode, ensures the security of the data, reduces the coupling degree between core modules, is easy to expand, and is convenient for the integration of the existing system and the migration of the database.
The above description is only a preferred embodiment of the present invention and is not intended to limit the technical solution of the present invention, and it should be understood by those skilled in the art that the technical solution can be modified and changed in several simple ways without departing from the spirit and basic concept of the present invention, and the modified and changed technical solution also belongs to the protection scope covered by the claims.
Claims (4)
1. A universal encrypted database connector, comprising:
the communication connection module is used for communicating with the application layer, the database server and the encryption and decryption module;
the communication between the communication connection module and the application layer can be adapted to various types of database drives in the aspect of functions, and the database types are obtained in the handshake stage according to the protocol packet format obtained by analysis of the preprocessing layer in the protocol analysis module; in the aspects of availability and performance, the connector cluster exists in the form of a reverse proxy, and is added with a Linux virtual service-dynamic routing dual-computer hot standby layer and an Nginx load balancing layer;
when the communication connection module communicates with the database server, the connector is connected with the database of the type corresponding to the application, each type of database cluster is deployed, and the database operation is balanced;
when the communication connection module communicates with the encryption and decryption module, a protocol for packaging SQL statements and result sets is designed, and the protocol is divided into two conditions of application requests and database responses: if the request is an application request, writing an unencrypted SQL statement packet, and reading an encrypted SQL statement packet; if the result is the database response, writing out the undecrypted result set packet, and reading in the decrypted result set packet;
the protocol analysis module is used for analyzing important information in the protocol packet from various types of databases, wherein the important information comprises database driving information in a handshake stage, SQL sentences in a request packet and a result set in a response packet;
the protocol analysis module comprises a pretreatment layer, an analysis layer, a fault judgment layer and a delivery layer, wherein the pretreatment layer is used for analyzing the communication protocol format of the database; the analysis layer is used for analyzing the character strings in the communication protocol format of the database; the judgment layer is used for judging whether the extracted character string needs to be encrypted or decrypted, and comprises encryption judgment and decryption judgment, wherein the encryption judgment is used for matching according to the extracted SQL sentence and the pattern string, the decryption judgment is used for matching according to the extracted column metadata and the pattern string, if the white list matching is successful or the black list matching is failed, the encryption or the decryption is needed, otherwise, the encryption or the decryption is not needed; the delivery layer is used for sending SQL sentences or result sets which do not need to be encrypted or decrypted to the connection module for processing, and sending the SQL sentences or result sets which need to be encrypted or decrypted to the encryption and decryption module for processing;
the encryption and decryption module is used for encrypting or decrypting SQL statements or result sets which are submitted by a delivery layer in the protocol analysis module and need to be encrypted or decrypted, and transmitting the encrypted text sets or the decrypted plaintext sets to the protocol recombination module for recombination;
the protocol recombination module is used for recombining the SQL sentences into SQL requests and recombining the result sets into SQL responses;
the service module is used for providing an auxiliary function;
the service module comprises a configuration analysis sub-module, a keepalive sub-module and a log sub-module; when the connector service is started, the configuration analysis submodule reads the content in the configuration file into the memory and structurizes the content; the keepalive sub-module is used for judging whether the main LVS dispatcher survives from the LVS dispatcher and judging whether the database server survives from the main connector service, the main LVS dispatcher executes virtual address drifting operation if the main LVS dispatcher does not survive, and the database server is switched to the next database server if the database server does not survive; the log submodule records logs in a grading mode, the logs are embedded into all the modules, the slave servers of the database can be recovered, and the number of the database servers can be expanded dynamically.
2. The universal encrypted database connector according to claim 1, wherein: the encryption and decryption module carries out serial processing on the SQL statement or the result set, wherein the processing comprises SQL statement recombination, key group query and encryption and decryption calculation; SQL sentence recombination is to replace the column attribute needing encryption or decryption with a special format which can be understood by a ciphertext database; in the stage of generating a syntax tree for the SQL statement, determining a corresponding encryption or decryption algorithm according to the data type of the column attribute to be encrypted or decrypted, and acquiring a key group corresponding to a database table from a key manager; according to the SQL statement recombination format, the data is encrypted by using an encryption algorithm and an encryption key group, the ciphertext data is returned to the delivery layer, the ciphertext data is decrypted by firstly obtaining a result set sent by the delivery layer, then the ciphertext data is decrypted by using a decryption algorithm and a decryption key group, and finally the plaintext data is returned to the delivery layer.
3. The universal cryptographic database connector of claim 1, wherein:
the protocol recombination module obtains a protocol packet format analyzed in a preprocessing stage according to the type of a currently processed database, then analyzes a processed SQL statement or result set from a packet sent by the encryption and decryption module, completes the replacement and filling of a byte stream according to the format of the protocol packet, and finally sends the byte stream to the connection module to be sent to a receiver.
4. A method for setting a universal encryption database connector is characterized by comprising the following steps:
establishing communication with the application layer, the database server and the encryption and decryption module;
when communication is established with an application layer, receiving a data packet sent by the application layer by using a database driver, if the data packet is still in a handshake stage, obtaining a packet serial number according to a protocol packet format obtained by analysis of a preprocessing layer in a protocol analysis module, judging whether a current packet belongs to a request packet or a response packet according to the packet serial number, analyzing database client information from the request packet, assigning the database type, judging whether handshake is successful or not from the response packet, recording, and if handshake is successful, skipping the handshake stage;
when communication is established with a database server, the operation reading and writing of the database are separated, and the operation load of the same type of database is loaded to different database servers by using a hash load balancing algorithm based on an ip address;
when the communication is established with the encryption and decryption module, a protocol for encapsulating SQL statements and result sets is designed, and the protocol is divided into two conditions of application requests and database responses: if the request is an application request, writing an unencrypted SQL statement packet, and reading an encrypted SQL statement packet; if the result is the database response, writing out the undecrypted result set packet, and reading in the decrypted result set packet;
analyzing database driving information of a handshake phase, SQL sentences in a request packet and result sets in a response packet for various types of databases, and analyzing whether the SQL sentences need to be encrypted and whether the result sets need to be decrypted; whether the SQL sentences need to be encrypted or not is analyzed, matching is carried out according to the extracted SQL sentences and the pattern strings, whether the analysis result set needs to be decrypted or not is carried out according to the extracted metadata of the columns and the pattern strings, if the white list matching is successful or the black list matching is failed, encryption or decryption is needed, and if not, encryption or decryption is not needed; carrying out serial processing on the SQL statement or result set by encryption or decryption processing;
encrypting or decrypting SQL sentences or result sets which need to be encrypted or decrypted, and recombining encrypted text sets or plaintext sets after encryption or decryption;
recombining the SQL statements into SQL requests and the result sets into SQL responses;
and, the auxiliary service in the above steps;
the auxiliary service comprises reading the content in the configuration file into the memory and structuring when the connector service is started; judging whether the master LVS scheduler is alive or not and judging whether the database server is alive or not from the master connector service by the slave LVS scheduler, executing virtual address drifting operation if the master LVS scheduler is not alive, and switching to the next database server if the database server is not alive; and (4) recording logs in a grading manner, embedding the logs into each module, recovering the slave servers of the database, and dynamically expanding the number of the servers of the database.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010707286.2A CN111859426B (en) | 2020-07-21 | 2020-07-21 | Universal encrypted database connector and setting method thereof |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010707286.2A CN111859426B (en) | 2020-07-21 | 2020-07-21 | Universal encrypted database connector and setting method thereof |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111859426A CN111859426A (en) | 2020-10-30 |
| CN111859426B true CN111859426B (en) | 2023-04-07 |
Family
ID=73001453
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010707286.2A Active CN111859426B (en) | 2020-07-21 | 2020-07-21 | Universal encrypted database connector and setting method thereof |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111859426B (en) |
Families Citing this family (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112711762A (en) * | 2020-12-22 | 2021-04-27 | 航天信息股份有限公司 | Transparent encryption method for database |
| WO2023010273A1 (en) * | 2021-08-03 | 2023-02-09 | 浙江大学 | Database encryption method that supports combinable sql query |
| CN115563088B (en) * | 2022-12-08 | 2023-07-25 | 广东睿江云计算股份有限公司 | Data migration method and migration system for databases of different types |
| CN116915387A (en) * | 2023-09-14 | 2023-10-20 | 山东三未信安信息科技有限公司 | Extensible database transparent encryption device and method |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2016078423A1 (en) * | 2014-11-17 | 2016-05-26 | 中兴通讯股份有限公司 | Transaction processing method and apparatus for distributed database system |
| CN107370725A (en) * | 2017-06-21 | 2017-11-21 | 西安电子科技大学 | The access method and system of general encrypting database under a kind of cloud environment |
| CN108509805A (en) * | 2018-03-21 | 2018-09-07 | 深圳天源迪科信息技术股份有限公司 | Data encrypting and deciphering and desensitization runtime engine and its working method |
| CN108734023A (en) * | 2018-04-28 | 2018-11-02 | 西安电子科技大学 | A kind of access of Encrypted Database System and integrated system and method |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20090182707A1 (en) * | 2008-01-10 | 2009-07-16 | Dbix Corporation | Database changeset management system and method |
-
2020
- 2020-07-21 CN CN202010707286.2A patent/CN111859426B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2016078423A1 (en) * | 2014-11-17 | 2016-05-26 | 中兴通讯股份有限公司 | Transaction processing method and apparatus for distributed database system |
| CN107370725A (en) * | 2017-06-21 | 2017-11-21 | 西安电子科技大学 | The access method and system of general encrypting database under a kind of cloud environment |
| CN108509805A (en) * | 2018-03-21 | 2018-09-07 | 深圳天源迪科信息技术股份有限公司 | Data encrypting and deciphering and desensitization runtime engine and its working method |
| CN108734023A (en) * | 2018-04-28 | 2018-11-02 | 西安电子科技大学 | A kind of access of Encrypted Database System and integrated system and method |
Non-Patent Citations (3)
| Title |
|---|
| Manyi Cai等.A Protocol for Extending Analytics Capability of SQL Database.《2016 7th International Conference on Cloud Computing and Big Data (CCBD)》.2017, * |
| Oracle通信TNS协议中请求报文的解析;侯方杰等;《计算机系统应用》;20181015(第10期);全文 * |
| 可搜索数据库加密系统的设计与实现;汪海伟等;《计算机技术与发展》;20170831(第08期);全文 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111859426A (en) | 2020-10-30 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111859426B (en) | Universal encrypted database connector and setting method thereof | |
| CN106874461B (en) | A kind of workflow engine supports multi-data source configuration security access system and method | |
| US6788648B1 (en) | Method and apparatus for load balancing a distributed processing system | |
| US6975595B2 (en) | Method and apparatus for monitoring and logging the operation of a distributed processing system | |
| US7646776B2 (en) | Method and apparatus for generating unique ID packets in a distributed processing system | |
| US8116456B2 (en) | Techniques for managing heterogeneous key stores | |
| US8880596B2 (en) | Software platform and method for processing unstructured data | |
| US20120210396A1 (en) | Processing extensible markup language security messages using delta parsing technology | |
| EP2171614A2 (en) | Transporting table valued parameter over tabular data stream protocol | |
| CN103647636B (en) | The method and device of security access data | |
| JP2009099151A (en) | User query processing system and method by query encryption transformation in database including encrypted column | |
| CN1625179A (en) | Send by reference in a customizable, tag-based protocol | |
| CN108734023B (en) | System and method for accessing and integrating ciphertext database system | |
| CN112416908A (en) | Method and system for analyzing prefix sublibrary storage data based on Handle identification | |
| JP4006214B2 (en) | Data search system, data relay server, database server, and database access method | |
| CN104618410B (en) | Resource supplying method and apparatus | |
| CN115208665A (en) | Block chain-based germplasm resource data secure sharing method and system | |
| CN113190348B (en) | Cross-platform virtual resource allocation method, device, equipment and storage medium | |
| CN100504770C (en) | Method for packing data, and unpacking packed data | |
| CN103957173A (en) | Semantic switch | |
| CN114793244B (en) | Resource processing method, device, equipment and medium for block chain | |
| WO2004023322A1 (en) | Method and apparatus for converting data between two dissimilar systems | |
| CN115146245B (en) | Hive series data encryption method and system with dynamically managed key authority | |
| CN107491361A (en) | The method for being classified other redundant storage is carried out to row in tables of data | |
| CN107480286A (en) | A kind of message processing method and trusted systems |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |