CN114153856A - Host data double-writing method and device - Google Patents

Host data double-writing method and device Download PDF

Info

Publication number
CN114153856A
CN114153856A CN202111403038.XA CN202111403038A CN114153856A CN 114153856 A CN114153856 A CN 114153856A CN 202111403038 A CN202111403038 A CN 202111403038A CN 114153856 A CN114153856 A CN 114153856A
Authority
CN
China
Prior art keywords
update
updating
request
database
data
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202111403038.XA
Other languages
Chinese (zh)
Inventor
田小龙
罗剑平
蓝振杰
马瑞岳
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Industrial and Commercial Bank of China Ltd ICBC
Original Assignee
Industrial and Commercial Bank of China Ltd ICBC
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Industrial and Commercial Bank of China Ltd ICBC filed Critical Industrial and Commercial Bank of China Ltd ICBC
Priority to CN202111403038.XA priority Critical patent/CN114153856A/en
Publication of CN114153856A publication Critical patent/CN114153856A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/23Updating
    • G06F16/235Update request formulation
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/24Querying
    • G06F16/242Query formulation
    • G06F16/2433Query languages
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/24Querying
    • G06F16/245Query processing
    • G06F16/2455Query execution
    • G06F16/24552Database cache management
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/27Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Databases & Information Systems (AREA)
  • Data Mining & Analysis (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Mathematical Physics (AREA)
  • Computational Linguistics (AREA)
  • Computing Systems (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)

Abstract

The embodiment of the application provides a host data double-writing method and a device, which can be used in the field of finance, and the method comprises the following steps: intercepting a database updating operation request in an online transaction request when the online transaction request is monitored, and analyzing the intercepted database updating operation request; generating an updating record query request according to the updating operation condition and the updating table name obtained by analysis, and acquiring updating record information of the database updating operation request before and after execution according to the updating record query request; intercepting the online transaction submitting transaction instruction when monitoring the online transaction submitting transaction instruction, generating a data updating service calling message of a host data service platform according to the updating record information, and executing host database updating operation according to the data updating service calling message; the method and the device can effectively improve the double-writing efficiency and accuracy of the host data.

Description

Host data double-writing method and device
Technical Field
The application relates to the field of data backup and can also be used in the field of finance, in particular to a method and a device for double writing of host data.
Background
In the process of building the lower host and the upper Platform, in order to ensure continuity of Service functions and support quick switching of the host when the Platform fails, in the prior art, a host Data update interface or MDSP (host Data Service Platform) Data update Service corresponding to each function is generally called in a Platform Service to perform double writing of a host database, so that a call code needs to be newly added, development cost is high, double writing processing needs to be called independently for each Service function, and a Platform Service logic code is invaded.
Disclosure of Invention
Aiming at the problems in the prior art, the application provides a host data double-writing method and device, which can effectively improve the efficiency and accuracy of host data double-writing.
In order to solve at least one of the above problems, the present application provides the following technical solutions:
in a first aspect, the present application provides a method for double writing of host data, including:
intercepting a database updating operation request in an online transaction request when the online transaction request is monitored, and analyzing the intercepted database updating operation request;
generating an updating record query request according to the updating operation condition and the updating table name obtained by analysis, and acquiring updating record information of the database updating operation request before and after execution according to the updating record query request;
and intercepting the online transaction submitting transaction instruction when monitoring that the online transaction submitting transaction instruction is submitted, generating a data updating service calling message of a host data service platform according to the updating record information, and executing host database updating operation according to the data updating service calling message.
Further, the generating an update record query request according to the update operation condition and the update table name obtained by the parsing, and acquiring update record information of the database update operation request before and after the execution according to the update record query request includes:
judging whether an update table corresponding to the database update operation request is a technical table without double writing, if so, updating the main storage system according to the database update operation request;
otherwise, generating an update record query request according to the update operation condition and the update table name obtained by analysis, and acquiring a field value corresponding to the update table and a field value corresponding to the update table after the database update operation request is executed;
and determining updated record information according to the updated table name, the updated type corresponding to the updated record query request and the field values before and after the database updating operation request is executed.
Further, intercepting a database update operation request in the online transaction request when the online transaction request is monitored includes:
and when an online transaction request is monitored for the first time in a set time period, putting the service transaction identifier into the context of the double-write task thread, and clearing the updating record information in the updating table.
Further, the generating a data update service call packet of a host data service platform according to the update record information, and executing a host database update operation according to the data update service call packet includes:
judging whether a business transaction identifier exists in the context of the double-write task thread corresponding to the data updating service calling message, and if not, returning a double-write failure signal;
otherwise, the updating record information in the updating table is obtained through traversal, and a data updating service calling message of a host data service platform is generated according to the name and the updating type of the updating table in the updating record information, the field values before and after the execution of the database updating operation request and the set message data format, so that the host data service platform updates the slave storage system according to the data updating service calling message.
Further, intercepting a database update operation request in the online transaction request when the online transaction request is monitored includes:
intercepting a database updating operation request in the online transaction request through an interceptor configured in a persistent layer framework configuration file when the online transaction request is monitored, wherein the database updating operation request is a request for performing data updating operation on an updating table of a main storage system.
Further, the parsing the intercepted database update operation request includes:
analyzing the database updating operation request through a preset connection pool component, replacing a parameter placeholder in a corresponding database operation statement with a preset binding parameter, and obtaining the database operation statement after parameter replacement, wherein the database operation statement comprises an updating operation condition and an updating table name.
In a second aspect, the present application provides a host data dual writing apparatus, including:
the transaction request intercepting module is used for intercepting a database updating operation request in the online transaction request when the online transaction request is monitored, and analyzing the intercepted database updating operation request;
the update information recording module is used for generating an update record query request according to the update operation condition and the update table name obtained by analysis, and acquiring update record information of the database update operation request before and after execution according to the update record query request;
and the data synchronization operation module is used for intercepting the online transaction submitting transaction instruction when the online transaction submitting transaction instruction is monitored, generating a data updating service calling message of a host data service platform according to the updating record information, and executing host database updating operation according to the data updating service calling message.
Further, the update information recording module includes:
the main storage operation unit is used for judging whether an update table corresponding to the database update operation request is a technical table without double writing, and if so, updating the main storage system according to the database update operation request;
an update record query unit, configured to generate an update record query request according to an update operation condition and an update table name obtained through parsing when it is determined that an update table corresponding to the database update operation request is not a technical table that does not need double writing, and obtain a field value corresponding to the update table and a field value corresponding to the update table after the execution of the database update operation request is completed according to the update record query request;
and the update record information determining unit is used for determining the update record information according to the update table name, the update type corresponding to the update record query request and the field values before and after the database update operation request is executed.
Further, the transaction request intercepting module comprises:
and the business transaction identification unit is used for placing the business transaction identification into the context of the double-write task thread and clearing the updating record information in the updating table when the online transaction request is monitored for the first time in a set time period.
Further, the data synchronization operation module includes:
a service transaction identifier judging unit, configured to judge whether a service transaction identifier exists in a dual-write task thread context corresponding to the data update service call packet, and if not, return a dual-write failure signal;
and the slave storage operation unit is used for traversing and acquiring the update record information in the update table when judging that the service transaction identifier exists in the context of the double-write task thread corresponding to the data update service call message, and generating a data update service call message of a host data service platform according to the update table name, the update type, the field values before and after the execution of the database update operation request and the set message data format in the update record information, so that the host data service platform performs update operation on the slave storage system according to the data update service call message.
In a third aspect, the present application provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the host data double-write method.
In a fourth aspect, the present application provides a computer readable storage medium having stored thereon a computer program which, when executed by a processor, performs the steps of the host data double write method described.
According to the technical scheme, the data updating service calling message of the host data service platform is synchronously generated by intercepting the online transaction request and the online transaction submitting transaction instruction in time, and the data double-writing operation of the host database master-slave storage system is accurately executed independent of the business processing logic, so that the efficiency and the accuracy of the data double-writing of the host can be effectively improved.
Drawings
In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings needed to be used in the description of the embodiments or the prior art will be briefly introduced below, and it is obvious that the drawings in the following description are some embodiments of the present application, and it is obvious for those skilled in the art to obtain other drawings based on these drawings without creative efforts.
FIG. 1 is a flowchart illustrating a method for double writing of host data according to an embodiment of the present application;
FIG. 2 is a second flowchart illustrating a method for host data double-writing according to an embodiment of the present application;
FIG. 3 is a third flowchart illustrating a host data double-writing method according to an embodiment of the present application;
FIG. 4 is a block diagram of a host data dual write apparatus according to an embodiment of the present application;
FIG. 5 is a second block diagram of a host data dual write apparatus according to an embodiment of the present application;
FIG. 6 is a third block diagram of a host data dual writing apparatus according to an embodiment of the present invention;
FIG. 7 is a fourth block diagram of a host data dual write apparatus according to an embodiment of the present application;
fig. 8 is a schematic structural diagram of an electronic device in an embodiment of the present application.
Detailed Description
In order to make the objects, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are some embodiments of the present application, but not all embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.
Considering that in order to ensure the continuity of Service functions and support fast switching back of a host when a Platform fails, in the prior art, a host Data update interface or an MDSP (host Data Service Platform) Data update Service corresponding to each function is generally called in a Platform Service to perform double writing of a host database, thereby causing the problems that a calling code needs to be added, the development cost is high, and each Service function needs to call double writing processing independently, and Platform Service logic codes are invaded, the application provides a host Data double writing method and a device, synchronously generates a Data update Service calling message of a host Data Service Platform by intercepting an online transaction request and an online transaction submitting transaction instruction in time, and accurately executes Data double writing operation of a master storage system and a slave storage system of the host database independently of Service processing logic, therefore, the efficiency and the accuracy of double writing of the host data can be effectively improved.
In order to effectively improve the efficiency and accuracy of the double writing of host data, the present application provides an embodiment of a method for double writing of host data, and referring to fig. 1, the method for double writing of host data specifically includes the following contents:
step S101: and intercepting a database updating operation request in the online transaction request when the online transaction request is monitored, and analyzing the intercepted database updating operation request.
Optionally, the present application may define an entry of an entity such as an online service or batch processing as a cut point of Spring (Java Web development framework), and trigger the cut plane when it is monitored that an online transaction occurs (i.e., an online transaction request is monitored) or a batch process starts, where optionally, the present application may place a service transaction identifier in the cut plane in a context of a dual-write thread, sequentially identify the dual-write thread to enter service transaction processing, and simultaneously clear record information in a set update table.
Optionally, in the step S101, an interceptor may be configured in a Mybatis (Java-based persistent layer framework) configuration file of the project, so as to intercept a database update operation request or an online transaction commit transaction instruction, and after intercepting an online transaction request sent by a user, the application may obtain or identify the database update operation request, where the database update operation request refers to a request for performing data update operations such as adding, deleting, and modifying on an update table (i.e., a target data table) of a main storage system.
Optionally, in step S101, the intercepted database update operation request may be analyzed, that is, a parameter placeholder in the database update operation request (that is, an SQL request) is replaced with a preset binding parameter, so as to obtain a finally executed SQL.
Specifically, the SQL statement can be parsed by a SQL parsing tool of the drive (JDBC connection pool component).
Step S102: and generating an updating record query request according to the updating operation condition and the updating table name obtained by analyzing, and acquiring updating record information of the database updating operation request before and after execution according to the updating record query request.
Optionally, after the database update operation request is analyzed, an update operation condition (e.g., a where condition) and an update table name (i.e., a table name of the target data table) may be obtained, and an update record query request may be generated accordingly, that is, before and after the update is performed, a "table name + where condition" is used for querying, and table record values before and after the update, that is, update record information, are obtained and cached in the dual-write thread context.
Step S103: and intercepting the online transaction submitting transaction instruction when monitoring that the online transaction submitting transaction instruction is submitted, generating a data updating service calling message of a host data service platform according to the updating record information, and executing host database updating operation according to the data updating service calling message.
Optionally, when the online transaction initiates a commit transaction instruction, the interceptor of the present application may intercept the online transaction commit transaction instruction, read the cached update record information, and assemble a message for invoking an MDSP (host data service platform) data update service, so as to invoke the MDSP service to implement data update of the host database from the storage system.
As can be seen from the above description, the host data double-write method provided in the embodiment of the present application can synchronously generate the data update service call packet of the host data service platform by intercepting the online transaction request and the online transaction commit transaction instruction in time, and accurately execute the data double-write operation of the master-slave storage system of the host database independently of the service processing logic, thereby effectively improving the efficiency and accuracy of the data double-write of the host.
In order to accurately obtain the update record information of the primary storage system before and after the update, in an embodiment of the host data double-writing method of the present application, referring to fig. 2, the step S102 may further specifically include the following steps:
step S201: and judging whether the updating table corresponding to the database updating operation request is a technical table without double writing, if so, updating the main storage system according to the database updating operation request.
Step S202: otherwise, generating an update record query request according to the update operation condition and the update table name obtained by analysis, and acquiring a field value corresponding to the update table and a field value corresponding to the update table after the database update operation request is executed.
Step S203: and determining updated record information according to the updated table name, the updated type corresponding to the updated record query request and the field values before and after the database updating operation request is executed.
Optionally, after the database update operation request is analyzed, an update operation condition (e.g., a where condition) and an update table name (i.e., a table name of the target data table) may be obtained, and an update record query request may be generated accordingly, that is, before and after the update is performed, a "table name + where condition" is used for querying, and table record values before and after the update, that is, update record information, are obtained and cached in the dual-write thread context.
For example, the SQL statement is first parsed by the SQL parsing tool of the druid.
And if the statement is judged to be updated after analysis, acquiring an updated where condition and an updated table name, and then inquiring by using the table name + where condition before and after updating to acquire the table record values before and after updating.
And if the inserted statement is judged after analysis, analyzing each column name and the corresponding column value through a managed SQL analysis tool, and then storing the column name and the corresponding column value into the HashMap.
And if the sentence is judged to be deleted after analysis, acquiring a deleted where condition and a deleted table name, and then inquiring by using the table name + where condition before updating is executed to acquire the content of the deleted record.
Then, the records before and after updating are converted into JSON format character strings, and the table name, the updating type (inserting, updating and deleting) and the field values before and after updating of the records of the updating table are saved in the context of the transaction thread according to the format shown in the following table 1:
table 1 update record information table format
Table name 1 Operation type 1 Pre-operation value 1 Value after operation 1
Table name 2 Operation type 2 Pre-operation value 2 Post-operation value 2
Table name 3 Operation type 3 Pre-operation value 3 Post-operation value 3
The actual row process stores are shown in table 2 below:
table 2 update record information presentation example
Figure BDA0003371388600000071
In order to perform data double-writing operation accurately for the online transaction request triggered by the service, in an embodiment of the host data double-writing method of the present application, the step S101 may further include the following steps:
and when an online transaction request is monitored for the first time in a set time period, putting the service transaction identifier into the context of the double-write task thread, and clearing the updating record information in the updating table.
Optionally, when an online transaction request is monitored for the first time within a set time period, the service transaction identifier may be placed in the context of the double-write task thread, and the update record information in the update table is cleared, so that when a subsequent intercepted transaction submitting action is performed, it is determined whether the service transaction identifier exists in the context of the double-write task thread, and if not, it indicates that non-service processing is triggered, and a double-write failure signal is directly returned.
In order to accurately double-write data to the slave storage system, in an embodiment of the host data double-write method of the present application, referring to fig. 3, the step S103 may further include the following steps:
step S301: and judging whether a business transaction identifier exists in the context of the double-write task thread corresponding to the data updating service calling message, and if not, returning a double-write failure signal.
Step S302: otherwise, the updating record information in the updating table is obtained through traversal, and a data updating service calling message of a host data service platform is generated according to the name and the updating type of the updating table in the updating record information, the field values before and after the execution of the database updating operation request and the set message data format, so that the host data service platform updates the slave storage system according to the data updating service calling message.
Optionally, when it is determined that the context of the double-write thread includes the service transaction identifier and the update record information is not empty, traversing the update record information, and assembling and calling the data packet of the MDSP service according to the update table name, the update type, the field value before and after the execution of the database update operation request, and the set packet data format in the update record information.
Optionally, after receiving the message sent by the platform, the MDSP may convert the JOSN message data into a table processing object, and when the chinese ASCII code is involved, to avoid a garbled code occurring when the data is inserted, the field type in the host table related to the ASCII code may be modified into a char for bit format, and at the same time, the field type in the sql program is defined as byte [ ].
Then, MDSP calls each table processor to update data, each table processor defines an adding, deleting and modifying method, wherein deleting operation is updated according to the unique index of the table, and modifying operation is updated according to the unique index of the table.
Meanwhile, if the update is abnormal, an error is returned, the platform end is synchronously rolled back, and the data consistency of the two ends is ensured.
In order to effectively improve the efficiency and accuracy of the host data double writing, the present application provides an embodiment of a host data double writing apparatus for implementing all or part of the contents of the host data double writing method, and referring to fig. 4, the host data double writing apparatus specifically includes the following contents:
the transaction request intercepting module 10 is configured to intercept a database update operation request in an online transaction request when the online transaction request is monitored, and analyze the intercepted database update operation request.
And the update information recording module 20 is configured to generate an update record query request according to the update operation condition and the update table name obtained through the analysis, and obtain update record information of the database update operation request before and after execution according to the update record query request.
And the data synchronization operation module 30 is configured to intercept the online transaction submitting transaction instruction when the online transaction submitting transaction instruction is monitored, generate a data update service call packet of a host data service platform according to the update record information, and execute a host database update operation according to the data update service call packet.
As can be seen from the above description, the host data double-write apparatus provided in the embodiment of the present application can synchronously generate the data update service call packet of the host data service platform by intercepting the online transaction request and the online transaction commit transaction instruction in time, and accurately execute the data double-write operation of the master-slave storage system of the host database independently of the service processing logic, thereby effectively improving the efficiency and accuracy of the data double-write of the host.
In order to accurately obtain the update record information of the primary storage system before and after the update, in an embodiment of the host data dual writing apparatus of the present application, referring to fig. 5, the update information recording module 20 includes:
and the main storage operation unit 21 is configured to determine whether an update table corresponding to the database update operation request is a technical table that does not need double writing, and if so, perform an update operation on the main storage system according to the database update operation request.
And the update record query unit 22 is configured to, when it is determined that the update table corresponding to the database update operation request is not a technical table that does not require double writing, generate an update record query request according to the update operation condition and the update table name obtained through parsing, and obtain a field value corresponding to the update table and a field value corresponding to the update table after the execution of the database update operation request is completed according to the update record query request.
An update record information determining unit 23, configured to determine update record information according to the update table name, the update type corresponding to the update record query request, and field values before and after the database update operation request is executed.
In order to perform data double-write operation accurately for the online transaction request triggered by the service, in an embodiment of the host data double-write apparatus of the present application, referring to fig. 6, the transaction request intercepting module 10 includes:
and the service transaction identification unit 11 is configured to, when an online transaction request is monitored for the first time within a set time period, place the service transaction identification in the context of the double-write task thread, and clear update record information in the update table.
In order to enable accurate double-writing of data to the slave storage system, in an embodiment of the host data double-writing apparatus of the present application, referring to fig. 7, the data synchronization operation module 30 includes:
and the service transaction identifier determining unit 31 is configured to determine whether a service transaction identifier exists in the context of the double-write task thread corresponding to the data update service call packet, and if not, return a double-write failure signal.
The slave storage operation unit 32 is configured to, when it is determined that a service transaction identifier exists in a context of a double-write task thread corresponding to the data update service call packet, traverse to obtain update record information in the update table, and generate a data update service call packet of a host data service platform according to an update table name, an update type, a field value before and after execution of the database update operation request, and a set packet data format in the update record information, so that the host data service platform performs an update operation on a slave storage system according to the data update service call packet.
To further illustrate the present solution, the present application further provides a specific application example of implementing the host data double-writing method by using the host data double-writing apparatus, which specifically includes the following contents:
1. initializing dual write thread contexts
(1) The entry to the online service or batch processing entity is defined as the Spring tangent point, and the tangent plane will be triggered when online transaction or batch start processing occurs.
(2) Defining the section, putting a service transaction mark into the thread context in the section, marking the thread to enter service transaction processing, and clearing the record content of the double write tables.
2. Intercepting update SQL
(1) And intercepting the update and submitting the transaction by an interceptor configured in the Mybatis configuration file of the project.
(2) And intercepting and updating the SQL, replacing the parameter placeholder in the SQL with the binding parameter, and acquiring the SQL which is finally executed.
(3) And if the table name of the current operation is judged to be the technical table without double writing, skipping the subsequent analysis processing and directly returning.
(4) Analyzing the update statement, analyzing the SQL statement through an SQL analysis tool of the pipeline, judging whether the SQL is updated, acquiring an updated where condition and an updated table name, then querying by using the table name + where condition before and after updating, and acquiring the table record values before and after updating.
(5) And analyzing the insert statement, analyzing each column name and the corresponding column value through an SQL (structured query language) analyzing tool of the druid, and then storing the column name and the corresponding column value into the HashMap.
(6) Analyzing the deleted statement, analyzing the SQL statement through an SQL analysis tool of the pipeline, judging whether the SQL is deleted, acquiring a deleted where condition and a deleted table name, and then inquiring by using the table name + where condition before updating is executed to acquire the content of the deleted record.
(7) And converting the records before and after updating into JSON format character strings.
(8) And saving the table name, the update type (insertion, update and deletion) and the field values before and after update of the update table record in the transaction thread context according to a set format.
3. Intercept transaction commit actions and invoke the MDSP service for double writes.
(1) And intercepting a transaction submitting action, judging whether the thread context has a double-write mark, if not, indicating non-business processing triggering, and directly returning.
(2) And judging whether the thread context updating content list is empty, traversing the updating records if the thread context updating content list is not empty, and assembling and calling the data message of the MDSP service.
4. MDSP service double write processing
(1) The receiving platform sends a message, and converts JOSN message data into a table processing object, wherein the table processing object defines the following format:
TableData{
private List<TableRecord>before;
private List<TableRecord>after;
}
(2) when Chinese ASCII code is related, in order to avoid messy code when data is inserted, the field type related to the ASCII code in the host table is modified into char for bit format, and the field type in the sqlj program is defined as byte [ ].
(3) And calling each table processor to update data, wherein each table processor defines an adding, deleting and modifying method, the deleting operation is updated according to the unique index of the table, and the modifying operation is updated according to the unique index of the table.
(4) If the update is abnormal, an error is returned, the platform end is synchronously rolled back, and the data of the two ends are ensured to be consistent.
As can be seen from the above, the present application can achieve at least the following technical effects:
1. by using a Spring section and a Mybatis interceptor mode, the method is transparent to a new platform function developer, and reduces the difficulty of realizing the double writing of the host.
2. The method is decoupled from specific business rules, is suitable for all lower host scenes, can realize the effects of component processing of the platform and implementation of MDSP programs in a toolization mode, realizes one-time writing and directly reuses each application.
In terms of hardware, in order to effectively improve the efficiency and accuracy of double writing of host data, the present application provides an embodiment of an electronic device for implementing all or part of the contents in the method for double writing of host data, where the electronic device specifically includes the following contents:
a processor (processor), a memory (memory), a communication Interface (Communications Interface), and a bus; the processor, the memory and the communication interface complete mutual communication through the bus; the communication interface is used for realizing information transmission between the host data double-writing device and relevant equipment such as a core service system, a user terminal, a relevant database and the like; the logic controller may be a desktop computer, a tablet computer, a mobile terminal, and the like, but the embodiment is not limited thereto. In this embodiment, the logic controller may be implemented with reference to the embodiment of the host data double-writing method and the embodiment of the host data double-writing apparatus in the embodiment, and the contents thereof are incorporated herein, and repeated descriptions are omitted.
It is understood that the user terminal may include a smart phone, a tablet electronic device, a network set-top box, a portable computer, a desktop computer, a Personal Digital Assistant (PDA), an in-vehicle device, a smart wearable device, and the like. Wherein, intelligence wearing equipment can include intelligent glasses, intelligent wrist-watch, intelligent bracelet etc..
In practical applications, part of the host data double-writing method may be performed on the electronic device side as described above, or all operations may be performed in the client device. The selection may be specifically performed according to the processing capability of the client device, the limitation of the user usage scenario, and the like. This is not a limitation of the present application. The client device may further include a processor if all operations are performed in the client device.
The client device may have a communication module (i.e., a communication unit), and may be communicatively connected to a remote server to implement data transmission with the server. The server may include a server on the task scheduling center side, and in other implementation scenarios, the server may also include a server on an intermediate platform, for example, a server on a third-party server platform that is communicatively linked to the task scheduling center server. The server may include a single computer device, or may include a server cluster formed by a plurality of servers, or a server structure of a distributed apparatus.
Fig. 8 is a schematic block diagram of a system configuration of an electronic device 9600 according to an embodiment of the present application. As shown in fig. 8, the electronic device 9600 can include a central processor 9100 and a memory 9140; the memory 9140 is coupled to the central processor 9100. Notably, this FIG. 8 is exemplary; other types of structures may also be used in addition to or in place of the structure to implement telecommunications or other functions.
In one embodiment, the host data double write method function may be integrated into the central processor 9100. The central processor 9100 may be configured to control as follows:
step S101: and intercepting a database updating operation request in the online transaction request when the online transaction request is monitored, and analyzing the intercepted database updating operation request.
Step S102: and generating an updating record query request according to the updating operation condition and the updating table name obtained by analyzing, and acquiring updating record information of the database updating operation request before and after execution according to the updating record query request.
Step S103: and intercepting the online transaction submitting transaction instruction when monitoring that the online transaction submitting transaction instruction is submitted, generating a data updating service calling message of a host data service platform according to the updating record information, and executing host database updating operation according to the data updating service calling message.
As can be seen from the above description, the electronic device provided in the embodiment of the present application intercepts the online transaction request and the online transaction submission transaction instruction in time, synchronously generates the data update service call packet of the host data service platform, and accurately executes the data double-write operation of the master storage system and the slave storage system of the host database independently of the service processing logic, thereby effectively improving the efficiency and the accuracy of the data double-write of the host.
In another embodiment, the host data double-writing device may be configured separately from the central processing unit 9100, for example, the host data double-writing device may be configured as a chip connected to the central processing unit 9100, and the host data double-writing method function is realized by the control of the central processing unit.
As shown in fig. 8, the electronic device 9600 may further include: a communication module 9110, an input unit 9120, an audio processor 9130, a display 9160, and a power supply 9170. It is noted that the electronic device 9600 also does not necessarily include all of the components shown in fig. 8; further, the electronic device 9600 may further include components not shown in fig. 8, which may be referred to in the art.
As shown in fig. 8, a central processor 9100, sometimes referred to as a controller or operational control, can include a microprocessor or other processor device and/or logic device, which central processor 9100 receives input and controls the operation of the various components of the electronic device 9600.
The memory 9140 can be, for example, one or more of a buffer, a flash memory, a hard drive, a removable media, a volatile memory, a non-volatile memory, or other suitable device. The information relating to the failure may be stored, and a program for executing the information may be stored. And the central processing unit 9100 can execute the program stored in the memory 9140 to realize information storage or processing, or the like.
The input unit 9120 provides input to the central processor 9100. The input unit 9120 is, for example, a key or a touch input device. Power supply 9170 is used to provide power to electronic device 9600. The display 9160 is used for displaying display objects such as images and characters. The display may be, for example, an LCD display, but is not limited thereto.
The memory 9140 can be a solid state memory, e.g., Read Only Memory (ROM), Random Access Memory (RAM), a SIM card, or the like. There may also be a memory that holds information even when power is off, can be selectively erased, and is provided with more data, an example of which is sometimes called an EPROM or the like. The memory 9140 could also be some other type of device. Memory 9140 includes a buffer memory 9141 (sometimes referred to as a buffer). The memory 9140 may include an application/function storage portion 9142, the application/function storage portion 9142 being used for storing application programs and function programs or for executing a flow of operations of the electronic device 9600 by the central processor 9100.
The memory 9140 can also include a data store 9143, the data store 9143 being used to store data, such as contacts, digital data, pictures, sounds, and/or any other data used by an electronic device. The driver storage portion 9144 of the memory 9140 may include various drivers for the electronic device for communication functions and/or for performing other functions of the electronic device (e.g., messaging applications, contact book applications, etc.).
The communication module 9110 is a transmitter/receiver 9110 that transmits and receives signals via an antenna 9111. The communication module (transmitter/receiver) 9110 is coupled to the central processor 9100 to provide input signals and receive output signals, which may be the same as in the case of a conventional mobile communication terminal.
Based on different communication technologies, a plurality of communication modules 9110, such as a cellular network module, a bluetooth module, and/or a wireless local area network module, may be provided in the same electronic device. The communication module (transmitter/receiver) 9110 is also coupled to a speaker 9131 and a microphone 9132 via an audio processor 9130 to provide audio output via the speaker 9131 and receive audio input from the microphone 9132, thereby implementing ordinary telecommunications functions. The audio processor 9130 may include any suitable buffers, decoders, amplifiers and so forth. In addition, the audio processor 9130 is also coupled to the central processor 9100, thereby enabling recording locally through the microphone 9132 and enabling locally stored sounds to be played through the speaker 9131.
An embodiment of the present application further provides a computer-readable storage medium capable of implementing all the steps in the host data double-writing method with the execution subject being the server or the client in the foregoing embodiments, where the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the computer program implements all the steps in the host data double-writing method with the execution subject being the server or the client in the foregoing embodiments, for example, when the processor executes the computer program, the processor implements the following steps:
step S101: and intercepting a database updating operation request in the online transaction request when the online transaction request is monitored, and analyzing the intercepted database updating operation request.
Step S102: and generating an updating record query request according to the updating operation condition and the updating table name obtained by analyzing, and acquiring updating record information of the database updating operation request before and after execution according to the updating record query request.
Step S103: and intercepting the online transaction submitting transaction instruction when monitoring that the online transaction submitting transaction instruction is submitted, generating a data updating service calling message of a host data service platform according to the updating record information, and executing host database updating operation according to the data updating service calling message.
As can be seen from the above description, the computer-readable storage medium provided in the embodiment of the present application intercepts the online transaction request and the online transaction commit transaction instruction in time, synchronously generates the data update service call packet of the host data service platform, and accurately executes the data double-write operation of the master-slave storage system of the host database independently of the service processing logic, thereby effectively improving the efficiency and accuracy of data double-write of the host.
As will be appreciated by one skilled in the art, embodiments of the present invention may be provided as a method, apparatus, or computer program product. Accordingly, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present invention may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.
The present invention is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (devices), and computer program products according to embodiments of the invention. It will be understood that each flow and/or block of the flow diagrams and/or block diagrams, and combinations of flows and/or blocks in the flow diagrams and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
The principle and the implementation mode of the invention are explained by applying specific embodiments in the invention, and the description of the embodiments is only used for helping to understand the method and the core idea of the invention; meanwhile, for a person skilled in the art, according to the idea of the present invention, there may be variations in the specific embodiments and the application scope, and in summary, the content of the present specification should not be construed as a limitation to the present invention.

Claims (10)

1. A method of double writing of host data, the method comprising:
intercepting a database updating operation request in an online transaction request when the online transaction request is monitored, and analyzing the intercepted database updating operation request;
generating an updating record query request according to the updating operation condition and the updating table name obtained by analysis, and acquiring updating record information of the database updating operation request before and after execution according to the updating record query request;
and intercepting the online transaction submitting transaction instruction when monitoring that the online transaction submitting transaction instruction is submitted, generating a data updating service calling message of a host data service platform according to the updating record information, and executing host database updating operation according to the data updating service calling message.
2. The method according to claim 1, wherein the generating an update record query request according to the parsed update operation condition and update table name, and obtaining the update record information of the database update operation request before and after execution according to the update record query request includes:
judging whether an update table corresponding to the database update operation request is a technical table without double writing, if so, updating the main storage system according to the database update operation request;
otherwise, generating an update record query request according to the update operation condition and the update table name obtained by analysis, and acquiring a field value corresponding to the update table and a field value corresponding to the update table after the database update operation request is executed;
and determining updated record information according to the updated table name, the updated type corresponding to the updated record query request and the field values before and after the database updating operation request is executed.
3. The method for host data double-writing according to claim 1, wherein intercepting a database update operation request in an online transaction request when the online transaction request is monitored comprises:
and when an online transaction request is monitored for the first time in a set time period, putting the service transaction identifier into the context of the double-write task thread, and clearing the updating record information in the updating table.
4. The method according to claim 3, wherein the generating a data update service call packet of a host data service platform according to the update record information, and performing a host database update operation according to the data update service call packet includes:
judging whether a business transaction identifier exists in the context of the double-write task thread corresponding to the data updating service calling message, and if not, returning a double-write failure signal;
otherwise, the updating record information in the updating table is obtained through traversal, and a data updating service calling message of a host data service platform is generated according to the name and the updating type of the updating table in the updating record information, the field values before and after the execution of the database updating operation request and the set message data format, so that the host data service platform updates the slave storage system according to the data updating service calling message.
5. The method for host data double-writing according to claim 1, wherein intercepting a database update operation request in an online transaction request when the online transaction request is monitored comprises:
intercepting a database updating operation request in the online transaction request through an interceptor configured in a persistent layer framework configuration file when the online transaction request is monitored, wherein the database updating operation request is a request for performing data updating operation on an updating table of a main storage system.
6. The method of claim 1, wherein the parsing the intercepted database update operation request comprises:
analyzing the database updating operation request through a preset connection pool component, replacing a parameter placeholder in a corresponding database operation statement with a preset binding parameter, and obtaining the database operation statement after parameter replacement, wherein the database operation statement comprises an updating operation condition and an updating table name.
7. A host data double write apparatus, comprising:
the transaction request intercepting module is used for intercepting a database updating operation request in the online transaction request when the online transaction request is monitored, and analyzing the intercepted database updating operation request;
the update information recording module is used for generating an update record query request according to the update operation condition and the update table name obtained by analysis, and acquiring update record information of the database update operation request before and after execution according to the update record query request;
and the data synchronization operation module is used for intercepting the online transaction submitting transaction instruction when the online transaction submitting transaction instruction is monitored, generating a data updating service calling message of a host data service platform according to the updating record information, and executing host database updating operation according to the data updating service calling message.
8. The apparatus of claim 7, wherein the update information recording module comprises:
the main storage operation unit is used for judging whether an update table corresponding to the database update operation request is a technical table without double writing, and if so, updating the main storage system according to the database update operation request;
an update record query unit, configured to generate an update record query request according to an update operation condition and an update table name obtained through parsing when it is determined that an update table corresponding to the database update operation request is not a technical table that does not need double writing, and obtain a field value corresponding to the update table and a field value corresponding to the update table after the execution of the database update operation request is completed according to the update record query request;
and the update record information determining unit is used for determining the update record information according to the update table name, the update type corresponding to the update record query request and the field values before and after the database update operation request is executed.
9. An electronic device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, wherein the steps of the host data double write method of any one of claims 1 to 6 are implemented when the program is executed by the processor.
10. A computer-readable storage medium, on which a computer program is stored, which, when being executed by a processor, carries out the steps of the host data double write method of any one of claims 1 to 6.
CN202111403038.XA 2021-11-24 2021-11-24 Host data double-writing method and device Pending CN114153856A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202111403038.XA CN114153856A (en) 2021-11-24 2021-11-24 Host data double-writing method and device

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202111403038.XA CN114153856A (en) 2021-11-24 2021-11-24 Host data double-writing method and device

Publications (1)

Publication Number Publication Date
CN114153856A true CN114153856A (en) 2022-03-08

Family

ID=80457380

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202111403038.XA Pending CN114153856A (en) 2021-11-24 2021-11-24 Host data double-writing method and device

Country Status (1)

Country Link
CN (1) CN114153856A (en)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114564485A (en) * 2022-04-28 2022-05-31 深圳竹云科技股份有限公司 User data processing method based on Elastic Search
CN116431669A (en) * 2023-06-12 2023-07-14 天津金城银行股份有限公司 Mybatis-based data processing method, mybatis-based data processing device, computer equipment and storage medium

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114564485A (en) * 2022-04-28 2022-05-31 深圳竹云科技股份有限公司 User data processing method based on Elastic Search
CN116431669A (en) * 2023-06-12 2023-07-14 天津金城银行股份有限公司 Mybatis-based data processing method, mybatis-based data processing device, computer equipment and storage medium
CN116431669B (en) * 2023-06-12 2023-08-15 天津金城银行股份有限公司 Mybatis-based data processing method, mybatis-based data processing device, computer equipment and storage medium

Similar Documents

Publication Publication Date Title
CN114153856A (en) Host data double-writing method and device
CN112636992B (en) Dynamic routing method, device, equipment and storage medium
CN111369247A (en) Cross-bank transaction data processing method and device
CN112463634B (en) Software testing method and device under micro-service architecture
CN104881454A (en) Updating method and system of parameter
CN104268229A (en) Resource obtaining method and device based on multi-process browser
CN112394932A (en) Automatic browser webpage skin changing method and device
CN111625250B (en) Application updating method and device, storage medium and electronic equipment
CN113791984B (en) Automatic interface testing method and device
CN115495519A (en) Report data processing method and device
CN114637531A (en) Method and device for dynamically generating application program interface
CN113515447B (en) Automatic testing method and device for system
CN114490402A (en) Method and device for checking validity of message field
CN111443945A (en) Component code modification method and device
CN112905464B (en) Application running environment data processing method and device
CN114840576A (en) Data standard matching method and device
CN113434423A (en) Interface test method and device
CN113392091A (en) Distributed cluster data migration method and device
CN113419957A (en) Rule-based big data offline batch processing performance capacity scanning method and device
CN112766827A (en) Automatic processing method and device for asset inventory data
CN112102058A (en) Management system parameter configuration method and device
CN113220760B (en) Data processing method, device, server and storage medium
CN111949355B (en) Column menu shielding method and device
CN115421976A (en) Remote disaster recovery data processing method and device
CN107729013B (en) Method for managing operation buttons on web page and computer-readable storage medium

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination