CN114372052A

CN114372052A - Data change record storage method and device, computer equipment and storage medium

Info

Publication number: CN114372052A
Application number: CN202111525889.1A
Authority: CN
Inventors: 朱师俊; 王健
Original assignee: Shenzhen Aozhe Network Technology Co ltd
Current assignee: Shenzhen Aozhe Network Technology Co ltd
Priority date: 2021-12-14
Filing date: 2021-12-14
Publication date: 2022-04-19

Abstract

The invention discloses a storage method and device of a data change record, computer equipment and a storage medium. The method comprises the following steps: the method comprises the steps of receiving a data change request, obtaining and changing original data from a first database, wherein the data change request comprises changed data, obtaining the first data from the changed data according to a plurality of preset first fields to generate a change message, storing the change message in the first database, obtaining second data from the original data and the changed data according to a plurality of preset second fields to generate a first change record, storing the first change record in a second database, synchronizing the change message to a preset message queue, enabling a server to consume the change message from the message queue to obtain the first change record, conducting standardization processing to obtain a second change record, and storing the second change record in the second database. The method and the device can avoid the limitation of the size of a single message of the message queue triggered by overlarge content of the first change record, and further improve the success rate of data change record storage.

Description

Data change record storage method and device, computer equipment and storage medium

Technical Field

The present invention relates to the field of data processing technologies, and in particular, to a method and an apparatus for storing a data change record, a computer device, and a storage medium.

Background

In a low-code platform, a user can modify data manually through a page or automatically through a workflow by configuring business rules, but for some important data, the changing process of the important data has significance, so that data change records need to be stored.

In the related art, the change message generated by the data change record is posted to the message queue for storage, but this may trigger the limitation of the size of a single message of the message queue due to the excessively large content of the data change record, thereby resulting in a low success rate of storing the data change record.

Disclosure of Invention

The present invention is directed to solving, at least to some extent, one of the technical problems in the related art. Therefore, the invention provides a data change record storage method and device, computer equipment and a storage medium, which can avoid the limitation of the size of a single message of a trigger message queue and improve the success rate of data change record storage.

In a first aspect, an embodiment of the present invention provides a method for storing a data change record, where the method includes:

receiving a data change request sent by a client; wherein the data change request comprises changed data;

acquiring original data corresponding to the changed data from a first database;

changing the original data according to the changed data to update the first database;

acquiring corresponding first data from the changed data according to a plurality of preset first fields;

generating a change message according to the first data, and storing the change message in the first database;

acquiring corresponding second data from the original data and the changed data according to a plurality of preset second fields;

generating a first change record according to the second data, and storing the first change record to a second database;

and synchronizing the change message to a preset message queue so that the server consumes the change message from the message queue to obtain the first change record, standardizing the first change record by the server to obtain a second change record, and storing the second change record in the second database by the server.

The technical scheme of the first aspect of the invention has at least one of the following advantages or beneficial effects: firstly, a service end acquires original data corresponding to changed data from a first database after receiving a data change request sent by a client, wherein the data change request comprises the changed data, then, the original data is changed according to the changed data to update the first database, corresponding first data is acquired from the changed data according to a plurality of preset first fields, a change message is generated according to the first data and is stored in the first database, meanwhile, corresponding second data is acquired from the original data and the changed data according to a plurality of preset second fields, a first change record is generated according to the second data and is stored in a second database, finally, the change message is synchronized into a preset message queue, the service end acquires the first change record from the message queue, then the service end performs standardized processing on the first change record to acquire a second change record, and storing the resulting second change record to a second database. According to the method and the device, the change message is sent to the message queue, the server side obtains the first change record by consuming the change message, then the first change record is subjected to standardization processing to obtain the finally required second change record, the second change record is stored in the second database, the first change record or the second change record is not directly sent to the message queue and then stored, the change message with smaller content is sent to the message queue instead of being sent to the message queue, and the first change record with larger content is obtained according to the change message, so that the limitation of the size of a single message of the message queue triggered by the fact that the content of the first change record is too large can be effectively avoided, and the success rate of data change record storage is improved.

In some embodiments of the invention, after the generating the first change record from the second data, the method further comprises:

storing the first change record to the second database to obtain a storage result; the storage result is a storage failure result;

and deleting the change message from the first database according to the storage failure result, and restoring the original data of the first database.

In a second aspect, an embodiment of the present invention further provides a method for storing a data change record, where the method includes:

acquiring a preset message queue;

obtaining a change message from the message queue to obtain an obtaining result; the change message is generated by the service end according to a data change request sent by the client, and the acquisition result is a successful acquisition result;

consuming the change message according to the successful acquisition result to obtain a first change record;

standardizing the first change record to obtain a second change record;

storing the second change record to a second database.

The technical scheme of the second aspect of the invention has at least one of the following advantages or beneficial effects: the server side obtains an obtaining result by obtaining a change message in a preset message queue, wherein the change message is generated by the server side according to a data change request sent by the client side, the obtaining result is a successful obtaining result, namely the server side consumes the change message to obtain a first change record after successfully obtaining the change message in the preset message queue, then carries out standardized processing on the first change record to obtain a second change record, and stores the finally obtained second change record in a second database. According to the method and the device, the first change record is obtained by consuming the change message in the message queue, the second change record is obtained by performing standardized processing on the first change record, and the finally obtained second change record is stored in the second database.

In some embodiments of the invention, before the normalizing the first change record to obtain the second change record, the method further comprises:

acquiring the change message from the message queue to obtain the acquisition result; wherein the acquisition result is an acquisition failure result;

acquiring a current time point according to the acquisition failure result;

acquiring the change message from the first database; wherein a storage time point of the change message is earlier than the current time point;

and acquiring the first change record according to the change message.

In some embodiments of the present invention, the modifying message includes a batch number, and the consuming the modifying message according to the successful obtaining result to obtain a first modification record includes:

acquiring the batch number in the change message; wherein the batch number is used to identify the change message;

and acquiring the first change record from the second database according to the batch number.

In some embodiments of the invention, after said storing the second change record to the second database, the method further comprises:

obtaining the second change record from the second database;

and sending the second change record to the client so that the client displays the second change record.

In a third aspect, an embodiment of the present invention further provides a first storage apparatus, where the apparatus includes:

a receiving module: the data change request is used for receiving a data change request sent by a client; wherein the data change request comprises changed data;

a first obtaining module: the data processing device is used for acquiring original data corresponding to the changed data from a first database;

a change module: the original data is changed according to the changed data so as to update the first database;

a second obtaining module: the device comprises a data acquisition module, a data processing module and a data processing module, wherein the data acquisition module is used for acquiring corresponding first data from the changed data according to a plurality of preset first fields;

a first storage module: the database server is used for generating a change message according to the first data and storing the change message to the first database;

a third obtaining module: the data processing device is used for acquiring corresponding second data from the original data and the changed data according to a plurality of preset second fields;

a second storage module: the first change record is generated according to the second data, and the first change record is stored in a second database;

a synchronization module: the server side is used for synchronizing the change message to a preset message queue so as to enable the server side to consume the change message from the message queue to obtain the first change record, enabling the server side to conduct standardization processing on the first change record to obtain a second change record, and enabling the server side to store the second change record in the second database.

The technical scheme of the third aspect of the invention has at least one of the following advantages or beneficial effects: the first storage device comprises a receiving module, a first obtaining module, a changing module, a second obtaining module, a first storage module, a third obtaining module, a second storage module and a synchronizing module. The receiving module receives a data change request sent by a client, wherein the data change request comprises change data, a first obtaining module obtains original data corresponding to the change data from a first database, then, the change module changes the original data to update the first database, a second obtaining module obtains corresponding first data from the change data according to a plurality of preset first fields, a first storage module stores change information generated according to the first data into the first database, meanwhile, a third obtaining module obtains corresponding second data from the original data and the change data according to a plurality of preset second fields, a second storage module generates a first change record according to the second data and stores the first change record into a second database, finally, a synchronization module synchronizes the change information into a preset message queue, and a server obtains the first change record from the message queue by consuming the change information, and then, carrying out standardization processing on the first change record to obtain a second change record, and storing the finally obtained second change record into a second database. According to the method and the device, the change message is sent to the message queue through the synchronization module, the server side obtains the first change record through consuming the change message, then the first change record is subjected to standardization processing to obtain the finally needed second change record, the second change record is stored in the second database, the first change record or the second change record is not directly sent to the message queue for storage, the change message with smaller content is sent to the message queue instead of the first change record or the second change record, and the first change record with larger content is obtained according to the change message, so that the limitation that the size of a single message of the message queue is triggered due to the fact that the content of the first change record is too large can be effectively avoided, and the success rate of data change record storage is improved.

In a fourth aspect, an embodiment of the present invention further provides a second storage device, where the second storage device includes:

a fourth obtaining module: the method comprises the steps of obtaining a preset message queue;

a fifth obtaining module: the message queue is used for acquiring the change message from the message queue to obtain an acquisition result; the change message is generated by the service end according to a data change request sent by the client, and the acquisition result is a successful acquisition result;

a consumption module: the system is used for consuming the change message according to the successful acquisition result to obtain a first change record;

a standardization processing module: the first change record is standardized to obtain a second change record;

a third storage module: for storing the second change record to a second database.

The technical scheme of the fourth aspect of the invention has at least one of the following advantages or beneficial effects: the second storage device comprises a fourth acquisition module, a fifth acquisition module, a consumption module, a standardization processing module and a third storage module. The server side obtains a change message in a preset message queue through a fourth obtaining module to obtain an obtaining result, wherein the change message is generated by the service side according to a data change request sent by the client side, the obtaining result is a successful obtaining result, namely after the server side successfully obtains the change message in the preset message queue, a consumption module consumes the change message to obtain a first change record, a standardization processing module carries out standardization processing on the first change record to obtain a second change record, and then a third storage module stores the finally obtained second change record in a second database. According to the method and the device, the consumption module consumes the change message in the message queue to obtain the first change record, the standardization processing module is used for conducting standardization processing on the first change record to obtain the second change record, and the third storage module is used for storing the finally obtained second change record into the second database.

In a fifth aspect, an embodiment of the present invention further provides a computer device, where the computer device includes a memory and a processor, where the memory stores a computer program, and the processor is configured to, when executed by the processor, perform: the method as in any embodiment of the first or second aspects.

In a sixth aspect, an embodiment of the present invention further provides a storage medium, where the storage medium is a computer-readable storage medium, and is characterized in that a computer program is stored in the computer-readable storage medium, and when the computer program is executed by a computer, the computer is configured to perform: the method as in any embodiment of the first or second aspects.

Additional features and advantages of the invention will be set forth in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention.

Drawings

The accompanying drawings are included to provide a further understanding of the invention and are incorporated in and constitute a part of this specification, illustrate embodiments of the invention and together with the example serve to explain the principles of the invention and not to limit the invention.

FIG. 1 is a first flow diagram of a method for storing a data change record provided by some embodiments of the present invention;

FIG. 2 is a second flowchart of a method of storing a data change record provided by some embodiments of the present invention;

FIG. 3 is a third flowchart of a method for storing a data change record according to some embodiments of the present invention;

FIG. 4 is a fourth flowchart of a method of storing a data change record according to some embodiments of the present invention;

FIG. 5 is a flowchart of a refinement step of step S330 in FIG. 3;

FIG. 6 is a fifth flowchart of a method of storing a data change record according to some embodiments of the present invention;

FIG. 7 is a flow chart of an actual application of a method for storing a data change record according to some embodiments of the present invention;

FIG. 8 is a block diagram of a first memory device according to some embodiments of the invention;

fig. 9 is a block diagram of a second storage device according to some embodiments of the present invention.

Detailed Description

Reference will now be made in detail to embodiments of the present invention, examples of which are illustrated in the accompanying drawings, wherein like or similar reference numerals refer to the same or similar elements or elements having the same or similar function throughout. The embodiments described below with reference to the accompanying drawings are illustrative only for the purpose of explaining the present invention, and are not to be construed as limiting the present invention.

In the description of the present invention, the meaning of a plurality is one or more, the meaning of a plurality is two or more, and the above, below, exceeding, etc. are understood as excluding the present numbers, and the above, below, within, etc. are understood as including the present numbers. If the first and second are described for the purpose of distinguishing technical features, they are not to be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated or implicitly indicating the precedence of the technical features indicated.

In the description of the present invention, reference to the description of the terms "one embodiment," "some embodiments," "an illustrative embodiment," "an example," "a specific example," or "some examples," etc., means that a particular feature, structure, material, or characteristic described in connection with the embodiment or example is included in at least one embodiment or example of the present invention. In this specification, the schematic representations of the terms used above do not necessarily refer to the same embodiment or example. Furthermore, the particular features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples.

In a low-code platform, a user can modify data manually through a page or automatically through a workflow by configuring business rules, but for some important data, the modification process is also significant, for example, in a scenario where multiple persons collaboratively perform form editing in a financial system, an important data entry error like money amount can be traced back to the work error of the user according to the modification history of the data, and for example, in a project management system, a task originally completed within a planning time is not expected to be modified, and the user can analyze the change rate of the task and the like according to the data change record because the expected completion time is not expected to be modified for several times, so that the data change record needs to be stored.

In the related art, the data change message generated by the data change record is posted to the message queue for storage, but this may trigger the limitation of the size of a single message of the message queue due to the content of the data change record being too large, thereby causing the failure of the storage of the data change record.

Based on this, the invention provides a method for storing data change records, which comprises the steps of firstly, after a service end receives a data change request sent by a client, obtaining original data corresponding to changed data from a first database, wherein the data change request comprises the changed data, then, changing the original data according to the changed data to update the first database, then obtaining corresponding first data from the changed data according to a plurality of preset first fields, generating a change message according to the first data, storing the change message in the first database, simultaneously, obtaining corresponding second data from the original data and the changed data according to a plurality of preset second fields, generating a first change record according to the second data, storing the first change record in a second database, and finally, synchronizing the change message in a preset message queue, the server side consumes the change message from the message queue to obtain a first change record, then the server side carries out standardization processing on the first change record to obtain a second change record, and the finally obtained second change record is stored in a second database. The change message of the present application does not include data that may include a large amount of content, such as data before field change or data after field change, so as to control the length of the change message. According to the method and the device, the change message is sent to the message queue, the server side obtains the first change record by consuming the change message, then the first change record is subjected to standardization processing to obtain the finally required second change record, the second change record is stored in the second database, the first change record or the second change record is not directly sent to the message queue for storage, the first change record is not sent to the message queue any more, but the change message with smaller content is sent to the message queue, and the first change record with larger content is obtained according to the change message, so that the limitation that the size of a single message of the message queue is triggered due to the fact that the content of the first change record is too large can be effectively avoided, and the success rate of data change record storage is further improved.

The embodiments of the present invention will be further explained with reference to the drawings.

Referring to fig. 1, some embodiments of the present invention provide a method for storing a data change record, which includes, but is not limited to, step S110, step S120, step S130, step S140, step S150, step S160, step S170, and step S180.

Step S110, receiving a data change request sent by the client.

In some embodiments, the service end receives a data change request sent by the client end so as to perform subsequent data change, storage operation and the like, wherein the data change request comprises changed data. It is understood that the client may be a low-code development platform, the change data is change request parameters included in the data change request, the change data includes, but is not limited to, data after field change, an operator, an operation event type, an enterprise code, a data table identification code, a metadata version, a batch number, an operation time, and the like, the operation event type includes operations of adding, deleting, modifying, and the like, and subsequent operations are facilitated by using the change data.

Step S120 is to acquire original data corresponding to the changed data from the first database.

In some embodiments, after receiving the data change request, the service end searches and obtains original data corresponding to the changed data from a first database according to the data change request, where the original data includes, but is not limited to, data before field change, an operator, an operation event type, an operation time, and the like, and the first database is a relational database including Oracle, SQLServer, DB2, Mysql, and the like. It should be noted that the first database and the second database mentioned later belong to different types of databases, the first database is a relational database for using transaction characteristics of the relational database to maintain data consistency, and the second database is a non-relational database for storing a large amount of data. It can be understood that the original data may be determined according to a part of data in the data change request, such as identification data, and the original data may be data in one data table, or may be a combination of data in multiple data tables, and specific data content of the original data acquired through the data change request may be acquired according to the required data.

Step S130, the original data is changed according to the changed data so as to update the first database.

In some embodiments, the original data in the first database corresponding to the changed data is changed to update the first database, and the change operation includes adding, deleting, modifying and the like.

Step S140, obtaining corresponding first data from the changed data according to a plurality of preset first fields.

In some embodiments, after the first database is updated, the corresponding first data is obtained from the changed data according to a plurality of preset first fields, where the plurality of preset first fields include, but are not limited to, an enterprise code, a data table identification code, a metadata version, a batch number, an operation time, and other main information.

In some embodiments, the data in the first database may be composed of data for multiple enterprises, with enterprise code used to distinguish data for different enterprises; the data table identification codes comprise identification codes of all data tables and are used for distinguishing and searching different data tables; the metadata version is used for matching the metadata before updating of the first database with the updated metadata, so that the metadata corresponding to the fields which cannot be changed in the first database is avoided, the consistency of the data is convenient to maintain, and the updated metadata version can change, wherein the metadata is description of the field configuration in the data table; the batch number is an identifier of a data change request, and all data records operated after receiving the corresponding data change request can be searched according to the batch number, namely the data records with the same batch number represent changes performed when the data change request is the same. It should be noted that the above explanation on some fields in the first field is not to be construed as limiting the present invention, and the first field may be set according to specific needs, and the role and meaning of the first field may also be customized according to the needs.

And step S150, generating a change message according to the first data, and storing the change message in a first database.

In some embodiments, a change message is generated according to the acquired first data, and the change message is stored in the first database, it is understood that the change message includes specific data in the first data, such as, but not limited to, enterprise code, data table identification code, metadata version, batch number, operation time, and other main data, the batch number is also used to identify the change message, and the change message does not include data that may include a large amount of content, such as data before data field change or data after field change, so as to control the length of the change message. It will also be appreciated that the first data may be grouped into a plurality of data records based on the batch number and the data table identification code, i.e., a change message may include a plurality of data records, and a batch number and a data table identification code may uniquely identify a data record in a change message.

Step S160, obtaining corresponding second data from the original data and the modified data according to a plurality of preset second fields.

In some embodiments, after the first database is updated, corresponding second data is obtained from the original data and the changed data according to a plurality of preset second fields, where the plurality of preset second fields include: the second data includes, but is not limited to, specific data corresponding to a plurality of second fields in the original data and the changed data.

In some embodiments, the data record number is used to identify the data record, one data record number may uniquely identify one data record, the data authority field is used to set the viewing authority of the user, only the user meeting the preset authority can view the data change record subsequently stored in the second database, and the data authority field includes, but is not limited to, department, authority level, and the like. It should be noted that the above explanation on some fields in the second field is not to be construed as limiting the present invention, and the second field may be set according to specific needs, and the function and meaning of the second field may also be customized according to the needs.

Step S170, generating a first change record according to the second data, and storing the first change record in a second database.

In some embodiments, a first change record is generated according to the obtained second data, and the first change record is stored in the second database, it is understood that the first change record includes a plurality of data records, and the first change record includes specific contents in the second data, such as, but not limited to, an enterprise code, a data table identification code, a metadata version, a batch number, an operation time, an operator, an operation event type, data before field change, data after field change, a data permission field, and other specific information. It should be noted that the second database is a non-relational database, which is easy to expand and has very high read-write performance, and the second database is used for storing a first change record and a second change record obtained subsequently, and meets the requirement of mass data growth, that is, can store a large number of first change records and second change records, and includes, but is not limited to, CouchDB, mongoddb, and the like, and can meet the requirement of mass data growth, where MongoDb and CouchDB are both document-oriented databases, and are suitable for storing large data. In addition, the first change record and the second change record can be asynchronously written into the second database, and in order to better store a large amount of data, the fragmentation technology of the second database can be started, so that the data is dispersedly stored on a plurality of devices.

Step S180, synchronizing the change message to a preset message queue.

In some embodiments, the service end synchronizes the generated change message to a preset message queue, that is, delivers the change message to the preset message queue, may be implemented by using message middleware such as rockmq, kafka, nsq, and the like, where the message middleware performs asynchronous data transmission by using an efficient and reliable message delivery mechanism, and performs integration of a distributed system based on data communication, and by providing a message queue model and a message delivery mechanism, the message middleware may extend communication between processes in a distributed environment, and it may provide functions of application decoupling, elastic scaling, redundant storage, traffic peak clipping, asynchronous communication, data synchronization, and the like in the distributed environment. The server side subscribes the change information of the information queue, consumes the change information to obtain a first change record, searches and obtains the first change record from the second database according to the change information queue, and then, standardizing the obtained first change record to obtain a second change record, wherein the standardization refers to the data processing of the first change record according to a preset storage format, for example, the operator field of the first change record corresponds to the operator ID, the operator name is obtained by searching the operator ID, it is understood that the standardized processing may be logically processed according to actual requirements, and the above examples are not to be construed as limiting the present invention, and finally the second change record obtained by the processing is stored in the second database, and the second change record is the finally required data change record. According to the technical scheme of the embodiment, the change message with smaller content is sent to the message queue, the first change record with larger content is obtained according to the change message instead of directly delivering the first change record to the message queue, so that the limitation of the size of a single message of the message queue triggered by the overlarge content of the first change record can be effectively avoided, and the success rate of storing the data change record is further improved.

In some embodiments, as shown in fig. 2, after the first change record is generated according to the second data, the method for storing the data change record further includes, but is not limited to, step S210 and step S220.

Step S210, storing the first change record into a second database to obtain a storage result;

step S220, deleting the change message from the first database according to the storage failure result, and restoring the original data of the first database.

Specifically, the generated first change record is stored in the second database to obtain a storage result, the storage result is a storage failure result, and then the change message is deleted from the first database according to the storage failure result, and the original data of the first database is restored. It should be noted that, after the storage of the first change record fails, the transaction characteristics of the relational database are used to perform a data rollback operation, which includes but is not limited to: and deleting the change message stored in the first database, and recovering the original data modified by the first database.

Referring to fig. 3, some embodiments of the present invention further provide a method for storing a data change record, which includes, but is not limited to, step S310, step S320, step S330, step S340, and step S350.

Step S310, a preset message queue is obtained.

In some embodiments, the server subscribes to a preset message queue to consume the messages in the message queue.

Step S320, obtaining the change message from the message queue to obtain the obtaining result.

In some embodiments, the server side obtains the change message from a preset message queue to obtain an obtaining result, the change message is generated by the service side according to the data change request sent by the client side, the obtaining result is a successful obtaining result, and the change message is consumed after the obtaining is successful.

And step S330, consuming the change message according to the successful acquisition result to obtain a first change record.

In some embodiments, after the change message is successfully obtained from the message queue, the change message is consumed to obtain the first change record, that is, the stored first change record is searched and obtained from the second database according to the change message.

In step S340, the first change log is normalized to obtain a second change log.

In some embodiments, the obtained first change record is standardized to obtain the second change record, that is, the first change record is subjected to data processing according to a preset storage format to obtain the second change record in an expected form.

Step S350, storing the second change record in the second database.

In some embodiments, the second change record is stored in the second database, and the second change record is the finally required data change record, and the client can query and display the data change record, so that the user can clearly see the data change record at the client.

In some embodiments, as shown in fig. 4, before the first change record is standardized to obtain the second change record, the method for storing the data change record further includes, but is not limited to, step S410, step S420, step S430, and step S440.

Step S410, obtaining the change message from the message queue to obtain the obtaining result;

step S420, acquiring a current time point according to the acquisition failure result;

step S430, obtaining a change message from a first database;

step S440, obtain the first change record according to the change message.

Specifically, the server side obtains the change message from the message queue to obtain an obtaining result, wherein the obtaining result is a failure obtaining result, that is, the server side does not obtain the change message from the message queue, then obtains the current time point, obtains all the change messages with the storage time point earlier than the current time point from the first database according to the current time point, and obtains the first change record according to the obtained change messages. The change messages are stored in the first database in advance, so that the problem that the first change record cannot be acquired due to the loss of the messages in the message queue, and further the storage of the data change record fails is solved.

In some embodiments, as shown in fig. 5, fig. 5 is a flowchart of the refining step of step S330, and step S330 includes, but is not limited to, step S510, step S520.

Step S510, acquiring a batch number in the change message;

step S520, obtain the first change record from the second database according to the batch number.

Specifically, the change message includes a batch number, where the batch number is used to identify the change message, and after the change message is successfully acquired from the message queue, the batch number in the change message is acquired, and then the stored first change record is acquired from the second database according to the batch number, where the first change record includes all data records corresponding to the batch number, that is, the first change record includes all data records stored in the second database under the same data change request.

In some embodiments, as shown in fig. 6, after the second change record is stored in the first database, the method for storing a data change record provided by the present invention further includes, but is not limited to, step S610 and step S620.

Step S610, acquiring a second change record from a second database;

step S620, sending the second change record to the client, so that the client displays the second change record.

Specifically, the server side obtains the second change record from the second database, and sends the second change record to the client side for displaying, so that the user can check the second change record.

In some embodiments, the client may further search a corresponding second change record, i.e., a data change record, from the second database according to a partial field, and display the corresponding second change record, i.e., the data change record, on a corresponding interface for a user to view, where the partial field includes a batch number, a data table identification code, an operation time, and the like.

In some embodiments, after the second change record is successfully stored in the second database, the change message successfully consumed in the process of executing the method in the past is deleted from the first database, the data amount stored in the first database is reduced, and the server is prevented from finding too many messages successfully consumed when the server acquires the change message from the first database, so that the acquisition efficiency is improved.

It can be understood that, in some embodiments, the change message that is failed to be sent may be automatically retried by using a retry mechanism for sending failure of the message queue, so as to improve the probability of successful sending of the message, the change message that is failed to be consumed may also be automatically retried by using a retry mechanism for consuming failure of the message queue, so as to improve the probability of successful consumption of the message, and the change message may also be sent at regular time by using a timing mechanism of the message queue, so as to enable the server to consume, and the method is suitable for a scenario of sending messages at regular time, and is used for executing a timing task.

In some embodiments, as shown in fig. 7, fig. 7 is a flowchart illustrating an actual application of a method for storing a data change record according to some embodiments of the present invention.

Specifically, the service end receives the data change request, acquires original data from a first database according to change data in the data change request, then changes the original data according to the change data to update the first database, acquires corresponding first data from the change data according to a plurality of preset first fields, then generates a change message according to the first data, stores the change message in the first database, simultaneously acquires corresponding second data from the original data and the change data according to a plurality of preset second fields, generates a first change record according to the second data, and writes the first change record into a second database, namely stores the first change record in the second database. If the first change record is successfully written into the second database, synchronizing the change message into a preset message queue; and if the writing fails, executing a rollback operation, namely deleting the change message stored in the first database and recovering the original data for modifying the first database. Then, the server subscribes the message queue, acquires the change message from the message queue, and if the change message is successfully acquired, consumes the change message to acquire first change data from a second database and generates a first change record according to the first change data; if the obtaining fails, obtaining the change information which is earlier than the current time point from the first database, consuming the change information to obtain the first change data from the second database, and generating a first change record according to the first change data, so that the situation that the change information in the information queue is lost and the first change record cannot be obtained is avoided. Then, the first change record is standardized to obtain a second change record, the second change record is written into a second database, namely the second change record is stored into the second database, and finally the change message stored in the first database is deleted. According to the method and the device, the first change record is not sent to the message queue, but the change message with smaller content is sent to the message queue, and then the server side obtains the first change record with larger content according to the change message, so that the limitation of the size of a single message of the message queue triggered by the overlarge content of the first change record can be effectively avoided, the success rate of storing the data change record is further improved, and the success rate of storing the second change record is also improved.

Referring to fig. 8, some embodiments of the present invention also provide a first storage apparatus including a receiving module 800, a first obtaining module 810, a changing module 820, a second obtaining module 830, a first storage module 840, a third obtaining module 850, a second storage module 860, and a synchronizing module 870. The receiving module 800 receives a data change request sent by a client, where the data change request includes change data, the first obtaining module 810 obtains original data corresponding to the change data from a first database, then the changing module 820 changes the original data to update the first database, the second obtaining module 830 obtains corresponding first data from the change data according to a plurality of preset first fields, the first storing module 840 stores a change message generated according to the first data in the first database, meanwhile, the third obtaining module 850 obtains corresponding second data from the original data and the change data according to a plurality of preset second fields, the second storing module 860 generates a first change record according to the second data and stores the first change record in the second database, and finally, the synchronizing module 870 synchronizes the change message to a preset message queue, and the server side consumes the change message from the message queue to obtain a first change record, then carries out standardization processing on the first change record to obtain a second change record, and stores the finally obtained second change record in a second database. According to the application, the change message is sent to the message queue through the synchronization module 870, the server side obtains a first change record through consuming the change message, then the first change record is standardized to obtain a finally needed second change record, and the second change record is stored in the second database instead of directly sending the first change record or the second change record to the message queue for storage. According to the method and the device, the first change record is not sent to the message queue any more, but the change message with smaller content is sent to the message queue, and the first change record with larger content is obtained according to the change message, so that the limitation of the size of a single message of the message queue triggered by the overlarge content of the first change record can be effectively avoided, and the success rate of data change record storage is further improved.

Referring to fig. 9, some embodiments of the present invention also provide a second storage apparatus including a fourth obtaining module 900, a fifth obtaining module 910, a consuming module 920, a normalization processing module 930, and a third storage module 940. The server obtains a preset message queue through the fourth obtaining module 900, and then obtains a change message in the message queue through the fifth obtaining module 910, to obtain an obtaining result, where the change message is generated by the service according to a data change request sent by the client, the obtaining result is a successful obtaining result, that is, after the server successfully obtains the change message in the preset message queue, the consumption module 920 consumes the change message to obtain a first change record, the standardization processing module 930 performs standardization processing on the first change record to obtain a second change record, and then the third storage module 940 stores the finally obtained second change record in the second database. According to the application, the consumption module 920 consumes the change message in the message queue to obtain the first change record, the standardization processing module 930 is used for conducting standardization processing on the first change record to obtain the second change record, and the third storage module 940 is used for storing the finally obtained second change record into the second database.

Some embodiments of the present invention also provide a computer device, which includes a memory and a processor, wherein the memory stores a program, and the program is used for executing the data change record storage method in any of the above embodiments when executed by the processor, for example, executing the above-described method steps S110 to S180 in fig. 1, method steps S210 to S220 in fig. 2, method steps S310 to S350 in fig. 3, method steps S410 to S440 in fig. 4, method steps S510 to S520 in fig. 5, and method steps S610 to S620 in fig. 6.

In addition, some embodiments of the present invention also provide a storage medium, which is a computer-readable storage medium, in which a computer program is stored, and when the computer program is executed by a computer, the computer is configured to execute the storage method of the data change record in any of the above embodiments, for example, the method steps S110 to S180 in fig. 1, the method steps S210 to S220 in fig. 2, the method steps S310 to S350 in fig. 3, the method steps S410 to S440 in fig. 4, the method steps S510 to S520 in fig. 5, and the method steps S610 to S620 in fig. 6, which are described above, are executed.

One of ordinary skill in the art will appreciate that all or some of the steps, systems, and methods disclosed above may be implemented as software, firmware, hardware, and suitable combinations thereof. Some or all of the physical components may be implemented as software executed by a processor, such as a central processing unit, digital signal processor, or microprocessor, or as hardware, or as an integrated circuit, such as an application specific integrated circuit. Such software may be distributed on computer readable media, which may include computer storage media (or non-transitory media) and communication media (or transitory media). The term computer storage media includes volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules or other data, as is well known to those of ordinary skill in the art. Computer storage media includes, but is not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, Digital Versatile Disks (DVD) or other optical disk storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other medium which can be used to store the desired information and which can accessed by a computer. In addition, communication media typically embodies computer readable instructions, data structures, program modules or other data in a modulated data signal such as a carrier wave or other transport mechanism and includes any information delivery media as known to those skilled in the art.

The embodiments of the present invention have been described in detail with reference to the accompanying drawings, but the present invention is not limited to the above embodiments, and various changes can be made within the knowledge of those skilled in the art without departing from the gist of the present invention. Furthermore, the embodiments of the present invention and the features of the embodiments may be combined with each other without conflict.

Claims

1. The method for storing the data change record is characterized by comprising the following steps:

2. The method of claim 1, wherein after the generating a first change record from the second data, the method further comprises:

3. The method for storing the data change record is characterized by comprising the following steps:

acquiring a preset message queue;

standardizing the first change record to obtain a second change record;

storing the second change record to a second database.

4. The method of claim 3, wherein prior to the normalizing the first change record to obtain a second change record, the method further comprises:

acquiring a current time point according to the acquisition failure result;

and acquiring the first change record according to the change message.

5. The method of claim 3, wherein the change message includes a batch number, and wherein consuming the change message according to the successful result of the obtaining to obtain a first change record comprises:

6. The method of any of claims 3 to 5, wherein after said storing the second change record to a second database, the method further comprises:

obtaining the second change record from the second database;

7. A first storage device, comprising:

8. A second storage device, comprising:

9. Computer arrangement, characterized in that the computer arrangement comprises a memory and a processor, wherein the memory has stored therein a computer program, which when executed by the processor is adapted to perform: the method of any one of claims 1 to 6.

10. A storage medium which is a computer-readable storage medium, wherein a computer program is stored in the computer-readable storage medium, and when the computer program is executed by a computer, the computer is configured to execute: the method of any one of claims 1 to 6.