CN117520449A - Data operation management method, device, equipment and storage medium - Google Patents
Data operation management method, device, equipment and storage medium Download PDFInfo
- Publication number
- CN117520449A CN117520449A CN202311498489.5A CN202311498489A CN117520449A CN 117520449 A CN117520449 A CN 117520449A CN 202311498489 A CN202311498489 A CN 202311498489A CN 117520449 A CN117520449 A CN 117520449A
- Authority
- CN
- China
- Prior art keywords
- synchronization
- target data
- resource
- node
- distributed node
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000007726 management method Methods 0.000 title claims abstract description 49
- 230000001360 synchronised effect Effects 0.000 claims abstract description 88
- 238000000034 method Methods 0.000 claims abstract description 70
- 238000012545 processing Methods 0.000 claims abstract description 28
- 230000008569 process Effects 0.000 claims abstract description 14
- 230000004044 response Effects 0.000 claims description 35
- 230000009471 action Effects 0.000 claims description 18
- 238000010586 diagram Methods 0.000 description 7
- 238000012986 modification Methods 0.000 description 3
- 230000004048 modification Effects 0.000 description 3
- 238000004891 communication Methods 0.000 description 2
- 238000004590 computer program Methods 0.000 description 2
- 238000013523 data management Methods 0.000 description 2
- 230000006978 adaptation Effects 0.000 description 1
- 238000007792 addition Methods 0.000 description 1
- 238000004458 analytical method Methods 0.000 description 1
- 238000013475 authorization Methods 0.000 description 1
- 238000010276 construction Methods 0.000 description 1
- 238000012217 deletion Methods 0.000 description 1
- 230000037430 deletion Effects 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 238000000802 evaporation-induced self-assembly Methods 0.000 description 1
- 238000004806 packaging method and process Methods 0.000 description 1
- 230000002093 peripheral effect Effects 0.000 description 1
- 238000013439 planning Methods 0.000 description 1
- 230000009466 transformation Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/27—Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/52—Program synchronisation; Mutual exclusion, e.g. by means of semaphores
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/54—Interprogram communication
- G06F9/546—Message passing systems or structures, e.g. queues
Abstract
The application provides a data operation management method, a device, equipment and a storage medium. The method comprises the following steps: the central node acquires a resource creation request, wherein the resource creation request comprises a site identifier, a resource identifier and a resource value; the central node creates target data corresponding to the resource identifier and the resource value according to the resource creation request, carries the target data in the synchronous message, and sends the target data to the distributed node corresponding to the site identifier through the unidirectional synchronous channel corresponding to the distributed node corresponding to the site identifier; the distributed node acquires the synchronous message through the unidirectional synchronous channel, and sends the target data in the synchronous message to a database in the distributed node according to the synchronous message so as to enable the database to synchronously process the target data, so that the distributed node acquires the target data corresponding to the distributed node in the process of acquiring the target data from the database to perform corresponding processing. The method improves the management efficiency of data operation.
Description
Technical Field
The present disclosure relates to the field of data synchronization technologies, and in particular, to a data operation management method, device, equipment, and storage medium.
Background
In response to customer service demands of all areas, the Pass platform needs to quickly build sceneries suitable for all areas, so that digital transformation is realized. Based on the coexistence requirement of intensive operation and hierarchical operation, the Pass platform provides an operation architecture system of hierarchical deployment to manage all levels of data.
In the prior art, the Pass platform provides an operation architecture system of hierarchical deployment, which mainly comprises: a central node and a plurality of distributed nodes. Specifically, the Pass platform operation management mainly includes: the central node is mainly used for carrying out operation management on data and mainly comprises the following steps: store, find, modify, delete, etc. The distributed nodes do not need to store the data, but only receive the needed data from the central node and display the data. Because all data are stored in the central node, the volume of the data is relatively large, so that the central node can search and send the data slowly, and the requirement of quickly constructing scenes is not met. In addition, when each distributed node needs to display data, a central node needs to be requested to acquire corresponding data, so that the operation management efficiency of the current Pass platform is relatively poor.
Based on the above, the platform operation data management in the prior art has the problems of poor management efficiency and incapability of realizing the requirement of quickly building the scene product.
Disclosure of Invention
The application provides a data operation management method, a device, equipment and a storage medium, which are used for solving the technical problems that the management efficiency of platform operation data management in the prior art is poor and the requirement of quickly constructing a scene product cannot be met.
In a first aspect, the present application provides a data operation management method, including:
the central node acquires a resource creation request, wherein the resource creation request comprises a site identifier, a resource identifier and a resource value;
the central node creates target data corresponding to the resource identifier and the resource value according to the resource creation request, carries the target data in the synchronous message, and sends the target data to the distributed node corresponding to the site identifier through the unidirectional synchronous channel corresponding to the distributed node corresponding to the site identifier;
the distributed node acquires the synchronous message through the unidirectional synchronous channel, and sends the target data in the synchronous message to a database in the distributed node according to the synchronous message so as to enable the database to synchronously process the target data, so that the distributed node acquires the target data corresponding to the distributed node in the process of acquiring the target data from the database to perform corresponding processing.
Optionally, in the method above, the synchronization message further includes: transaction identification, and synchronization actions; wherein the synchronization action includes adding, modifying and/or deleting;
then, according to the synchronization message, the target data in the synchronization message is sent to a database in the distributed node, including:
and adding the target data to the database in the distributed node according to the synchronous action in the synchronous message, and/or modifying or deleting the target data in the database in the distributed node.
Optionally, the method as above further comprises:
the distributed node acquires a storage response fed back by the database and generates a feedback message when the storage response is a storage failure response;
the distributed node sends the feedback information to the central node, so that the central node can send the target data to the distributed node again through the unidirectional synchronous channel according to the feedback information.
Optionally, the method as above, the distributed node obtaining a storage response of the database feedback, and generating the feedback message when the storage response is a storage failure response, includes:
the distributed node acquires a storage response fed back by the database, and acquires a corresponding synchronization method, synchronization time, synchronization state, operator identification, operation time and synchronization dependence when the database performs synchronization processing on target data when the storage response is a storage failure response; the method comprises the steps that a synchronization method characterizes a method for adding, modifying and deleting target data by a distributed node, and the synchronization state comprises success or failure; the synchronization dependence characterizes the relationship between the target data;
the distributed node generates a feedback message according to the synchronization method, the synchronization time, the synchronization state, the operator identification, the operation time and the synchronization dependence.
Optionally, in the method above, the center node re-sends the target data to the distributed node through the unidirectional synchronization channel according to the feedback message, including:
if the central node determines that the synchronization state in the feedback message is failed, determining re-synchronized target data based on the feedback message, and starting timing so as to send the re-synchronized target data to the distributed nodes through the unidirectional synchronization channel when the timing time reaches the preset synchronization time;
or,
if the central node determines that the synchronization state in the feedback message is failure, determining re-synchronization target data based on the feedback message, and sending the re-synchronization target data to the distributed nodes through the unidirectional synchronization channel when receiving the synchronization request.
Optionally, the method as above, the central node obtains a resource creation request, including:
the central node receives a resource creation request sent by a user terminal;
or,
the central node receives a resource creation request uploaded by the distributed node, wherein the resource creation request is a request sent to the distributed node by a user terminal.
Optionally, the method as above, the resource identification includes: user resource identification, account resource identification, and/or model resource identification. In a second aspect, the present application provides a data operation management apparatus, including:
the acquisition module is used for acquiring a resource creation request by the central node, wherein the resource creation request comprises a site identifier, a resource identifier and a resource value;
the synchronous module is used for creating target data corresponding to the resource identifier and the resource value according to the resource creation request by the central node, carrying the target data in the synchronous message, and sending the synchronous message to the distributed node corresponding to the site identifier through a unidirectional synchronous channel corresponding to the distributed node corresponding to the site identifier;
the processing module is used for the distributed node to acquire the synchronous message through the unidirectional synchronous channel, and according to the synchronous message, the target data in the synchronous message is sent to the database in the distributed node so as to carry out synchronous processing on the target data by the database, so that the distributed node acquires the target data corresponding to the distributed node, and the target data is acquired from the database to carry out corresponding processing. In a third aspect, the present application contemplates an electronic device comprising a processor, and a memory communicatively coupled to the processor;
the memory stores computer-executable instructions;
the processor executes the computer-executable instructions stored in the memory to implement the methods as described herein.
In a fourth aspect, the present application proposes a computer readable storage medium having stored therein computer executable instructions for implementing a method as the present application when executed by a processor.
According to the data operation management method, the device, the equipment and the storage medium, a resource creation request is acquired through a central node, target data corresponding to a resource identifier and a resource value are created according to the resource creation request, the target data are carried in a synchronous message, the synchronous message is sent to a distributed node corresponding to a site identifier through a unidirectional synchronous channel corresponding to the distributed node corresponding to the site identifier, the distributed node acquires the synchronous message through the unidirectional synchronous channel, and the target data in the synchronous message are sent to a database in the distributed node according to the synchronous message so that the database can synchronously process the target data, and accordingly the distributed node can acquire the target data corresponding to the distributed node and process the target data correspondingly from the database. After the central node creates the target data, the target data is actively synchronized to the database in the distributed node through the unidirectional synchronization channel, so that the distributed node does not need to request the central node to acquire corresponding data when the data is required to be displayed, the corresponding data can be directly acquired from the internal database, and the data operation management efficiency is improved. Meanwhile, because a unified single-flow-direction synchronous channel (namely a unidirectional synchronous channel) is constructed between the central node and the distributed nodes, the complexity of synchronous logic is greatly reduced, and the condition that the data of the central node and the distributed nodes of the hierarchical operation scene are inconsistent is avoided, so that the operation management efficiency of the Pass platform is effectively improved.
Drawings
The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the application and together with the description, serve to explain the principles of the application.
FIG. 1 is a schematic diagram of a hierarchical operation scenario architecture of a Pass platform on which the data operation management method of the present application is based;
fig. 2 is a flow chart of a data operation management method according to an embodiment of the present application;
fig. 3 is a flow chart of a data operation management method according to an embodiment of the present application;
fig. 4 is a flow chart of a data operation management method provided in an embodiment of the present application;
fig. 5 is a flow chart of a data operation management method according to an embodiment of the present application;
fig. 6A is a flow chart of a data operation management method according to an embodiment of the present application;
fig. 6B is a flow chart of a data operation management method according to an embodiment of the present application;
fig. 7 is a flow chart of another data operation management method according to an embodiment of the present application;
fig. 7A is a schematic flow chart of obtaining a resource creation request according to an embodiment of the present application;
FIG. 7B is a flowchart illustrating another method for obtaining a resource creation request according to an embodiment of the present application;
fig. 8 is a schematic structural diagram of a data operation management device according to an embodiment of the present application;
fig. 9 is a schematic structural diagram of an electronic device according to an embodiment of the present application.
Specific embodiments thereof have been shown by way of example in the drawings and will herein be described in more detail. These drawings and the written description are not intended to limit the scope of the inventive concepts in any way, but to illustrate the concepts of the present application to those skilled in the art by reference to specific embodiments.
Detailed Description
Reference will now be made in detail to exemplary embodiments, examples of which are illustrated in the accompanying drawings. When the following description refers to the accompanying drawings, the same numbers in different drawings refer to the same or similar elements, unless otherwise indicated. The implementations described in the following exemplary examples are not representative of all implementations consistent with the present application. Rather, they are merely examples of apparatus and methods consistent with some aspects of the present application as detailed in the accompanying claims.
It should be noted that, the user information (including but not limited to user equipment information, user personal information, etc.) and the data (including but not limited to data for analysis, stored data, presented data, etc.) referred to in the present application are information and data authorized by the user or fully authorized by each party, and the collection, use and processing of the related data need to comply with the related laws and regulations and standards, and provide corresponding operation entries for the user to select authorization or rejection.
In the prior art, data operation management has coexistence requirements of intensive operation and hierarchical operation, so that a hierarchical deployment operation architecture system is constructed, and the system comprises: a central node and a plurality of distributed nodes; and the data synchronization efficiency of the central node and the plurality of distributed nodes can influence the data operation management.
However, at present, the data is mainly managed by the central node, but the distributed nodes do not need to manage the data, but only accept the data from the central node for display. Because all data are stored in the central node, the volume of the data is large, the central node can spend more time searching and sending the data corresponding to the distributed node each time, and the distributed node needs to send a request to the central node each time when the distributed node needs to acquire the data, so that the data operation management efficiency is low.
Based on the method, the data are stored in the central node in a concentrated manner, and the problem that the data operation management efficiency is low only in a mode of displaying the data in the distributed nodes and the rapid construction of the scene product cannot be realized exists.
In order to solve the above problems, the inventive concept of the present application is: how to simply and efficiently realize the data synchronization between the central node and the distributed nodes so as to realize the requirement of quickly constructing the scenerised product.
Fig. 1 is a schematic diagram of a hierarchical operation scenario of a Pass platform based on the data operation management method of the present application, and as shown in fig. 1, the Pass platform mainly includes: a central node 11, and a plurality of distributed nodes 12. The central node 11 may be a server, and the distributed nodes 12 may be clients. Specifically, the central node 11 establishes a unidirectional synchronization channel with each distributed node 12 in a Socket manner, so as to realize that the central node 11 performs data synchronization to each distributed node 12.
The following describes the technical solutions of the present application and how the technical solutions of the present application solve the above technical problems in detail with specific embodiments. The following embodiments may be combined with each other, and the same or similar concepts or processes may not be described in detail in some embodiments. Embodiments of the present application will be described below with reference to the accompanying drawings.
Fig. 2 is a flow chart of a data operation management method according to an embodiment of the present application. As shown in fig. 2, the method may include:
s201, a central node acquires a resource creation request, wherein the resource creation request comprises a site identifier, a resource identifier and a resource value.
In this embodiment, optionally, the central node may receive a resource creation request sent by the user terminal. Or the central node receives a resource creation request uploaded by the distributed node, wherein the resource creation request is a request sent to the distributed node by the user terminal.
In the embodiment, both the central node and the distributed nodes can initiate resource creation requests, but the resource creation requests are received by the central node and sent to the corresponding distributed nodes through unidirectional channels, and the data synchronization mode ensures that the data of the central node and the distributed nodes are always consistent, so that the possibility of occurrence of problems in data operation management due to different data is avoided.
More optionally, the resource identifier includes: user resource identification, account resource identification, and/or model resource identification.
In this embodiment, the user resource identifier may refer to a unique identifier that determines the user; the account resource identification may refer to a unique identification that determines an account, for example, the account may be classified as an administrator account, a general account; the model resource identification may refer to an identification characterizing device information.
S202, the center node creates target data corresponding to the resource identifier and the resource value according to the resource creation request, carries the resource identifier and the resource value in the synchronous message, and sends the synchronous message to the distributed node corresponding to the site identifier through the unidirectional synchronous channel corresponding to the distributed node corresponding to the site identifier.
In this embodiment, the resource identifier refers to an identifier corresponding to data to be created by the central node based on a resource creation request, and the resource value refers to content of the data created by the central node. Thus, the central node may create the target data based on the resource identification and the resource value in the resource creation request.
In addition, the central node also needs to synchronize data to the distributed nodes after creating the target data. Because the site identifier is a unique identifier corresponding to the distributed node, the distributed node needing to perform data synchronization can be determined through the site identifier. After determining the distributed node corresponding to the site identifier, the central node carries the resource identifier and the resource value in a synchronous message through a unidirectional synchronous channel between the central node and the distributed node and sends the synchronous message to the distributed node.
For example, the central node receives two resource creation requests respectively, the first resource creation request including: a site identification of the first distributed site, a resource identification of the first resource, and a resource value; the second resource creation request includes: the site identification of the second distributed site, the resource identification of the second resource, and the resource value. The central node creates first target data according to the first resource creation request, and then sends the first target data to the first distributed site in the synchronization message with the resource identifier and the resource value of the first resource, so that the first distributed site can perform first target data synchronization processing in the database based on the resource identifier and the resource value of the first resource in the synchronization message. In addition, the central node can also create second target data according to the second resource creation request, and then send the second distributed site with the resource identifier and the resource value of the second resource in the synchronization message, so that the second distributed site can perform second target data synchronization processing in the database based on the resource identifier and the resource value of the second resource in the synchronization message.
For example, the site identifier of the first distributed site may be 1001, the resource identifier of the first resource is 2001, and the resource value is a. The site identification of the second distributed site may be 1002, the resource identification of the second resource is 2002, and the resource value is a. From the site identification 1001, the resource identification 2001, the resource value a, first target data is created, the first target data including: resource identification 2001, resource value a, by which it can be determined that resource value a for resource identification 2001 is to be synchronized to site number 1. Similarly, according to the site identifier 1002, the resource identifier 2002 and the resource value A, second target data is created, wherein the second target data comprises: resource identification 2002, resource value a, by which it can be determined that resource value a for resource identification 2002 is to be synchronized to site number 2. Thus, different distributed sites can be identified by site identification, different resources can be identified by resource identification, and resource values characterize the content of the resources.
S203, the distributed node acquires the synchronous message through the unidirectional synchronous channel, and sends the target data in the synchronous message to a database in the distributed node according to the synchronous message so as to enable the database to synchronously process the target data, so that the distributed node acquires the target data corresponding to the distributed node in the process of acquiring the target data from the database to perform corresponding processing.
In this embodiment, for example, the request body corresponding to the synchronization message mainly includes:
| field identification | Field name | Field type | Description of the invention |
| site_id | Site identification | String | For identifying distributed sites |
| site_key | Site value | String | Name of distributed site |
| resource_id | Resource identification | String | For identifying data |
| resourece_key | Resource value | String | Content of data |
In this embodiment, after the distributed node receives the synchronization message illustrated by the above example, it is determined whether the station identifier in the synchronization message is consistent with the station identifier corresponding to the distributed node, and if the station identifier is consistent with the station identifier, the resource identifier and the target data corresponding to the resource identifier are sent to the database in the distributed node, so that the database performs synchronization processing on the target data.
In this embodiment, the central node obtains a resource creation request, creates target data corresponding to a resource identifier and a resource value according to the resource creation request, carries the target data in a synchronization message, sends the synchronization message to a distributed node corresponding to a site identifier through a unidirectional synchronization channel corresponding to a distributed node corresponding to the site identifier, and sends the target data in the synchronization message to a database in the distributed node according to the synchronization message, so that the database performs synchronization processing on the target data, so that the distributed node obtains the target data corresponding to the distributed node, and obtains the target data from the database to perform corresponding processing. After the central node creates the target data, the target data is actively synchronized to the database in the distributed node through the unidirectional synchronization channel, so that the distributed node does not need to request the central node to acquire corresponding data when the data is required to be displayed, the corresponding data can be directly acquired from the internal database, and the data operation management efficiency is improved. Meanwhile, because a unified single-flow-direction synchronous channel (namely a unidirectional synchronous channel) is constructed between the central node and the distributed nodes, the complexity of synchronous logic is greatly reduced, and the condition that the data of the central node and the distributed nodes of the hierarchical operation scene are inconsistent is avoided, so that the operation management efficiency of the Pass platform is effectively improved.
Fig. 3 is a flow chart of a data operation management method according to a second embodiment of the present application. Based on the embodiment shown in fig. 2, as shown in fig. 3, a specific implementation manner of the step S203 is as follows:
s301, the distributed node acquires the synchronous message through a unidirectional synchronous channel, and adds the target data to a database in the distributed node according to the synchronous action in the synchronous message, and/or modifies or deletes the target data in the database in the distributed node.
In this embodiment, for example, the request body of the synchronization message may further include:
in this embodiment, the transaction identifier may refer to a primary key of a synchronization message, and is used to identify different synchronization messages.
Wherein the synchronization message may further include: the synchronization action, in particular, may include, but is not limited to: new addition, modification and deletion. For example, if the synchronization action is newly added, storing the resource identifier and the target data to a database in the distributed node; if the synchronous action is modification, the data is found in a database of the distributed node according to the resource identification, and the modification is carried out according to the target data; if the synchronization action is deleting, the data is found in the database of the distributed node according to the resource identification, and the data is deleted. And the data in the database in the distributed node is synchronously processed according to the synchronous action in the synchronous message, so that the data of the distributed node and the data of the central node can be kept consistent, and the data autonomy of the distributed node is realized.
Fig. 4 is a flow chart of a data operation management method according to a third embodiment of the present application. On the basis of the embodiment shown in fig. 2, as shown in fig. 4, after the step S203, the data operation management method further includes:
s401, the distributed node acquires a storage response fed back by the database, and generates a feedback message when the storage response is a storage failure response.
In this embodiment, the storage response fed back by the database includes success and failure, and if the storage response fed back by the database is successful, the data synchronization is successfully represented, and the synchronization is ended; if the storage response fed back by the database is failure, the data synchronization is represented to be failed, and a feedback message needs to be generated to inform the central node.
And S402, the distributed node sends the feedback message to the central node, so that the central node can send the target data to the distributed node again through the unidirectional synchronous channel according to the feedback message.
In this embodiment, the feedback message may determine that the synchronization of the target data fails, and inform the central node to send the target data to the distributed node again through the unidirectional synchronization channel, so that the distributed node continuously completes the data synchronization operation, and the problem that the data of the central node and the distributed node is inconsistent due to the synchronization failure is avoided.
Fig. 5 is a flow chart of a data operation management method according to a fourth embodiment of the present application, and on the basis of the embodiment shown in fig. 4, as shown in fig. 5, a specific implementation manner of the step S401 is as follows:
s501, the distributed node acquires a storage response fed back by the database, and acquires a corresponding synchronization method, synchronization time, synchronization state, operator identification, operation time and synchronization dependence when the database performs synchronization processing on target data when the storage response is a storage failure response; the method comprises the steps that a synchronization method characterizes a method for adding, modifying and deleting target data by a distributed node, and the synchronization state comprises success or failure; synchronization dependencies characterize relationships between target data.
In this embodiment, for example, if the first administrator of the newly added distributed node succeeds, the first administrator of the distributed node fails to newly add the first user of the distributed node, the distributed node receives a feedback message of a database in the distributed node about the failure of the newly added first user, and obtains a corresponding synchronization method, synchronization time, synchronization status, operator identifier, operator time and synchronization dependency, where the corresponding synchronization method is the newly added user method, the synchronization time is the time for executing the synchronization operation, the synchronization status is the failure, the operator is the first administrator, the operation time is the time of the newly added first user of the distributed node, and the synchronization dependency is the newly added first user of the first administrator.
S502, the distributed node generates a feedback message according to the synchronization method, the synchronization time, the synchronization state, the operator identification, the operation time and the synchronization dependence.
In this embodiment, the feedback message includes, for example:
in this embodiment, generating the feedback message may refer to acquiring a synchronization method, synchronization time, synchronization status, operator identifier, operation time and synchronization dependency of the data with synchronization failure, and encapsulating the data into a feedback message body.
Fig. 6A to fig. 6B are respectively schematic flow diagrams of a data operation management method according to a fifth embodiment of the present application, and based on the embodiment shown in fig. 4, as shown in fig. 6A, a specific implementation manner of the step S402 is as follows:
and S601a, if the central node determines that the synchronous state in the feedback message is failed.
S602a, determining re-synchronous target data based on the feedback message.
And S603a, starting timing so as to send the re-synchronized target data to the distributed nodes through the unidirectional synchronization channel when the timing time reaches the preset synchronization time.
In this embodiment, for example, the preset synchronization time is 8 per day: 00, time reaches 8: and 00, the center node acquires feedback information with failed synchronization state from the database corresponding to the center node, determines corresponding target data according to the feedback information, and performs resynchronization operation. By executing the resynchronization work at regular time, the workload of synchronization can be reduced, and the data operation management efficiency can be improved.
Alternatively, as shown in fig. 6B, another implementation manner of the step S402 is:
and S601b, if the central node determines that the synchronous state in the feedback message is failed.
S602b, determining the re-synchronous target data based on the feedback message.
And S603b, when receiving the synchronization request, sending the re-synchronized target data to the distributed nodes through the unidirectional synchronization channel.
In this embodiment, according to receiving the synchronization request, real-time data synchronization can be performed on the target data with synchronization failure, so that the problem that data with real-time requirements cannot be synchronized in time is avoided.
Fig. 7 is a flow chart of another data operation management method provided in the embodiment of the present application, and as shown in fig. 7, the data operation management method includes:
s701, receiving a resource creation request through a synchronous event receiving service preset by a central node.
In this embodiment, as shown in fig. 7A, the center node acquires a resource creation request of the user terminal of the center node, or, as shown in fig. 7B, the center node acquires a resource creation request of the distributed node.
S702, acquiring a corresponding capability interface in synchronous event receiving service preset by a center node according to a resource creating request, and completing resource creation of the center node.
In this embodiment, the capability interface may refer to an interface that provides resource operation capability for the central node, including, but not limited to, a new user resource interface, a new account resource interface, and a new object model resource interface. For example, if the resource creation request is a newly added user resource, the corresponding capability interface is a newly created user resource interface.
S703, acquiring target data in the resource creation request, and packaging the target data into a request body preset by the synchronous interface to obtain a synchronous message.
In this embodiment, the synchronization message includes a transaction identifier, a site value, a resource identifier, a resource value, a synchronization action, and a synchronization result, where the transaction identifier characterizes a unique identifier of the synchronization message, the site identifier characterizes a unique identifier of the distributed site, the site value characterizes a name of the distributed site, the resource identifier characterizes a unique identifier of data to be synchronized, the resource value characterizes a content of the data to be synchronized, the synchronization action includes adding, modifying, or deleting, and an initial value of the synchronization result is a null value.
S704, sending the synchronous message from the central node to the corresponding distributed node through a preset unidirectional synchronous channel between the central node and the distributed node.
In this embodiment, the preset unidirectional synchronization channel between the central node and the distributed node may refer to that the central node is used as a server, the distributed node is used as a client, the central node and the distributed node establish long connection in a Socket (Socket) manner, and only data is allowed to be synchronized from the central node to the distributed node, where Socket is an abstract layer through which an application program can send or receive data, and can perform operations such as opening, reading, writing, closing, and the like, which are similar to a file, on the application program.
And S705, the corresponding distributed node completes the synchronization of the target data according to the synchronization message, and feeds back the synchronization result to the central node.
In this embodiment, if the synchronization of the target data is successful, the synchronization result is successful, and if the synchronization of the data is failed, the synchronization result is failed, and the synchronization result is returned to the central node.
S706, if the synchronization result is failure, recording the synchronization result information, and re-executing the synchronization operation of the synchronization message in response to the synchronization request.
In this embodiment, the synchronization result information includes a synchronization user, a synchronization method, a synchronization time, a synchronization state, a synchronization dependency, an operator, an operation time, and the like, where the synchronization user may refer to a user of the central node or a user of the distributed node; the synchronization method may refer to a method of the distributed node performing the synchronization action, for example, including an add method, a modify method, a delete method, and the like; the synchronization time may refer to the time of executing the synchronization operation, the synchronization state is initially a null value, and after the synchronization operation is completed, the synchronization state is determined to be successful or failed according to the synchronization result; synchronization dependencies may refer to relationships between synchronization data; the operator may refer to a user who creates the synchronization action, and the operation time may refer to a time at which the synchronization action is performed.
And S707, if the synchronization result is that the synchronization fails, recording the synchronization result information, and re-executing the synchronization operation of the synchronization message in a preset period.
Fig. 8 is a schematic structural diagram of a data operation management device according to an embodiment of the present application. As shown in fig. 8, the data operation management device 80 includes: an acquisition module 801, a synchronization module 802, and a processing module 803. Wherein:
an obtaining module 801, configured to obtain, by a central node, a resource creation request, where the resource creation request includes a site identifier, a resource identifier, and a resource value;
the synchronization module 802 is configured to create, by the central node, target data corresponding to the resource identifier and the resource value according to the resource creation request, and carry the target data in a synchronization message, and send the synchronization message to a distributed node corresponding to the site identifier through a unidirectional synchronization channel corresponding to the distributed node corresponding to the site identifier;
the processing module 803 is configured to obtain the synchronization message by the distributed node through the unidirectional synchronization channel, and send the target data in the synchronization message to a database in the distributed node according to the synchronization message, so that the database performs synchronization processing on the target data, so that the distributed node obtains the target data corresponding to the distributed node, and performs corresponding processing on the target data obtained from the database.
In the embodiment of the present application, the obtaining module 801 may be further specifically configured to:
the central node receives a resource creation request sent by a user terminal;
or,
the central node receives a resource creation request uploaded by the distributed node, wherein the resource creation request is a request sent to the distributed node by a user terminal.
In the embodiment of the present application, the synchronization module 802 may be specifically further configured to:
if the central node determines that the synchronization state in the feedback message is failed, determining re-synchronized target data based on the feedback message, and starting timing so as to send the re-synchronized target data to the distributed nodes through the unidirectional synchronization channel when the timing time reaches the preset synchronization time;
or,
if the central node determines that the synchronization state in the feedback message is failure, determining re-synchronization target data based on the feedback message, and sending the re-synchronization target data to the distributed nodes through the unidirectional synchronization channel when receiving the synchronization request.
In the embodiment of the present application, the processing module 803 may further specifically be configured to:
and adding the resource identification and the target data to a database in the distributed node according to the synchronous action in the synchronous message, and/or modifying or deleting the resource identification and the target data in the database in the distributed node.
In the embodiment of the present application, the processing module 803 may further specifically be configured to:
the distributed node acquires a storage response fed back by the database and generates a feedback message when the storage response is a storage failure response;
the distributed node sends the feedback information to the central node, so that the central node can send the target data to the distributed node again through the unidirectional synchronous channel according to the feedback information.
In the embodiment of the present application, the processing module 803 may further specifically be configured to:
the distributed node acquires a storage response fed back by the database, and acquires a corresponding synchronization method, synchronization time, synchronization state, operator identification, operation time and synchronization dependence when the database performs synchronization processing on target data when the storage response is a storage failure response; the method comprises the steps that a synchronization method characterizes a method for adding, modifying and deleting target data by a distributed node, and the synchronization state comprises success or failure; the synchronization dependence characterizes the relationship between the target data;
the distributed node generates a feedback message according to the synchronization method, the synchronization time, the synchronization state, the operator identification, the operation time and the synchronization dependence.
Fig. 9 is a schematic structural diagram of an electronic device according to an embodiment of the present application. As shown in fig. 9, the electronic device 90 includes:
the electronic device 90 may include one or more processing cores 'processors 901, one or more computer-readable storage media's memory 902, communication components 903, and the like. The processor 901, the memory 902, and the communication unit 903 are connected via a bus 904.
In a specific implementation, at least one processor 901 executes computer-executable instructions stored in a memory 902, such that the at least one processor 901 performs a method of resource planning as described above.
The specific implementation process of the processor 901 may refer to the above-mentioned method embodiment, and its implementation principle and technical effects are similar, and this embodiment will not be described herein again.
In the embodiment shown in fig. 9, it should be understood that the processor may be a central processing unit (english: central Processing Unit, abbreviated as CPU), or may be other general purpose processors, digital signal processors (english: digital Signal Processor, abbreviated as DSP), application specific integrated circuits (english: application Specific Integrated Circuit, abbreviated as ASIC), or the like. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like. The steps of a method disclosed in connection with the present invention may be embodied directly in a hardware processor for execution, or in a combination of hardware and software modules in a processor for execution.
The Memory may comprise high-speed Memory (Random Access Memory, RAM) or may further comprise Non-volatile Memory (NVM), such as at least one disk Memory.
The bus may be an industry standard architecture (Industry Standard Architecture, ISA) bus, an external device interconnect (Peripheral Component, PCI) bus, or an extended industry standard architecture (Extended Industry Standard Architecture, EISA) bus, among others. The buses may be divided into address buses, data buses, control buses, etc. For ease of illustration, the buses in the drawings of the present application are not limited to only one bus or one type of bus.
In some embodiments, a computer program product is also proposed, comprising a computer program or instructions which, when executed by a processor, implement the steps of any of the data operation management methods described above.
The specific implementation of each operation above may be referred to the previous embodiments, and will not be described herein.
Other embodiments of the present application will be apparent to those skilled in the art from consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of the application following, in general, the principles of the application and including such departures from the present disclosure as come within known or customary practice within the art to which the application pertains. It is intended that the specification and examples be considered as exemplary only, with a true scope and spirit of the application being indicated by the following claims.
Claims (10)
1. The data operation management method is characterized in that the method is applied to an operation management platform, and the operation management platform comprises: the system comprises a central node and a plurality of distributed nodes, wherein the central node respectively establishes a unidirectional synchronous channel with each distributed node; the method comprises the following steps:
the central node acquires a resource creation request, wherein the resource creation request comprises a site identifier, a resource identifier and a resource value;
the center node creates target data corresponding to the resource identifier and the resource value according to the resource creation request, carries the target data in a synchronous message, and sends the target data to a distributed node corresponding to the site identifier through a unidirectional synchronous channel corresponding to the distributed node corresponding to the site identifier;
the distributed node acquires the synchronous message through the unidirectional synchronous channel, and sends the target data in the synchronous message to a database in the distributed node according to the synchronous message so as to enable the database to synchronously process the target data, so that the distributed node acquires the target data corresponding to the distributed node in the process of acquiring the target data from the database and correspondingly processes the target data.
2. The method of claim 1, wherein the synchronization message further comprises: transaction identification, and synchronization actions; wherein the synchronization action includes adding, modifying and/or deleting;
said sending said target data in said synchronization message to a database in said distributed node according to said synchronization message, comprising:
and adding the target data to a database in the distributed node according to the synchronous action in the synchronous message, and/or modifying or deleting the target data in the database in the distributed node.
3. The method as recited in claim 1, further comprising:
the distributed node acquires a storage response fed back by the database, and generates a feedback message when the storage response is a storage failure response;
and the distributed node sends the feedback message to the central node so that the central node can send the target data to the distributed node again through the unidirectional synchronous channel according to the feedback message.
4. The method of claim 3, wherein the distributed node obtains a store response of the database feedback and generates a feedback message when the store response is a store failure response, comprising:
the distributed node acquires a storage response fed back by the database, and acquires a corresponding synchronization method, synchronization time, synchronization state, operator identification, operation time and synchronization dependence when the database performs synchronization processing on the target data when the storage response is a storage failure response; the synchronization method characterizes a method for the distributed node to add, modify and delete the target data, and the synchronization state comprises success or failure; the synchronization dependencies characterize relationships between the target data;
and the distributed node generates the feedback message according to the synchronization method, the synchronization time, the synchronization state, the operator identification, the operation time and the synchronization dependence.
5. A method according to claim 3, wherein the central node resends the target data to the distributed nodes via the unidirectional synchronization channel in accordance with the feedback message, comprising:
if the central node determines that the synchronization state in the feedback message is failed, determining re-synchronization target data based on the feedback message, and starting timing so as to send the re-synchronization target data to the distributed nodes through the unidirectional synchronization channel when the timing time reaches the preset synchronization time;
or,
and if the central node determines that the synchronization state in the feedback message is failure, determining re-synchronization target data based on the feedback message, and sending the re-synchronization target data to the distributed nodes through the unidirectional synchronization channel when a synchronization request is received.
6. The method according to any one of claims 1 to 5, wherein the central node obtaining a resource creation request comprises:
the central node receives a resource creation request sent by a user terminal;
or,
and the central node receives a resource creation request uploaded by the distributed node, wherein the resource creation request is a request sent to the distributed node by a user terminal.
7. The method according to any one of claims 1 to 5, wherein the resource identification comprises: user resource identification, account resource identification, and/or model resource identification.
8. A data operation management apparatus, comprising:
the acquisition module is used for acquiring a resource creation request by the central node, wherein the resource creation request comprises a site identifier, a resource identifier and a resource value;
the synchronization module is used for creating target data corresponding to the resource identifier and the resource value according to the resource creation request by the central node, carrying the target data in the synchronization message, and sending the target data to the distributed node corresponding to the site identifier through a unidirectional synchronization channel corresponding to the distributed node corresponding to the site identifier;
the processing module is used for the distributed node to acquire the synchronous message through the unidirectional synchronous channel, and according to the synchronous message, the target data in the synchronous message is sent to a database in the distributed node so that the database can synchronously process the target data, and therefore the distributed node can acquire the target data corresponding to the distributed node from the database to perform corresponding processing.
9. An electronic device, comprising: a processor, and a memory communicatively coupled to the processor;
the memory stores computer-executable instructions;
the processor executes computer-executable instructions stored in the memory to implement the method of any one of claims 1 to 7.
10. A computer readable storage medium having stored therein computer executable instructions which when executed by a processor are adapted to carry out the method of any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311498489.5A CN117520449A (en) | 2023-11-10 | 2023-11-10 | Data operation management method, device, equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311498489.5A CN117520449A (en) | 2023-11-10 | 2023-11-10 | Data operation management method, device, equipment and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN117520449A true CN117520449A (en) | 2024-02-06 |
Family
ID=89758072
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202311498489.5A Pending CN117520449A (en) | 2023-11-10 | 2023-11-10 | Data operation management method, device, equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117520449A (en) |
-
2023
- 2023-11-10 CN CN202311498489.5A patent/CN117520449A/en active Pending
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108683668B (en) | Resource checking method, device, storage medium and equipment in content distribution network | |
| CN110046062B (en) | Distributed data processing method and system | |
| CN112036125B (en) | Document management method and device and computer equipment | |
| CN113094430B (en) | Data processing method, device, equipment and storage medium | |
| CN108228581B (en) | Zookeeper compatible communication method, server and system | |
| CN113010549A (en) | Data processing method based on remote multi-active system, related equipment and storage medium | |
| EP3974960B1 (en) | Method and system for automated testing of web service apis | |
| CN111399993A (en) | Cross-chain implementation method, device, equipment and medium for associated transaction request | |
| CN112035062B (en) | Migration method of local storage of cloud computing, computer equipment and storage medium | |
| CN115221156A (en) | Database cluster capacity expansion method and device, computer equipment and storage medium | |
| CN111327680B (en) | Authentication data synchronization method, device, system, computer equipment and storage medium | |
| CN107819729B (en) | Data request method and system, access device, storage device and storage medium | |
| CN112866406A (en) | Data storage method, system, device, equipment and storage medium | |
| CN117520449A (en) | Data operation management method, device, equipment and storage medium | |
| CN109389271B (en) | Application performance management method and system | |
| CN110609707A (en) | Online data processing system generation method, device and equipment | |
| CN115238006A (en) | Retrieval data synchronization method, device, equipment and computer storage medium | |
| CN111586438B (en) | Method, device and system for processing service data | |
| CN117010889A (en) | Data processing method, device, equipment, medium and product | |
| CN107704557B (en) | Processing method and device for operating mutually exclusive data, computer equipment and storage medium | |
| CN115129779A (en) | Database synchronization method, device and readable medium | |
| CN112463879B (en) | Data volume synchronization method and equipment | |
| CN113379542B (en) | Block chain transaction query method, device, medium and electronic equipment | |
| CN117171419B (en) | Content processing method, device, electronic apparatus, storage medium, and program product | |
| CN116954474A (en) | Method, apparatus, device, storage medium and program product for data reading |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |