CN113448647B - Resource synchronization method, implementation equipment and electronic equipment - Google Patents

Resource synchronization method, implementation equipment and electronic equipment Download PDF

Info

Publication number
CN113448647B
CN113448647B CN202110717257.9A CN202110717257A CN113448647B CN 113448647 B CN113448647 B CN 113448647B CN 202110717257 A CN202110717257 A CN 202110717257A CN 113448647 B CN113448647 B CN 113448647B
Authority
CN
China
Prior art keywords
local
plug
opposite
resource
cursor
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202110717257.9A
Other languages
Chinese (zh)
Other versions
CN113448647A (en
Inventor
徐智成
刘飞
杨波
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Huayun Data Holding Group Co ltd
Original Assignee
Huayun Data Holding Group Co ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Huayun Data Holding Group Co ltd filed Critical Huayun Data Holding Group Co ltd
Priority to CN202110717257.9A priority Critical patent/CN113448647B/en
Publication of CN113448647A publication Critical patent/CN113448647A/en
Application granted granted Critical
Publication of CN113448647B publication Critical patent/CN113448647B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00Arrangements for program control, e.g. control units
    • G06F9/06Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/44Arrangements for executing specific programs
    • G06F9/445Program loading or initiating
    • G06F9/44505Configuring for program initiating, e.g. using registry, configuration files
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00Arrangements for program control, e.g. control units
    • G06F9/06Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/46Multiprogramming arrangements
    • G06F9/50Allocation of resources, e.g. of the central processing unit [CPU]
    • G06F9/5005Allocation of resources, e.g. of the central processing unit [CPU] to service a request
    • G06F9/5027Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resource being a machine, e.g. CPUs, Servers, Terminals
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00Network arrangements or protocols for supporting network services or applications
    • H04L67/01Protocols
    • H04L67/10Protocols in which an application is distributed across nodes in the network
    • H04L67/1095Replication or mirroring of data, e.g. scheduling or transport for data synchronisation between network nodes

Landscapes

  • Engineering & Computer Science (AREA)
  • Software Systems (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Signal Processing (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)

Abstract

The invention provides a resource synchronization method, an implementation device and an electronic device, wherein the resource synchronization method is used for executing synchronization on resources respectively provided by a local device and an opposite terminal device which are logically isolated, and comprises the steps of starting a plug-in unit arranged between the local device and the terminal device, wherein the plug-in unit is controlled by the local device and pulls an opposite terminal cursor formed by the opposite terminal device due to resource fluctuation through the plug-in unit; the plug-in pulls a local cursor formed by the current resource of the local device, compares the local cursor with the opposite-end cursor in consistency, and only when the opposite-end cursor forms an increment event relative to the local cursor, performs data filling on the increment event to determine increment data. The method for incremental synchronization of the incremental data based on resource variation between the local equipment and the opposite terminal equipment reduces the longer resource synchronization process executed by the local equipment and the opposite terminal equipment, overcomes the defect of overlarge performance cost in the synchronization process, and realizes the strong consistency of the resources between the local equipment and the opposite terminal equipment.

Description

Resource synchronization method, implementation equipment and electronic equipment
Technical Field
The present invention relates to the field of computer technologies, and in particular, to a resource synchronization method, a device for implementing resource synchronization, and an electronic device.
Background
The resource is used as an object to be synchronized between devices, and the interaction and the access between the client and the database are required to be frequently performed between the client and the server so as to ensure the consistency of the resource between the devices. In the prior art, a resource synchronization means (hereinafter referred to as "periodic full-scale synchronization") combining periodic synchronization and full-scale synchronization is generally adopted. However, when the resource synchronization is performed on the local device and the opposite terminal device by adopting a periodic full-volume synchronization mode, the local device can only communicate with the opposite terminal device in a fixed window period (namely, an operation time period adopting full-volume synchronization) so as to synchronize the resource, so that the problem of untimely resource synchronization still exists in a non-window period;
meanwhile, in the prior art, resource expense of the local equipment opposite terminal equipment is overlarge by adopting a resource synchronization means of periodic full synchronization, so that the efficiency of executing resource synchronization among the equipment is lower and the resource consistency among the equipment is poorer; in addition, when a cloud platform or a computer cluster logically performs operations on a bottom-layer device (for example, a database) and a top-layer device (for example, a client) logically located on the top layer at the same time, a resource (for example, a data disk or a system disk of a virtual machine) is also easy to generate inconsistency between the resources respectively located between the bottom-layer device and the bottom-layer device.
Finally, during the period of performing the periodic full-quantity synchronization, if the top device performs operations such as deletion, copy, migration, etc. on a certain resource from the bottom device, the method for synchronizing resources in the prior art may cause a defect that a resource difference exists between the local device and the peer device after the periodic full-quantity synchronization is performed.
In view of this, there is a need for an improvement in the resource synchronization method in the prior art to solve the above-mentioned problems.
Disclosure of Invention
The invention aims to disclose a resource synchronization method, a resource synchronization realization device and an electronic device, which are used for solving the defects of long time consumption, overlarge performance cost and inconsistent resources of a local device and an opposite terminal device in the synchronization process by adopting a timed full-quantity synchronization resource synchronization method in the prior art so as to realize strong consistency of the resources of the local device and the opposite terminal device respectively.
To achieve one of the above objects, the present invention provides a resource synchronization method,
synchronization is performed on resources that logically isolated local devices and peer devices have separately,
comprising the following steps:
starting a plug-in unit arranged between the local equipment and the opposite terminal equipment, wherein the plug-in unit is controlled by the local equipment and pulls an opposite terminal cursor formed by the opposite terminal equipment due to resource variation through the plug-in unit;
The plug-in pulls a local cursor formed by the current resource of the local device, compares the local cursor with the opposite-end cursor in consistency, and only when the opposite-end cursor forms an increment event relative to the local cursor, performs data filling on the increment event to determine increment data.
As a further improvement of the present invention, the resources include one or any combination of several of computing resources, network resources, storage resources, configuration files, script files, data or messages; the local equipment and the opposite terminal equipment are logically isolated physical servers, virtual servers, nodes or software in a computer cluster.
As a further improvement of the invention, the plug-in circularly compares the consistency of the local cursor with that of the opposite-end cursor in a set circulation period, and sets a global clock for the plug-in so as to sort the increment events through the global clock;
wherein the cycle period is a floating period.
As a further improvement of the present invention, the cycle period is configured periodically to the plug-in by a third party entity controlling the peer device;
the third party body is logically independent of the local equipment, the opposite terminal equipment and the plug-in.
As a further improvement of the present invention, after the data filling is performed on the incremental event, the method further includes:
and loading the resources corresponding to the incremental data to the local equipment, and storing the incremental data formed by executing the data filling to a local equipment database.
As a further improvement of the present invention, after performing data population on the incremental event to determine the incremental data, further comprising:
performing arbitration on the incremental data using the plugin to determine the authenticity of the incremental data;
and performing format conversion processing on the incremental data by using a plug-in to convert the incremental data into format-converted incremental data which can be recognized by the local equipment, notifying the local equipment of the format-converted incremental data and finally storing the format-converted incremental data into a local equipment database, guiding the plug-in to enter a sleep period, circularly performing consistency comparison on the local vernier and the opposite vernier by the plug-in after the sleep period passes, and performing data filling on the incremental event to determine the incremental data only when the opposite vernier forms the incremental event relative to the local vernier.
As a further improvement of the invention, the opposite cursor formed by the plug-in pulling opposite equipment due to resource variation is specifically:
And the plug-in reversely pulls the opposite-end cursor formed by the opposite-end equipment due to resource variation according to the time axis.
As a further improvement of the present invention, after the plug-in is started, the plug-in further includes:
when the plug-in is first created, the plug-in pulls an initial opposite-end cursor formed by the opposite-end device, extracts a plurality of differences between the opposite-end cursor formed by resource variation and the initial opposite-end cursor formed by the opposite-end device based on a time axis as an increment event, and copies the resources of the opposite-end device to the local device in a full-scale synchronous mode.
As a further improvement of the invention, the consistency comparison between the local cursor and the opposite cursor is specifically:
the plug-in pulls the opposite-end cursor formed by the opposite-end equipment due to resource variation and forms an event queue of the opposite-end equipment, and the plug-in pulls the local cursor formed by the current resource of the local equipment and forms a plug-in event queue; and executing a search operation on the local cursor in the current state in the event queue of the opposite terminal equipment to determine the logic position of the local cursor in the current state in the event queue of the opposite terminal equipment, and identifying that the local cursor is inconsistent with the opposite terminal cursor when the logic position of the local cursor in the current state in the event queue of the plug-in unit is deviated from the logic position of the local cursor in the current state in the event queue of the opposite terminal equipment.
As a further improvement of the present invention, the resource possessed by the local device is obtained from the opposite terminal device in one direction, and a resource synchronization event is formed between the local device and the opposite terminal device.
As a further improvement of the present invention, there is also included:
judging whether the opposite terminal cursor formed by the plug-in unit from the opposite terminal equipment due to resource variation is the latest opposite terminal equipment cursor;
if yes, executing cleaning processing on the plug-in event queue, and locking resources corresponding to the resource acquisition requests initiated by the local equipment to the opposite terminal equipment by the plug-in, wherein the opposite terminal equipment refuses to respond to the resource acquisition requests initiated by the local equipment to the opposite terminal equipment in the current cycle period;
if not, triggering the event of re-pulling the latest opposite terminal cursor formed by the opposite terminal equipment due to resource variation when the waiting cycle period is reached.
As a further improvement of the present invention, there is also included: and monitoring the resource change of the opposite terminal equipment by using the query module so as to respond to a request of pulling the opposite terminal cursor formed by the opposite terminal equipment due to the resource change in the cycle period.
Meanwhile, based on the same thought, the invention also discloses a device for realizing resource synchronization, which comprises:
at least one local device, at least one peer device, and at least one plug-in disposed between the logically isolated local device and the peer device;
The device for realizing the resource synchronization executes synchronization on the resources respectively possessed by the logically isolated local device and the opposite terminal device through the resource synchronization method created by any one of the above inventions.
As a further improvement of the invention, the plug-in is deployed and operated in a database, a memory or a virtual machine of the opposite terminal equipment, or is deployed and operated in a database, a memory or a virtual machine which is logically independent from the local equipment and the opposite terminal equipment.
Finally, based on the same inventive concept, the invention also discloses an electronic device, comprising:
a processor, a memory device composed of at least one memory unit, and
a communication bus establishing a communication connection between the processor and the memory device;
the processor is configured to execute one or more programs stored in the storage device to implement the resource synchronization method according to any of the above inventions.
Compared with the prior art, the invention has the beneficial effects that:
firstly, in the application, a local cursor formed by the current resource of the local equipment is pulled by a plug-in a circulation period, and consistency comparison is carried out on the local cursor and an opposite-end cursor, so that real-time detection of incremental data generated by resource variation between the local equipment and the opposite-end equipment is realized, real-time synchronization of the resources between the local equipment and the opposite-end equipment is realized, and the defects of long time consumption, overlarge performance cost and inconsistent resources of the local equipment and the opposite-end equipment in a synchronization process by adopting a timing full-volume synchronization method in the prior art are overcome, so that strong consistency of the resources of the local equipment and the opposite-end equipment is realized;
Meanwhile, in the method, based on the judgment of whether the opposite-end cursor formed by the opposite-end equipment due to resource variation is the latest opposite-end equipment cursor, the possibility of inconsistent resources between the local equipment and the opposite-end equipment caused by the fact that the local equipment calls resources from the opposite-end equipment forcedly in the same cycle period is effectively avoided, and the reliability and the operation fineness of resource synchronization are further ensured.
Drawings
FIG. 1 is an overall flow chart of a resource synchronization method of the present invention;
FIG. 2 is a timing diagram of a resource synchronization method according to the present invention when performing resource synchronization between a peer device and a local device;
FIG. 3 is a schematic diagram of a controller and a scheduler for scheduling resources in a peer device, when the resources in the peer device change, by monitoring AMQP (Advanced Message Queuing Protocol, advanced message queue protocol) to determine whether the resources of the peer device change;
FIG. 4 is a topology diagram of an apparatus for implementing resource synchronization according to the present invention;
FIG. 5 is a detailed flow chart of a plug-in pulling a local cursor formed by the current resource of the local device, and performing consistency comparison between the local cursor and an opposite-end cursor;
FIG. 6 is a schematic diagram showing a plug-in pulling a peer cursor formed by a peer device due to resource variation and forming a peer device event queue, and a plug-in pulling a local cursor formed by a current resource of a local device and forming a plug-in event queue, where the local cursor in a current state is configured to identify that the local cursor is inconsistent with the peer cursor when a logic position in the peer device event queue is shifted;
FIG. 7 is a logic diagram of an implementation device for resource synchronization of the present invention shown in FIG. 4;
FIG. 8 is a schematic diagram of an event triggering pulling of a latest peer cursor formed by a peer device due to resource variation when a cycle period for waiting for a local cursor to perform consistency comparison with the peer cursor is reached in an example in which the peer cursor formed by a plug-in from the peer device due to resource variation is not the latest peer device cursor;
FIG. 9 is a schematic diagram of performing a cleaning process on a plug-in event queue and locking the resources of the opposite device included in the plug-in event queue after the cleaning process, in an example where the opposite device cursor formed by the plug-in device due to resource variation is the latest opposite device cursor;
FIG. 10 is a topology diagram of a device for implementing resource synchronization according to a modification of the present invention;
Fig. 11 is a topology of an electronic device according to the present invention.
Detailed Description
The present invention will be described in detail below with reference to the embodiments shown in the drawings, but it should be understood that the embodiments are not limited to the present invention, and functional, method, or structural equivalents and alternatives according to the embodiments are within the scope of protection of the present invention by those skilled in the art.
Before explaining the various embodiments of the invention in detail, the meanings of the main technical terms in the various embodiments are necessarily explained.
The term'Local equipment"and term"Opposite terminal equipment"is just a relative concept and any carrier of computer-executable code can be executed. For example, a local device may be considered a data center and a peer device may be considered a virtual device; for another example, the local device may be considered a physical server and the peer device a software (e.g., APP); also for example, both the local device and the peer device can be considered as onePlanting software; alternatively, the local device is considered as a cloud management platform, and the peer device is considered as a virtual server.
The term'Logic isolation"refers to the absence of an execution hierarchy of computer code between two devices/systems. However, the logically isolated local device and the opposite device may establish a data transmission/reception relationship based on a specific call interface (API), a protocol, or the like.
Embodiment one:
referring to fig. 1 and fig. 2, this embodiment discloses a resource synchronization method, which performs synchronization on resources respectively owned by a logically isolated local device and a peer device. The resources comprise one or any combination of a plurality of computing resources, network resources, storage resources, configuration files, script files, data or messages; the local device 22 is a logically isolated physical server, virtual server, node in a computer cluster, or software from the peer device 24. In the context of the present embodiment, the local device 22 may be considered a producer and the peer device 24 may be considered a consumer. Events such as access requests initiated by the local device 22 (e.g., making calls to data in the peer device 24, creating virtual machines, etc.) are handled by the peer device 24 in response to access requests initiated by the local device 22. Alternatively, the user (or administrator) synchronizes the change in data or resources in the local device 22 and/or the peer device 24, respectively, via a unified access interface (API).
The resource synchronization method comprises the steps S1 to S2.
Firstly, step S1 is executed to start a plug-in unit 23 (plug in) disposed between the local device 22 and the peer device 24, where the plug-in unit 23 is controlled by the local device 22 and pulls the peer cursor formed by the peer device 24 due to resource variation through the plug-in unit 23.
In this embodiment, the query module 230 is used to monitor the resource variation of the peer device 24, so as to respond to the request of the plug-in 23 to pull the peer cursor formed by the peer device 24 due to the resource variation in the cycle period. The plug-in 23 may be disposed on the opposite device 24 (but logically independent of devices_1-i and device_1-j), or the plug-in 23 may be directly distinguished from the opposite device 24 as a separate component and interact with the respective devices constituting the opposite device 24 through the adapted query module 230. Referring to fig. 4, devices_1 to device_i, devices_1 to device_j may be virtualization platforms, and plug-in 23 may complete subsequent resource synchronization operations by reading events generated by devices_1 to device_i and/or devices_1 to device_j based on resource changes. Both the parameter i and the parameter j take positive integers which are larger than or equal to 1. The task console 231 listens for events generated by device_1 to device_i based on resource changes, or listens for events generated by device_1 to device_j based on resource changes through the message middleware 232. Specifically, for devices such as vSphere, the plug-in 23 executes, via the task console 231, peer cursors formed by resource changes to one or more peer devices 24; for an OpenStack or xenerver virtualization platform, such as open source based, plug-in 23 executes, via message middleware 232, peer cursors formed by resource changes to one or more peer devices 24. Plug-in 23 may configure mutually adapted plug-ins according to different types of peer devices 24, such as the peer devices defined by the vSphere device or the peer devices defined by the xenerver virtualization platform. As shown in connection with fig. 7, plug-in 23 may be a vSphere plug-in 233 to interact with the vSphere353 as a peer device on the character; alternatively, plug-in 23 may also be an OpenStack plug-in 234 to interact with OpenStack354 that is a peer device in the role, and plug-in 23 may also be other peer device plug-ins 235 to interact with other peer devices 355 that are peer devices in the role. Meanwhile, one or more plug-ins may be independently configured between the local device 22 and the peer device 24. The plug-in event queues formed by the vSphere plug-in 233, the OpenStack plug-in 234, and the other peer device plug-ins 235, respectively, form an event queue pool.
Further, as shown in fig. 3, for a cloud platform formed by an OpenStack or xenerver virtualization platform, storage nodes, network nodes and computing nodes are typically deployed, where the storage nodes are managed by a Scheduler 14 through a data volume controller 10, the network nodes are managed by the Scheduler 14 through a network controller 11, and the computing nodes are managed by the Scheduler 14 (Scheduler) through a computing controller 12. The event of resource change of any one of the storage node, the network node and the computing node can be perceived by the plugin 23 through the mode of monitoring the AMQP between the data volume controller 10, the network controller 11 and the computing controller 12 and the scheduler 14 respectively, and monitored in real time.
In a set cycle period (refer to Loop200 in fig. 2), after the plugin 23 pulls the local cursor and the opposite cursor at the same moment from the local device 22 and the opposite device 24 respectively, the plugin 23 circularly executes consistency comparison on the local cursor and the opposite cursor in the set cycle period, and sets a global clock for the plugin 23 so as to sequence incremental events through the global clock; wherein the cycle period is a floating period. The cycle period is configured to the plug-in 23 periodically by the third party body 50 controlling the peer device 24 or configured to the plug-in 23 directly by the user; wherein the third party body 50 is logically independent of the local device 22, the peer device 24 and the plug-in 23. The third party body 50 may be a database and the plug-in 23 is run in the third party body 50. The third party body 50 is logically isolated from the peer device 24 and the local device 22. By setting the global clock, the order of pulling the local cursor and the opposite cursor can be received by the local device 22 and the opposite device 24, and an accurate and unique time axis can be formed. Meanwhile, the possibility that each event formed by the local device 22 or the opposite terminal device 24 after receiving the access request of resource scheduling initiated by the user is in an unordered state based on the time sequence of the time axis can be omitted, so that the problem of higher time delay caused by the subsequent execution of the resource synchronization process is effectively avoided.
The local Cursor and the opposite Cursor are both cursors (cursors). The cursor functions to temporarily store data blocks extracted from a database or device (e.g., virtual machine data disk interacting with the data volume controller 10). In some cases, it is necessary to transfer the data from the table stored on the disk to the memory of the computer for processing, and finally display the processing results or finally write the processing results back to the database or the device. Meanwhile, the floating time period of the circulation period is logically built in the plugin 23, so that flexible conditions can be provided according to the actual requirements of the resource specification or the resource synchronization frequency of the local equipment 22 and/or the opposite-end equipment 24, and the circulation period can be directly issued or modified in the plugin 23 by a user or an administrator in a mode of remotely accessing or locally accessing a third party main body (such as a database) running the plugin 23, thereby realizing more flexible resource synchronization strategy between the local equipment 22 and the opposite-end equipment 24 through the plugin 23 and improving the timeliness of resource synchronization.
Then, step S2 is executed, and the plug-in 23 pulls the local cursor formed by the current resource of the local device 22, and performs consistency comparison between the local cursor and the opposite-end cursor, and performs data filling on the increment event only when the opposite-end cursor forms the increment event with respect to the local cursor, so as to determine the increment data. After performing the data population on the incremental event, the method further comprises: the local device 22 is loaded with the resources corresponding to the incremental data, and the incremental data formed by the execution of the data population is notified to the local device 22 and finally saved to the local device database 21, so that the synchronization operation is completed when the resources between the local device 22 and the peer device 24 are changed. The process of performing consistency comparison between the local cursor and the opposite cursor by the plugin 23 essentially belongs to the difference between events respectively possessed by the local device 22 and the opposite device 24 at a specific time point, and is based on the opposite device event queue and the plugin event queue in fig. 6, so that the local device change event formed by the local device 22 in fig. 8 and fig. 9 due to resource change occurs is read by the plugin 23 and compared with the same time point in the opposite device event list, so as to determine whether the resources between the local device 22 and the opposite device 24 have consistency or not according to whether the positions of the plugin event queue formed by the plugin 23 on the time axis are the same. The opposite cursor formed by the plug-in 23 pulling the opposite device 24 due to resource variation is specifically: the plug-in 23 pulls the opposite cursor formed by the opposite device 24 due to the resource variation in the reverse direction according to the time axis. Referring to fig. 6, the arrow to the left represents a time axis on which a plurality of events are formed based on a cycle period. For example, the event B in the event queue of the opposite device is changed to the event a (the latest event), the event corresponding to the opposite cursor pulled up by the opposite device 24 at the cursor position of the plug-in event queue is the event C, and the event C is an event already formed in the event queue of the opposite device along the time axis and at a certain historical time of the current time axis, and then the logical position of the event C contained in the event queue of the opposite device can be found according to the event C in the event queue of the plug-in.
It should be noted that, the plug-in 23 is equivalent in role to an arbitrator and collects local cursors and peer cursors formed by the local device 22 and the peer device 24 respectively, and at the same time, orders the local cursors and the peer cursors based on the order of occurrence of the time axis. The plugin 23 pulls the peer cursor formed by the peer device due to the resource variation reversely along the time axis from the same cursor position (for example, event C in the peer device event queue and event C in the plugin event queue in fig. 6). In fig. 6, the event list of the peer device includes an event a, an event B, an event C, an existing event D, an existing event E, and the like. The local device 22 and the opposite device 24 always have the same resource, so that the cursor position of the plug-in unit moves by the same step width along the extending direction of the time axis. If the resources of the local device 22 and the counterpart device 24 change, the cursor position of the plug-in is moved by a different step in the extending direction of the time axis. Of course, in the first and only full synchronization, the cursor position of the current event in the peer device 24 points to the logical position of event a (the most recent) then the peer device event queue is formed relative to the plug-in event queue such that event a and event B are new events relative to the local device 22, corresponding to the local device 22, and form incremental data, and need to be subjected to resource synchronization operations. In general, the data corresponding to the incremental event formed by the peer device 24 based on the resource variation is used as trusted reference data, so that the data corresponding to the incremental event formed by the resource variation in the local device is compared with the reference data, and the resource synchronization is performed only for the incremental data between the local device 22 and the peer device 24 in a manner of performing the incremental synchronization, so as to reduce the resource consumption and the synchronization event of the resource synchronization.
The local cursor and the peer cursor may respectively characterize the resource status of the local device 22 and the peer device 24 at a specific point in time. In the present embodiment, the operational decision of whether to perform resource synchronization between the local device 22 and the one or more peer devices 24 is determined by determining whether the local cursor is consistent with the peer cursor. After performing data population on the incremental event to determine incremental data, further comprising: arbitration is performed on the delta data using plug-in 23 to determine the authenticity of the delta data. Thereby effectively preventing false delta data from being wrongly written into the local device database 21.
As shown in fig. 2 and fig. 5 in combination, in this embodiment, after the plug-in 23 is started, the method further includes: the plug-in 23 pulls the initial peer-to-peer cursor formed by the peer-to-peer device 24 when the plug-in 23 is first created, extracts the difference between the peer-to-peer device 24 and the initial peer-to-peer cursor formed by a plurality of peer-to-peer cursors due to resource variation formed by the peer-to-peer device 24 based on the time axis as an incremental event, copies the resources of the peer-to-peer device 24 to the local device 22 in a full-scale synchronous manner, and finally copies the resources to the local device database 21 in a full-scale synchronous manner. After full synchronization, a Loop cycle Loop200 is performed. After performing the aforementioned consistency comparison, the method further comprises: the plug-in 23 is used to perform format conversion processing on the incremental data to convert the format-converted incremental data recognizable by the local device 22, and to notify the local device of the format-converted incremental data and finally save it to the local device database 21, and to guide the plug-in 23 into a sleep period, and to perform consistency comparison of the local cursor and the opposite cursor cyclically by the plug-in after the sleep period has elapsed, and to perform data population on the incremental event to determine the incremental data only when the opposite cursor forms an incremental event with respect to the local cursor. After the sleep period has elapsed, the arrow in fig. 2 is again cyclically executed by the plug-in 23 in the set cycle period: time record of the opposite device is read to arrow in time sequence: the respective execution steps of updating the delta data. The setting of the sleep period may be the processing time of the resource execution schedule of the local device or the peer device, which indirectly improves the reliability and stability of the resource synchronization method in this embodiment for the resource synchronization process between the local device and the peer device.
In the present embodiment, the plug-in 23 reads event records of the peer devices in time order (i.e., reverse order of the time axis) to determine a plurality of peer cursors at each time recording point (located on the time axis), and consistency comparison is performed by the plug-in 23. Referring to fig. 5, the consistency comparison between the local cursor and the opposite cursor is specifically:
step 111, pull the local cursor and the opposite cursor.
Step 112, judging whether the local cursor is consistent with the opposite-end cursor; if yes, the direct jump is ended; if not, go to step 113. Specifically, in this embodiment, the determining logic in step 112 specifically includes: the plug-in 23 pulls the opposite cursor formed by the opposite device due to resource variation and forms an event queue of the opposite device, and the plug-in pulls the local cursor formed by the current resource of the local device 22 and forms a plug-in event queue.
Step 113, performing a lookup operation on the local cursor in the current state in the event queue of the opposite device, so as to determine a logical position of the local cursor in the current state in the event queue of the opposite device.
Step 114, pull the event of the latest peer cursor that the peer device has traveled due to the resource change. And when the logic position of the local cursor in the current state in the plug-in event queue is deviated from the logic position of the local cursor in the current state in the event queue of the opposite terminal equipment, identifying that the local cursor is inconsistent with the opposite terminal cursor. Therefore, the event corresponding to the inconsistent cursor formed by the local cursor and the opposite-end cursor is regarded as an increment event.
Step 115, performing data population on the delta event to determine delta data.
Step 116, notify the local device 22 and update the resources in the local device 22.
In this embodiment, the resources of the local device 22 are obtained from the peer device 24 in one direction, and the resource synchronization event formed between the local device 22 and the peer device 24 ends.
Compared with the resource synchronization method adopting the periodic full-quantity synchronization in the prior art, the resource synchronization method disclosed in the embodiment greatly reduces the calculation cost of the comparison process generated in the synchronization process based on the resource variation, solves the defects of long time consumption and overlarge performance cost of the synchronization process and inconsistent resources of the local equipment 22 and the opposite equipment 24 in the resource synchronization method adopting the periodic full-quantity synchronization in the prior art, and realizes the strong consistency of the resources of the local equipment 22 and the opposite equipment 24 respectively.
In this embodiment, the resource synchronization method further includes:
it is determined whether the opposite-end cursor formed by the plug-in 23 from the opposite-end device 24 due to the resource variation is the latest opposite-end device cursor.
If (in combination with scenario two shown in fig. 9), the cleaning process is performed on the plug-in event queue, where the plug-in 23 locks the resource corresponding to the resource acquisition request initiated by the local device 22 to the peer device 24, and the peer device 24 refuses to respond to the resource acquisition request initiated by the local device 22 to the peer device 24 in the current cycle period. Thus, the plug-in 23 will reject the request initiated by the local device 22 for the resource change. For example, if the local device 22 and the peer device 24 are synchronized for the first time and only once, if a certain resource in the peer device 24 has been called or occupied, then at this time, the local device 22 may be considered as invalid if the call is performed again by the plug-in 23 to the called or occupied resource to perform the synchronization operation of the resource. Accordingly, an event of an access request initiated by a user through the local device 22 to a resource that has been invoked or occupied will be identified as an invalid event, thereby avoiding the local device 22 from maliciously performing a resource synchronization operation, so that the synchronization operation and access request between the local device 22 and the peer device 24 with respect to the resource are more transparent. In fig. 9, a local device change event A1 generated by a local device 22 due to resource fluctuation does not belong to the same event as an event a in the event queue of the peer device. If not (in connection with scenario one of fig. 8), the event of restarting the peer device 2424 to form the latest peer cursor due to the resource change is triggered when the waiting cycle is reached. In fig. 8, a local device change event A1 generated by a local device 22 due to resource fluctuation and an event a in the event queue of the peer device belong to the same event. Therefore, the method realizes that the access request initiated by the user to the local equipment 22 cannot be effectively responded and effectively fed back in the same time point of the time axis due to the fact that the resources of the local equipment and the opposite equipment are not synchronous in the process of changing the resources respectively belonging to the local equipment 22 and the opposite equipment 24, and reduces the interactive process of synchronizing the resources of the local equipment, the plug-in unit and the opposite equipment.
By the resource synchronization method disclosed by the embodiment, the synchronization time between the local equipment 22 and the opposite terminal equipment 24 in the process of executing resource synchronization is reduced, the defect of overlarge performance expenditure in the synchronization process is overcome, and the strong consistency of resources between the local equipment 22 and the opposite terminal equipment 24 is realized.
Embodiment two:
referring to fig. 2, fig. 4, fig. 7, and fig. 10, this embodiment further discloses a device for implementing resource synchronization, including: at least one local device 22, at least one peer device 24, and at least one plug-in 23 disposed between the logically isolated local device 22 and the peer device 24. The implementation device of resource synchronization performs synchronization on resources respectively possessed by the logically isolated local device 22 and the peer device 24 by the resource synchronization method described in the first embodiment, and such resource synchronization is incremental synchronization.
Plug-in 23 is deployed and runs in a database, memory, or virtual machine of the peer device 24, or in a database, memory, or virtual machine logically independent of the local device 22 and the peer device 24. Although only one local device 22 is shown in fig. 7, one skilled in the art can reasonably envision that the number of local devices 22 interacting with the various plug-ins (e.g., the vSphere plug-in 233, the OpenStack plug-in 234, or other peer-device plug-ins 235 disclosed in embodiment one) can be one, or two, three, or even more.
Referring to fig. 2, fig. 4, and fig. 7, the plug-in 23 uses the query module 230 to monitor the resource variation of the peer device 24, so as to respond to the request of the plug-in 23 to pull the peer cursor formed by the peer device 24 due to the resource variation in the cycle period. The plug-in 23 may be disposed on the opposite device 24 (but logically independent of devices_1-i and device_1-j), or the plug-in 23 may be directly distinguished from the opposite device 24 as a separate component and interact with the respective devices constituting the opposite device 24 through the adapted query module 230. Referring to fig. 4, devices_1 to device_i, devices_1 to device_j may be virtualization platforms, and plug-in 23 may complete subsequent resource synchronization operations by reading events generated by devices_1 to device_i and/or devices_1 to device_j based on resource changes. Both the parameter i and the parameter j take positive integers which are larger than or equal to 1. The task console 231 listens for events generated by device_1 to device_i based on resource changes, or listens for events generated by device_1 to device_j based on resource changes through the message middleware 232. Specifically, for devices such as vSphere, the plug-in 23 executes, via the task console 231, peer cursors formed by resource changes to one or more peer devices 24; for an OpenStack or xenerver virtualization platform, such as open source based, plug-in 23 executes, via message middleware 232, peer cursors formed by resource changes to one or more peer devices 24. Plug-in 23 may configure mutually adapted plug-ins according to different types of peer devices 24, such as the peer devices defined by the vSphere device or the peer devices defined by the xenerver virtualization platform. As shown in connection with fig. 7, plug-in 23 may be a vSphere plug-in 233 to interact with the vSphere353 as a peer device on the character; alternatively, plug-in 23 may also be an OpenStack plug-in 234 to interact with OpenStack354 that is a peer device in the role, and plug-in 23 may also be other peer device plug-ins 235 to interact with other peer devices 355 that are peer devices in the role. Meanwhile, one or more plug-ins may be independently configured between the local device 22 and the peer device 24. The plug-in event queues formed by the vSphere plug-in 233, the OpenStack plug-in 234, and the other peer device plug-ins 235, respectively, form an event queue pool.
As a reasonable modification, as shown in fig. 2 and 10, the cycle period is configured to the plug-in 23 periodically by the third party body 50 of the control-end device 24 or configured to the plug-in 23 directly by the user; wherein the third party body 50 is logically independent of the local device 22, the peer device 24 and the plug-in 23. The third party body 50 may be a database and the plug-in 23 is run in the third party body 50. The third party body 50 is logically isolated from the peer device 24 and the local device 22. The third party entity 50 in fig. 10 may also be considered a computer system/device capable of creating and running plug-ins 23 and controlled by a user who performs the following operations to plug-ins 23 by way of UI interfaces or script configuration: creating plug-ins, modifying plug-ins, deleting plug-ins, configuring loop cycles, and the like.
The implementation device for resource synchronization disclosed in this embodiment has the same part as that in the first embodiment, and is shown in the first embodiment, and will not be described herein.
Embodiment III:
a specific implementation of an electronic device 500 is shown with reference to the present embodiment shown in fig. 11.
In this embodiment, the electronic device 500 includes: a processor 51, a memory device 52 consisting of at least one memory unit, and a communication bus 53 establishing a communication connection between the processor 51 and the memory device 52. The processor 51 is configured to execute one or more programs stored in the storage 52 to implement the specific implementation of the resource synchronization method as disclosed in the first embodiment.
Specifically, the storage device 52 may be composed of storage units 521 to 52k, where the parameter k is a positive integer greater than or equal to 1. The processor 51 may be ASIC, FPGA, CPU, MCU or other physical hardware or virtual devices with instruction processing functions. The form of the communication bus 53 is not particularly limited, I 2 C bus, SPI bus, SCI bus, PCI-E bus, ISA bus, etc., and can be reasonably used according to the specific type and application scenario requirements of the electronic device 500And (5) changing. The communication bus 53 is not the point of the present invention and is not described in detail in the present application.
The storage device 52 may be based on a distributed file system such as Ceph or glumerfs, or may be a RAID 0-RAID 7 Disk array, or may be configured as one or more hard disks or removable storage devices, a database server, an SSD (Solid-state Disk), a NAS storage system, or a SAN storage system. Specifically, in the present embodiment, the electronic device 500 may be configured as a computer, a server, a data center, a virtual cluster, a portable mobile terminal, a Web system, a financial payment platform or ERP system, a virtual online payment platform/system, and the like.
The integrated units, if implemented in the form of software functional units and sold or used as stand-alone products, may be stored in a computer readable storage medium. Based on such understanding, the technical solution of the present invention may be embodied in essence or a part contributing to the prior art or all or part of the technical solution in the form of a software product stored in a storage medium, including several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) or a processor (processor) to execute all or part of the steps of the method according to the embodiments of the present invention. The aforementioned storage medium includes: a U-disk, a removable hard disk, a Read-Only Memory (ROM), a random access Memory (RAM, random Access Memory), a magnetic disk, or an optical disk, or other various media capable of storing program codes.
The technical solutions of the electronic device disclosed in this embodiment and the first and/or second embodiments have the same parts, and are not described in detail herein with reference to the first and/or second embodiments.
The above list of detailed descriptions is only specific to practical embodiments of the present invention, and they are not intended to limit the scope of the present invention, and all equivalent embodiments or modifications that do not depart from the spirit of the present invention should be included in the scope of the present invention.
It will be evident to those skilled in the art that the invention is not limited to the details of the foregoing illustrative embodiments, and that the present invention may be embodied in other specific forms without departing from the spirit or essential characteristics thereof. The present embodiments are, therefore, to be considered in all respects as illustrative and not restrictive, the scope of the invention being indicated by the appended claims rather than by the foregoing description, and all changes which come within the meaning and range of equivalency of the claims are therefore intended to be embraced therein. Any reference sign in a claim should not be construed as limiting the claim concerned.
Furthermore, it should be understood that although the present disclosure describes embodiments, not every embodiment is provided with a separate embodiment, and that this description is provided for clarity only, and that the disclosure is not limited to the embodiments described in detail below, and that the embodiments described in the examples may be combined as appropriate to form other embodiments that will be apparent to those skilled in the art.

Claims (14)

1. A method for synchronizing a resource, which comprises the steps of,
synchronization is performed on resources that logically isolated local devices and peer devices have separately,
Characterized by comprising the following steps:
starting a plug-in unit arranged between the local equipment and the opposite terminal equipment, wherein the plug-in unit is controlled by the local equipment and pulls an opposite terminal cursor formed by the opposite terminal equipment due to resource variation through the plug-in unit;
the plug-in pulls a local cursor formed by the current resource of the local equipment, compares the local cursor with the opposite-end cursor in consistency, and only when the opposite-end cursor forms an increment event relative to the local cursor, performs data filling on the increment event to determine increment data;
the consistency comparison between the local cursor and the opposite-end cursor is specifically as follows:
the plug-in pulls the opposite-end cursor formed by the opposite-end equipment due to resource variation and forms an event queue of the opposite-end equipment, and the plug-in pulls the local cursor formed by the current resource of the local equipment and forms a plug-in event queue; and executing a search operation on the local cursor in the current state in the event queue of the opposite terminal equipment to determine the logic position of the local cursor in the current state in the event queue of the opposite terminal equipment, and identifying that the local cursor is inconsistent with the opposite terminal cursor when the logic position of the local cursor in the current state in the event queue of the plug-in unit is deviated from the logic position of the local cursor in the current state in the event queue of the opposite terminal equipment.
2. The resource synchronization method of claim 1, wherein the resources comprise one or any combination of several of computing resources, network resources, storage resources, configuration files, script files, data, or messages; the local equipment and the opposite terminal equipment are logically isolated physical servers, virtual servers, nodes or software in a computer cluster.
3. The resource synchronization method according to claim 1, wherein the plugin cyclically performs consistency comparison between a local cursor and an opposite cursor in a set cycle period, and sets a global clock for the plugin, so as to order the increment events by the global clock;
wherein the cycle period is a floating period.
4. A resource synchronization method according to claim 3, wherein the cycle period is configured to the plug-in periodically by a third party entity controlling the peer device;
the third party body is logically independent of the local equipment, the opposite terminal equipment and the plug-in.
5. The resource synchronization method of claim 3, wherein after the performing the data filling on the incremental event, further comprising:
And loading the resources corresponding to the incremental data to the local equipment, and storing the incremental data formed by executing the data filling to a local equipment database.
6. The resource synchronization method of claim 5, wherein after performing data population on the incremental event to determine the incremental data, further comprising:
performing arbitration on the incremental data using the plugin to determine the authenticity of the incremental data;
and performing format conversion processing on the incremental data by using a plug-in to convert the incremental data into format-converted incremental data which can be recognized by the local equipment, notifying the local equipment of the format-converted incremental data and finally storing the format-converted incremental data into a local equipment database, guiding the plug-in to enter a sleep period, circularly performing consistency comparison on the local vernier and the opposite vernier by the plug-in after the sleep period passes, and performing data filling on the incremental event to determine the incremental data only when the opposite vernier forms the incremental event relative to the local vernier.
7. The resource synchronization method according to claim 1, wherein the opposite cursor formed by the plug-in pulling opposite device due to resource variation is specifically:
and the plug-in reversely pulls the opposite-end cursor formed by the opposite-end equipment due to resource variation according to the time axis.
8. The resource synchronization method of claim 7, wherein after the plug-in is started, further comprising:
when the plug-in is first created, the plug-in pulls an initial opposite-end cursor formed by the opposite-end device, extracts a plurality of differences between the opposite-end cursor formed by resource variation and the initial opposite-end cursor formed by the opposite-end device based on a time axis as an increment event, and copies the resources of the opposite-end device to the local device in a full-scale synchronous mode.
9. The resource synchronization method according to claim 1, wherein the local device has resources obtained from the peer device in one direction, and the resource synchronization event is formed between the local device and the peer device.
10. The resource synchronization method of claim 9, further comprising:
judging whether the opposite terminal cursor formed by the plug-in unit from the opposite terminal equipment due to resource variation is the latest opposite terminal equipment cursor;
if yes, executing cleaning processing on the plug-in event queue, and locking resources corresponding to the resource acquisition requests initiated by the local equipment to the opposite terminal equipment by the plug-in, wherein the opposite terminal equipment refuses to respond to the resource acquisition requests initiated by the local equipment to the opposite terminal equipment in the current cycle period;
If not, triggering the event of re-pulling the latest opposite terminal cursor formed by the opposite terminal equipment due to resource variation when the waiting cycle period is reached.
11. The resource synchronization method of claim 10, further comprising: and monitoring the resource change of the opposite terminal equipment by using the query module so as to respond to a request of pulling the opposite terminal cursor formed by the opposite terminal equipment due to the resource change in the cycle period.
12. An implementation device for resource synchronization, comprising:
at least one local device, at least one peer device, and at least one plug-in disposed between the logically isolated local device and the peer device;
the device for implementing resource synchronization performs synchronization on resources respectively owned by the logically isolated local device and the peer device by using the resource synchronization method according to any one of claims 1 to 11.
13. The implementation device according to claim 12, wherein the plug-in is deployed and run in a database, a memory or a virtual machine of the peer device or in a database, a memory or a virtual machine logically independent of the local device and the peer device.
14. An electronic device, comprising:
A processor, a memory device composed of at least one memory unit, and
a communication bus establishing a communication connection between the processor and the memory device;
the processor is configured to execute one or more programs stored in the storage device to implement the resource synchronization method of any one of claims 1 to 11.
CN202110717257.9A 2021-06-28 2021-06-28 Resource synchronization method, implementation equipment and electronic equipment Active CN113448647B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202110717257.9A CN113448647B (en) 2021-06-28 2021-06-28 Resource synchronization method, implementation equipment and electronic equipment

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202110717257.9A CN113448647B (en) 2021-06-28 2021-06-28 Resource synchronization method, implementation equipment and electronic equipment

Publications (2)

Publication Number Publication Date
CN113448647A CN113448647A (en) 2021-09-28
CN113448647B true CN113448647B (en) 2024-02-09

Family

ID=77813119

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202110717257.9A Active CN113448647B (en) 2021-06-28 2021-06-28 Resource synchronization method, implementation equipment and electronic equipment

Country Status (1)

Country Link
CN (1) CN113448647B (en)

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116909759B (en) * 2023-09-12 2024-01-19 苏州浪潮智能科技有限公司 Information processing method, device and equipment, medium and controller cluster

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US9973374B1 (en) * 2013-11-07 2018-05-15 Cole Asher Ratias Systems and methods for synchronizing content and information on multiple computing devices
CN111309745A (en) * 2020-02-10 2020-06-19 腾讯科技(深圳)有限公司 Virtual resource processing method and device, electronic equipment and storage medium
CN112527899A (en) * 2020-12-03 2021-03-19 北京百度网讯科技有限公司 Data synchronization method, device, equipment and storage medium

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US9973374B1 (en) * 2013-11-07 2018-05-15 Cole Asher Ratias Systems and methods for synchronizing content and information on multiple computing devices
CN111309745A (en) * 2020-02-10 2020-06-19 腾讯科技(深圳)有限公司 Virtual resource processing method and device, electronic equipment and storage medium
CN112527899A (en) * 2020-12-03 2021-03-19 北京百度网讯科技有限公司 Data synchronization method, device, equipment and storage medium

Also Published As

Publication number Publication date
CN113448647A (en) 2021-09-28

Similar Documents

Publication Publication Date Title
US20200265009A1 (en) Blockchain-based data migration method and apparatus
EP3754514A1 (en) Distributed database cluster system, data synchronization method and storage medium
US8930309B2 (en) Interval-controlled replication
US9619544B2 (en) Distributed state management using dynamic replication graphs
CN105335251B (en) A kind of fault recovery method and system
US8949828B2 (en) Single point, scalable data synchronization for management of a virtual input/output server cluster
US20150379100A1 (en) Coordinated suspension of replication groups
US20180004777A1 (en) Data distribution across nodes of a distributed database base system
US20150378775A1 (en) Log-based transaction constraint management
US10135929B2 (en) Fault-tolerant data processing computer system and method for implementing a distributed two-tier state machine
CN112654978A (en) Method, equipment and system for checking data consistency in distributed heterogeneous storage system in real time
CN103581332A (en) HDFS framework and pressure decomposition method for NameNodes in HDFS framework
CN109582459A (en) The method and device that the trustship process of application is migrated
CN113448647B (en) Resource synchronization method, implementation equipment and electronic equipment
CN112190924A (en) Data disaster tolerance method, device and computer readable medium
CN111666266A (en) Data migration method and related equipment
CN114253743A (en) Message synchronization method, device, node and readable storage medium
EP3696658A1 (en) Log management method, server and database system
CN114064217B (en) OpenStack-based node virtual machine migration method and device
CN111597079A (en) Method and system for detecting and recovering MySQL Galera cluster fault
US20180309702A1 (en) Method and device for processing data after restart of node
CN114697194B (en) Blocking event notification method and device
CN111400098B (en) Copy management method and device, electronic equipment and storage medium
CN106850283B (en) Event-driven cloud AC alarm processing system and method
CN111813501A (en) Data deleting method, device, equipment and storage medium

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant