CN110012064B

CN110012064B - Data synchronization method and system

Info

Publication number: CN110012064B
Application number: CN201910134354.8A
Authority: CN
Inventors: 周兴; 方小帅; 沈军; 李云鹏
Original assignee: Visionvera Information Technology Co Ltd
Current assignee: Visionvera Information Technology Co Ltd
Priority date: 2019-02-22
Filing date: 2019-02-22
Publication date: 2022-12-23
Anticipated expiration: 2039-02-22
Also published as: CN110012064A

Abstract

The embodiment of the application provides a data synchronization method and a corresponding data synchronization system, wherein the method and the system are applied to a video network, the video network comprises a protocol conversion server and a monitoring unified management mserver server which is in communication connection with the protocol conversion server, and the mserver server comprises a database; the corotation server caches a plurality of change data generated in the current period, the operation type identifier is deleted, the operation type identifier is stored in a modified or newly added mode, and then the cached change data in the current period and the stored change data are sent to the mserver server, so that the mserver server deletes the data with the operation type identifier as the deleted identifier in the database, data consistency of the corotation server and the mserver server is improved, useless data redundancy in the database of the mserver server is reduced, and response capability of the mserver server is guaranteed.

Description

Data synchronization method and system

Technical Field

The present application relates to the field of video networking technologies, and in particular, to a method and a system for data synchronization.

Background

In the video network, when the corotation server aims at deletion operation, the data deleted by the deletion operation can be cleared in real time, and the deleted data cannot be reported to the mserver server, so that the mserver server cannot delete corresponding data in a database, the data in the database of the mserver server is larger and larger, and the response capability of the mserver server is lower and lower; meanwhile, real-time synchronization is adopted by the co-rotating server and the mserver server, namely, the changed data is synchronized when the change occurs, although the real-time performance is good, the problem of large load of the co-rotating server and the mserver server is caused by frequent transmission.

Disclosure of Invention

In view of the above, embodiments of the present application are proposed to provide a method of data synchronization and a corresponding system of data synchronization that overcome or at least partially solve the above problems.

In order to solve the above problems, the present application discloses a method for data synchronization, which is applied to a video network, where the video network includes a protocol conversion server and a monitoring unified management mserver server in communication connection with the protocol conversion server, and the mserver server includes a database; the method comprises the following steps:

the protocol conversion server generates a plurality of change data aiming at a plurality of resource change operations occurring in the current synchronization period; each change data is provided with an operation type identifier, and the operation type identifier comprises a deletion identifier, a modification identifier and a new addition identifier;

the protocol conversion server caches the first change data with the deletion identification and stores the second change data with the modification identification and the new addition identification aiming at the change data;

when the co-rotation server receives a data subscription instruction sent by the mserver server at the ending moment of the current synchronization time period, extracting all cached first change data and all stored second change data;

the co-rotation server sends the first change data and the second change data to the mserver server; the mserver server is used for deleting the data corresponding to the first change data in the database and storing each second change data in the database.

Preferably, the step of sending the first change data and the second change data to the mserver server by the collaboration server includes:

the protocol conversion server encapsulates the first change data and the second change data into third change data; wherein the third variation data has data size information;

and the cooperative conversion server sends the third data to the mserver server.

Preferably, after the step of sending the first change data and the second change data to the mserver server by the coordination server, the method further includes:

when the corotation server receives a synchronization completion signaling returned by the mserver server, deleting the cached first change data; the synchronous completion signaling is generated by the mserver server when judging that the size information of the data is consistent with the size of the received third change data;

or when the cooperation server receives a retransmission signaling returned by the mserver server, the third change data is sent to the mserver server again; and the retransmission signaling is generated by the mserver server when judging that the size information of the data is inconsistent with the size of the received third changed data.

Preferably, the collaboration server performs local persistent storage on the second change data.

Preferably, after the step of sending the first change data and the second change data to the mserver server by the corotation server, the method further includes:

the protocol conversion server deletes all first change data cached in the first N synchronization periods adjacent to the current synchronization period; n is a positive integer.

In order to solve the technical problem, the application also correspondingly discloses a system for data synchronization, which is applied to a video network, wherein the video network comprises a protocol conversion server and a monitoring unified management mserver server in communication connection with the protocol conversion server, and the mserver server comprises a database; the system is located in the cooperative conversion server and comprises:

the periodic data generation module is used for generating a plurality of change data aiming at a plurality of resource change operations generated in the current synchronization period; each piece of change data is provided with an operation type identifier, and the operation type identifier comprises a deletion identifier, a modification identifier and a new addition identifier;

a change data classification storage module, configured to cache, for the multiple change data, first change data with the deletion identifier, and store second change data with the modification identifier and the new addition identifier;

a first instruction receiving module, configured to receive a data subscription instruction sent by the mserver server at an end time of the current synchronization time period;

the data extraction module is used for extracting all the cached first change data and all the stored second change data from the received data subscription instruction;

and the change data sending module is used for sending the first change data and the second change data to the mserver server so as to inform the mserver server to delete the data corresponding to the first change data in the database and store the second change data in the database.

Preferably, the system further comprises:

the change data packaging module is used for packaging the first change data and the second change data into third change data; wherein the third variation data has data size information;

the change data sending module is further configured to send the third data to the mserver server; and the mserver server is used for generating a synchronous completion signaling when judging that the size information of the data is consistent with the size of the received third change data, and generating a retransmission signaling if judging that the size information of the data is inconsistent with the size of the received third change data.

Preferably, the system further comprises:

a second instruction receiving module, configured to receive the synchronization completion signaling and the retransmission signaling returned by the mserver server;

a first buffer data deleting module, configured to delete the buffered first change data according to the synchronization completion signaling;

and a retransmission module, configured to retransmit the third changed data to the mserver server according to the retransmission signaling.

Preferably, the system further comprises:

and the local persistence module is used for carrying out local persistence storage on the second change data.

Preferably, the system further comprises:

the second cache data deleting module is used for deleting all first change data cached in the first N synchronization cycles adjacent to the current synchronization cycle; n is a positive integer.

Compared with the prior art, the embodiment of the application has the following advantages:

firstly, the characteristic of the video networking is applied, and the protocol server and the mserver server synchronize the change data in a periodic synchronization mode, so that the protocol server sends the change data to the mserver server at regular time, the frequency of frequently synchronizing the data between the protocol server and the mserver server is reduced, and the operation load of the protocol server is reduced; when the collaboration server aims at the deletion operation, the deletion data aimed at by the deletion operation is temporarily cached and sent to the mserver server, so that the deleted data is deleted in the database with the mserver server, the data consistency of the collaboration server and the mserver server is improved, the useless data redundancy in the database of the mserver server is reduced, and the response capability of the mserver server is ensured.

Secondly, when synchronizing data, the protocol conversion server informs the mserver server of the total size of the change data of the current synchronization period together, so that the mserver server can judge whether complete synchronous data is received or not according to the size information of the data informed by the protocol conversion server, resynchronizes the data if the complete synchronous data is received, and deletes the deleted data marked as deleted by the cache of the protocol conversion server if the complete synchronous data is received, so that the cache space of the protocol conversion server is released, and the efficiency of caching and deleting the data by the protocol conversion server again is improved.

In addition, in the embodiment of the application, the cooperative conversion server performs classification processing on the changed data generated by different operations, temporarily caches the deleted data generated by the deletion operation and then sends the cached deleted data, and performs local persistent storage on the modified or newly added data, so that the classification management capability of the cooperative conversion server on the data generated by various operations is improved.

Drawings

FIG. 1 is a networking schematic of a video network of the present application;

FIG. 2 is a schematic diagram of a hardware architecture of a node server according to the present application;

fig. 3 is a schematic diagram of a hardware architecture of an access switch of the present application;

fig. 4 is a schematic diagram of a hardware structure of an ethernet protocol conversion gateway according to the present application;

FIG. 5 is a flow chart of the steps of method embodiment 1 of data synchronization of the present application;

FIG. 6 is a diagram of an application environment of embodiment 1 of a data synchronization method of the present application;

FIG. 7 is a flow chart of the steps of method embodiment 2 of the present application for data synchronization;

fig. 8 is a block diagram of a data synchronization apparatus according to embodiment 3 of the present application.

Detailed Description

In order to make the aforementioned objects, features and advantages of the present application more comprehensible, the present application is described in further detail with reference to the accompanying drawings and the detailed description.

The video networking is an important milestone for network development, is a real-time network, can realize high-definition video real-time transmission, and pushes a plurality of internet applications to high-definition video, and high-definition faces each other.

The video networking adopts a real-time high-definition video exchange technology, can integrate required services such as dozens of services of video, voice, pictures, characters, communication, data and the like on a system platform on a network platform, such as high-definition video conference, video monitoring, intelligent monitoring analysis, emergency command, digital broadcast television, delayed television, network teaching, live broadcast, VOD on demand, television mail, personal Video Recorder (PVR), intranet (self-office) channels, intelligent video broadcast control, information distribution and the like, and realizes high-definition quality video broadcast through a television or a computer.

To better understand the embodiments of the present application, the following description refers to the internet of view:

some of the techniques applied by the video network are as follows:

network technology (network technology)

Network innovations in video networking have improved the traditional Ethernet (Ethernet) to face the potentially enormous video traffic on the network. Unlike pure network Packet Switching (Packet Switching) or network Circuit Switching (Circuit Switching), the Packet Switching is adopted by the technology of the video networking to meet the Streaming requirement. The video networking technology has the advantages of flexibility, simplicity and low price of packet switching, and simultaneously has the quality and safety guarantee of circuit switching, thereby realizing the seamless connection of the whole network switching type virtual circuit and the data format.

Switching Technology (Switching Technology)

The video network adopts two advantages of asynchronism and packet switching of the Ethernet, eliminates the defects of the Ethernet on the premise of full compatibility, has end-to-end seamless connection of the whole network, is directly communicated with a user terminal, and directly bears an IP data packet. The user data does not require any format conversion across the entire network. The video network is a higher-level form of the Ethernet, is a real-time exchange platform, can realize the large-scale high-definition video real-time transmission of the whole network which can not be realized by the current Internet, and pushes a plurality of network video applications to high-definition and unification.

Server technology (Servertechnology)

The server technology on the video networking and unified video platform is different from the traditional server, the streaming media transmission of the video networking and unified video platform is established on the basis of connection orientation, the data processing capacity of the video networking and unified video platform is independent of flow and communication time, and a single network layer can contain signaling and data transmission. For voice and video services, the complexity of video networking and unified video platform streaming media processing is much simpler than that of data processing, and the efficiency is greatly improved by more than one hundred times compared with that of a traditional server.

Storage Technology (Storage Technology)

The super-high speed storage technology of the unified video platform adopts the most advanced real-time operating system in order to adapt to the media content with super-large capacity and super-large flow, the program information in the server instruction is mapped to the specific hard disk space, the media content is not passed through the server any more, and is directly sent to the user terminal instantly, and the general waiting time of the user is less than 0.2 second. The optimized sector distribution greatly reduces the mechanical motion of the magnetic head track seeking of the hard disk, the resource consumption only accounts for 20% of that of the IP internet of the same grade, but concurrent flow which is 3 times larger than that of the traditional hard disk array is generated, and the comprehensive efficiency is improved by more than 10 times.

Network Security Technology (Network Security Technology)

The structural design of the video network completely eliminates the network security problem troubling the internet structurally by the modes of independent service permission control each time, complete isolation of equipment and user data and the like, generally does not need antivirus programs and firewalls, avoids the attack of hackers and viruses, and provides a structural carefree security network for users.

Service Innovation Technology (Service Innovation Technology)

The unified video platform integrates services and transmission, and is not only automatically connected once whether a single user, a private network user or a network aggregate. The user terminal, the set-top box or the PC are directly connected to the unified video platform to obtain various multimedia video services in various forms. The unified video platform adopts a menu type configuration table mode to replace the traditional complex application programming, can realize complex application by using very few codes, and realizes infinite new service innovation.

Networking of the video network is as follows:

the video network is a centralized control network structure, and the network can be a tree network, a star network, a ring network and the like, but on the basis of the centralized control node, the whole network is controlled by the centralized control node in the network.

As shown in fig. 1, the video network is divided into an access network and a metropolitan network.

The devices of the access network part can be mainly classified into 3 types: node server, access switch, terminal (including various set-top boxes, coding boards, memories, etc.). The node server is connected to an access switch, which may be connected to a plurality of terminals and may be connected to an ethernet network.

The node server is a node which plays a centralized control function in the access network and can control the access switch and the terminal. The node server may be directly connected to the access switch or may be directly connected to the terminal.

Similarly, devices of the metropolitan network portion may also be classified into 3 types: metropolitan area server, node switch, node server. The metro server is connected to a node switch, which may be connected to a plurality of node servers.

The node server is a node server of the access network part, namely the node server belongs to both the access network part and the metropolitan area network part.

The metropolitan area server is a node which plays a centralized control function in the metropolitan area network and can control a node switch and a node server. The metropolitan area server can be directly connected with the node switch or directly connected with the node server.

Therefore, the whole video network is a network structure with layered centralized control, and the network controlled by the node server and the metropolitan area server can be in various structures such as tree, star and ring.

The access network part can form a unified video platform (the part in the dotted circle), and a plurality of unified video platforms can form a video network; each unified video platform may be interconnected via metropolitan area and wide area video networking.

Video networking device classification

1.1 devices in the video network of the embodiment of the present application can be mainly classified into 3 types: servers, switches (including ethernet gateways), terminals (including various set-top boxes, code boards, memories, etc.). The video network as a whole can be divided into a metropolitan area network (or national network, global network, etc.) and an access network.

1.2 wherein the devices of the access network part can be mainly classified into 3 types: node servers, access switches (including ethernet gateways), terminals (including various set-top boxes, code boards, memories, etc.).

The specific hardware structure of each access network device is as follows:

a node server:

as shown in fig. 2, the system mainly includes a network interface module 201, a switching engine module 202, a CPU module 203, and a disk array module 204;

the packets coming from the network interface module 201, the cpu module 203 and the disk array module 204 all enter the switching engine module 202; the switching engine module 202 performs an operation of looking up the address table 205 on the incoming packet, thereby obtaining the direction information of the packet; and stores the packet in a queue of the corresponding packet buffer 206 based on the packet's steering information; if the queue of the packet buffer 206 is nearly full, it is discarded; the switching engine module 202 polls all packet buffer queues for forwarding if the following conditions are met: 1) The port send buffer is not full; 2) The queue packet counter is greater than zero. The disk array module 204 mainly implements control over the hard disk, including initialization, read-write, and other operations on the hard disk; the CPU module 203 is mainly responsible for protocol processing with an access switch and a terminal (not shown in the figure), configuring an address table 205 (including a downlink protocol packet address table, an uplink protocol packet address table, and a data packet address table), and configuring the disk array module 204.

The access switch:

as shown in fig. 3, the network interface module mainly includes a network interface module (a downlink network interface module 301 and an uplink network interface module 302), a switching engine module 303 and a CPU module 304;

wherein, the packet (uplink data) coming from the downlink network interface module 301 enters the packet detection module 305; the packet detection module 305 detects whether the Destination Address (DA), the Source Address (SA), the packet type, and the packet length of the packet meet the requirements, if so, allocates a corresponding stream identifier (stream-id) and enters the switch engine module 303, otherwise, discards the stream identifier; the packet (downstream data) coming from the upstream network interface module 302 enters the switching engine module 303; the data packet coming from the CPU module 204 enters the switching engine module 303; the switching engine module 303 performs an operation of looking up the address table 306 on the incoming packet, thereby obtaining the direction information of the packet; if the packet entering the switching engine module 303 is from the downstream network interface to the upstream network interface, the packet is stored in the queue of the corresponding packet buffer 307 in association with the stream-id; if the queue of the packet buffer 307 is nearly full, it is discarded; if the packet entering the switching engine module 303 is not from the downlink network interface to the uplink network interface, the data packet is stored in the queue of the corresponding packet buffer 307 according to the guiding information of the packet; if the queue of the packet buffer 307 is nearly full, it is discarded.

The switching engine module 303 polls all packet buffer queues, which in this embodiment is divided into two cases:

if the queue is from the downlink network interface to the uplink network interface, the following conditions are met for forwarding: 1) The port send buffer is not full; 2) The queued packet counter is greater than zero; 3) Obtaining a token generated by a code rate operation module;

if the queue is not from the downlink network interface to the uplink network interface, the following conditions are met for forwarding: 1) The port send buffer is not full; 2) The queued packet counter is greater than zero.

The rate operation module 208 is configured by the CPU module 204, and generates tokens for packet buffer queues from all downstream network interfaces to upstream network interfaces at programmable intervals to control the rate of upstream forwarding.

The CPU module 304 is mainly responsible for protocol processing with the node server, configuration of the address table 306, and configuration of the code rate operation module 308.

Ethernet protocol conversion gateway：

As shown in fig. 4, the apparatus mainly includes a network interface module (a downlink network interface module 401 and an uplink network interface module 402), a switching engine module 403, a CPU module 404, a packet detection module 405, a code rate operation module 408, an address table 406, a packet buffer 407, a MAC adding module 409, and a MAC deleting module 410.

Wherein, the data packet coming from the downlink network interface module 401 enters the packet detection module 405; the packet detection module 405 detects whether the ethernet MAC DA, the ethernet MAC SA, the ethernet length or frame type, the video network destination address DA, the video network source address SA, the video network packet type, and the packet length of the packet meet the requirements, and if so, allocates a corresponding stream identifier (stream-id); then, the MAC deletion module 410 subtracts MAC DA, MAC SA, length or frame type (2 byte) and enters the corresponding receiving buffer, otherwise, discards it;

the downlink network interface module 401 detects the sending buffer of the port, and if there is a packet, obtains the ethernet MAC DA of the corresponding terminal according to the destination address DA of the packet, adds the ethernet MAC DA of the terminal, the MAC SA of the ethernet protocol gateway, and the ethernet length or frame type, and sends the packet.

The other modules in the ethernet protocol gateway function similarly to the access switch.

A terminal:

the system mainly comprises a network interface module, a service processing module and a CPU module; for example, the set-top box mainly comprises a network interface module, a video and audio coding and decoding engine module and a CPU module; the coding board mainly comprises a network interface module, a video and audio coding engine module and a CPU module; the memory mainly comprises a network interface module, a CPU module and a disk array module.

1.3 devices of the metropolitan area network part can be largely classified into 2 types: node server, node exchanger, metropolitan area server. The node switch mainly comprises a network interface module, a switching engine module and a CPU module; the metropolitan area server mainly comprises a network interface module, a switching engine module and a CPU module.

2. Vission networking data packet definition

2.1 Access network packet definition

The data packet of the access network mainly comprises the following parts: destination Address (DA), source Address (SA), reserved byte, payload (PDU), CRC.

As shown in the following table, the data packet of the access network mainly includes the following parts:

DA

SA

Reserved

Payload

CRC

wherein:

the Destination Address (DA) is composed of 8 bytes (byte), the first byte represents the type of the data packet (such as various protocol packets, multicast data packets, unicast data packets, etc.), there are 256 possibilities at most, the second byte to the sixth byte are metropolitan area network addresses, and the seventh byte and the eighth byte are access network addresses;

the Source Address (SA) is also composed of 8 bytes (byte), defined as the same as the Destination Address (DA);

the reserved byte consists of 2 bytes;

the payload part has different lengths according to the types of different datagrams, 64 bytes if various protocol packets, 32+1024=1056 bytes if single-multicast data packets, and certainly not limited to the above 2 types;

the CRC consists of 4 bytes and is calculated in accordance with the standard ethernet CRC algorithm.

2.2 metropolitan area network packet definition

The topology of a metropolitan area network is a graph and there may be 2, or even more than 2, connections between two devices, i.e., more than 2 connections between a node switch and a node server, between a node switch and a node switch, and between a node switch and a node server. However, the metro network address of the metro network device is unique, and in order to accurately describe the connection relationship between the metro network devices, parameters are introduced in the embodiment of the present application: a label to uniquely describe a metropolitan area network device.

In this specification, the definition of the Label is similar to that of the Label of MPLS (Multi-Protocol Label Switch), and assuming that there are two connections between the device a and the device B, there are 2 labels for the packet from the device a to the device B, and 2 labels for the packet from the device B to the device a. The label is classified into an incoming label and an outgoing label, and assuming that the label (incoming label) of the packet entering the device a is 0x0000, the label (outgoing label) of the packet leaving the device a may become 0x0001. The network access process of the metro network is a network access process under centralized control, that is, address allocation and label allocation of the metro network are both dominated by the metro server, and the node switch and the node server are both passively executed, which is different from label allocation of MPLS, and label allocation of MPLS is a result of mutual negotiation between the switch and the server.

As shown in the following table, the data packet of the metro network mainly includes the following parts:

DA

SA

Reserved

label (R)

Payload

CRC

Namely Destination Address (DA), source Address (SA), reserved byte (Reserved), tag, payload (PDU), CRC. The format of the tag may be defined by reference to the following: the tag is 32 bits with the upper 16 bits reserved and only the lower 16 bits used, and its position is between the reserved bytes and payload of the packet.

Based on the characteristics of the video networking, one of the core concepts of the embodiment of the application is provided, and the protocol of the video networking is followed, and the protocol of the video networking is adopted, so that the protocol server synchronizes the change data with the mserver server in a periodic synchronization mode, and the protocol server sends the change data to the mserver server at regular time, thereby reducing the frequency of frequently sending data between the protocol server and the mserver server and reducing the operation load of the protocol server; when the corotation server aims at the deletion operation, the deletion data aimed at by the deletion operation is temporarily cached and sent to the mserver server, so that the data deleted by the corotation server and the mserver server are deleted in the database, the data consistency of the corotation server and the mserver server is improved, useless data redundancy in the database of the mserver server is reduced, and the response capability of the mserver server is ensured.

Example one

Referring to fig. 5, a flowchart of steps of embodiment 1 of a data synchronization method according to the present application is shown, in this embodiment of the present application, the method may be applied to a video network, where the video network may include a collaboration server 601 and a monitoring unified management mserver server 602 communicatively connected to the collaboration server 601, and the mserver server 602 includes a database 603.

An application environment of a method for data synchronization according to an embodiment of the present application is shown in fig. 6. The collaboration server 601 may be understood as a server connected to a plurality of monitoring devices for performing management control on the monitoring devices; the mserver server 602 may be understood as a server connected to a plurality of assist servers, and configured to perform management control on the cooperation server 601 and maintain monitoring device information and resource change data uploaded by the cooperation server 601; the database 603 may be understood as a database 603 for storing and managing monitoring device information and resource change data, and may be a mySQL type database 603.

The data synchronization method of the embodiment of the application may specifically include the following steps:

in step 501, the collaboration server 601 generates a plurality of change data for a plurality of resource change operations occurring in the current synchronization cycle.

Each change data has an operation type identifier, and the operation type identifier includes a deletion identifier, a modification identifier and a new addition identifier.

In the embodiment of the application, the cooperative server 601 periodically synchronizes the change data to the mserver server 602, wherein the synchronization period may be set to 1 hour or 30 minutes, and the specific time may be flexibly set according to the actual situation; in actual implementation, both the mserver server 602 and the corotation server 601 set the same synchronization period, for example, the corotation server 601 may set a synchronization period starting from 11/2018, and the synchronization period is 1 hour; the mserver server 602 may also set the same information so that the mserver server 602 and the collaboration server 601 achieve synchronization of transmission and reception.

In practice, the resource changing operation may be an operation of canceling a certain monitoring device, an operation of adding a certain monitoring device, or an operation of modifying information of a certain monitoring device, and accordingly, the operation type identifier may be understood as an identifier for distinguishing a specific form of the operation, for example, a delete identifier indicates that the changed data is deleted, an add identifier indicates that the changed data is added, and a modify identifier indicates that the changed data is modified according to the original data. In practice, each resource changing operation may generate only one change data, or may generate a plurality of change data.

Step 502, the coordination server 601 caches the first changed data with the deletion identifier and stores the second changed data with the modification identifier and the new addition identifier for the plurality of changed data.

In practice, the first change data can be understood as data relating to a monitoring device when the monitoring device is cancelled on the assistance server. In specific implementation, when a monitoring device is deleted, the corotation server 601 deletes the data in the position where the monitoring device information and the data are stored, and at the same time, marks the data with a deletion identifier and then places the data into a cache, where performing the cache may be understood as placing the first change data into a memory for temporary storage. And storing the second change data generated by the modification operation and the addition operation into the position for storing the monitoring equipment information and data.

In step 503, when receiving a data subscription instruction sent by the mserver server 602 at the ending time of the current synchronization time period, the collaboration server 601 extracts all the cached first change data and all the stored second change data.

When a synchronization period ends, the mserver server 602 sends a data subscription instruction, so that the collaboration server 601 sends the change data in the synchronization period to the mserver server 602. In practice, the change data for the resource change operation all have time identifiers, and when the change data of the current synchronization period is extracted, which data belong to the change data in the current period can be determined according to the time identifiers of the change data.

For example, the current synchronization cycle is 11 months, 2 days, 12 in 2018: 11/month/2/14 between 00 and 2018: 00, time identification is 12 in 2018, 11, 2, and 12:00 and 2018, 11, 2, 14: the data between 00 is the variation data of the current synchronization period.

In practice, the cooperative server 601 may record the start-stop time value of each synchronization cycle, so as to accurately extract the change data of the corresponding cycle without missing the change data.

In step 504, the collaboration server 601 sends the first change data and the second change data to the mserver server 602.

The mserver server 602 is configured to delete the data corresponding to the first changed data in the database 603, and store each second changed data in the database 603.

In practice, the first change data is data to be deleted, and when the mserver server 602 receives the first change data, it is known that the data needs to be deleted according to the identifier of the first change data, and then the mserver server 602 deletes the data corresponding to the first change data in the database 603; if the second changed data is new or modified data, the mserver server 602 adds the new or modified data to the database 603 in the database 603.

For example, if the first change data is data for a monitoring device, the mserver server 602 deletes the data related to the a monitoring device in the database 603; if the second change data is data for the B monitoring device, the mserver server 602 adds the second change data to the database 603 in the database 603.

In specific implementation, after deleting the data corresponding to the first change data in the database 603, the mserver server 602 may also delete the first change data without buffering the first change data, so that the storage space of the mserver server 602 is not occupied by the useless data, and the data processing response capability of the mserver server 602 in the data synchronization process is improved.

As an optional example of the embodiment of the present application, this step 504 may specifically include the following sub-steps:

in sub-step 5041, the collaboration server 601 packages the first change data and the second change data into third change data. Wherein the third variation data has data size information.

The third change data may be compressed data obtained by packing the first change data and the second change data. The data size information may be understood as a total data size of the first variation data and the second variation data.

Substep 5042, the collaboration server 601 sends the third data to the mserver server 602.

As an optional example of the embodiment of the present application, after step 504, the following step may be further included:

step 505, when receiving the synchronization completion signaling returned by the mserver server 602, the collaboration server 601 deletes the cached first change data.

The synchronization completion signaling is generated by the mserver server 602 when it is determined that the size information of the data is consistent with the size of the received third change data.

When the mserver server 602 successfully receives all the first change data and the second change data, a synchronization completion signaling is generated, and the synchronization completion signaling may be character information or coding information. When receiving the synchronization completion signaling, the cooperative server 601 may delete the temporarily cached first change data of the current synchronization cycle, thereby releasing the cache space.

In practice, each time the cooperative conversion server 601 successfully sends the change data in one synchronization period, the first change data temporarily cached in the synchronization period is deleted, so that the effect of immediate transmission and deletion is achieved, and the real-time response efficiency of the cooperative conversion server 601 is improved.

Or, in step 506, when receiving the retransmission signaling returned by the mserver server 602, the coordination server 601 retransmits the third change data to the mserver server 602.

The retransmission signaling is generated by the mserver server 602 when it is determined that the size information of the data is inconsistent with the size of the received third change data.

If the mserver server 602 does not receive all the first change data and the second change data, a retransmission signaling is sent out, so that the corotation server 601 is resynchronized, the mserver server 602 can receive complete synchronous data, and the high consistency of data synchronization of the mserver server 602 and the corotation server 601 is improved.

In practice, after resynchronization, if complete data is received, the mserver server 602 still generates a synchronization completion signaling, so that the coordination server 601 may delete the temporarily cached first change data after receiving the synchronization completion signaling.

As an optional example of the embodiment of the present application, in step 502, the collaboration server 601 performs local persistent storage on the second change data.

Local persistent storage may be understood as saving the second change data to the hard disk so that it is still accessible after the corotation server 601 is restarted or upgraded.

In the embodiment of the application, the protocol server 601 and the mserver server 602 synchronize the change data in a periodic synchronization manner, so that the protocol server 601 sends the change data to the mserver server 602 at regular time, the frequency of frequently synchronizing the data between the protocol server 601 and the mserver server 602 is reduced, and the operation load of the protocol server 601 is reduced; when the collaboration server 601 aims at the deletion operation, the deletion data aimed at by the deletion operation is temporarily cached and sent to the mserver server 602, so that the deleted data is deleted in the database 603 with the mserver server 602, the data consistency of the collaboration server 601 and the mserver server 602 is improved, the useless data redundancy in the database 603 of the mserver server 602 is reduced, and the response capability of the mserver server 602 is ensured.

Example two

Referring to fig. 7, a flowchart of steps of method embodiment 2 of data synchronization according to the present application is shown, in this application embodiment, the method may be applied to a video network, where the video network may include a collaboration server 601 and a monitoring unified management mserver server 602 communicatively connected to the collaboration server 601, and the mserver server 602 includes a database 603.

in step 701, the collaboration server 601 generates a plurality of change data for a plurality of resource change operations occurring in the current synchronization cycle.

In the embodiment of the application, the cooperative server 601 periodically synchronizes the change data to the mserver server 602, wherein the synchronization period can be set to 20 minutes, and the specific time can be flexibly set according to the actual situation; in actual implementation, after the mserver server 602 sets the synchronization period, the synchronization period may be notified to the protocol server 601, and the protocol server 601 records each synchronization period. So that the mserver server 602 and the corotation server 601 achieve synchronization of transmission and reception.

Step 702, the corotation server 601 caches the first changed data with the deletion identifier and stores the second changed data with the modification identifier and the new addition identifier for the plurality of changed data.

The specific process of this step may be as described in step 502 of embodiment 1.

In step 703, when receiving the data subscription instruction sent by the mserver server 602 at the end time of the current synchronization time period, the collaboration server 601 extracts all the cached first change data and all the stored second change data.

The specific process of this step may be as described in step 503 of example 1.

Step 704, the collaboration server 601 sends the first change data and the second change data to the mserver server 602.

The mserver server 602 is configured to delete the data corresponding to the first change data in the database 603, and store each second change data in the database 603.

The specific process of this step may refer to the description of step 504 in example 1.

In an optional example of the embodiment of the present application, after step 504, the following steps may also be included:

step 705, the cooperative server 601 deletes all the first change data cached in the first N synchronization cycles adjacent to the current synchronization cycle; n is a positive integer.

In practice, after sending the data of the current synchronization cycle, the collaboration server 601 may delete the first change data cached in the current synchronization cycle and all the first change data cached in the previous N synchronization cycles, where N is 2, and the current synchronization cycle is 11, 2, 12 in 2018: 11/month/2/14 between 00 and 2018: 00, after the cooperative server 601 sends the first change data of the synchronization period, the first change data of the synchronization period is deleted from 2018, 11, 2, 8: 11/h, 2/h 12 between 00 and 2018: 00, where 11/s/2/s/8 in 2018: 00 to 2018, 11, 2, 10:00 is the first second sync period, 11 months, 2 days, 10 in 2018: 11/h, 2/h 12 between 00 and 2018: 00 is the previous synchronization period.

The cooperative conversion server 601 deletes the first change data to release the cache space, so that the response speed of the cooperative conversion server 601 is improved; meanwhile, the cooperative conversion server 601 may actively delete the first change data cached in the N synchronization cycles that have been synchronized before, and N may be set according to an actual situation, so that when N is set to be greater than 2, the number of operations for deleting the first change data may be reduced, and the operating efficiency of the cooperative conversion server 601 is improved.

In the embodiment of the application, the change data is synchronized between the corotation server 601 and the mserver server 602 in a periodic synchronization manner, so that the corotation server 601 sends the change data to the mserver server 602 at regular time, the frequency of frequently synchronizing the data between the corotation server 601 and the mserver server 602 is reduced, and the operation load of the corotation server 601 is reduced; when the collaboration server 601 aims at the deletion operation, the deletion data aimed at by the deletion operation is temporarily cached and sent to the mserver server 602, so that the deleted data is deleted in the database 603 with the mserver server 602, the data consistency of the collaboration server 601 and the mserver server 602 is improved, the useless data redundancy in the database 603 of the mserver server 602 is reduced, and the response capability of the mserver server 602 is ensured. Meanwhile, the cooperative conversion server 601 can actively delete the first change data cached in the synchronization cycle which is completed synchronously, so that the response capability of the cooperative conversion server 601 is improved.

EXAMPLE III

As shown in fig. 8, a system for data synchronization according to embodiment 3 of the present application, in contrast to the processing method of embodiment 1, the system may be applied to a video network, where the video network includes a collaboration server 601 and a monitoring unified management mserver server 602 communicatively connected to the collaboration server 601, and the mserver server 602 may include a database 603; the system may be located in the collaboration server 601, and specifically may include the following modules:

a periodic data generation module 801, configured to generate a plurality of change data for a plurality of resource change operations occurring in a current synchronization period; each piece of change data is provided with an operation type identifier, and the operation type identifier comprises a deletion identifier, a modification identifier and a new addition identifier;

a change data classification storage module 802, configured to cache, for the multiple change data, the first change data with the deletion identifier, and store the second change data with the modification identifier and the new addition identifier;

a first instruction receiving module 803, configured to receive a data subscription instruction sent by the mserver server 602 at the expiration time of the current synchronization time period;

a data extracting module 804, configured to extract all cached first variation data and all stored second variation data according to the received data subscription instruction;

a change data sending module 806, configured to send the first change data and the second change data to the mserver server 602, so as to notify the mserver server 602 to delete the data corresponding to the first change data in the database 603, and store each second change data in the database 603.

As an optional example of the embodiment of the present application, the system further includes the following modules:

a change data packaging module 805, configured to package the first change data and the second change data into third change data; wherein the third variation data has data size information;

the change data sending module 806 is further configured to send the third data to the mserver server 602; the mserver server 602 is configured to generate a synchronization completion signaling when the size information of the data is determined to be consistent with the size of the received third change data, and generate a retransmission signaling when the size information of the data is determined to be inconsistent with the size of the received third change data.

a second instruction receiving module 807, configured to receive the synchronization completion signaling and the retransmission signaling returned by the mserver server 602;

a first buffer data deleting module 809, configured to delete the buffered first change data according to the synchronization completion signaling;

a resending module 808, configured to resend the third changed data to the mserver server 602 according to the retransmission signaling.

a local persistence module 811, configured to locally persist the second change data.

a second cache data deleting module 810, configured to delete all first change data cached in the first N synchronization cycles adjacent to the current synchronization cycle; n is a positive integer.

For the apparatus embodiment, since it is substantially similar to the method embodiment, the description is relatively simple, and reference may be made to the partial description of the method embodiment for relevant points.

The embodiments in the present specification are described in a progressive manner, each embodiment focuses on differences from other embodiments, and the same and similar parts among the embodiments are referred to each other.

As will be appreciated by one of skill in the art, embodiments of the present application may be provided as a method, apparatus, or computer program product. Accordingly, embodiments of the present application may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, embodiments of the present application may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.

Embodiments of the present application are described with reference to flowchart illustrations and/or block diagrams of methods, terminal devices (systems), and computer program products according to embodiments of the application. It will be understood that each flow and/or block of the flow diagrams and/or block diagrams, and combinations of flows and/or blocks in the flow diagrams and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing terminal to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing terminal, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.

These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing terminal to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.

These computer program instructions may also be loaded onto a computer or other programmable data processing terminal to cause a series of operational steps to be performed on the computer or other programmable terminal to produce a computer implemented process such that the instructions which execute on the computer or other programmable terminal provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.

While preferred embodiments of the present application have been described, additional variations and modifications of these embodiments may occur to those skilled in the art once they learn of the basic inventive concepts. Therefore, it is intended that the appended claims be interpreted as including the preferred embodiment and all changes and modifications that fall within the true scope of the embodiments of the present application.

Finally, it should also be noted that, herein, relational terms such as first and second, and the like may be used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Also, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or terminal that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or terminal. Without further limitation, an element defined by the phrase "comprising a … …" does not exclude the presence of another identical element in a process, method, article, or terminal apparatus that comprises the element.

The method for data synchronization and the corresponding system for data synchronization provided by the present application are introduced in detail above, and specific examples are applied herein to illustrate the principles and embodiments of the present application, and the descriptions of the above embodiments are only used to help understand the method and the core ideas of the present application; meanwhile, for a person skilled in the art, according to the idea of the present application, the specific implementation manner and the application scope may be changed, and in summary, the content of the present specification should not be construed as a limitation to the present application.

Claims

1. The method is characterized in that the method is applied to a video network, the video network comprises a protocol conversion server and a monitoring unified management mserver server which is in communication connection with the protocol conversion server, and the mserver server comprises a database; the mserver server and the protocol conversion server set the same synchronization period; the method comprises the following steps:

the protocol conversion server generates a plurality of change data aiming at a plurality of resource change operations occurring in the current synchronization period; each piece of change data is provided with an operation type identifier, and the operation type identifier comprises a deletion identifier, a modification identifier and a new addition identifier;

the coordination server caches the first change data with the deletion identifier according to the change data, stores the second change data with the modification identifier and the new identifier to a position for storing monitoring equipment information and data so as to perform local persistent storage, wherein the second change data is new or modified data; when the coordination server deletes data to be deleted in the position for storing the monitoring equipment information and the data, the first change data is obtained by marking a deletion identifier on the data;

when the co-rotation server receives a data subscription instruction sent by the mserver server at the ending moment of the current synchronization period, extracting all cached first change data and all stored second change data according to the time identification of the change data;

the co-rotation server sends the first change data and the second change data to the mserver server; the mserver server is used for deleting data corresponding to the first change data in the database and storing each second change data in the database;

and the protocol conversion server deletes the temporarily cached first change data in the synchronization period every time the change data in one synchronization period is successfully sent.

2. The method of claim 1, wherein the step of the corotation server sending the first change data and the second change data to the mserver server comprises:

the corotation server encapsulates the first change data and the second change data into third change data; wherein the third variation data has data size information;

and the cooperation server sends the third change data to the mserver server.

3. The method of claim 2, wherein after the step of the collaboration server sending the first change data and the second change data to the mserver server, further comprising:

when the protocol server receives a synchronization completion signaling returned by the mserver server, deleting the cached first change data; the synchronous completion signaling is generated by the mserver server when judging that the size information of the data is consistent with the size of the received third change data;

or, when receiving a retransmission signaling returned by the mserver server, the coordination server retransmits the third change data to the mserver server; and the retransmission signaling is generated by the mserver server when judging that the size information of the data is inconsistent with the size of the received third change data.

4. The method of claim 1, wherein after the step of the collaboration server sending the first change data and the second change data to the mserver server, further comprising:

5. The system is characterized in that the system is applied to a video network, the video network comprises a protocol conversion server and a monitoring unified management mserver server which is in communication connection with the protocol conversion server, and the mserver server comprises a database; the mserver server and the protocol conversion server set the same synchronization period; the system is located in the cooperative conversion server and comprises:

the change data classification storage module is used for caching the first change data with the deletion identifier according to the change data, storing the second change data with the modification identifier and the newly added identifier to a position for storing monitoring equipment information and data so as to perform local persistent storage, wherein the second change data is newly added or modified data; when the coordination server deletes the data to be deleted in the position for storing the monitoring equipment information and the data, the first change data is obtained by marking the data with a deletion identifier;

a first instruction receiving module, configured to receive a data subscription instruction sent by the mserver server at the end time of the current synchronization period;

the data extraction module is used for extracting all the cached first change data and all the stored second change data according to the received data subscription instruction and the time identification of the change data;

a change data sending module, configured to send the first change data and the second change data to the mserver server, so as to notify the mserver server to delete data corresponding to the first change data in the database, and store each second change data in the database; and the protocol conversion server deletes the temporarily cached first change data in the synchronization period every time the change data in one synchronization period is successfully sent.

6. The system of claim 5, further comprising:

the change data sending module is further configured to send the third change data to the mserver server; and the mserver server is used for generating a synchronous completion signaling when judging that the size information of the data is consistent with the size of the received third change data, and generating a retransmission signaling if judging that the size information of the data is inconsistent with the size of the received third change data.

7. The system of claim 6, further comprising:

8. The system of claim 5, further comprising:

9. The system of claim 5, further comprising: