CN111385334B - Data distribution method, device, equipment and storage medium - Google Patents
Data distribution method, device, equipment and storage medium Download PDFInfo
- Publication number
- CN111385334B CN111385334B CN201811643691.1A CN201811643691A CN111385334B CN 111385334 B CN111385334 B CN 111385334B CN 201811643691 A CN201811643691 A CN 201811643691A CN 111385334 B CN111385334 B CN 111385334B
- Authority
- CN
- China
- Prior art keywords
- data
- information
- meta
- server
- changed
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 54
- 238000009826 distribution Methods 0.000 title claims abstract description 49
- 238000003860 storage Methods 0.000 title claims abstract description 24
- 238000013500 data storage Methods 0.000 claims abstract description 67
- 238000012544 monitoring process Methods 0.000 claims abstract description 51
- 238000004590 computer program Methods 0.000 claims description 6
- 230000008859 change Effects 0.000 abstract description 12
- 230000006978 adaptation Effects 0.000 abstract description 7
- 239000003795 chemical substances by application Substances 0.000 description 44
- 238000010586 diagram Methods 0.000 description 10
- 238000012545 processing Methods 0.000 description 7
- 230000003287 optical effect Effects 0.000 description 6
- 230000005540 biological transmission Effects 0.000 description 4
- 238000007726 management method Methods 0.000 description 4
- 230000006870 function Effects 0.000 description 3
- 230000008569 process Effects 0.000 description 3
- 230000009471 action Effects 0.000 description 2
- 239000013307 optical fiber Substances 0.000 description 2
- 230000002093 peripheral effect Effects 0.000 description 2
- 230000000644 propagated effect Effects 0.000 description 2
- 238000012546 transfer Methods 0.000 description 2
- 238000004458 analytical method Methods 0.000 description 1
- 238000003491 array Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000008901 benefit Effects 0.000 description 1
- 238000004891 communication Methods 0.000 description 1
- 238000009434 installation Methods 0.000 description 1
- 230000014759 maintenance of location Effects 0.000 description 1
- 238000013507 mapping Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000005457 optimization Methods 0.000 description 1
- 230000008520 organization Effects 0.000 description 1
- 230000008707 rearrangement Effects 0.000 description 1
- 238000005096 rolling process Methods 0.000 description 1
- 239000004065 semiconductor Substances 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/10—Protocols in which an application is distributed across nodes in the network
- H04L67/1095—Replication or mirroring of data, e.g. scheduling or transport for data synchronisation between network nodes
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L41/00—Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks
- H04L41/08—Configuration management of networks or network elements
- H04L41/0893—Assignment of logical groups to network elements
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/06—Protocols specially adapted for file transfer, e.g. file transfer protocol [FTP]
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/50—Network services
- H04L67/56—Provisioning of proxy services
Landscapes
- Engineering & Computer Science (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Information Transfer Between Computers (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention discloses a data distribution method, a data distribution device, data distribution equipment and a storage medium. The method comprises the following steps: monitoring data meta-information on a server, wherein the data meta-information is generated by the server according to data sent by a service party, a data storage type and a data storage address; when the monitored data meta-information is changed, determining whether the data corresponding to the changed data meta-information exists in the local machine or not; if the local is not present, the data corresponding to the changed data meta information is obtained from the server, and the automatic adaptation of the data distribution along with the dynamic change of the network node can be realized through the technical scheme of the invention.
Description
Technical Field
The embodiment of the invention relates to the technical field of data distribution, in particular to a data distribution method, a data distribution device, data distribution equipment and a storage medium.
Background
Data distribution primarily refers to the transmission of data from a source to one or more destinations. In a complex online service system, the main difficulties faced by data distribution are:
1. the data storage logic is complex. Different data may need to be written to different storage media, such as to a local disk file, or to kv storage. A piece of data may also be written to both the local disk and the kv storage.
2. The network node relationship is complex. The data to be distributed is usually distributed to some network nodes only, and the mapping relation between the data and the network nodes needs an effective management organization mode. Meanwhile, the network nodes are dynamically changed, and the nodes requiring data distribution are dynamically changed. For example, when a new node is added, the data can be automatically distributed to the new node, and the data cannot be manually transmitted for the new node once.
3. Typically the data will have version iteration requirements. When a certain version of data is in error, a need needs to be provided to quickly revert to the previous version.
Disclosure of Invention
The embodiment of the invention provides a data distribution method, a device, equipment and a storage medium, so as to realize automatic adaptation of data distribution along with dynamic change of network nodes.
In a first aspect, an embodiment of the present invention provides a data distribution method, including:
monitoring data meta-information on a server, wherein the data meta-information is generated by the server according to data sent by a service party, a data storage type and a data storage address;
when the monitored data meta-information is changed, determining whether the data corresponding to the changed data meta-information exists in the local machine or not;
and if the local device does not exist, acquiring the data corresponding to the changed data meta-information from the server.
Further, before the monitoring the data meta-information on the server, the method further includes:
the resource agent for listening is configured.
Further, the data meta information includes: data service name, version number, data storage type, data storage address, and data type.
Further, the monitoring the data meta-information on the server includes:
and monitoring the data meta-information on the server according to the data service name.
Further, the monitored data meta information is changed, including:
the version number in the monitored data meta information is changed.
Further, after obtaining the data corresponding to the changed data meta information from the server, the method further includes:
and storing or decompressing the acquired data according to the data type.
Further, after obtaining the data corresponding to the changed data meta information from the server, the method further includes:
the configured resource agent for listening is deleted.
In a second aspect, an embodiment of the present invention further provides a data distribution device, including:
the monitoring module is used for monitoring the data meta-information on the server, wherein the data meta-information is generated by the server according to the data sent by the service party, the data storage type and the data storage address;
the determining module is used for determining whether the data corresponding to the changed data meta information exists in the host or not when the monitored data meta information is changed;
and the acquisition module is used for acquiring the data corresponding to the changed data meta-information from the server if the local device does not exist.
Further, the method further comprises the following steps:
and the configuration module is used for configuring the resource agent for monitoring before the data meta-information on the monitoring server.
Further, the data meta information includes: data service name, version number, data storage type, data storage address, and data type.
Further, the monitoring module includes:
and the data meta information monitoring unit is used for monitoring the data meta information on the server according to the data service name.
Further, the determining module is specifically configured to:
the version number in the monitored data meta information is changed.
Further, the method further comprises the following steps:
and the processing unit is used for storing or decompressing the acquired data according to the data type after acquiring the data corresponding to the changed data meta-information from the server.
Further, the method further comprises the following steps:
and the deleting module is used for deleting the configured resource agent for monitoring after acquiring the data corresponding to the changed data meta-information from the server.
In a third aspect, an embodiment of the present invention further provides a computer device, including a memory, a processor, and a computer program stored in the memory and capable of running on the processor, where the processor executes the program to implement a data distribution method according to any one of the embodiments of the present invention.
In a fourth aspect, embodiments of the present invention further provide a computer readable storage medium having stored thereon a computer program which, when executed by a processor, implements a data distribution method according to any of the embodiments of the present invention.
In a fifth aspect, an embodiment of the present invention further provides a data distribution system, including: a network node, a service party and a server, the network node being provided with a computer device as described in the third aspect.
The embodiment of the invention monitors the data meta-information on the server, wherein the data meta-information is generated by the server according to the data sent by the service party, the data storage type and the data storage address; when the monitored data meta-information is changed, determining whether the data corresponding to the changed data meta-information exists in the local machine or not; and if the local machine does not exist, acquiring the data corresponding to the changed data meta-information from the server, monitoring the data meta-information by the network node, and when the data meta-information is changed, acquiring the data corresponding to the changed data meta-information from the server if the network node does not exist, so that the automatic adaptation of data distribution along with the dynamic change of the network node can be realized.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings that are needed in the embodiments will be briefly described below, it being understood that the following drawings only illustrate some embodiments of the present invention and therefore should not be considered as limiting the scope, and other related drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.
FIG. 1A is a schematic diagram of a data distribution system according to an embodiment of the present invention;
FIG. 1B is a flow chart of a data distribution method according to a first embodiment of the invention;
FIG. 2A is a flow chart of a data distribution method according to a second embodiment of the present invention;
FIG. 2B is a schematic diagram of a data distribution process according to a second embodiment of the present invention;
FIG. 2C is a schematic diagram of a resource agent deployment procedure in a second embodiment of the present invention;
fig. 2D is a schematic diagram of a data meta-information listening process in a second embodiment of the present invention;
FIG. 3 is a schematic diagram of a data distribution device according to a third embodiment of the present invention;
fig. 4 is a schematic structural diagram of a computer device in a fourth embodiment of the present invention.
Detailed Description
The invention is described in further detail below with reference to the drawings and examples. It is to be understood that the specific embodiments described herein are merely illustrative of the invention and are not limiting thereof. It should be further noted that, for convenience of description, only some, but not all of the structures related to the present invention are shown in the drawings.
It should be noted that: like reference numerals and letters denote like items in the following figures, and thus once an item is defined in one figure, no further definition or explanation thereof is necessary in the following figures. Meanwhile, in the description of the present invention, the terms "first", "second", and the like are used only to distinguish the description, and are not to be construed as indicating or implying relative importance.
Fig. 1A is a schematic diagram of a data distribution system according to an embodiment of the present invention. Specifically, as shown in fig. 1A, the data distribution system includes: a service party 100, a network node 200 and a server 300.
The service side 100 is mainly configured to generate data and send the data to the server 300; the server 300 acquires the data storage type and the data storage address, and determines the data service name, version number, and data storage type according to the data generated by the service party 100. According to the data sent by the sender 100, the data storage type and the data storage address server 300, data meta-information is generated, the service party 100 deploys a resource agent on at least one network node 200 according to the requirement, and the at least one network node 200 configures the resource agent according to the data attribute which needs to be concerned. The network node 200 listens for data meta-information on the server 300 through the resource agent.
It should be noted that, in this embodiment, the number of the network nodes 200 may be specifically determined according to the enterprise side, and the enterprise side 100 may increase the network nodes 200 or decrease the network nodes 200 by deploying resource agents or deleting resource agents in the network nodes 200, which is not limited to this, and meanwhile, the network nodes 200 in this embodiment respectively execute the data distribution methods corresponding to the embodiments described below, and specific execution processes are further described in the following data distribution methods, which are not described in detail herein.
Example 1
Fig. 1B is a flowchart of a data distribution method according to an embodiment of the present invention, where the method may be implemented by a data distribution device according to an embodiment of the present invention, and the device may be implemented in software and/or hardware, as shown in fig. 1B, and the method specifically includes the following steps:
s110, monitoring data meta-information on a server, wherein the data meta-information is generated by the server according to data sent by a service party, a data storage type and a data storage address.
The data meta information is generated according to data sent by the service party, a data storage type and a data storage address, for example, the service party generates data and sends the data to the server, the server feeds back the data storage type and the data storage address, and the data service name, the version number and the data storage type are determined according to the data generated by the service party.
The data storage type and the data storage address are information fed back by the server after the service side sends the data to the server.
The method for monitoring the data meta-information on the server can be used for configuring a resource agent for each network node, and the resource agent is used for monitoring the data meta-information on the server.
Specifically, the service side generates data and sends the data to the server, the server feeds back the data storage type and the data storage address, and generates data element information according to the data sent by the sender, the data storage type and the data storage address, and the network node monitors the data element information on the server.
S120, when the monitored data meta-information is changed, determining whether the data corresponding to the changed data meta-information exists in the local machine.
The change of the data meta-information may be a change of a part of the data meta-information, for example, if the data meta-information includes a data service name and a version number, the change of the data meta-information may be a change of a version number.
The data may be installation package data or text data, and the type of the data is not limited in the embodiment of the invention.
The method for determining whether the data corresponding to the changed data meta-information exists in the local machine may be that the data corresponding to the changed data meta-information is matched with the data stored in the local machine, so as to determine whether the data corresponding to the changed data meta-information exists in the local machine, or that the attribute information of the data corresponding to the changed data meta-information is matched with the attribute information of the data stored in the local machine, so as to determine whether the data corresponding to the changed data meta-information exists in the local machine.
Specifically, when it is monitored that the data meta-information is changed, whether the local machine exists in the data corresponding to the changed data meta-information is determined according to the changed data meta-information, for example, if the data meta-information includes a version number, when it is monitored that the version number is changed, whether the local machine exists in the data corresponding to the data meta-information after the version number is determined.
And S130, if the local device does not exist, acquiring the data corresponding to the changed data meta-information from the server.
The method for obtaining the changed data meta-information from the server may be that when it is monitored that the data meta-information is changed, a data obtaining instruction is generated, and the data corresponding to the changed data meta-information is obtained from the server.
Specifically, if the data corresponding to the changed data meta-information is not stored in the local device, the data corresponding to the changed data meta-information may be obtained from the server, for example, if the version number in the data meta-information is changed, and if the data corresponding to the data meta-information with the changed version number is not stored in the local device corresponding to the network node, the data corresponding to the data meta-information with the changed version number is obtained from the server.
According to the technical scheme, the data meta-information on the server is monitored, wherein the data meta-information is generated by the server according to data sent by a service party, a data storage type and a data storage address; when the monitored data meta-information is changed, determining whether the data corresponding to the changed data meta-information exists in the local machine or not; and if the local machine does not exist, acquiring the data corresponding to the changed data meta-information from the server, monitoring the data meta-information by the network node, and when the data meta-information is changed, acquiring the data corresponding to the changed data meta-information from the server if the network node does not exist, so that the automatic adaptation of data distribution along with the dynamic change of the network node can be realized.
Example two
Fig. 2A is a flowchart of a data distribution method according to a second embodiment of the present invention, where the optimization is performed based on the foregoing embodiment, and before the monitoring of the data meta information on the server, the method further includes: the resource agent for listening is configured.
As shown in fig. 2A, the method of this embodiment specifically includes the following steps:
s210, configuring a resource agent for monitoring.
The resource agent is used for monitoring the data meta-information on the server.
The resource agent may be a resource agent.
Specifically, the service side deploys a resource agent on at least one network node according to the requirement, and the at least one network node configures the resource agent according to the data attribute that needs to be concerned, for example, if the network node needs to be concerned with data of the data service name a, the resource agent for monitoring is configured, and the data meta-information of the data service name a is monitored.
S220, monitoring data meta-information on the server, wherein the data meta-information is generated by the server according to data sent by a service party, a data storage type and a data storage address.
Optionally, the data meta information includes: data service name, version number, data storage type, data storage address, and data type.
The data service name, the version number and the data storage type are acquired according to data generated by a service party.
The data storage address and the data type are fed back by the server after the service side sends the data to the server.
Optionally, the monitoring the data meta-information on the server includes:
and monitoring the data meta-information on the server according to the data service name.
Specifically, the monitoring of the data meta information on the server according to the data service name may be, for example, if the resource agent configured for monitoring monitors the data meta information of the data service name a, then the monitoring of the data meta information of the data service name a on the server.
S230, when the monitored data meta-information is changed, whether the data corresponding to the changed data meta-information exists in the local machine or not is determined.
Optionally, the monitored data meta-information is changed, including:
the version number in the monitored data meta information is changed.
The version number may be changed from a low version number to a high version number, or may be changed from a high version number to a low version number by rolling back.
The reason why the version number in the data meta-information is changed may be that a service party initiates new data distribution, so that the version number in the data meta-information is changed, or an interface for selecting the version number may be present at one end of the service party, and after the service party selects the version number, the data meta-information in the corresponding server may be correspondingly modified, for example, if there is an interface at one end of the service party: and icons corresponding to the version numbers 6, 7 and 8, selecting the icon corresponding to the version number 6 by the service party, and changing the data meta-information in the server into the data meta-information corresponding to the version number 6 according to the touch operation of the service party on the version number 6 if the version number corresponding to the data meta-information in the server is the version number 8 before the service party does not operate.
Specifically, according to the data element information on the data service name monitoring server, the version number in the data element information is monitored to be changed. For example, if the resource agent configured for monitoring monitors the data meta information of the data service name a, the data meta information of the data service name a is monitored, and the version number in the data meta information of the data service name a is monitored to be changed from the version number 7 to the version number 8.
S240, if the local device does not exist, acquiring the data corresponding to the changed data meta-information from the server.
Optionally, after obtaining the data corresponding to the changed data meta information from the server, the method further includes:
and storing or decompressing the acquired data according to the data type.
The data type may be a compressed file or a common file, which is not limited in the embodiment of the present invention.
Specifically, if the data type is a compressed file, after the resource agent acquires the data corresponding to the changed data meta-information from the server, a solution operation is required; if the data type is a common file, after the resource agent acquires the data corresponding to the changed data meta-information from the server, the data is directly stored.
Optionally, after obtaining the data corresponding to the changed data meta information from the server, the method further includes:
the configured resource agent for listening is deleted.
Specifically, if a network node needs to be added or deleted, only a resource agent needs to be deployed or deleted at the network node.
In one specific example, data distribution requires data to be transferred between different machines over a network. Currently, there are scp, rsync, and anstable tools commonly used to transfer data between two machines. scp is a shorthand for secure copy, a command for remote copying of files under Linux, and its similar command has cp, but cp cannot be copied across servers only locally, and scp transfer is encrypted. scp supports password authentication or ssh authentication. rsync is an application software under Unix that can update files and directories of two computers synchronously, and properly uses the rsync algorithm to reduce the amount of data transferred. Compared with scp, rsync can achieve breakpoint resume and file delta update. The transmission efficiency is higher than scp. Anstable is an automated operation tool, which supports multi-node publishing and remote task execution in a manner of calling ssh remote commands. One advantage of stable is that it provides ip packet management functionality, some ips can be attributed under one or more packets. Convenient management of batch machines can be achieved by means of an stable. scp and rsync are both tools for comparing the bottom layers, and cannot well achieve the problem of data synchronization between batch nodes. And the conventional support machine grouping management, so that file batch inter-node transmission can be realized. But cannot meet the dynamic change of the network node.
In this embodiment, as shown in fig. 2B, the data meta information mainly includes the following: data service name: a globally unique name for identifying the service. Version number: the data set may need to be updated continuously, and each update marks a unique version for the data generated at this time. Data storage type and address: the data generated by the business party needs to be saved to a file service accessible by the whole network, such as ftp or hdfs, and the storage address is informed. And adopting a corresponding pulling strategy by Resource angent according to the remote storage type and the address. Data type: the data type is used to tell the resource agent how to save the data to the local after pulling the data. If the data may be saved using a compressed file, the resource agent needs to be decompressed after the downloading is completed. When a service side initiates a new data distribution each time, the service side firstly stores the data to a server accessible by the whole network, and then registers meta-information of the data in the configuration. The Resource agent is mainly responsible for pulling data content from a remote file server, taking corresponding analysis actions to save the data to a local file system or writing in other storages such as redis. Each Resource agent is configured with which data it needs to care about, and monitors changes in data meta-information through the data service name. And automatically triggering the transmission action once the data meta-information is changed. Each network node deploys a resource agent. When one network node is added or deleted, the corresponding network node is only required to be deployed or deleted at the network node. The Resource agent will locally store the most recent version of the original content snapshot for each business datum. When the data needs to be rolled back to an old version, the service side only needs to update the corresponding data meta-information to the data meta-information of the version data in the configuration center. The Resource agent will first check if the version data already exists locally, and if not, re-download. The efficiency of the data rollback operation is provided.
As shown in fig. 2C, the service side deploys the resource agent on at least one network node according to the requirements, and the at least one network node configures the resource agent according to the data attribute that needs to be concerned. The network node monitors the data meta-information on the server through the resource agent. As shown in fig. 2D, the service side generates data and transmits the data to the server; the server acquires the data storage type and the data storage address, and determines the data service name, version number and data storage type according to the data generated by the service party. According to the data sent by the sender, the data storage type and the data storage address server, generating data element information, monitoring the data element information on the server by the network node through the resource agent, after monitoring that the data element information is changed, determining whether the data corresponding to the changed data element information exists in the network node, and if the data corresponding to the changed data element information does not exist in the network node, acquiring the data corresponding to the changed data element information from the server.
According to the embodiment of the invention, the resource agent monitors the data meta-information, so that the automatic adaptation of data distribution along with the dynamic change of the network node is realized. Version control is introduced to the data, so that the data can be conveniently and iteratively released.
According to the technical scheme, a resource agent for monitoring is configured, and data meta-information on a server is monitored, wherein the data meta-information is generated by the server according to data sent by a service party, a data storage type and a data storage address; when the monitored data meta-information is changed, determining whether the data corresponding to the changed data meta-information exists in the local machine or not; if the local machine does not exist, the data corresponding to the changed data element information is obtained from the server, the enterprise side deploys a resource agent on at least one network node according to the requirement, the network node configures the resource agent according to the data attribute which needs to be concerned, the network node monitors the data element information through the resource agent, and when the data element information is changed, if the data corresponding to the changed data element information does not exist in the network node, the data corresponding to the changed data element information is obtained from the server, so that the automatic adaptation of the data distribution along with the dynamic change of the network node can be realized.
Example III
Fig. 3 is a schematic structural diagram of a data distribution device according to a third embodiment of the present invention. The present embodiment may be applied to the case of data distribution, where the apparatus may be implemented in software and/or hardware, and the apparatus may be integrated into any device that provides a function of data distribution, as shown in fig. 3, where the data distribution apparatus specifically includes: listening module 310, determining module 320, and obtaining module 330.
The monitoring module 310 is configured to monitor data meta information on a server, where the data meta information is generated by the server according to data sent by a service party, a data storage type and a data storage address;
a determining module 320, configured to determine whether the data corresponding to the changed data meta information exists in the host when the monitored data meta information is changed;
and the obtaining module 330 is configured to obtain, from the server, data corresponding to the changed data meta information if the local device does not exist.
Optionally, the method further comprises:
and the configuration module is used for configuring the resource agent for monitoring before the data meta-information on the monitoring server.
Optionally, the data meta information includes: data service name, version number, data storage type, data storage address, and data type.
Optionally, the monitoring module includes:
and the data meta information monitoring unit is used for monitoring the data meta information on the server according to the data service name.
Optionally, the determining module is specifically configured to:
the version number in the monitored data meta information is changed.
Optionally, the method further comprises:
and the processing unit is used for storing or decompressing the acquired data according to the data type after acquiring the data corresponding to the changed data meta-information from the server.
Optionally, the method further comprises:
and the deleting module is used for deleting the configured resource agent for monitoring after acquiring the data corresponding to the changed data meta-information from the server.
The product can execute the method provided by any embodiment of the invention, and has the corresponding functional modules and beneficial effects of the execution method.
According to the technical scheme, the data meta-information on the server is monitored, wherein the data meta-information is generated by the server according to data sent by a service party, a data storage type and a data storage address; when the monitored data meta-information is changed, determining whether the data corresponding to the changed data meta-information exists in the local machine or not; and if the local machine does not exist, acquiring the data corresponding to the changed data meta-information from the server, monitoring the data meta-information by the network node, and when the data meta-information is changed, acquiring the data corresponding to the changed data meta-information from the server if the network node does not exist, so that the automatic adaptation of data distribution along with the dynamic change of the network node can be realized.
Example IV
Fig. 4 is a schematic structural diagram of a computer device according to a fourth embodiment of the present invention. Fig. 4 illustrates a block diagram of an exemplary computer device 12 suitable for use in implementing embodiments of the present invention. The computer device 12 shown in fig. 4 is merely an example and should not be construed as limiting the functionality and scope of use of embodiments of the present invention.
As shown in FIG. 4, the computer device 12 is in the form of a general purpose computing device. Components of computer device 12 may include, but are not limited to: one or more processors or processing units 16, a system memory 28, a bus 18 that connects the various system components, including the system memory 28 and the processing units 16.
The system memory 28 may include computer system readable media in the form of volatile memory, such as Random Access Memory (RAM) 30 and/or cache memory 32. The computer device 12 may further include other removable/non-removable, volatile/nonvolatile computer system storage media. By way of example only, storage system 34 may be used to read from or write to non-removable, nonvolatile magnetic media (not shown in FIG. 4, commonly referred to as a "hard disk drive"). Although not shown in fig. 4, a magnetic disk drive for reading from and writing to a removable non-volatile magnetic disk (e.g., a "floppy disk"), and an optical disk drive for reading from or writing to a removable non-volatile optical disk (e.g., a CD-ROM, DVD-ROM, or other optical media) may be provided. In such cases, each drive may be coupled to bus 18 through one or more data medium interfaces. Memory 28 may include at least one program product having a set (e.g., at least one) of program modules configured to carry out the functions of embodiments of the invention.
A program/utility 40 having a set (at least one) of program modules 42 may be stored in, for example, memory 28, such program modules 42 including, but not limited to, an operating system, one or more application programs, other program modules, and program data, each or some combination of which may include an implementation of a network environment. Program modules 42 generally perform the functions and/or methods of the embodiments described herein.
The computer device 12 may also communicate with one or more external devices 14 (e.g., keyboard, pointing device, display 24, etc.), one or more devices that enable a user to interact with the computer device 12, and/or any devices (e.g., network card, modem, etc.) that enable the computer device 12 to communicate with one or more other computing devices. Such communication may occur through an input/output (I/O) interface 22. In addition, in the computer device 12 of the present embodiment, the display 24 is not present as a separate body but is embedded in the mirror surface, and the display surface of the display 24 and the mirror surface are visually integrated when the display surface of the display 24 is not displayed. Moreover, computer device 12 may also communicate with one or more networks such as a Local Area Network (LAN), a Wide Area Network (WAN) and/or a public network, such as the Internet, through network adapter 20. As shown, network adapter 20 communicates with other modules of computer device 12 via bus 18. It should be appreciated that although not shown, other hardware and/or software modules may be used in connection with computer device 12, including, but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, data backup storage systems, and the like.
The processing unit 16 executes various functional applications and data processing by running programs stored in the system memory 28, for example, implementing a data distribution method provided by an embodiment of the present invention: monitoring data meta-information on a server, wherein the data meta-information is generated by the server according to data sent by a service party, a data storage type and a data storage address; when the monitored data meta-information is changed, determining whether the data corresponding to the changed data meta-information exists in the local machine or not; and if the local device does not exist, acquiring the data corresponding to the changed data meta-information from the server.
Example five
A fifth embodiment of the present invention provides a computer readable storage medium having stored thereon a computer program which, when executed by a processor, implements a data distribution method as provided in all the embodiments of the present invention: monitoring data meta-information on a server, wherein the data meta-information is generated by the server according to data sent by a service party, a data storage type and a data storage address; when the monitored data meta-information is changed, determining whether the data corresponding to the changed data meta-information exists in the local machine or not; and if the local device does not exist, acquiring the data corresponding to the changed data meta-information from the server.
Any combination of one or more computer readable media may be employed. The computer readable medium may be a computer readable signal medium or a computer readable storage medium. The computer readable storage medium can be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or a combination of any of the foregoing. More specific examples (a non-exhaustive list) of the computer-readable storage medium would include the following: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In this document, a computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device.
The computer readable signal medium may include a propagated data signal with computer readable program code embodied therein, either in baseband or as part of a carrier wave. Such a propagated data signal may take any of a variety of forms, including, but not limited to, electro-magnetic, optical, or any suitable combination of the foregoing. A computer readable signal medium may also be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device.
Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.
Computer program code for carrying out operations of the present invention may be written in any combination of one or more programming languages, including an object oriented programming language such as Java, smalltalk, C ++ and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any kind of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or may be connected to an external computer (for example, through the Internet using an Internet service provider).
It should be noted that the above description is only a preferred embodiment of the present invention. It will be understood by those skilled in the art that the present invention is not limited to the particular embodiments described herein, but is capable of various obvious changes, rearrangements and substitutions without departing from the scope of the invention. Therefore, while the invention has been described in connection with what is presently considered to be the most practical and preferred embodiment, it is to be understood that the invention is not to be limited to the disclosed embodiment, but on the contrary, is intended to cover various equivalent embodiments and modifications and equivalent arrangements included within the spirit and scope of the appended claims.
Claims (11)
1. A data distribution method, wherein the method is applied to a network node, and comprises:
monitoring data meta-information on a server, wherein the data meta-information is generated by the server according to data sent by a service party, a data storage type and a data storage address;
when the monitored data meta-information is changed, determining whether the data corresponding to the changed data meta-information exists in the local machine or not;
if the local machine does not exist, acquiring data corresponding to the changed data meta-information from the server;
the data storage type and the data storage address are information fed back by the server after the service side sends the data to the server;
the method for monitoring the data meta-information on the server is that a resource agent is configured at each network node and is used for monitoring the data meta-information on the server.
2. The method of claim 1, wherein prior to listening for the data meta-information on the server, further comprising:
the resource agent for listening is configured.
3. The method of claim 2, wherein the data meta-information comprises: data service name, version number, data storage type, data storage address, and data type.
4. The method of claim 3, wherein the listening for data meta-information on the server comprises:
and monitoring the data meta-information on the server according to the data service name.
5. A method according to claim 3, wherein the monitored data meta-information is changed, comprising:
the version number in the monitored data meta information is changed.
6. A method according to claim 3, further comprising, after obtaining the data corresponding to the changed data meta information from the server:
and storing or decompressing the acquired data according to the data type.
7. The method according to any one of claims 1 to 6, further comprising, after obtaining the data corresponding to the changed data meta information from the server:
the configured resource agent for listening is deleted.
8. A data distribution device, comprising:
the monitoring module is used for monitoring the data meta-information on the server, wherein the data meta-information is generated by the server according to the data sent by the service party, the data storage type and the data storage address;
the determining module is used for determining whether the data corresponding to the changed data meta information exists in the host or not when the monitored data meta information is changed;
the acquisition module is used for acquiring data corresponding to the changed data meta-information from the server if the local device does not exist;
the data storage type and the data storage address are information fed back by the server after the service side sends the data to the server;
the method for monitoring the data meta-information on the server is that a resource agent is configured at each network node and is used for monitoring the data meta-information on the server.
9. A computer device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, wherein the processor implements the method of any of claims 1-7 when the program is executed by the processor.
10. A computer readable storage medium, on which a computer program is stored, characterized in that the program, when being executed by a processor, implements the method according to any of claims 1-7.
11. A data distribution system, comprising: network node, a service party and a server, the network node being provided with a computer device according to claim 9.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811643691.1A CN111385334B (en) | 2018-12-29 | 2018-12-29 | Data distribution method, device, equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811643691.1A CN111385334B (en) | 2018-12-29 | 2018-12-29 | Data distribution method, device, equipment and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111385334A CN111385334A (en) | 2020-07-07 |
| CN111385334B true CN111385334B (en) | 2023-06-30 |
Family
ID=71216076
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201811643691.1A Active CN111385334B (en) | 2018-12-29 | 2018-12-29 | Data distribution method, device, equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111385334B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112685503A (en) * | 2021-01-04 | 2021-04-20 | 上海圣剑网络科技股份有限公司 | Data processing method, device and system based on automatic operation and maintenance tool |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2007201719A (en) * | 2006-01-25 | 2007-08-09 | Sii Data Service Kk | Simultaneous data distribution system and simultaneous data distribution method |
| CN102347989A (en) * | 2011-10-25 | 2012-02-08 | 百度在线网络技术（北京）有限公司 | Data distribution method and system based on resource description symbols |
| CN105988881A (en) * | 2015-02-10 | 2016-10-05 | 阿里巴巴集团控股有限公司 | Method and equipment for processing resource access operation information |
| CN108108286A (en) * | 2018-01-03 | 2018-06-01 | 上海众人网络安全技术有限公司 | Method of data capture and device, server, storage medium |
Family Cites Families (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP5982260B2 (en) * | 2012-11-09 | 2016-08-31 | 株式会社日立製作所 | Data structure management device |
| CN103561314A (en) * | 2013-10-12 | 2014-02-05 | 深圳市同洲电子股份有限公司 | Method and terminal for updating program information |
| CN107818118B (en) * | 2016-09-14 | 2019-04-30 | 北京百度网讯科技有限公司 | Date storage method and device |
| CN109479046A (en) * | 2016-11-30 | 2019-03-15 | 华为技术有限公司 | Data transmission method and equipment |
-
2018
- 2018-12-29 CN CN201811643691.1A patent/CN111385334B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2007201719A (en) * | 2006-01-25 | 2007-08-09 | Sii Data Service Kk | Simultaneous data distribution system and simultaneous data distribution method |
| CN102347989A (en) * | 2011-10-25 | 2012-02-08 | 百度在线网络技术（北京）有限公司 | Data distribution method and system based on resource description symbols |
| CN105988881A (en) * | 2015-02-10 | 2016-10-05 | 阿里巴巴集团控股有限公司 | Method and equipment for processing resource access operation information |
| CN108108286A (en) * | 2018-01-03 | 2018-06-01 | 上海众人网络安全技术有限公司 | Method of data capture and device, server, storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111385334A (en) | 2020-07-07 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN112099918B (en) | Live migration of clusters in a containerized environment | |
| US8255359B2 (en) | System and methods for asynchronous synchronization | |
| US8407688B2 (en) | Methods and apparatus for storing and transmitting historical configuration data associated with information technology assets | |
| WO2021082675A1 (en) | Database information backup method and recovery method, electronic device, and computer readable storage medium | |
| US10216601B2 (en) | Agent dynamic service | |
| US10055309B1 (en) | Parallel restoration of a virtual machine's virtual machine disks | |
| JP2012520504A (en) | A programming model for synchronizing the browser cache across devices and web services | |
| US9128627B1 (en) | Method and system for virtual machine backup | |
| CN107797887B (en) | Data backup and recovery method and device, storage medium and electronic equipment | |
| CN111818145B (en) | File transmission method, device, system, equipment and storage medium | |
| JP2021515304A (en) | Methods, computer programs, and equipment for post-failure recovery using checkpoints in key-value stores in a time-series log structure in the system. | |
| US9971611B2 (en) | Monitoring a mobile device application | |
| CN110221910B (en) | Method and apparatus for performing MPI jobs | |
| US9075809B1 (en) | Methods and systems for application cluster virtual nodes | |
| US20070038959A1 (en) | Resource presentation convergence | |
| CN111385334B (en) | Data distribution method, device, equipment and storage medium | |
| US8738873B2 (en) | Interfacing with a point-in-time copy service architecture | |
| US11595493B2 (en) | System and method for namespace masking in an integration flow | |
| CN107526838B (en) | Method and device for database cluster capacity expansion | |
| CN111031126B (en) | Cluster cache sharing method, system, equipment and storage medium | |
| CN111190878B (en) | Method, device, equipment and storage medium for sharing access NAS snapshot | |
| KR102665749B1 (en) | Method and apparatus for ensuring continued device operational reliability in cloud-degraded mode | |
| US10073689B2 (en) | Managing application lifecycles within a federation of distributed software applications | |
| CN114661420B (en) | Application protection method, device and system based on Kubernetes container platform | |
| US7574458B2 (en) | Computer backups using un-used disk space |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| TR01 | Transfer of patent right |
Effective date of registration: 20231008 Address after: 31a, 15 / F, building 30, maple mall, bangrang Road, Brazil, Singapore Patentee after: Baiguoyuan Technology (Singapore) Co.,Ltd. Address before: 511400 floor 23-39, building B-1, Wanda Plaza North, Wanbo business district, 79 Wanbo 2nd Road, Nancun Town, Panyu District, Guangzhou City, Guangdong Province Patentee before: GUANGZHOU BAIGUOYUAN INFORMATION TECHNOLOGY Co.,Ltd. |
|
| TR01 | Transfer of patent right |