CN112769897A - Synchronization method and device for edge calculation message, electronic equipment and storage medium - Google Patents
Synchronization method and device for edge calculation message, electronic equipment and storage medium Download PDFInfo
- Publication number
- CN112769897A CN112769897A CN202011520048.7A CN202011520048A CN112769897A CN 112769897 A CN112769897 A CN 112769897A CN 202011520048 A CN202011520048 A CN 202011520048A CN 112769897 A CN112769897 A CN 112769897A
- Authority
- CN
- China
- Prior art keywords
- message
- edge calculation
- target
- module
- target address
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/10—Protocols in which an application is distributed across nodes in the network
- H04L67/1095—Replication or mirroring of data, e.g. scheduling or transport for data synchronisation between network nodes
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/27—Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor
Abstract
The application discloses a synchronization method and device of edge computing messages, electronic equipment and a storage medium, and relates to the technical field of computers, in particular to the technical field of artificial intelligence such as cloud computing, cloud storage and big data processing. The specific implementation scheme is as follows: acquiring an edge calculation message acquired by target terminal equipment; determining a target address corresponding to the edge calculation message, wherein the target address corresponds to a server in the stream processing platform; performing protocol conversion on the edge calculation message according to a data transmission protocol of the stream processing platform; and the edge calculation message after the protocol conversion is synchronized to the server corresponding to the target address by adopting a data transmission protocol, so that the edge calculation message can be uploaded to an external stream processing platform, the application scene of the edge calculation message synchronization is expanded, and the personalized use requirement of a user is met.
Description
Technical Field
The application relates to the technical field of computers, in particular to the technical field of artificial intelligence such as cloud computing, cloud storage and big data processing, and particularly relates to a synchronization method and device for edge computing messages, electronic equipment and a storage medium.
Background
Artificial intelligence is the subject of research that makes computers simulate some human mental processes and intelligent behaviors (such as learning, reasoning, thinking, planning, etc.), both at the hardware level and at the software level. Artificial intelligence hardware technologies generally include technologies such as sensors, dedicated artificial intelligence chips, cloud computing, distributed storage, big data processing, and the like; the artificial intelligence software technology mainly comprises a computer vision technology, a voice recognition technology, a natural language processing technology, machine learning/deep learning, a big data processing technology, a knowledge map technology and the like.
In the scenes of internet of things and edge computing, a device with an edge computing function usually acquires a large number of device parameters and running state information of the device, and these messages may be referred to as edge computing messages, and the edge computing messages are usually synchronized to a cloud server through Message Queue Telemetry Transport (MQTT).
Disclosure of Invention
A synchronization method, an apparatus, an electronic device, a storage medium, and a computer program product for edge calculation messages are provided.
According to a first aspect, there is provided a method for synchronizing edge calculation messages, comprising: acquiring an edge calculation message acquired by target terminal equipment; determining a target address corresponding to the edge computing message, the target address corresponding to a server in a stream processing platform; performing protocol conversion on the edge calculation message according to a data transmission protocol of the stream processing platform; and synchronizing the edge calculation message after protocol conversion to a server corresponding to the target address by adopting the data transmission protocol.
According to a second aspect, there is provided a synchronization apparatus for edge calculation messages, comprising: the first acquisition module is used for acquiring the edge calculation message acquired by the target terminal equipment; a determining module, configured to determine a target address corresponding to the edge computing message, where the target address corresponds to a server in a stream processing platform; the processing module is used for carrying out protocol conversion on the edge calculation message according to a data transmission protocol of the stream processing platform; and the transmission module is used for synchronizing the edge calculation message after protocol conversion to the server corresponding to the target address by adopting the data transmission protocol.
According to a third aspect, there is provided an electronic device comprising: at least one processor; and a memory communicatively coupled to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute the method for synchronizing edge calculation messages according to the embodiment of the present application.
According to a fourth aspect, a non-transitory computer-readable storage medium is proposed, having stored thereon computer instructions for causing the computer to perform the method for synchronizing edge calculation messages disclosed in embodiments of the present application.
According to a fifth aspect, a computer program product is proposed, comprising a computer program which, when executed by a processor, implements the synchronization method of edge computation messages disclosed in embodiments of the present application.
It should be understood that the statements in this section do not necessarily identify key or critical features of the embodiments of the present application, nor do they limit the scope of the present application. Other features of the present application will become apparent from the following description.
Drawings
The drawings are included to provide a better understanding of the present solution and are not intended to limit the present application. Wherein:
FIG. 1 is a schematic diagram according to a first embodiment of the present application;
FIG. 2 is a schematic diagram according to a second embodiment of the present application;
FIG. 3 is a schematic diagram of an architecture of a synchronization apparatus for edge computing messages according to an embodiment of the present application;
FIG. 4 is a schematic illustration according to a third embodiment of the present application;
FIG. 5 is a schematic illustration according to a fourth embodiment of the present application;
fig. 6 is a block diagram of an electronic device for implementing a synchronization method of edge calculation messages according to an embodiment of the present application.
Detailed Description
The following description of the exemplary embodiments of the present application, taken in conjunction with the accompanying drawings, includes various details of the embodiments of the application for the understanding of the same, which are to be considered exemplary only. Accordingly, those of ordinary skill in the art will recognize that various changes and modifications of the embodiments described herein can be made without departing from the scope and spirit of the present application. Also, descriptions of well-known functions and constructions are omitted in the following description for clarity and conciseness.
Fig. 1 is a schematic diagram according to a first embodiment of the present application.
It should be noted that an execution main body of the synchronization method for edge calculation messages in this embodiment is a synchronization apparatus for edge calculation messages, the apparatus may be implemented in a software and/or hardware manner, the apparatus may be configured in an electronic device, and the electronic device may include but is not limited to a terminal, a server, and the like.
The embodiment of the application relates to the technical field of artificial intelligence such as cloud computing, cloud storage and big data processing.
Wherein, Artificial Intelligence (Artificial Intelligence), english is abbreviated as AI. The method is a new technical science for researching and developing theories, methods, technologies and application systems for simulating, extending and expanding human intelligence.
Cloud computing (cloud computing) refers to a technology architecture that accesses a flexibly extensible shared physical or virtual resource pool through a network, where resources may include servers, operating systems, networks, software, applications, storage devices, and the like, and may be deployed and managed in an on-demand, self-service manner. Through the cloud computing technology, high-efficiency and strong data processing capacity can be provided for technical application and model training of artificial intelligence, block chains and the like.
Cloud storage is a mode of online storage, i.e., data is stored in multiple virtual servers, which are usually hosted by third parties, instead of on dedicated servers.
The big data processing refers to a process of analyzing and processing large-scale data in an artificial intelligence mode, and the big data can be summarized into 5V, and has large data Volume (Volume), high speed (Velocity), multiple types (Velocity), Value (Value) and authenticity (Veracity).
As shown in fig. 1, the method for synchronizing the edge calculation messages includes:
s101: and acquiring the edge calculation message collected by the target terminal equipment.
The terminal device that needs to perform synchronization processing on the edge calculation message collected by the terminal device may be referred to as a target terminal device, and the target terminal device may be a device having an edge calculation function in a scene of an internet of things and edge calculation, such as a computer device having an edge calculation function.
The edge calculation message, such as the device parameters collected by the device with the edge calculation function and the running state information of the device, is not limited.
The stream processing platform in the embodiment of the present application may be, for example, an open source stream processing platform Kafka, or may also be other big data computing and storing platforms having a stream data processing function, which is not limited in this respect.
In the embodiment of the present application, the stream processing platform is an open source stream processing platform Kafka.
In some embodiments, after obtaining the edge calculation message acquired by the target terminal device, the method may further perform encryption processing on the edge calculation message after the protocol conversion according to the encryption certificate, and then synchronize the edge calculation message after the protocol conversion to a server corresponding to the target address, including: and synchronizing the encrypted edge calculation message to a server corresponding to the target address.
The encryption certificate is an encryption certificate required by the connection stream processing platform, and the encryption certificate may include some encryption algorithms, so that when the edge calculation message after the protocol conversion is encrypted according to the encryption certificate, the edge calculation message after the protocol conversion may be encrypted by using the encryption algorithm, and when the edge calculation message is synchronized subsequently, the edge calculation message after the protocol conversion may be specifically subjected to synchronous encryption processing.
Because the edge calculation message after the protocol conversion is encrypted in advance according to the encryption certificate, the transmission safety of the edge calculation message can be effectively ensured, the content leakage of the message is avoided, and the safety performance of the synchronization of the edge calculation message is improved.
S102: a destination address corresponding to the edge computation message is determined, the destination address corresponding to a server in the stream processing platform.
The above-mentioned opening source flow processing platform Kafka may be a cluster server architecture, and different server nodes in the cluster server correspond to one address, where the address corresponding to the server node which is subscribed in advance for processing the edge computation message of the target terminal device may be referred to as a target address.
The server node which is subscribed in advance and used for processing the edge calculation message of the target terminal device may be understood as an internal data storage structure, and is consistent with the data storage structure of the edge calculation message of the target terminal device, and may be a server node which is subscribed in advance based on a required data storage structure, so that after the edge calculation message acquired by the target terminal device is acquired, a target address corresponding to the edge calculation message is determined directly according to a pre-subscribed relationship, and the target address corresponds to a server in the stream processing platform, which is not limited thereto.
In some embodiments, after determining the target address corresponding to the edge computing message, an encryption certificate required for connecting the stream processing platform may also be obtained; according to the encrypted certificate, checking the connectivity of the target address; synchronizing the edge calculation message after the protocol conversion to the server corresponding to the target address, or synchronizing the edge calculation message after the protocol conversion to the server corresponding to the target address after determining that the connectivity check passes.
That is to say, the method for verifying the connectivity of the target address is provided in the embodiments of the present application, before transmitting the edge calculation message to the target address, the connectivity of the target address is first verified, and after the connectivity is verified, the edge calculation message is synchronized, and if the verification fails, some reconnection methods may be adopted to update the unavailable target address, so that the availability of the target address can be effectively ensured, and the probability that the target address is unavailable after the transmission rule is created is effectively reduced.
S103: and carrying out protocol conversion on the edge calculation message according to a data transmission protocol of the stream processing platform.
After the edge calculation message acquired by the target terminal device is acquired and the target address corresponding to the edge calculation message is determined, the edge calculation message may be subjected to protocol conversion according to a data transmission protocol of the stream processing platform.
For example, if the open source stream processing platform Kafka supports a data transmission protocol, the protocol of the initial format of the edge calculation message may be converted into the data transmission protocol supported by the open source stream processing platform Kafka, and then the subsequent steps are triggered.
S104: and synchronizing the edge calculation message after protocol conversion to a server corresponding to the target address by adopting a data transmission protocol.
After the protocol of the initial format of the edge calculation message is converted into the data transmission protocol supported by the open source stream processing platform Kafka, the data transmission protocol supported by the open source stream processing platform Kafka can be directly adopted to synchronize the edge calculation message after the protocol conversion to the server corresponding to the target address, so that the edge calculation message is uploaded to an external stream processing platform, the application scene of the synchronization of the edge calculation message is expanded, and the personalized use requirement of a user is met.
In this embodiment, the edge calculation message acquired by the target terminal device is acquired, the target address corresponding to the edge calculation message is determined, the target address corresponds to the server in the stream processing platform, the edge calculation message is subjected to protocol conversion according to the data transmission protocol of the stream processing platform, and the edge calculation message after the protocol conversion is synchronized to the server corresponding to the target address by using the data transmission protocol, so that the edge calculation message can be uploaded to an external stream processing platform, an application scenario of the edge calculation message synchronization is expanded, and personalized use requirements of users are met.
Fig. 2 is a schematic diagram according to a second embodiment of the present application.
As shown in fig. 2, the method for synchronizing the edge calculation messages includes:
s201: a target terminal device is determined from among a plurality of terminal devices.
S202: a target topic category is determined.
S203: and storing the target subject category and the identification of the target terminal equipment in a correlation manner.
That is, it may be a data storage structure based on requirements to subscribe the server nodes in advance, and takes the topic category corresponding to the data storage structure supported by the subscribed server node as the target topic category (the target topic category indicates the data storage structure supported by the server node), the target topic category, as described above, may be the topic category corresponding to the data storage structure of the edge computation message to be synchronized, that is, the data storage structure of the edge computation messages to be synchronized is divided into a topic class, which identifies the data storage structure of the server to which the edge computation messages are synchronized, the edge calculation message and the data storage structure of the server to which it needs to be synchronized have a correspondence, in order to ensure efficient storage of the edge calculation messages, the edge calculation messages are generally synchronized to the server nodes corresponding to the topic categories.
In this embodiment of the present application, a terminal device that performs an edge calculation function may be determined first and used as a target terminal device, then a data storage structure of an edge calculation message obtained by the target terminal device is determined, a target topic category corresponding to the data storage structure is determined, and then the target topic category and an identifier of the target terminal device are stored in an associated manner.
S204: and acquiring the edge calculation message which belongs to the target subject category and is acquired by the target terminal equipment according to the identification.
The target subject category and the identification of the target terminal device are stored in an associated manner, the initial edge calculation message acquired by the target terminal device can be directly acquired according to the identification, and then the data storage structure corresponding to the target subject category is adopted to adjust the data storage structure of the initial edge calculation message in the message rule engine based on the end side, so that the edge calculation message of the target subject category is formed, the efficient acquisition and storage of the edge calculation message can be effectively guaranteed, and the synchronization performance of the edge calculation message is improved.
S205: a destination address corresponding to the edge computation message is determined, the destination address corresponding to a server in the stream processing platform.
S206: and carrying out protocol conversion on the edge calculation message according to a data transmission protocol of the stream processing platform.
S205 and S206 may specifically refer to the above embodiments, and are not described herein again.
S207: and acquiring the calculation resource consumption value required by the synchronous edge calculation message.
In this embodiment, in order to solve the technical problems of weak hardware capability, limited resources, and the like of the edge computing device, a resource flexible scheduling algorithm may be executed to dynamically expand or reduce the number of instances of the message transmission module according to the value of the consumption of the computing resource, so as to adjust the system resources occupied by the message transmission module as needed and reduce the operating pressure of the edge device.
S208: and dynamically expanding or contracting the number of instances of the message transmission module according to the calculation resource consumption value, wherein the message transmission module is used for synchronizing the edge calculation message after protocol conversion to a server corresponding to the target address by using the instances.
For example, the computation resource consumption values (CPU usage, memory usage, and data concurrency amount) required by the synchronous edge computation message may be obtained in real time, so as to dynamically perform capacity expansion or capacity reduction Processing on the number of instances of the message transmission module according to the CPU usage, the memory usage, and the data concurrency amount.
Optionally, in some embodiments, the computational resource consumption value may be compared with a set threshold range, where the set threshold range includes a maximum consumption value and a minimum consumption value; if the calculation resource consumption value is smaller than the minimum consumption value, carrying out capacity expansion processing on the number of the instances of the message transmission module; if the calculation resource consumption value is larger than the maximum consumption value, the capacity reduction processing is carried out on the number of the instances of the message transmission module, the realization is simple and convenient, and a better resource management effect is achieved.
The set threshold range may be set in advance according to the transmission performance of the message transmission module, and the set threshold range may be adaptively adjusted, where the set threshold range includes a value range between a maximum consumption value and a minimum consumption value, the maximum consumption value indicates the maximum CPU usage rate, the memory usage rate, the data concurrency amount, and the like that can be borne by the data transmission module, and the minimum consumption value indicates the minimum CPU usage rate, the memory usage rate, the data concurrency amount, and the like that can be borne by the data transmission module.
Since the message transmission module is configured to synchronize the edge calculation message after the protocol conversion to the server corresponding to the target address by using the instance, and a plurality of instances are configured in the message transmission module, in this embodiment, it is supported that when the operation resource consumption value is less than the minimum consumption value (indicating that the operation resource consumption value consumed by transmitting the edge calculation message does not excessively occupy resources), capacity expansion processing is performed on the number of instances of the message transmission module, and when the operation resource consumption value is greater than the maximum consumption value (indicating that the operation resource consumption value consumed by transmitting the edge calculation message occupies resources relatively, capacity reduction processing is performed on the number of instances of the message transmission module.
S209: and synchronizing the edge calculation message after protocol conversion to a server corresponding to the target address by adopting a data transmission protocol.
For the description of S209, reference may be made to the above embodiments specifically, and details are not repeated here.
In this embodiment, the edge calculation message acquired by the target terminal device is acquired, the target address corresponding to the edge calculation message is determined, the target address corresponds to the server in the stream processing platform, the edge calculation message is subjected to protocol conversion according to the data transmission protocol of the stream processing platform, and the edge calculation message after the protocol conversion is synchronized to the server corresponding to the target address by using the data transmission protocol, so that the edge calculation message can be uploaded to an external stream processing platform, an application scenario of the edge calculation message synchronization is expanded, and personalized use requirements of users are met. The target subject category and the identification of the target terminal equipment are stored in a correlated mode, the initial edge calculation information collected by the target terminal equipment can be directly obtained according to the identification, then, a data storage structure corresponding to the target subject category is adopted, and the data storage structure of the initial edge calculation information in the message rule engine based on the end side is adjusted to form the edge calculation information of the target subject category, so that the efficient collection and storage of the edge calculation information can be effectively guaranteed, and the synchronization performance of the edge calculation information is improved. In order to solve the technical problems of weak hardware capability, limited resources and the like of the edge computing equipment, a resource flexible scheduling algorithm can be executed to dynamically expand or reduce the number of instances of the message transmission module according to the calculation resource consumption value, so that the system resources occupied by the message transmission module can be adjusted as required, and the operating pressure of the edge side equipment is reduced.
A more specific example is given below:
referring to fig. 3, fig. 3 is a schematic diagram of an architecture of a synchronization apparatus for edge computation messages in this embodiment, an example is performed by using a stream processing platform as an open source stream processing platform Kafka, and a hardware architecture of the synchronization apparatus for edge computation messages in this embodiment may include: the system comprises a message bus module, a message rule engine based on an end side, a secure connection module, a Kafka destination configuration module, a protocol conversion module, an external Kafka connectivity test module, an external Kafka data uploading module and a resource flexible scheduling module, so that the synchronization of an edge calculation message to a server corresponding to a target address of the external Kafka in a data encryption mode is supported.
In addition, a data acquisition module may also be configured in the hardware architecture, the data acquisition module acquires the edge calculation message acquired by the target terminal device (the number of the target terminal devices may be one or more), and transmits the edge calculation message to the message rule engine based on the message bus module, and the message rule engine based on the end side triggers the subsequent transmission of the edge calculation message.
The description for the hardware architecture described above may be as follows:
the data collection module, the source of the collected message and the category of the target topic sent to the local message center module, the message bus module, a Message acquisition module is connected in an abutting mode and is simultaneously forwarded to an end-side rule engine module, a safety connection module, an encryption certificate required by supporting and connecting configuration with an external Kafka, an external Kafka destination configuration module is supported to load the encryption certificate in the safety connection module to carry out external Kafka connectivity test, a protocol conversion module is supported to convert Message Queue Telemetry Transmission (MQTT) into a Kafka Message supported by an open source flow processing platform Kafka, an external Kafka data uploading module, the method can receive the edge calculation message sent by the rule engine module, upload the edge calculation message to an external open source stream processing platform Kafka, and research and develop a resource flexible scheduling module.
The data collection module may collect the Message source periodically according to the configuration, and then send the Message to the appointed topic of the local Message center module, and the Message transmission protocol includes, but is not limited to, a standard Message Queue Telemetry Transport (MQTT) protocol, etc.
The message bus module is responsible for transmitting edge calculation messages on the end side.
And the rule engine module at the end side is used for subscribing the subject of the local message bus and forwarding the subject to the external Kafka data uploading module.
The Secure connection module supports connection establishment with an external Kafka based on an SSL (Secure Sockets Layer) certificate (encryption certificate), ensures security of connection and data transmission process, and verifies validity of the encryption certificate in advance.
The user adds the external Kafka connection address, grouping information, subject category and the like to the encrypted certificate, and provides information required for connecting the external Kafka.
By the external Kafka connectivity test, a user can obtain the connectivity of the corresponding destination address in time, the availability of the destination address is ensured, and the probability of unavailability of the destination address after the rule is created is reduced.
The protocol conversion module supports the conversion of MQTT messages into messages (such as Kafka messages) corresponding to the data transmission protocol supported by the open source stream processing platform Kafka, and the valid information of the messages includes, but is not limited to, message coding information, message subject, message quality, message load, and the like.
The external Kafka data uploading module receives the message of the rule engine module, calls the message conversion module to convert the MQTT message into a Kafka message, and uploads the Kafka message to the external Kafka.
The above-mentioned configuration resource flexible scheduling module sets the maximum number of instances and the minimum number of instances of the external Kafka data uploading module, and the upper and lower limit thresholds (maximum and minimum values) of the reference index (calculation resource consumption value), so that during operation, the resource flexible scheduling module collects and counts the CPU utilization, the memory utilization, and the data concurrency number of the external Kafka data uploading module, if the number exceeds/is lower than the set threshold, the number of instances of the flow calculation engine module (i.e. the external Kafka data uploading module, i.e. the above-mentioned message transmission module) is subjected to capacity expansion processing or capacity reduction processing, and the minimum number of instances is set to 0, so as to reduce the system resource consumption to the maximum extent.
Fig. 4 is a schematic diagram according to a third embodiment of the present application.
As shown in fig. 4, the synchronization apparatus 40 for edge calculation messages includes:
a first obtaining module 401, configured to obtain an edge calculation message collected by a target terminal device;
a determining module 402, configured to determine a target address corresponding to the edge calculation message, where the target address corresponds to a server in the stream processing platform;
a processing module 403, configured to perform protocol conversion on the edge calculation message according to a data transmission protocol of the stream processing platform;
a transmission module 404, configured to synchronize the edge calculation message after protocol conversion to a server corresponding to the target address by using a data transmission protocol.
In some embodiments of the present application, as shown in fig. 5, fig. 5 is a schematic diagram according to a fourth embodiment of the present application, where the synchronization apparatus 50 for edge calculation messages, the first obtaining module 501, the determining module 502, the processing module 503, and the transmitting module 504 further include:
a checking module 505, configured to obtain an encryption certificate required by the connection stream processing platform after determining a target address corresponding to the edge calculation message; according to the encrypted certificate, checking the connectivity of the target address;
the transmission module 504 is specifically configured to:
and if the connectivity check is passed, synchronizing the edge calculation message after the protocol conversion to a server corresponding to the target address.
In some embodiments of the present application, as shown in fig. 5, further comprising:
a second obtaining module 506, configured to obtain an operation resource consumption value required for synchronizing the edge calculation message before synchronizing the edge calculation message after the protocol conversion to the server corresponding to the target address;
and a management module 507, configured to dynamically perform capacity expansion or capacity reduction processing on the number of instances of the message transmission module according to the computational resource consumption value, where the message transmission module is configured to synchronize the edge calculation message after protocol conversion to a server corresponding to the target address by using the instances.
In some embodiments of the present application, the management module 507 is specifically configured to:
comparing the calculation resource consumption value with a set threshold range, wherein the set threshold range comprises a maximum consumption value and a minimum consumption value;
if the calculation resource consumption value is smaller than the minimum consumption value, carrying out capacity expansion processing on the number of the instances of the message transmission module;
and if the calculation resource consumption value is larger than the maximum consumption value, carrying out capacity reduction processing on the number of the instances of the message transmission module.
In some embodiments of the present application, as shown in fig. 5, the method further includes:
the encryption module 508 is configured to encrypt the edge calculation message after the protocol conversion according to the encryption certificate after obtaining the edge calculation message acquired by the target terminal device;
the transmission module 504 is specifically configured to:
and synchronizing the encrypted edge calculation message to a server corresponding to the target address.
In some embodiments of the present application, as shown in fig. 5, further comprising:
an association module 509, configured to determine a target terminal device from among the plurality of terminal devices, and determine a target subject category; and storing the target subject category and the identification of the target terminal equipment in a correlation manner;
the first obtaining module 501 is specifically configured to:
and acquiring the edge calculation message which belongs to the target subject category and is acquired by the target terminal equipment according to the identification.
It is understood that the synchronization apparatus 50 of the edge calculation message in fig. 5 of the present embodiment and the synchronization apparatus 40 of the edge calculation message in the foregoing embodiment, the first obtaining module 501 and the first obtaining module 401 in the foregoing embodiment, the determining module 502 and the determining module 402 in the foregoing embodiment, the processing module 503 and the processing module 403 in the foregoing embodiment, and the transmission module 504 and the transmission module 404 in the foregoing embodiment may have the same functions and structures.
It should be noted that the foregoing explanation of the synchronization method for edge calculation messages is also applicable to the synchronization apparatus for edge calculation messages in this embodiment, and is not repeated herein.
In this embodiment, the edge calculation message acquired by the target terminal device is acquired, the target address corresponding to the edge calculation message is determined, the target address corresponds to the server in the stream processing platform, the edge calculation message is subjected to protocol conversion according to the data transmission protocol of the stream processing platform, and the edge calculation message after the protocol conversion is synchronized to the server corresponding to the target address by using the data transmission protocol, so that the edge calculation message can be uploaded to an external stream processing platform, an application scenario of the edge calculation message synchronization is expanded, and personalized use requirements of users are met.
There is also provided, in accordance with an embodiment of the present application, an electronic device, a readable storage medium, and a computer program product.
Fig. 6 is a block diagram of an electronic device for implementing a synchronization method of edge calculation messages according to an embodiment of the present application. Electronic devices are intended to represent various forms of digital computers, such as laptops, desktops, workstations, personal digital assistants, servers, blade servers, mainframes, and other appropriate computers. The electronic device may also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions, are meant to be examples only, and are not meant to limit implementations of the present application that are described and/or claimed herein.
As shown in fig. 6, the apparatus 600 includes a computing unit 601, which can perform various appropriate actions and processes according to a computer program stored in a Read Only Memory (ROM)602 or a computer program loaded from a storage unit 608 into a Random Access Memory (RAM) 603. In the RAM 603, various programs and data required for the operation of the device 600 can also be stored. The calculation unit 601, the ROM 602, and the RAM 603 are connected to each other via a bus 604. An input/output (I/O) interface 605 is also connected to bus 604.
A number of components in the device 600 are connected to the I/O interface 605, including: an input unit 606 such as a keyboard, a mouse, or the like; an output unit 607 such as various types of displays, speakers, and the like; a storage unit 608, such as a magnetic disk, optical disk, or the like; and a communication unit 609 such as a network card, modem, wireless communication transceiver, etc. The communication unit 609 allows the device 600 to exchange information/data with other devices via a computer network such as the internet and/or various telecommunication networks.
The computing unit 601 may be a variety of general and/or special purpose processing components having processing and computing capabilities. Some examples of the computing unit 601 include, but are not limited to, a Central Processing Unit (CPU), a Graphics Processing Unit (GPU), various dedicated Artificial Intelligence (AI) computing chips, various computing units running machine learning model algorithms, a Digital Signal Processor (DSP), and any suitable processor, controller, microcontroller, and so forth. The calculation unit 601 performs the respective methods and processes described above, for example, a synchronization method of edge calculation messages.
For example, in some embodiments, the synchronization method of edge computation messages may be implemented as a computer software program tangibly embodied in a machine-readable medium, such as storage unit 608. In some embodiments, part or all of the computer program may be loaded and/or installed onto the device 600 via the ROM 602 and/or the communication unit 609. When the computer program is loaded into the RAM 603 and executed by the computing unit 601, one or more steps of the above described synchronization method of edge computation messages may be performed. Alternatively, in other embodiments, the computation unit 601 may be configured by any other suitable means (e.g., by means of firmware) to perform the synchronization method of the edge computation messages.
Various implementations of the systems and techniques described here above may be implemented in digital electronic circuitry, integrated circuitry, Field Programmable Gate Arrays (FPGAs), Application Specific Integrated Circuits (ASICs), Application Specific Standard Products (ASSPs), system on a chip (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and/or combinations thereof. These various embodiments may include: implemented in one or more computer programs that are executable and/or interpretable on a programmable system including at least one programmable processor, which may be special or general purpose, receiving data and instructions from, and transmitting data and instructions to, a storage system, at least one input device, and at least one output device.
Program code for implementing the synchronization method of edge computing messages of the present application may be written in any combination of one or more programming languages. These program codes may be provided to a processor or controller of a general purpose computer, special purpose computer, or other programmable data processing apparatus, such that the program codes, when executed by the processor or controller, cause the functions/operations specified in the flowchart and/or block diagram to be performed. The program code may execute entirely on the machine, partly on the machine, as a stand-alone software package partly on the machine and partly on a remote machine or entirely on the remote machine or server.
In the context of this application, a machine-readable medium may be a tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. The machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples of a machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.
To provide for interaction with a user, the systems and techniques described here can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to a user; and a keyboard and a pointing device (e.g., a mouse or a trackball) by which a user can provide input to the computer. Other kinds of devices may also be used to provide for interaction with a user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form, including acoustic, speech, or tactile input.
The systems and techniques described here can be implemented in a computing system that includes a back-end component (e.g., as a data server), or that includes a middleware component (e.g., an application server), or that includes a front-end component (e.g., a user computer having a graphical user interface or a web browser through which a user can interact with an implementation of the systems and techniques described here), or any combination of such back-end, middleware, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: local Area Networks (LANs), Wide Area Networks (WANs), the internet, and blockchain networks.
The computer system may include clients and servers. A client and server are generally remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other. The Server can be a cloud Server, also called a cloud computing Server or a cloud host, and is a host product in a cloud computing service system, so as to solve the defects of high management difficulty and weak service expansibility in the traditional physical host and VPS service ("Virtual Private Server", or simply "VPS"). The server may also be a server of a distributed system, or a server incorporating a blockchain.
It should be understood that various forms of the flows shown above may be used, with steps reordered, added, or deleted. For example, the steps described in the present disclosure may be executed in parallel, sequentially, or in different orders, and are not limited herein as long as the desired results of the technical solutions disclosed in the present disclosure can be achieved.
The above-described embodiments should not be construed as limiting the scope of the present application. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions may be made in accordance with design requirements and other factors. Any modification, equivalent replacement, and improvement made within the spirit and principle of the present application shall be included in the protection scope of the present application.
Claims (15)
1. A method of synchronizing edge computation messages, comprising:
acquiring an edge calculation message acquired by target terminal equipment;
determining a target address corresponding to the edge computing message, the target address corresponding to a server in a stream processing platform;
performing protocol conversion on the edge calculation message according to a data transmission protocol of the stream processing platform; and
and synchronizing the edge calculation message after protocol conversion to a server corresponding to the target address by adopting the data transmission protocol.
2. The method of claim 1, further comprising, after the determining a destination address corresponding to the edge computation message:
acquiring an encryption certificate required by connecting the stream processing platform;
according to the encrypted certificate, checking the connectivity of the target address;
wherein the synchronizing the edge calculation message after the protocol conversion to the server corresponding to the target address includes:
and if the connectivity check is passed, synchronizing the edge calculation message after the protocol conversion to a server corresponding to the target address.
3. The method of claim 1, prior to said synchronizing the protocol converted edge computation message to the server corresponding to the destination address, comprising:
acquiring a calculation resource consumption value required by synchronizing the edge calculation message;
and dynamically expanding or contracting the number of instances of the message transmission module according to the operation resource consumption value, wherein the message transmission module is used for synchronizing the edge calculation message after protocol conversion to a server corresponding to the target address by adopting the instances.
4. The method according to claim 3, wherein the dynamically expanding or contracting the number of instances of the message transmission module according to the value of the computational resource consumption comprises:
comparing the computing resource consumption value with a set threshold range, wherein the set threshold range comprises a maximum consumption value and a minimum consumption value;
if the operation resource consumption value is smaller than the minimum consumption value, carrying out capacity expansion processing on the number of the instances of the message transmission module;
and if the calculation resource consumption value is larger than the maximum consumption value, carrying out capacity reduction processing on the number of the instances of the message transmission module.
5. The method of claim 2, wherein after the obtaining the edge calculation message collected by the target terminal device, further comprising:
according to the encryption certificate, carrying out encryption processing on the edge calculation message after the protocol conversion;
wherein the synchronizing the edge calculation message after the protocol conversion to the server corresponding to the target address includes:
and synchronizing the encrypted edge calculation message to a server corresponding to the target address.
6. The method according to claim 1, further comprising, before the obtaining the edge calculation message collected by the target terminal device:
determining a target terminal device from among a plurality of terminal devices;
determining a target topic category; and
storing the target subject category and the identification of the target terminal equipment in a correlation manner;
the acquiring of the edge calculation message collected by the target terminal device includes:
and acquiring the edge calculation message which is acquired by the target terminal equipment and belongs to the target subject category according to the identification.
7. A synchronization apparatus of an edge calculation message, comprising:
the first acquisition module is used for acquiring the edge calculation message acquired by the target terminal equipment;
a determining module, configured to determine a target address corresponding to the edge computing message, where the target address corresponds to a server in a stream processing platform;
the processing module is used for carrying out protocol conversion on the edge calculation message according to a data transmission protocol of the stream processing platform; and
and the transmission module is used for synchronizing the edge calculation message after protocol conversion to the server corresponding to the target address by adopting the data transmission protocol.
8. The apparatus of claim 7, further comprising:
the verification module is used for acquiring an encryption certificate required by connecting the stream processing platform after the target address corresponding to the edge calculation message is determined; according to the encrypted certificate, checking the connectivity of the target address;
wherein, the transmission module is specifically configured to:
and if the connectivity check is passed, synchronizing the edge calculation message after the protocol conversion to a server corresponding to the target address.
9. The apparatus of claim 7, further comprising:
a second obtaining module, configured to obtain a computation resource consumption value required for synchronizing the edge calculation message before synchronizing the edge calculation message after the protocol conversion to the server corresponding to the target address;
and the management module is used for dynamically carrying out capacity expansion or capacity reduction processing on the number of instances of the message transmission module according to the operation resource consumption value, wherein the message transmission module is used for synchronizing the edge calculation message after protocol conversion to the server corresponding to the target address by adopting the instances.
10. The apparatus according to claim 9, wherein the management module is specifically configured to:
comparing the computing resource consumption value with a set threshold range, wherein the set threshold range comprises a maximum consumption value and a minimum consumption value;
if the operation resource consumption value is smaller than the minimum consumption value, carrying out capacity expansion processing on the number of the instances of the message transmission module;
and if the calculation resource consumption value is larger than the maximum consumption value, carrying out capacity reduction processing on the number of the instances of the message transmission module.
11. The apparatus of claim 8, further comprising:
the encryption module is used for encrypting the edge calculation message after the protocol conversion according to the encryption certificate after the edge calculation message acquired by the target terminal equipment is acquired;
wherein, the transmission module is specifically configured to:
and synchronizing the encrypted edge calculation message to a server corresponding to the target address.
12. The apparatus of claim 7, further comprising:
the association module is used for determining target terminal equipment from a plurality of terminal equipment and determining a target subject category; and storing the target subject category in association with the identifier of the target terminal device;
the first obtaining module is specifically configured to:
and acquiring the edge calculation message which is acquired by the target terminal equipment and belongs to the target subject category according to the identification.
13. An electronic device, comprising:
at least one processor; and
a memory communicatively coupled to the at least one processor; wherein the content of the first and second substances,
the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-6.
14. A non-transitory computer readable storage medium having stored thereon computer instructions for causing the computer to perform the method of any one of claims 1-6.
15. A computer program product comprising a computer program which, when executed by a processor, implements the method according to any one of claims 1-6.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011520048.7A CN112769897B (en) | 2020-12-21 | 2020-12-21 | Synchronization method and device of edge calculation message, electronic equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011520048.7A CN112769897B (en) | 2020-12-21 | 2020-12-21 | Synchronization method and device of edge calculation message, electronic equipment and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112769897A true CN112769897A (en) | 2021-05-07 |
| CN112769897B CN112769897B (en) | 2023-04-18 |
Family
ID=75695164
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011520048.7A Active CN112769897B (en) | 2020-12-21 | 2020-12-21 | Synchronization method and device of edge calculation message, electronic equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112769897B (en) |
Cited By (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113420048A (en) * | 2021-05-19 | 2021-09-21 | 中交公规土木大数据信息技术（北京）有限公司 | Data aggregation method and device, electronic equipment and storage medium |
| CN113423088A (en) * | 2021-06-18 | 2021-09-21 | 安徽星云互联科技有限责任公司 | Information distribution method, device, equipment, system and storage medium |
| CN113438318A (en) * | 2021-07-13 | 2021-09-24 | 阿波罗智联(北京)科技有限公司 | Performance test system and method of cloud control platform, electronic equipment and storage medium |
| CN113515237A (en) * | 2021-07-22 | 2021-10-19 | 中移（杭州）信息技术有限公司 | Edge node capacity expansion method and device, edge node and program product |
| CN113630465A (en) * | 2021-08-10 | 2021-11-09 | 北京百度网讯科技有限公司 | Message processing method, device, equipment and storage medium |
| CN113783954A (en) * | 2021-09-07 | 2021-12-10 | 中控智网(北京)能源技术有限公司 | Oil and gas pipeline component data transmission system and method |
| CN114726871A (en) * | 2022-03-22 | 2022-07-08 | 阿里巴巴（中国）有限公司 | Data synchronization method, device and network system |
| CN115297093A (en) * | 2022-08-02 | 2022-11-04 | 北京百度网讯科技有限公司 | Data access method for edge device, related device and computer program product |
| CN116737482A (en) * | 2023-08-10 | 2023-09-12 | 上海孤波科技有限公司 | Method and device for collecting chip test data in real time and electronic equipment |
Citations (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20190208007A1 (en) * | 2018-01-03 | 2019-07-04 | Verizon Patent And Licensing Inc. | Edge Compute Systems and Methods |
| CN110266776A (en) * | 2019-06-05 | 2019-09-20 | 广州云华智慧科技有限公司 | Cloud layer communication system |
| CN110430281A (en) * | 2019-08-15 | 2019-11-08 | 深圳市般若大数据技术有限公司 | Method, apparatus, equipment and the storage medium of public cloud acquisition private clound data |
| CN110460521A (en) * | 2019-09-19 | 2019-11-15 | 北京中电普华信息技术有限公司 | A kind of edge calculations AnyRouter |
| CN110545307A (en) * | 2019-07-19 | 2019-12-06 | 中移（杭州）信息技术有限公司 | Edge computing platform, calling method and computer readable storage medium |
| CN110557387A (en) * | 2019-08-29 | 2019-12-10 | 浙江大搜车软件技术有限公司 | cross-network equipment communication method, device, system, server and readable storage medium |
| CN110769039A (en) * | 2019-10-09 | 2020-02-07 | 腾讯科技（深圳）有限公司 | Resource scheduling method and device, electronic equipment and computer readable storage medium |
| CN111314339A (en) * | 2020-02-12 | 2020-06-19 | 腾讯科技（深圳）有限公司 | Data transmission method and device |
| CN111800282A (en) * | 2019-04-08 | 2020-10-20 | 阿里巴巴集团控股有限公司 | Network system, instance management and control method, device and storage medium |
| WO2020216038A1 (en) * | 2019-04-22 | 2020-10-29 | 华为技术有限公司 | Data management method, lepm, and mepm |
-
2020
- 2020-12-21 CN CN202011520048.7A patent/CN112769897B/en active Active
Patent Citations (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20190208007A1 (en) * | 2018-01-03 | 2019-07-04 | Verizon Patent And Licensing Inc. | Edge Compute Systems and Methods |
| CN111800282A (en) * | 2019-04-08 | 2020-10-20 | 阿里巴巴集团控股有限公司 | Network system, instance management and control method, device and storage medium |
| WO2020216038A1 (en) * | 2019-04-22 | 2020-10-29 | 华为技术有限公司 | Data management method, lepm, and mepm |
| CN110266776A (en) * | 2019-06-05 | 2019-09-20 | 广州云华智慧科技有限公司 | Cloud layer communication system |
| CN110545307A (en) * | 2019-07-19 | 2019-12-06 | 中移（杭州）信息技术有限公司 | Edge computing platform, calling method and computer readable storage medium |
| CN110430281A (en) * | 2019-08-15 | 2019-11-08 | 深圳市般若大数据技术有限公司 | Method, apparatus, equipment and the storage medium of public cloud acquisition private clound data |
| CN110557387A (en) * | 2019-08-29 | 2019-12-10 | 浙江大搜车软件技术有限公司 | cross-network equipment communication method, device, system, server and readable storage medium |
| CN110460521A (en) * | 2019-09-19 | 2019-11-15 | 北京中电普华信息技术有限公司 | A kind of edge calculations AnyRouter |
| CN110769039A (en) * | 2019-10-09 | 2020-02-07 | 腾讯科技（深圳）有限公司 | Resource scheduling method and device, electronic equipment and computer readable storage medium |
| CN111314339A (en) * | 2020-02-12 | 2020-06-19 | 腾讯科技（深圳）有限公司 | Data transmission method and device |
Non-Patent Citations (3)
| Title |
|---|
| S. YANG;L. CUI; SHENZHEN UNIVERSITY;M. XU; TSINGHUA UNIVERSITY; Y. YANG; TONGJI/YALE;R. HUANG; RESEARCH INSTITUTE OF TSINGHUA UNIV: "Delivering Functions over Networks: Traffic and Performance Optimization for Edge Computing using ALTO" * |
| 王旭亮;武宇亭;任宏丹;: "基于5G+MEC的数字化PaaS赋能平台在智慧安防的应用研究" * |
| 陆璐;: "面向5G网络的边缘计算技术研究" * |
Cited By (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113420048A (en) * | 2021-05-19 | 2021-09-21 | 中交公规土木大数据信息技术（北京）有限公司 | Data aggregation method and device, electronic equipment and storage medium |
| CN113423088A (en) * | 2021-06-18 | 2021-09-21 | 安徽星云互联科技有限责任公司 | Information distribution method, device, equipment, system and storage medium |
| CN113438318A (en) * | 2021-07-13 | 2021-09-24 | 阿波罗智联(北京)科技有限公司 | Performance test system and method of cloud control platform, electronic equipment and storage medium |
| CN113515237A (en) * | 2021-07-22 | 2021-10-19 | 中移（杭州）信息技术有限公司 | Edge node capacity expansion method and device, edge node and program product |
| CN113630465A (en) * | 2021-08-10 | 2021-11-09 | 北京百度网讯科技有限公司 | Message processing method, device, equipment and storage medium |
| CN113630465B (en) * | 2021-08-10 | 2024-03-12 | 北京百度网讯科技有限公司 | Message processing method, device, equipment and storage medium |
| CN113783954A (en) * | 2021-09-07 | 2021-12-10 | 中控智网(北京)能源技术有限公司 | Oil and gas pipeline component data transmission system and method |
| CN114726871A (en) * | 2022-03-22 | 2022-07-08 | 阿里巴巴（中国）有限公司 | Data synchronization method, device and network system |
| CN115297093A (en) * | 2022-08-02 | 2022-11-04 | 北京百度网讯科技有限公司 | Data access method for edge device, related device and computer program product |
| CN116737482A (en) * | 2023-08-10 | 2023-09-12 | 上海孤波科技有限公司 | Method and device for collecting chip test data in real time and electronic equipment |
Also Published As
| Publication number | Publication date |
|---|---|
| CN112769897B (en) | 2023-04-18 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN112769897B (en) | Synchronization method and device of edge calculation message, electronic equipment and storage medium | |
| US11071905B1 (en) | System, method and apparatus for collecting and utilizing big data for online gameplay | |
| CN110808922B (en) | Message processing method and device, storage medium and electronic equipment | |
| CN110839084A (en) | Session management method, device, equipment and medium | |
| CN113766487B (en) | Cloud mobile phone information acquisition method, device, equipment and medium | |
| CN113724398A (en) | Augmented reality method, apparatus, device and storage medium | |
| CN113726881B (en) | Communication connection establishment method, related device and computer readable storage medium | |
| US10250515B2 (en) | Method and device for forwarding data messages | |
| CN114253710A (en) | Processing method of computing request, intelligent terminal, cloud server, equipment and medium | |
| CN115269145A (en) | High-energy-efficiency heterogeneous multi-core scheduling method and device for offshore unmanned equipment | |
| CN114553548A (en) | Communication method, device, equipment and storage medium | |
| CN114579311A (en) | Method, apparatus, device and storage medium for executing distributed computing task | |
| CN113778645A (en) | Task scheduling method, device and equipment based on edge calculation and storage medium | |
| CN114268799A (en) | Streaming media transmission method and device, electronic equipment and medium | |
| CN113935069A (en) | Data verification method, device and equipment based on block chain and storage medium | |
| EP4170974A1 (en) | Slice service processing method and apparatus, network device, and readable storage medium | |
| CN112671970B (en) | Control method and control device for mobile equipment and cloud mobile phone, electronic equipment, mobile equipment, cloud server and medium | |
| CN116662276B (en) | Data processing method, device, electronic equipment and storage medium | |
| CN115589391B (en) | Instant messaging processing method, device and equipment based on block chain and storage medium | |
| CN116614367A (en) | Grouping method and model updating method of terminal equipment | |
| CN116382852A (en) | Micro-service deployment method and device, electronic equipment and medium | |
| CN115440257A (en) | Vehicle audio processing method and device, vehicle, electronic equipment and storage medium | |
| CN114971349A (en) | Information generation method, device, equipment, storage medium and program product | |
| CN116614402A (en) | Model transmission method, device, electronic equipment and storage medium | |
| CN116938996A (en) | Method, device and equipment for loading model by user in meta-universe environment |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |