CN112769897B - Synchronization method and device of edge calculation message, electronic equipment and storage medium - Google Patents

Synchronization method and device of edge calculation message, electronic equipment and storage medium Download PDF

Info

Publication number
CN112769897B
CN112769897B CN202011520048.7A CN202011520048A CN112769897B CN 112769897 B CN112769897 B CN 112769897B CN 202011520048 A CN202011520048 A CN 202011520048A CN 112769897 B CN112769897 B CN 112769897B
Authority
CN
China
Prior art keywords
message
edge calculation
target
target address
consumption value
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202011520048.7A
Other languages
Chinese (zh)
Other versions
CN112769897A (en
Inventor
陈�胜
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Beijing Baidu Netcom Science and Technology Co Ltd
Original Assignee
Beijing Baidu Netcom Science and Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Beijing Baidu Netcom Science and Technology Co Ltd filed Critical Beijing Baidu Netcom Science and Technology Co Ltd
Priority to CN202011520048.7A priority Critical patent/CN112769897B/en
Publication of CN112769897A publication Critical patent/CN112769897A/en
Application granted granted Critical
Publication of CN112769897B publication Critical patent/CN112769897B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00Network arrangements or protocols for supporting network services or applications
    • H04L67/01Protocols
    • H04L67/10Protocols in which an application is distributed across nodes in the network
    • H04L67/1095Replication or mirroring of data, e.g. scheduling or transport for data synchronisation between network nodes
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/27Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor

Landscapes

  • Engineering & Computer Science (AREA)
  • Databases & Information Systems (AREA)
  • Theoretical Computer Science (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Signal Processing (AREA)
  • Computing Systems (AREA)
  • Data Mining & Analysis (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Information Transfer Between Computers (AREA)
  • Computer And Data Communications (AREA)

Abstract

The application discloses a synchronization method and device of edge computing messages, electronic equipment and a storage medium, and relates to the technical field of computers, in particular to the technical field of artificial intelligence such as cloud computing, cloud storage and big data processing. The specific implementation scheme is as follows: acquiring an edge calculation message acquired by target terminal equipment; determining a target address corresponding to the edge calculation message, wherein the target address corresponds to a server in the stream processing platform; performing protocol conversion on the edge calculation message according to a data transmission protocol of the stream processing platform; and the edge calculation message after the protocol conversion is synchronized to the server corresponding to the target address by adopting a data transmission protocol, so that the edge calculation message can be uploaded to an external stream processing platform, the application scene of the edge calculation message synchronization is expanded, and the personalized use requirement of a user is met.

Description

Synchronization method and device for edge calculation message, electronic equipment and storage medium
Technical Field
The application relates to the technical field of computers, in particular to the technical field of artificial intelligence such as cloud computing, cloud storage and big data processing, and particularly relates to a synchronization method and device for edge computing messages, electronic equipment and a storage medium.
Background
Artificial intelligence is the subject of research that makes computers simulate some human mental processes and intelligent behaviors (such as learning, reasoning, thinking, planning, etc.), both at the hardware level and at the software level. Artificial intelligence hardware technologies generally include technologies such as sensors, dedicated artificial intelligence chips, cloud computing, distributed storage, big data processing, and the like; the artificial intelligence software technology mainly comprises a computer vision technology, a voice recognition technology, a natural language processing technology, machine learning/deep learning, a big data processing technology, a knowledge map technology and the like.
In the scenes of internet of things and edge computing, a device with an edge computing function usually acquires a large number of device parameters and running state information of the device, and these messages may be referred to as edge computing messages, and the edge computing messages are usually synchronized to a cloud server through Message Queue Telemetry Transport (MQTT).
Disclosure of Invention
A synchronization method, an apparatus, an electronic device, a storage medium, and a computer program product for edge calculation messages are provided.
According to a first aspect, there is provided a method for synchronizing edge calculation messages, comprising: acquiring an edge calculation message acquired by target terminal equipment; determining a target address corresponding to the edge computing message, the target address corresponding to a server in a stream processing platform; performing protocol conversion on the edge calculation message according to a data transmission protocol of the stream processing platform; and synchronizing the edge calculation message after protocol conversion to a server corresponding to the target address by adopting the data transmission protocol.
According to a second aspect, there is provided a synchronization apparatus for edge calculation messages, comprising: the first acquisition module is used for acquiring the edge calculation message acquired by the target terminal equipment; a determining module, configured to determine a target address corresponding to the edge computing message, where the target address corresponds to a server in a stream processing platform; the processing module is used for carrying out protocol conversion on the edge calculation message according to a data transmission protocol of the stream processing platform; and the transmission module is used for synchronizing the edge calculation message after protocol conversion to the server corresponding to the target address by adopting the data transmission protocol.
According to a third aspect, there is provided an electronic device comprising: at least one processor; and a memory communicatively coupled to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute the method for synchronizing edge calculation messages according to the embodiment of the present application.
According to a fourth aspect, a non-transitory computer-readable storage medium is proposed, having stored thereon computer instructions for causing the computer to perform the method for synchronizing edge calculation messages disclosed in embodiments of the present application.
According to a fifth aspect, a computer program product is proposed, comprising a computer program which, when executed by a processor, implements the synchronization method of edge computation messages disclosed in embodiments of the present application.
It should be understood that the statements in this section do not necessarily identify key or critical features of the embodiments of the present application, nor do they limit the scope of the present application. Other features of the present application will become apparent from the following description.
Drawings
The drawings are included to provide a better understanding of the present solution and are not to be considered limiting of the present application. Wherein:
FIG. 1 is a schematic diagram according to a first embodiment of the present application;
FIG. 2 is a schematic diagram according to a second embodiment of the present application;
FIG. 3 is a schematic diagram of an architecture of a synchronization apparatus for edge computing messages according to an embodiment of the present application;
FIG. 4 is a schematic illustration according to a third embodiment of the present application;
FIG. 5 is a schematic illustration according to a fourth embodiment of the present application;
fig. 6 is a block diagram of an electronic device for implementing a synchronization method of edge calculation messages according to an embodiment of the present application.
Detailed Description
The following description of the exemplary embodiments of the present application, taken in conjunction with the accompanying drawings, includes various details of the embodiments of the application for the understanding of the same, which are to be considered exemplary only. Accordingly, those of ordinary skill in the art will recognize that various changes and modifications of the embodiments described herein can be made without departing from the scope and spirit of the present application. Also, descriptions of well-known functions and constructions are omitted in the following description for clarity and conciseness.
Fig. 1 is a schematic diagram according to a first embodiment of the present application.
It should be noted that an execution subject of the synchronization method for edge calculation messages in this embodiment is a synchronization apparatus for edge calculation messages, and the apparatus may be implemented in software and/or hardware, and the apparatus may be configured in an electronic device, and the electronic device may include, but is not limited to, a terminal, a server, and the like.
The embodiment of the application relates to the technical field of artificial intelligence such as cloud computing, cloud storage and big data processing.
Wherein, artificial Intelligence (Artificial Intelligence), english is abbreviated as AI. The method is a new technical science for researching and developing theories, methods, technologies and application systems for simulating, extending and expanding human intelligence.
Cloud computing (cloud computing) refers to a technology architecture that accesses a flexibly extensible shared physical or virtual resource pool through a network, where resources may include servers, operating systems, networks, software, applications, storage devices, and the like, and may be deployed and managed in an on-demand, self-service manner. Through the cloud computing technology, high-efficiency and strong data processing capacity can be provided for technical application such as artificial intelligence and block chains and model training.
Cloud storage is a mode of online storage on the internet, that is, data is stored in a plurality of virtual servers usually hosted by third parties, rather than on dedicated servers.
The big data processing refers to a process of analyzing and processing large-scale data in an artificial intelligence mode, and the big data can be summarized into 5V, and has large data Volume (Volume), high speed (Velocity), multiple types (Velocity), value (Value) and authenticity (Veracity).
As shown in fig. 1, the method for synchronizing the edge calculation messages includes:
s101: and acquiring the edge calculation message acquired by the target terminal equipment.
The terminal device that needs to perform synchronization processing on the edge calculation message collected by the terminal device may be referred to as a target terminal device, and the target terminal device may be a device having an edge calculation function in a scene of an internet of things and edge calculation, such as a computer device having an edge calculation function.
The edge calculation message, such as the device parameters acquired by the device having the edge calculation function and the running state information of the device, is not limited.
The stream processing platform in the embodiment of the present application may be, for example, an open source stream processing platform Kafka, or may also be other big data computing and storing platforms having a stream data processing function, which is not limited in this respect.
In the embodiment of the present application, the stream processing platform is an open source stream processing platform Kafka.
In some embodiments, after obtaining the edge calculation message acquired by the target terminal device, the method may further perform encryption processing on the edge calculation message after the protocol conversion according to the encryption certificate, and then synchronize the edge calculation message after the protocol conversion to a server corresponding to the target address, including: and synchronizing the encrypted edge calculation message to a server corresponding to the target address.
The encryption certificate is an encryption certificate required by the connection stream processing platform, and the encryption certificate may include some encryption algorithms, so that when the edge calculation message after the protocol conversion is encrypted according to the encryption certificate, the edge calculation message after the protocol conversion may be encrypted by using the encryption algorithm, and when the edge calculation message is synchronized subsequently, the edge calculation message after the protocol conversion may be specifically subjected to synchronous encryption processing.
Because the edge calculation message after the protocol conversion is encrypted in advance according to the encryption certificate, the transmission safety of the edge calculation message can be effectively ensured, the content leakage of the message is avoided, and the safety performance of the synchronization of the edge calculation message is improved.
S102: a destination address corresponding to the edge computation message is determined, the destination address corresponding to a server in the stream processing platform.
The above-mentioned opening source flow processing platform Kafka may be a cluster server architecture, and different server nodes in the cluster server correspond to one address, where the address corresponding to the server node which is subscribed in advance for processing the edge computation message of the target terminal device may be referred to as a target address.
The server node which is subscribed in advance and used for processing the edge calculation message of the target terminal device may be understood as an internal data storage structure, and is consistent with the data storage structure of the edge calculation message of the target terminal device, and may be a server node which is subscribed in advance based on a required data storage structure, so that after the edge calculation message acquired by the target terminal device is acquired, a target address corresponding to the edge calculation message is determined directly according to a pre-subscribed relationship, and the target address corresponds to a server in the stream processing platform, which is not limited thereto.
In some embodiments, after determining the destination address corresponding to the edge computing message, the encryption certificate required for connecting to the stream processing platform may also be obtained; according to the encrypted certificate, checking the connectivity of the target address; synchronizing the edge calculation message after the protocol conversion to the server corresponding to the target address, or synchronizing the edge calculation message after the protocol conversion to the server corresponding to the target address after determining that the connectivity check passes.
That is to say, the embodiments of the present application provide a method for verifying connectivity of a target address, where before transmitting an edge calculation message to the target address, connectivity of the target address is first verified, and after the connectivity is verified, synchronization is performed on the edge calculation message, and if the connectivity is not verified, some reconnection methods may be adopted to update an unavailable target address, so as to effectively ensure availability of the target address and effectively reduce the probability that the target address is unavailable after a transmission rule is created.
S103: and carrying out protocol conversion on the edge calculation message according to a data transmission protocol of the stream processing platform.
After the edge calculation message acquired by the target terminal device is acquired and the target address corresponding to the edge calculation message is determined, the edge calculation message may be subjected to protocol conversion according to a data transmission protocol of the stream processing platform.
For example, if the open source stream processing platform Kafka supports one data transmission protocol, the protocol of the initial format of the edge calculation message may be converted into the data transmission protocol supported by the open source stream processing platform Kafka, and then the subsequent steps may be triggered.
S104: and synchronizing the edge calculation message after the protocol conversion to a server corresponding to the target address by adopting a data transmission protocol.
After the protocol of the initial format of the edge calculation message is converted into the data transmission protocol supported by the open source stream processing platform Kafka, the data transmission protocol supported by the open source stream processing platform Kafka can be directly adopted to synchronize the edge calculation message after the protocol conversion to the server corresponding to the target address, so that the edge calculation message is uploaded to an external stream processing platform, the application scene of the synchronization of the edge calculation message is expanded, and the personalized use requirement of a user is met.
In this embodiment, the edge calculation message acquired by the target terminal device is acquired, the target address corresponding to the edge calculation message is determined, the target address corresponds to the server in the stream processing platform, the edge calculation message is subjected to protocol conversion according to the data transmission protocol of the stream processing platform, and the edge calculation message after the protocol conversion is synchronized to the server corresponding to the target address by using the data transmission protocol, so that the edge calculation message can be uploaded to an external stream processing platform, an application scenario of the edge calculation message synchronization is expanded, and personalized use requirements of users are met.
Fig. 2 is a schematic diagram according to a second embodiment of the present application.
As shown in fig. 2, the method for synchronizing the edge calculation messages includes:
s201: a target terminal device is determined from among a plurality of terminal devices.
S202: a target topic category is determined.
S203: and storing the target subject category and the identification of the target terminal equipment in a correlation manner.
That is, the server nodes may be subscribed in advance based on the required data storage structure, and the topic category corresponding to the data storage structure supported by the subscribed server node is used as the target topic category (the target topic category indicates the data storage structure supported by the server node), and then the target topic category may be the topic category corresponding to the data storage structure of the edge computation message to be synchronized, that is, the data storage structure of the edge computation message to be synchronized is divided into one topic category, and the topic category is used for identifying the data storage structure of the server to which the edge computation message is synchronized, so that the edge computation message and the data storage structure of the server to which the edge computation message needs to be synchronized have a corresponding relationship, and in order to ensure efficient storage of the edge computation message, the edge computation message is generally synchronized into the server nodes corresponding to the topic category.
In this embodiment of the present application, a terminal device that performs an edge calculation function may be determined first and used as a target terminal device, then a data storage structure of an edge calculation message obtained by the target terminal device is determined, a target topic category corresponding to the data storage structure is determined, and then the target topic category and an identifier of the target terminal device are stored in an associated manner.
S204: and acquiring the edge calculation message which belongs to the target subject category and is acquired by the target terminal equipment according to the identification.
The target subject category and the identification of the target terminal device are stored in an associated manner, the initial edge calculation message acquired by the target terminal device can be directly acquired according to the identification, and then the data storage structure corresponding to the target subject category is adopted to adjust the data storage structure of the initial edge calculation message in the message rule engine based on the end side, so that the edge calculation message of the target subject category is formed, the efficient acquisition and storage of the edge calculation message can be effectively guaranteed, and the synchronization performance of the edge calculation message is improved.
S205: a destination address corresponding to the edge computation message is determined, the destination address corresponding to a server in the stream processing platform.
S206: and carrying out protocol conversion on the edge calculation message according to a data transmission protocol of the stream processing platform.
S205 and S206 may specifically refer to the above embodiments, and are not described herein again.
S207: and acquiring the calculation resource consumption value required by the synchronous edge calculation message.
In this embodiment, in order to solve the technical problems of weak hardware capability, limited resources, and the like of the edge computing device, a resource flexible scheduling algorithm may be executed to dynamically expand or reduce the number of instances of the message transmission module according to the value of the consumption of the computing resource, so as to adjust the system resources occupied by the message transmission module as needed and reduce the operating pressure of the edge device.
S208: and dynamically carrying out capacity expansion or capacity reduction processing on the number of the instances of the message transmission module according to the operation resource consumption value, wherein the message transmission module is used for synchronizing the edge calculation message after the protocol conversion to a server corresponding to the target address by adopting the instances.
For example, the computation resource consumption values (CPU usage, memory usage, and data concurrency amount) required by the synchronous edge computation message may be obtained in real time, so as to dynamically perform capacity expansion or capacity reduction Processing on the number of instances of the message transmission module according to the CPU usage, the memory usage, and the data concurrency amount.
Optionally, in some embodiments, the computational resource consumption value may be compared with a set threshold range, where the set threshold range includes a maximum consumption value and a minimum consumption value; if the calculation resource consumption value is smaller than the minimum consumption value, carrying out capacity expansion processing on the number of the instances of the message transmission module; if the calculation resource consumption value is larger than the maximum consumption value, the capacity reduction processing is carried out on the number of the instances of the message transmission module, the realization is simple and convenient, and the resource management effect is better.
The set threshold range may be set in advance according to the transmission performance of the message transmission module, and the set threshold range may be adaptively adjusted, where the set threshold range includes a value range between a maximum consumption value and a minimum consumption value, the maximum consumption value indicates the maximum CPU usage rate, the memory usage rate, the data concurrency amount, and the like that can be borne by the data transmission module, and the minimum consumption value indicates the minimum CPU usage rate, the memory usage rate, the data concurrency amount, and the like that can be borne by the data transmission module.
Since the message transmission module is configured to synchronize the edge calculation message after the protocol conversion to the server corresponding to the target address by using the example, and a plurality of examples are configured in the message transmission module, in this embodiment, it is supported that when the operation resource consumption value is less than the minimum consumption value (indicating that the operation resource consumption value consumed by transmitting the edge calculation message does not excessively occupy resources), capacity expansion processing is performed on the number of examples of the message transmission module, and when the operation resource consumption value is greater than the maximum consumption value (indicating that the operation resource consumption value consumed by transmitting the edge calculation message occupies resources relatively more resources), capacity reduction processing is performed on the number of examples of the message transmission module.
S209: and synchronizing the edge calculation message after protocol conversion to a server corresponding to the target address by adopting a data transmission protocol.
For the description of S209, reference may be made to the foregoing embodiments specifically, and details are not repeated here.
In this embodiment, the edge calculation message acquired by the target terminal device is acquired, the target address corresponding to the edge calculation message is determined, the target address corresponds to a server in the stream processing platform, the edge calculation message is subjected to protocol conversion according to a data transmission protocol of the stream processing platform, and the data transmission protocol is adopted to synchronize the edge calculation message after the protocol conversion to the server corresponding to the target address, so that the edge calculation message can be uploaded to an external stream processing platform, an application scenario of the edge calculation message synchronization is expanded, and personalized use requirements of users are met. The target subject category and the identification of the target terminal equipment are stored in a correlated mode, the initial edge calculation information collected by the target terminal equipment can be directly obtained according to the identification, then, a data storage structure corresponding to the target subject category is adopted, and the data storage structure of the initial edge calculation information in the message rule engine based on the end side is adjusted to form the edge calculation information of the target subject category, so that the efficient collection and storage of the edge calculation information can be effectively guaranteed, and the synchronization performance of the edge calculation information is improved. In order to solve the technical problems of weak hardware capability, limited resources and the like of the edge computing equipment, a resource flexible scheduling algorithm can be executed to dynamically expand or reduce the number of instances of the message transmission module according to the calculation resource consumption value, so that the system resources occupied by the message transmission module can be adjusted as required, and the operating pressure of the edge side equipment is reduced.
A more specific example is given below:
referring to fig. 3, fig. 3 is a schematic diagram of an architecture of a synchronization apparatus for edge computation messages in this embodiment, an example is performed by using a stream processing platform as an open source stream processing platform Kafka, and a hardware architecture of the synchronization apparatus for edge computation messages in this embodiment may include: the system comprises a message bus module, a message rule engine based on an end side, a secure connection module, a Kafka destination configuration module, a protocol conversion module, an external Kafka connectivity test module, an external Kafka data uploading module and a resource flexible scheduling module, so that the synchronization of an edge calculation message to a server corresponding to a target address of the external Kafka in a data encryption mode is supported.
In addition, a data acquisition module may also be configured in the hardware architecture, the data acquisition module acquires the edge calculation message acquired by the target terminal device (the number of the target terminal devices may be one or more), and transmits the edge calculation message to the message rule engine based on the message bus module, and the message rule engine based on the end side triggers the subsequent transmission of the edge calculation message.
The description for the hardware architecture described above may be as follows:
the system comprises a data acquisition module, an acquired Message source, a target subject category sent to a local Message center module, a Message bus module, a butt joint Message acquisition module, a rule engine module, an end-side rule engine module, a safety connection module, an encryption certificate required by supporting configuration connection of an external Kafka, an external Kafka destination configuration module, a protocol conversion module, a Kafka Message uploading module, an external Kafka data uploading module, and a resource elastic scheduling module, wherein the data acquisition module, the acquired Message source and the target subject category sent to the local Message center module can be configured in advance, the butt joint Message acquisition module can simultaneously forward the encryption certificate to the end-side rule engine module, the safety connection module can support configuration of an encryption certificate required by connection of the external Kafka, the external Kafka destination configuration module can support loading of the encryption certificate in the safety connection module to perform external Kafka connectivity test of the Kafka, the protocol conversion module can support conversion of Message Telemetry queue transmission (MQTT) into a Kafka Message supported by an open source flow processing platform Kafka, the external Kafka data uploading module can receive an edge calculation Message sent by the rule engine module, and upload the edge calculation Message to the local Message source flow processing platform to the external Kafka local Message center module, and the resource elastic scheduling module can receive the edge calculation Message bus module to develop the resource elastic scheduling module.
The data collection module may collect Message sources periodically according to configuration, and then send the Message to a contracted topic of the local Message center module, and the Message transmission protocol includes, but is not limited to, a standard Message Queue Telemetry Transport (MQTT) protocol, and the like.
The message bus module is responsible for transmitting the edge calculation message at the end side.
And the rule engine module at the end side is used for subscribing the theme of the local message bus and forwarding the theme to the external Kafka data uploading module.
The Secure connection module supports connection establishment with an external Kafka based on an SSL (Secure Sockets Layer) certificate (encryption certificate), ensures security of connection and data transmission process, and verifies validity of the encryption certificate in advance.
The user adds the external Kafka connection address, grouping information, subject category and the like to the encrypted certificate, and provides information required for connecting the external Kafka.
By the external Kafka connectivity test, a user can obtain the connectivity of the corresponding destination address in time, the availability of the destination address is ensured, and the probability of unavailability of the destination address after the rule is created is reduced.
The protocol conversion module supports the conversion of MQTT messages into messages (such as Kafka messages) corresponding to the data transmission protocol supported by the open source stream processing platform Kafka, and the valid information of the messages includes, but is not limited to, message coding information, message subject, message quality, message load, and the like.
The external Kafka data uploading module receives the message of the rule engine module, calls the message conversion module to convert the MQTT message into a Kafka message, and uploads the Kafka message to the external Kafka.
The above-mentioned flexible resource allocation scheduling module sets the maximum number of instances and the minimum number of instances of the external Kafka data uploading module, and the upper and lower limit thresholds (maximum and minimum values) of the reference index (calculation resource consumption value), so that during operation, the flexible resource scheduling module collects and counts the CPU utilization, memory utilization, and data concurrency number of the external Kafka data uploading module, and if the number exceeds/is lower than the set threshold, performs capacity expansion processing or capacity reduction processing on the number of instances of the flow type calculation engine module (i.e. the external Kafka data uploading module, i.e. the above-mentioned message transmission module), and the minimum number of instances support is set to 0, so as to reduce the consumption of system resources to the greatest extent.
Fig. 4 is a schematic diagram according to a third embodiment of the present application.
As shown in fig. 4, the synchronization apparatus 40 for edge calculation messages includes:
a first obtaining module 401, configured to obtain an edge calculation message collected by a target terminal device;
a determining module 402, configured to determine a target address corresponding to the edge calculation message, where the target address corresponds to a server in the stream processing platform;
a processing module 403, configured to perform protocol conversion on the edge computing message according to a data transmission protocol of the stream processing platform;
a transmission module 404, configured to synchronize the edge calculation message after protocol conversion to a server corresponding to the target address by using a data transmission protocol.
In some embodiments of the present application, as shown in fig. 5, fig. 5 is a schematic diagram according to a fourth embodiment of the present application, where the synchronization apparatus 50 for edge calculation messages, the first obtaining module 501, the determining module 502, the processing module 503, and the transmitting module 504 further include:
a checking module 505, configured to obtain an encryption certificate required by the connection stream processing platform after determining a target address corresponding to the edge calculation message; according to the encrypted certificate, checking the connectivity of the target address;
the transmission module 504 is specifically configured to:
and if the connectivity check is passed, synchronizing the edge calculation message after the protocol conversion to a server corresponding to the target address.
In some embodiments of the present application, as shown in fig. 5, further comprising:
a second obtaining module 506, configured to obtain an operation resource consumption value required for synchronizing the edge calculation message before synchronizing the edge calculation message after the protocol conversion to the server corresponding to the target address;
and a management module 507, configured to dynamically perform capacity expansion or capacity reduction processing on the number of instances of the message transmission module according to the computational resource consumption value, where the message transmission module is configured to synchronize the edge calculation message after protocol conversion to a server corresponding to the target address by using the instances.
In some embodiments of the present application, the management module 507 is specifically configured to:
comparing the calculation resource consumption value with a set threshold range, wherein the set threshold range comprises a maximum consumption value and a minimum consumption value;
if the operation resource consumption value is less than the minimum consumption value, carrying out capacity expansion processing on the number of the instances of the message transmission module;
and if the calculation resource consumption value is larger than the maximum consumption value, carrying out capacity reduction processing on the number of the instances of the message transmission module.
In some embodiments of the present application, as shown in fig. 5, the method further includes:
the encryption module 508 is configured to encrypt the edge calculation message after the protocol conversion according to the encryption certificate after obtaining the edge calculation message acquired by the target terminal device;
the transmission module 504 is specifically configured to:
and synchronizing the encrypted edge calculation message to a server corresponding to the target address.
In some embodiments of the present application, as shown in fig. 5, further comprising:
an association module 509, configured to determine a target terminal device from among the plurality of terminal devices, and determine a target subject category; and storing the target theme category and the identification of the target terminal equipment in a correlation manner;
the first obtaining module 501 is specifically configured to:
and acquiring the edge calculation message which belongs to the target subject category and is acquired by the target terminal equipment according to the identifier.
It is understood that the synchronization apparatus 50 of the edge calculation message in fig. 5 of the present embodiment and the synchronization apparatus 40 of the edge calculation message in the foregoing embodiment, the first obtaining module 501 and the first obtaining module 401 in the foregoing embodiment, the determining module 502 and the determining module 402 in the foregoing embodiment, the processing module 503 and the processing module 403 in the foregoing embodiment, and the transmission module 504 and the transmission module 404 in the foregoing embodiment may have the same functions and structures.
It should be noted that the foregoing explanation of the synchronization method for the edge calculation message is also applicable to the synchronization apparatus for the edge calculation message in this embodiment, and is not described herein again.
In this embodiment, the edge calculation message acquired by the target terminal device is acquired, the target address corresponding to the edge calculation message is determined, the target address corresponds to the server in the stream processing platform, the edge calculation message is subjected to protocol conversion according to the data transmission protocol of the stream processing platform, and the edge calculation message after the protocol conversion is synchronized to the server corresponding to the target address by using the data transmission protocol, so that the edge calculation message can be uploaded to an external stream processing platform, an application scenario of the edge calculation message synchronization is expanded, and personalized use requirements of users are met.
According to embodiments of the present application, an electronic device, a readable storage medium, and a computer program product are also provided.
Fig. 6 is a block diagram of an electronic device for implementing a synchronization method of edge calculation messages according to an embodiment of the present application. Electronic devices are intended to represent various forms of digital computers, such as laptops, desktops, workstations, personal digital assistants, servers, blade servers, mainframes, and other appropriate computers. The electronic device may also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions, are meant to be examples only, and are not meant to limit implementations of the present application that are described and/or claimed herein.
As shown in fig. 6, the apparatus 600 includes a computing unit 601, which can perform various appropriate actions and processes according to a computer program stored in a Read Only Memory (ROM) 602 or a computer program loaded from a storage unit 608 into a Random Access Memory (RAM) 603. In the RAM 603, various programs and data necessary for the operation of the device 600 can also be stored. The calculation unit 601, the ROM 602, and the RAM 603 are connected to each other via a bus 604. An input/output (I/O) interface 605 is also connected to bus 604.
A number of components in the device 600 are connected to the I/O interface 605, including: an input unit 606 such as a keyboard, a mouse, and the like; an output unit 607 such as various types of displays, speakers, and the like; a storage unit 608, such as a magnetic disk, optical disk, or the like; and a communication unit 609 such as a network card, modem, wireless communication transceiver, etc. The communication unit 609 allows the device 600 to exchange information/data with other devices via a computer network such as the internet and/or various telecommunication networks.
The computing unit 601 may be a variety of general and/or special purpose processing components having processing and computing capabilities. Some examples of the computing unit 601 include, but are not limited to, a Central Processing Unit (CPU), a Graphics Processing Unit (GPU), various dedicated Artificial Intelligence (AI) computing chips, various computing units running machine learning model algorithms, a Digital Signal Processor (DSP), and any suitable processor, controller, microcontroller, and so forth. The calculation unit 601 performs the respective methods and processes described above, for example, a synchronization method of edge calculation messages.
For example, in some embodiments, the synchronization method of edge computation messages may be implemented as a computer software program tangibly embodied in a machine-readable medium, such as storage unit 608. In some embodiments, part or all of a computer program may be loaded onto and/or installed onto device 600 via ROM 602 and/or communications unit 609. When the computer program is loaded into the RAM 603 and executed by the computing unit 601, one or more steps of the above described synchronization method of edge computation messages may be performed. Alternatively, in other embodiments, the computation unit 601 may be configured by any other suitable means (e.g., by means of firmware) to perform the synchronization method of the edge computation messages.
Various implementations of the systems and techniques described here above may be implemented in digital electronic circuitry, integrated circuitry, field Programmable Gate Arrays (FPGAs), application Specific Integrated Circuits (ASICs), application Specific Standard Products (ASSPs), system on a chip (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and/or combinations thereof. These various embodiments may include: implemented in one or more computer programs that are executable and/or interpretable on a programmable system including at least one programmable processor, which may be special or general purpose, receiving data and instructions from, and transmitting data and instructions to, a storage system, at least one input device, and at least one output device.
Program code for implementing the synchronization method of edge computing messages of the present application may be written in any combination of one or more programming languages. These program codes may be provided to a processor or controller of a general purpose computer, special purpose computer, or other programmable data processing apparatus, such that the program codes, when executed by the processor or controller, cause the functions/operations specified in the flowchart and/or block diagram to be performed. The program code may execute entirely on the machine, partly on the machine, as a stand-alone software package partly on the machine and partly on a remote machine or entirely on the remote machine or server.
In the context of this application, a machine-readable medium may be a tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. The machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples of a machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.
To provide for interaction with a user, the systems and techniques described here can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to a user; and a keyboard and a pointing device (e.g., a mouse or a trackball) by which a user can provide input to the computer. Other kinds of devices may also be used to provide for interaction with a user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form, including acoustic, speech, or tactile input.
The systems and techniques described here can be implemented in a computing system that includes a back-end component (e.g., as a data server), or that includes a middleware component (e.g., an application server), or that includes a front-end component (e.g., a user computer having a graphical user interface or a web browser through which a user can interact with an implementation of the systems and techniques described here), or any combination of such back-end, middleware, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: local Area Networks (LANs), wide Area Networks (WANs), the Internet, and blockchain networks.
The computer system may include clients and servers. A client and server are generally remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other. The Server can be a cloud Server, also called a cloud computing Server or a cloud host, and is a host product in a cloud computing service system, so as to solve the defects of high management difficulty and weak service expansibility in the traditional physical host and VPS service ("Virtual Private Server", or simply "VPS"). The server may also be a server of a distributed system, or a server incorporating a blockchain.
It should be understood that various forms of the flows shown above may be used, with steps reordered, added, or deleted. For example, the steps described in the present disclosure may be executed in parallel, sequentially, or in different orders, and are not limited herein as long as the desired results of the technical solutions disclosed in the present disclosure can be achieved.
The above-described embodiments should not be construed as limiting the scope of the present application. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions may be made, depending on design requirements and other factors. Any modification, equivalent replacement, and improvement made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims (12)

1. A method of synchronizing edge computation messages, comprising:
acquiring an edge calculation message acquired by target terminal equipment;
determining a target address corresponding to the edge calculation message, wherein the target address corresponds to a server in a stream processing platform, and the target address is an address corresponding to a server node which is subscribed in advance and used for processing the edge calculation message;
performing protocol conversion on the edge calculation message according to a data transmission protocol of the stream processing platform; and
adopting the data transmission protocol to synchronize the edge calculation message after protocol conversion to a server corresponding to the target address;
before the synchronizing the protocol-converted edge computation message to the server corresponding to the target address, the method includes:
acquiring a calculation resource consumption value required by synchronizing the edge calculation message;
and dynamically performing capacity expansion or capacity reduction on the number of instances of the message transmission module according to the operation resource consumption value, wherein the operation resource consumption value is compared with a set threshold range, and the capacity expansion or capacity reduction is determined to be performed on the number of instances of the message transmission module according to a comparison result, and the message transmission module is used for synchronizing the edge calculation message after protocol conversion to a server corresponding to the target address by adopting the instances.
2. The method of claim 1, further comprising, after the determining a destination address corresponding to the edge computation message:
acquiring an encryption certificate required by connecting the stream processing platform;
according to the encrypted certificate, checking the connectivity of the target address;
wherein the synchronizing the edge calculation message after the protocol conversion to the server corresponding to the target address includes:
and if the connectivity check is passed, synchronizing the edge calculation message after the protocol conversion to a server corresponding to the target address.
3. The method according to claim 1, wherein the dynamically expanding or contracting the number of instances of the message transmission module according to the value of the computational resource consumption comprises:
comparing the calculation resource consumption value with a set threshold range, wherein the set threshold range comprises a maximum consumption value and a minimum consumption value;
if the operation resource consumption value is smaller than the minimum consumption value, carrying out capacity expansion processing on the number of the instances of the message transmission module;
and if the calculation resource consumption value is larger than the maximum consumption value, carrying out capacity reduction processing on the number of the instances of the message transmission module.
4. The method according to claim 2, wherein after the obtaining the edge calculation message collected by the target terminal device, further comprising:
according to the encryption certificate, carrying out encryption processing on the edge calculation message after the protocol conversion;
wherein the synchronizing the edge calculation message after the protocol conversion to the server corresponding to the target address includes:
and synchronizing the encrypted edge calculation message to a server corresponding to the target address.
5. The method according to claim 1, further comprising, before the obtaining the edge calculation message collected by the target terminal device:
determining a target terminal device from among a plurality of terminal devices;
determining a target topic category; and
storing the target subject category and the identification of the target terminal equipment in a correlation manner;
the acquiring of the edge calculation message collected by the target terminal device includes:
and acquiring the edge calculation message which is acquired by the target terminal equipment and belongs to the target subject category according to the identification.
6. A synchronization apparatus of an edge calculation message, comprising:
the first acquisition module is used for acquiring the edge calculation message acquired by the target terminal equipment;
a determining module, configured to determine a target address corresponding to the edge computing message, where the target address corresponds to a server in a stream processing platform, and the target address is an address corresponding to a server node that is subscribed in advance and used for processing the edge computing message;
the processing module is used for carrying out protocol conversion on the edge calculation message according to a data transmission protocol of the stream processing platform; and
the transmission module is used for synchronizing the edge calculation message after protocol conversion to the server corresponding to the target address by adopting the data transmission protocol;
a second obtaining module, configured to obtain a computation resource consumption value required for synchronizing the edge calculation message before synchronizing the edge calculation message after the protocol conversion to the server corresponding to the target address;
and the management module is used for dynamically carrying out capacity expansion or capacity reduction on the number of instances of the message transmission module according to the operation resource consumption value, comparing the operation resource consumption value with a set threshold range, and determining to carry out capacity expansion or capacity reduction on the number of instances of the message transmission module according to a comparison result, and the message transmission module is used for synchronizing the edge calculation message after protocol conversion to a server corresponding to the target address by adopting the instances.
7. The apparatus of claim 6, further comprising:
the verification module is used for acquiring an encryption certificate required by connecting the stream processing platform after the target address corresponding to the edge calculation message is determined; according to the encrypted certificate, checking the connectivity of the target address;
wherein, the transmission module is specifically configured to:
and if the connectivity check is passed, synchronizing the edge calculation message after the protocol conversion to a server corresponding to the target address.
8. The apparatus according to claim 6, wherein the management module is specifically configured to:
comparing the computing resource consumption value with a set threshold range, wherein the set threshold range comprises a maximum consumption value and a minimum consumption value;
if the operation resource consumption value is smaller than the minimum consumption value, carrying out capacity expansion processing on the number of the instances of the message transmission module;
and if the calculation resource consumption value is larger than the maximum consumption value, carrying out capacity reduction processing on the number of the instances of the message transmission module.
9. The apparatus of claim 7, further comprising:
the encryption module is used for encrypting the edge calculation message after the protocol conversion according to the encryption certificate after the edge calculation message acquired by the target terminal equipment is acquired;
wherein, the transmission module is specifically configured to:
and synchronizing the encrypted edge calculation message to a server corresponding to the target address.
10. The apparatus of claim 6, further comprising:
the association module is used for determining target terminal equipment from a plurality of terminal equipment and determining a target subject category; and storing the target subject category in association with the identifier of the target terminal device;
the first obtaining module is specifically configured to:
and acquiring the edge calculation message which belongs to the target subject category and is acquired by the target terminal equipment according to the identification.
11. An electronic device, comprising:
at least one processor; and
a memory communicatively coupled to the at least one processor; wherein,
the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-5.
12. A non-transitory computer readable storage medium having stored thereon computer instructions for causing the computer to perform the method of any one of claims 1-5.
CN202011520048.7A 2020-12-21 2020-12-21 Synchronization method and device of edge calculation message, electronic equipment and storage medium Active CN112769897B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202011520048.7A CN112769897B (en) 2020-12-21 2020-12-21 Synchronization method and device of edge calculation message, electronic equipment and storage medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202011520048.7A CN112769897B (en) 2020-12-21 2020-12-21 Synchronization method and device of edge calculation message, electronic equipment and storage medium

Publications (2)

Publication Number Publication Date
CN112769897A CN112769897A (en) 2021-05-07
CN112769897B true CN112769897B (en) 2023-04-18

Family

ID=75695164

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202011520048.7A Active CN112769897B (en) 2020-12-21 2020-12-21 Synchronization method and device of edge calculation message, electronic equipment and storage medium

Country Status (1)

Country Link
CN (1) CN112769897B (en)

Families Citing this family (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113420048A (en) * 2021-05-19 2021-09-21 中交公规土木大数据信息技术(北京)有限公司 Data aggregation method and device, electronic equipment and storage medium
CN113423088A (en) * 2021-06-18 2021-09-21 安徽星云互联科技有限责任公司 Information distribution method, device, equipment, system and storage medium
CN113438318B (en) * 2021-07-13 2023-04-07 阿波罗智联(北京)科技有限公司 Performance test system and method of cloud control platform, electronic equipment and storage medium
CN113515237B (en) * 2021-07-22 2024-09-20 中移(杭州)信息技术有限公司 Edge node capacity expansion method and device, edge node and program product
CN113630465B (en) * 2021-08-10 2024-03-12 北京百度网讯科技有限公司 Message processing method, device, equipment and storage medium
CN113783954B (en) * 2021-09-07 2024-10-18 中控创新(北京)能源技术有限公司 Oil and gas pipeline component data transmission system and method
CN114726871B (en) * 2022-03-22 2024-06-11 阿里巴巴(中国)有限公司 Data synchronization method, device and network system
CN115297093B (en) * 2022-08-02 2024-06-25 北京至为恒通企业管理有限公司 Edge device data access method, related device and computer program product
CN116016716B (en) * 2022-12-08 2024-10-11 江苏开拓信息与系统有限公司 Virtual intelligent box processing system and processing method for edge computing of Internet of things
CN116737482A (en) * 2023-08-10 2023-09-12 上海孤波科技有限公司 Method and device for collecting chip test data in real time and electronic equipment

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110430281A (en) * 2019-08-15 2019-11-08 深圳市般若大数据技术有限公司 Method, apparatus, equipment and the storage medium of public cloud acquisition private clound data
CN110460521A (en) * 2019-09-19 2019-11-15 北京中电普华信息技术有限公司 A kind of edge calculations AnyRouter
WO2020216038A1 (en) * 2019-04-22 2020-10-29 华为技术有限公司 Data management method, lepm, and mepm

Family Cites Families (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US10659526B2 (en) * 2018-01-03 2020-05-19 Verizon Patent And Licensing Inc. Edge compute systems and methods
CN111800282B (en) * 2019-04-08 2023-03-28 阿里巴巴集团控股有限公司 Network system, instance management and control method, device and storage medium
CN110266776A (en) * 2019-06-05 2019-09-20 广州云华智慧科技有限公司 Cloud layer communication system
CN110545307B (en) * 2019-07-19 2022-09-27 中移(杭州)信息技术有限公司 Edge computing platform, calling method and computer readable storage medium
CN110557387A (en) * 2019-08-29 2019-12-10 浙江大搜车软件技术有限公司 cross-network equipment communication method, device, system, server and readable storage medium
CN110769039B (en) * 2019-10-09 2021-12-10 腾讯科技(深圳)有限公司 Resource scheduling method and device, electronic equipment and computer readable storage medium
CN111314339B (en) * 2020-02-12 2021-09-10 腾讯科技(深圳)有限公司 Data transmission method and device

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2020216038A1 (en) * 2019-04-22 2020-10-29 华为技术有限公司 Data management method, lepm, and mepm
CN110430281A (en) * 2019-08-15 2019-11-08 深圳市般若大数据技术有限公司 Method, apparatus, equipment and the storage medium of public cloud acquisition private clound data
CN110460521A (en) * 2019-09-19 2019-11-15 北京中电普华信息技术有限公司 A kind of edge calculations AnyRouter

Non-Patent Citations (4)

* Cited by examiner, † Cited by third party
Title
.Delivering Functions over Networks: Traffic and Performance Optimization for Edge Computing using ALTO.IETF .2020,全文. *
S. Yang ; L. Cui ; Shenzhen University ; M. Xu ; Tsinghua University ; Y. Yang ; Tongji/Yale ; R. Huang ; Research Institute of Tsinghua University in Shenzhen *
王旭亮 ; 武宇亭 ; 任宏丹 ; .基于5G+MEC的数字化PaaS赋能平台在智慧安防的应用研究.电子技术应用.2020,(第06期),全文. *
陆璐 ; .面向5G网络的边缘计算技术研究.信息通信技术与政策.2018,(第11期),全文. *

Also Published As

Publication number Publication date
CN112769897A (en) 2021-05-07

Similar Documents

Publication Publication Date Title
CN112769897B (en) Synchronization method and device of edge calculation message, electronic equipment and storage medium
CN110839084B (en) Session management method, device, equipment and medium
US9641595B2 (en) System management apparatus, system management method, and storage medium
CN113724398A (en) Augmented reality method, apparatus, device and storage medium
CN114253710A (en) Processing method of computing request, intelligent terminal, cloud server, equipment and medium
CN114374703A (en) Method, device and equipment for acquiring cloud mobile phone information and storage medium
CN113961289A (en) Data processing method, device, equipment and storage medium
US10250515B2 (en) Method and device for forwarding data messages
CN115567602A (en) CDN node back-to-source method, device and computer readable storage medium
CN115269145A (en) High-energy-efficiency heterogeneous multi-core scheduling method and device for offshore unmanned equipment
CN113778645A (en) Task scheduling method, device and equipment based on edge calculation and storage medium
CN114553548A (en) Communication method, device, equipment and storage medium
CN104503846B (en) A kind of resource management system based on cloud computing system
CN113824689A (en) Edge computing network, data transmission method, device, equipment and storage medium
CN114139605A (en) Distributed model training method, system, device and storage medium
CN114201289A (en) Target detection method and system based on edge computing node and cloud server
CN113887435A (en) Face image processing method, device, equipment, storage medium and program product
CN115914380A (en) Communication delay optimization method of cloud computing resource manager based on ZLib compression algorithm
EP4170974A1 (en) Slice service processing method and apparatus, network device, and readable storage medium
CN112671970B (en) Control method and control device for mobile equipment and cloud mobile phone, electronic equipment, mobile equipment, cloud server and medium
CN115589391B (en) Instant messaging processing method, device and equipment based on block chain and storage medium
CN116662276B (en) Data processing method, device, electronic equipment and storage medium
CN117221127A (en) Method and device for using artificial intelligence service at communication network end side
CN116614367A (en) Grouping method and model updating method of terminal equipment
CN116382852A (en) Micro-service deployment method and device, electronic equipment and medium

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant