KR102127652B1 - Publish and subscribe communication system and method thereof - Google Patents

Abstract

A method of operating a publish and subscribe communication system according to an aspect of the inventive concept includes receiving a state vector for a state of a subscriber; Adjusting the waiting period to adjust the waiting cycle of the loop operation of the receiver based on the input state vector si and a reward function for adjusting the waiting period of the receiver Generating a value; And passing the generated waiting period adjustment value to the subscriber.

Description

PUBLISH AND SUBSCRIBE COMMUNICATION SYSTEM AND METHOD THEREOF}

The technical idea of the present invention relates to a publishing and subscribing communication system and a method for operating the same, and more particularly, to a publishing and subscribing communication system for adjusting a subscription cycle and a method for operating the same.

The publish-subscribe model is an asynchronous messaging paradigm.

In the publish-subscribe model, the message of the publisher is not specified for a specific subscriber.

The issued message is delivered to subscribers who have subscribed to each category according to a predetermined category.

The recipient can only receive the desired message without having any knowledge of the publisher.

The decoupling of these publishers and receivers allows for a more dynamic network topology and high scalability.

In a future network environment such as an edge network different from a network environment in which a predetermined number or a small number of clients exist, a dynamic environment in which a large number of clients exist is created.

The communication structure according to the publish-subscribe model is recognized as an effective protocol as an effective protocol in the Internet of Things (IoT) environment, but the workload increases when the number of recipients maintaining a connection to a broker increases.

In addition, in the process of the receiver maintaining a connection by performing a loop operation, unnecessary resources are consumed if a message for the same topic is not published to the broker.

The publishing and subscribing communication system and its operating method according to the technical idea of the present invention have an object to optimally control a waiting time of a receiver.

In addition, the present invention has an object to adjust the waiting period of the receiver using a deep neural network.

In addition, the present invention has an object to define a receiver's environment as a state vector and adjust a waiting time by applying a compensation function.

An object of the present invention is to improve the operation efficiency and energy efficiency of the receiver by optimally controlling the waiting time of the receiver.

The technical problem to be achieved by the publishing and subscribing communication system and its operation method according to the technical idea of the present invention is not limited to the above-mentioned task(s), and another task(s) not mentioned is from the following description Will be clearly understood.

A method of operating a publish and subscribe communication system according to an aspect of the inventive concept includes receiving a state vector for a state of a subscriber; Adjusting the waiting period to adjust the waiting cycle of the loop operation of the receiver based on the input state vector si and a reward function for adjusting the waiting period of the receiver Generating a value; And passing the generated waiting period adjustment value to the subscriber.

According to an exemplary embodiment, the state vector (si) is the number of messages (mi) corresponding to the topic (topic) specified by the recipient, the number of messages (fi) of high importance among the messages corresponding to the topic, and It may include a waiting time (t _i ) of the important message corresponding to the topic.

According to an exemplary embodiment, the compensation function is the following equation derived from the state vector

Can be

According to an exemplary embodiment, the generating of the waiting period adjustment value may include inputting the input state vector into a deep neural network, and determining the waiting period adjusting operation of the receiver by applying the compensation function, And calculating the waiting period value using the compensation function.

According to an exemplary embodiment, the deep neural network may be applied with at least one reinforcement learning method of a Q-learning method and a policy gradient method.

According to an exemplary embodiment, the step of delivering the generated waiting period adjustment value to the receiver may include transmitting the message when the receiver subscribes, and setting the next period of the receiver.

A broker included in a publishing and subscription communication system according to another aspect according to the technical spirit of the present invention includes at least one processor; And a memory electrically connected to the processor, wherein the memory, when the processor is running, receives a state vector for a state of a subscriber, and inputs the state vector (s _i ) and the recipient Based on a reward function for adjusting a waiting period, a waiting period adjustment value for adjusting a waiting period of a loop operation of the receiver is generated, and the generated waiting period adjustment value is generated. It is possible to store instructions to deliver the to the subscriber (subscriber).

According to an exemplary embodiment, the status vector (s _i ) is the number of messages (m _i ) corresponding to the topic (topic) designated by the recipient, and the number of messages with high importance (f _i) among the messages corresponding to the topic ), and the waiting time t _i of the important message corresponding to the topic.

According to an exemplary embodiment, the compensation function is the following equation derived from the state vector

Can be

According to an exemplary embodiment, the memory, when the processor is running, inputs the input state vector into a deep neural network, applies the compensation function to determine an operation of adjusting the waiting period of the receiver, and the compensation function Instructions for calculating the waiting period value may be stored by using.

According to an exemplary embodiment, the memory, when the processor is running, to apply at least one reinforcement learning method of the Q-learning (Q-learning) method and the policy gradient (Policy Gradient) method to the deep neural network You can save your instructions.

According to an exemplary embodiment, the memory may store instructions that, when executed by the processor, transmit the waiting period adjustment value when the receiver subscribes, and set for the next period of the receiver.

The publication and subscription communication system according to embodiments of the present invention and the method of operation thereof can optimally control the waiting time of a receiver.

The present invention can control the waiting period of the receiver by using the deep neural network.

In addition, the present invention can adjust the waiting time by defining the environment of the receiver as a state vector and applying a compensation function.

In addition, the present invention can optimally control the waiting time of the receiver, thereby improving the operational efficiency and energy efficiency of the receiver.

A brief description of each drawing is provided to better understand the drawings cited herein.
1 is a conceptual diagram of a publication and subscription communication system according to various embodiments of the present invention.
2 is a conceptual diagram of a publishing and subscribing communication system including a reinforcement learning agent according to various embodiments of the present invention.
3 is a flowchart of an operation for determining a waiting period adjustment according to various embodiments of the present invention.
4 is a flowchart of a standby period adjustment operation according to various embodiments of the present invention.
5 is an exemplary diagram of an adjusted cycle according to various embodiments of the present invention.
6 is a graph of experimental results according to an embodiment of the present invention.

The technical idea of the present invention can be applied to various changes and can have various embodiments, and specific embodiments will be illustrated in the drawings and described in detail through detailed description. However, this is not intended to limit the technical spirit of the present invention to specific embodiments, and should be understood to include all modifications, equivalents, and substitutes included in the scope of the technical spirit of the present invention.

In describing the technical spirit of the present invention, when it is determined that a detailed description of related known technologies may unnecessarily obscure the subject matter of the technical spirit of the present invention, the detailed description will be omitted. In addition, the numbers (for example, first, second, etc.) used in the description process of this specification are only identification symbols for distinguishing one component from other components.

In addition, in this specification, when one component is referred to as "connected" or "connected" with another component, the one component may be directly connected to the other component, or may be directly connected, but in particular It should be understood that, as long as there is no objection to the contrary, it may or may be connected via another component in the middle.

In addition, terms such as “~ unit”, “~ group”, “~ ruler”, and “~ module” described in the present specification mean a unit that processes at least one function or operation, which is a processor or microprocessor. Processor (Micro Processor), Application Processor (Application Processor), Micro Controller (Micro Controller), CPU (Central Processing Unit), GPU (Graphics Processing Unit), APU (Accelerate Processor Unit), DSP (Digital Signal Processor), ASIC ( It may be implemented by hardware or software such as an application specific integrated circuit (FPGA), a field programmable gate array (FPGA), or a combination of hardware and software.

In addition, it is intended to clarify that the division of the constituent parts in this specification is only classified according to main functions of each constituent part. That is, two or more components to be described below may be combined into one component, or one component may be divided into two or more for each subdivided function. In addition, each of the constituent parts to be described below may additionally perform some or all of the functions of other constituent parts in addition to the main functions of the constituent parts, and some of the main functions of the constituent parts are different. Needless to say, it may also be carried out exclusively by.

The present invention proposes a publish/subscribe structure based on reinforcement learning that is optimized for an event-driven traffic generation environment in a publish/subscribe communication structure of the MQTT protocol.

The present invention ensures efficient resource utilization of subscribers in the IoT environment through reinforcement learning in the existing loop structure.

In the present invention, the subscriber registers a topic and the broker saves the topic, and then the receiver accesses the broker at regular time intervals, and the receiver receives messages of topics of interest at a time. The purpose is to optimally control the waiting time.

In order to achieve this object, the present invention may define a receiver's state vector, and input a state into a deep neural network to adjust a reference value of latency. The present invention can determine the direction of the standby time adjustment by applying a reward function to the standby time adjustment.

Meanwhile, the present invention can be applied to various communication protocols of the publish-subscribe model other than the MQTT protocol. However, in the following description, for ease of explanation, the MQTT protocol will be described as an example.

The MQTT protocol enables efficient communication in a machine-to-machine (M2M) and Internet of Things environment using a publish/subscribe communication structure.

The MQTT protocol can communicate even in low power and low bandwidth environments.

The MQTT protocol has a process of connecting and disconnecting between the MQTT client and the MQTT broker. After the connection is established, it is basically subscribed, unsubscribed, and published. Connections and messages can be sent and received.

Hereinafter, embodiments according to the technical spirit of the present invention will be described in detail.

1 is a conceptual diagram of a publication and subscription communication system according to various embodiments of the present invention.

Referring to FIG. 1, a publishing and subscribing communication system may include a publisher (publisher, 100), a broker (broker, 200), and a receiver (subscriber, 300). For example, the publication and subscription communication system may be composed of one or more publishers 100 and one or more recipients 300 and brokers 200.

The issuer 100 can send a specific message, and the receiver 200 can receive the message. For example, the receiver 200 may receive a message for a subscribed topic.

In one embodiment, the issuer 100 may be a sensor, and the receiver 200 may be an actuator.

The receiver 300 may perform an internal loop operation continuously, maintain a connection, and subscribe to the broker 200 with a topic name to be received.

The broker 200 may store an issuance message transmitted by at least one issuer 100 based on a topic name written in the issuance message.

The receiver 300 may continuously search for the topic name of the publication message stored in the broker 200 through the loop operation, and when the publication message having the same topic name as the topic name to be received is confirmed, the confirmed publication message It may request the broker 200 to publish to the receiver (300).

The broker 200 may issue a message of the issuer 100 to the receiver 300 having the same topic, at the request of the receiver 300.

Such a publication and subscription communication system uses a topic that is a small sized identifier that can represent a message in an Internet of Things environment in which a certain number of clients exist and the message is sent/received for a limited purpose. By recognizing and transmitting and receiving messages, it can be used as a lightweight protocol than other protocols.

In order to adjust the waiting period of the receiver 300 according to the present invention, a reinforcement learning agent 400 may be included.

The reinforcement learning agent 400 will be described with reference to FIG. 2.

2 is a conceptual diagram of a publishing and subscribing communication system including a reinforcement learning agent according to various embodiments of the present invention.

Referring to FIG. 2, the present invention may include a reinforcement learning agent 400 for applying reinforcement learning, and the reinforcement learning agent 400 may be located on the broker 200.

The reinforcement learning agent 400 may 1) mount a deep network that has been previously learned, and 2) learn in real time from messages generated in real time from the receiver 300.

The reinforcement learning agent 400 may be connected to a waiting cycle adjustment module in the broker 200, input the current state of the receiver 300 into the deep neural network, and transmit the adjusted waiting cycle according to the input.

On the other hand, the present invention may determine whether the adjustment of the waiting period of the receiver 300 is necessary, and if it is determined that the adjustment is necessary, it is possible to reflect the waiting period adjustment value to be described later.

With reference to FIG. 3, the determination of the waiting period adjustment according to various embodiments of the present invention and the operation according thereto will be described.

3 is a flowchart of an operation for determining a waiting period adjustment according to various embodiments of the present invention.

The receiver 300 may register a topic to be subscribed (S110). The broker 200 may store topics registered by the recipient in a registry (S130). The broker 200 compares the waiting time of the subscription with the waiting period of the receiver 300 (S150). As a result of comparison, if the waiting period is shorter than the waiting time for the subscription, the receiver 300 may wait for the subscription (S160). In addition, the receiver 300 may receive a message for the registered topic according to the waiting period. When the message 300 is issued, the receiver 300 may perform a waiting period update that reflects the waiting period adjustment value (S170). For example, the receiver 300 may receive a waiting period adjustment value together with a message for a topic registered from the broker 200, and may change the waiting period by reflecting the received waiting period adjustment value. Accordingly, the receiver 300 may operate with a waiting period adjusted for the next period.

On the other hand, in step S150, if the subscription waiting time is longer than the waiting period, the receiver 300 may access the broker 200 according to the current waiting period and receive a message on the registered topic (S190).

As described above, the present invention compares the waiting period of the subscription 300 and the waiting period of the receiver 300 related to the period in which the publisher 100 transmits a message to the broker 200 for a topic, determines whether a waiting period update is necessary, In this case, the waiting period of the receiver 300 may be updated.

Hereinafter, contents of calculating a waiting period adjustment value for updating the waiting period of the receiver 300 according to various embodiments of the present invention will be described.

4 is a flowchart of a standby period adjustment operation according to various embodiments of the present invention.

Referring to FIG. 4, the reinforcement learning agent 400 may receive a state vector (S210).

For example, the reinforcement learning agent 400 on the broker 200 may receive a state vector related to the state of the receiver 300.

In one embodiment, the reinforcement learning agent 400 includes the number of messages (m _i ) corresponding to the topic specified by the receiver 300 (m _i ), the number of messages (f _i ) of high importance among the messages corresponding to the specified topic, and the specified topic. A state vector including the waiting time (t _i ) of an important message corresponding to may be received. In addition, such a state vector may be obtained from state information received from the receiver 300 by the broker 200.

The above-described state vector can be expressed as follows.

S _i = [m _i , f _i , t _i ]

The reinforcement learning agent 400 may input the input state vector into a deep neural network (S230).

The reward function for adjusting the waiting period of the receiver 300 may be derived from the state vector.

The following criteria can be applied to the design criteria of the compensation function.

1) The more messages you receive at once, the better

2) Loss of important messages

3) Damage occurs when important messages are delayed

Accordingly, the present invention can apply the following compensation function.

In the present invention, for example, the reinforcement learning agent 400 may operate the deep neural network based on the state vector and compensation function designed above.

For example, the present invention can be applied to at least one reinforcement learning method of the Q-learning (Q-learning) method and the policy gradient (Policy Gradient) method. Therefore, the present invention can apply both the Q-learning method and the policy gradient method.

In one embodiment, the present invention can operate the deep neural network in the following order according to the Q-learning method.

In another embodiment, the present invention may operate the deep neural network in the following order according to the policy gradient method.

The reinforcement learning agent 400 may determine a waiting period adjustment operation as a result of applying a deep neural network (S250).

For example, the reinforcement learning agent 400 may determine whether to reduce or increase the waiting period of the receiver 300.

Therefore, the determination of the waiting period adjustment operation of the present invention may mean determining the direction of the waiting time adjustment.

The reinforcement learning agent 400 may calculate a waiting period adjustment value using a compensation function (S270).

The reinforcement learning agent 400 may calculate a compensation using a compensation function, and may calculate a waiting period adjustment value based on the calculated compensation.

The calculated waiting period adjustment value may be transmitted by the broker 200 to the receiver 300, and the transmitted waiting period adjustment value may be reflected in the next period of the receiver 300.

Accordingly, as shown in FIG. 5, the waiting cycle of the receiver 300 may be adjusted, and the receiver 300 may access the broker 200 with an adjusted cycle.

Meanwhile, the above-described reinforcement learning agent 400 may be located on the broker 200 or may be a component included in the broker 200.

When the publish/subscribe structure according to the present invention is applied, the receiver 300 can flexibly change the waiting time period through a deep neural network to which a defined state vector and a reward function are applied. And in order to prove that the standby time period adjustment is optimally controlled according to the present invention, the experimental results of measuring the battery consumption of the receiver will be described with reference to FIG. 6.

6 is a graph of experimental results according to an embodiment of the present invention.

Referring to FIG. 6, when the average MQTT operation consumes an average of 1058.05 mW per unit time, when Q-learning and Policy Gradient are applied according to the publish/subscribe structure according to the present invention It can be seen that it consumes 1052.8mW and 1054.2mW, respectively. These results are 5.25mW and 3.85mW power savings, which can be seen as a small difference, but considering the power consumption in the idle state of the device corresponding to the receiver 300, 1049.84mW, The energy consumed for MQTT communication is 8.21 mW in general MQTT communication, and 2.96 mW when the Q-learning method of the present invention is applied, and 4.36 mW when the Policy Gradient method is applied, It can be concluded that communication energy savings of 63.9% and 47.0%, respectively, are possible.

Therefore, as can be seen from the experimental results to which the present invention is applied, according to the present invention, it is possible to ensure efficient resource utilization of the receiver 300, and thus it can be seen that the operation efficiency and energy efficiency of the receiver 300 are improved.

As such, the present invention can optimally control the waiting time of the receiver of the publication and subscription communication system.

Meanwhile, the above-described present invention can be applied to the publish/subscribe communication structure of the MQTT protocol, and can be applied to various protocols according to the publish/subscribe communication structure in addition to the MQTT protocol.

As described above, the technical idea of the present invention has been described in detail with various embodiments, but the technical idea of the present invention is not limited to the above embodiments, and has ordinary knowledge in the art within the scope of the technical idea of the present invention Various modifications and changes are possible.

Claims (12)

In the method of operation of the publishing (Publish) and subscription (subscribe) communication system,
Receiving a state vector for the state of the subscriber;
A waiting period for adjusting a waiting cycle of the loop operation of the receiver based on the input state vector s _i and a reward function for adjusting the waiting period of the receiver Generating an adjustment value; And
And transmitting the generated waiting period adjustment value to the subscriber.
How it works.
According to claim 1,
The state vector s _i is
The number of messages (m _i ) corresponding to the topic designated by the recipient,
The number of messages of high importance (f _i ) among the messages corresponding to the topic, and
Including the waiting time (t _i ) of the important message corresponding to the topic
How it works.
According to claim 2,
The compensation function
The following equation derived from the state vector

How it works.
According to claim 1,
The step of generating the standby period adjustment value is
Inputting the input state vector into a deep neural network;
Determining an operation of adjusting the waiting period of the receiver by applying the compensation function;
Comprising the step of calculating the waiting period adjustment value using the compensation function
How it works.
According to claim 4,
The deep neural network
At least one reinforcement learning method of Q-learning method and Policy Gradient method is applied.
How it works.
According to claim 1,
The step of delivering the generated waiting period adjustment value to the receiver is
It is delivered during the subscription operation of the receiver, and comprising the step of being set for the next cycle of the receiver
How it works.
In a broker included in a publishing and subscription communication system,
At least one processor; And
A memory electrically connected to the processor,
The memory, when the processor is running,
Receive a state vector for the state of the subscriber,
A waiting period for adjusting a waiting cycle of the loop operation of the receiver based on the input state vector s _i and a reward function for adjusting the waiting period of the receiver Create adjustment values,
Storing instructions for passing the generated waiting period adjustment value to the subscriber
broker.
The method of claim 7,
The state vector s _i is
The number of messages (m _i ) corresponding to the topic designated by the recipient,
The number of messages of high importance (f _i ) among the messages corresponding to the topic, and
Including the waiting time (t _i ) of the important message corresponding to the topic
broker.
The method of claim 8,
The compensation function
The following equation derived from the state vector

broker.
The method of claim 7,
The memory, when the processor is running,
Enter the input state vector into the deep neural network,
Applying the compensation function to determine the waiting period adjustment operation of the receiver,
Storing instructions for calculating the waiting period adjustment value using the compensation function
broker.
The method of claim 10,
The memory, when the processor is running,
Storing instructions for applying at least one reinforcement learning method of the Q-learning method and the policy gradient method to the deep neural network
broker.
The method of claim 7,
The memory, when the processor is running,
In the case of the subscription operation of the receiver, the standby cycle adjustment value is transmitted to store instructions to be set for the next cycle of the receiver
broker.

Images