CN114374708A - Intelligent factory data automatic subscription method based on collaborative filtering and MQTT - Google Patents
Intelligent factory data automatic subscription method based on collaborative filtering and MQTT Download PDFInfo
- Publication number
- CN114374708A CN114374708A CN202111482972.5A CN202111482972A CN114374708A CN 114374708 A CN114374708 A CN 114374708A CN 202111482972 A CN202111482972 A CN 202111482972A CN 114374708 A CN114374708 A CN 114374708A
- Authority
- CN
- China
- Prior art keywords
- subscription
- mqtt
- data
- subscriber
- automatic
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 title claims abstract description 52
- 238000001914 filtration Methods 0.000 title claims abstract description 25
- 230000008569 process Effects 0.000 claims abstract description 8
- 239000011159 matrix material Substances 0.000 claims description 32
- 238000004364 calculation method Methods 0.000 claims description 7
- 238000012986 modification Methods 0.000 claims description 3
- 230000004048 modification Effects 0.000 claims description 3
- 238000012163 sequencing technique Methods 0.000 claims description 2
- 230000005540 biological transmission Effects 0.000 abstract description 18
- 238000004422 calculation algorithm Methods 0.000 description 4
- 238000006243 chemical reaction Methods 0.000 description 4
- 238000012216 screening Methods 0.000 description 4
- 230000003068 static effect Effects 0.000 description 4
- 230000008901 benefit Effects 0.000 description 3
- 238000010586 diagram Methods 0.000 description 3
- 230000000694 effects Effects 0.000 description 3
- 238000005516 engineering process Methods 0.000 description 3
- 238000012545 processing Methods 0.000 description 3
- 238000004458 analytical method Methods 0.000 description 2
- 238000004519 manufacturing process Methods 0.000 description 2
- 230000002159 abnormal effect Effects 0.000 description 1
- 238000013473 artificial intelligence Methods 0.000 description 1
- 238000013528 artificial neural network Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000004891 communication Methods 0.000 description 1
- 238000003066 decision tree Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 238000005265 energy consumption Methods 0.000 description 1
- 238000002474 experimental method Methods 0.000 description 1
- 238000009776 industrial production Methods 0.000 description 1
- 239000000463 material Substances 0.000 description 1
- 230000008520 organization Effects 0.000 description 1
- 238000007637 random forest analysis Methods 0.000 description 1
- 238000005070 sampling Methods 0.000 description 1
- 238000012546 transfer Methods 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/12—Protocols specially adapted for proprietary or special-purpose networking environments, e.g. medical networks, sensor networks, networks in vehicles or remote metering networks
-
- G—PHYSICS
- G05—CONTROLLING; REGULATING
- G05B—CONTROL OR REGULATING SYSTEMS IN GENERAL; FUNCTIONAL ELEMENTS OF SUCH SYSTEMS; MONITORING OR TESTING ARRANGEMENTS FOR SUCH SYSTEMS OR ELEMENTS
- G05B19/00—Programme-control systems
- G05B19/02—Programme-control systems electric
- G05B19/418—Total factory control, i.e. centrally controlling a plurality of machines, e.g. direct or distributed numerical control [DNC], flexible manufacturing systems [FMS], integrated manufacturing systems [IMS] or computer integrated manufacturing [CIM]
- G05B19/4185—Total factory control, i.e. centrally controlling a plurality of machines, e.g. direct or distributed numerical control [DNC], flexible manufacturing systems [FMS], integrated manufacturing systems [IMS] or computer integrated manufacturing [CIM] characterised by the network communication
-
- G—PHYSICS
- G05—CONTROLLING; REGULATING
- G05B—CONTROL OR REGULATING SYSTEMS IN GENERAL; FUNCTIONAL ELEMENTS OF SUCH SYSTEMS; MONITORING OR TESTING ARRANGEMENTS FOR SUCH SYSTEMS OR ELEMENTS
- G05B19/00—Programme-control systems
- G05B19/02—Programme-control systems electric
- G05B19/418—Total factory control, i.e. centrally controlling a plurality of machines, e.g. direct or distributed numerical control [DNC], flexible manufacturing systems [FMS], integrated manufacturing systems [IMS] or computer integrated manufacturing [CIM]
- G05B19/41885—Total factory control, i.e. centrally controlling a plurality of machines, e.g. direct or distributed numerical control [DNC], flexible manufacturing systems [FMS], integrated manufacturing systems [IMS] or computer integrated manufacturing [CIM] characterised by modeling, simulation of the manufacturing system
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/90—Details of database functions independent of the retrieved data types
- G06F16/95—Retrieval from the web
- G06F16/953—Querying, e.g. by the use of web search engines
- G06F16/9536—Search customisation based on social or collaborative filtering
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L12/00—Data switching networks
- H04L12/66—Arrangements for connecting between networks having differing types of switching systems, e.g. gateways
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/14—Session management
- H04L67/146—Markers for unambiguous identification of a particular session, e.g. session cookie or URL-encoding
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/2866—Architectures; Arrangements
- H04L67/30—Profiles
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02P—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN THE PRODUCTION OR PROCESSING OF GOODS
- Y02P90/00—Enabling technologies with a potential contribution to greenhouse gas [GHG] emissions mitigation
- Y02P90/02—Total factory control, e.g. smart factories, flexible manufacturing systems [FMS] or integrated manufacturing systems [IMS]
Landscapes
- Engineering & Computer Science (AREA)
- Signal Processing (AREA)
- Computer Networks & Wireless Communication (AREA)
- Physics & Mathematics (AREA)
- Manufacturing & Machinery (AREA)
- General Physics & Mathematics (AREA)
- Databases & Information Systems (AREA)
- General Engineering & Computer Science (AREA)
- Automation & Control Theory (AREA)
- Quality & Reliability (AREA)
- Theoretical Computer Science (AREA)
- Health & Medical Sciences (AREA)
- Medical Informatics (AREA)
- General Health & Medical Sciences (AREA)
- Data Mining & Analysis (AREA)
- Computing Systems (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention discloses an intelligent factory data automatic subscription method based on collaborative filtering and MQTT, which relates to the field of intelligent factories and comprises the following steps: establishing an OPC UA information model on an edge gateway; the edge gateway processes the data collected by the sensor and puts the processed data into an OPC UA information model; the MQTT publisher acquires data and related information from the OPC UA information model and publishes the data and the related information to an MQTT agent in a theme mode; storing the historical subscription relation on the MQTT agent in a database; generating a data set by using a historical subscription relationship, generating an automatic subscription rule according to the data set by using an automatic subscription method, and sending data to a designated MQTT subscriber in a theme form by an MQTT agent according to the automatic subscription rule; the application parses the required data from the MQTT subscriber. The invention designs an automatic subscription framework and a method, combines collaborative filtering with MQTT, realizes automatic subscription of data, and can improve the automation degree and transmission efficiency, thereby being better suitable for large-scale dynamic scenes of intelligent factory data transmission.
Description
Technical Field
The invention relates to the field of intelligent factories, in particular to an intelligent factory data automatic subscription method based on collaborative filtering and MQTT.
Background
Against the background of china manufacturing 2025 and industry 4.0, both china and the world have intelligent manufacturing as the next generation of industrial form and strategic development target, and have invested a great deal of manpower, material resources and financial resources. The main objective of industry 4.0 is to realize a digital intelligent factory, which must rely on technologies such as industrial internet of things and artificial intelligence, and the core of the technology lies in data and calculation. The MQTT is one of the most popular data transmission technologies in the industrial Internet of things, has the characteristics and advantages of low energy consumption, small data transmission quantity, less network connection, asynchronous receiving and transmitting support and the like, and is suitable for the requirement of large-scale data transmission of intelligent factories.
MQTT has the above-mentioned advantage in that it employs a publish-subscribe rather than client-server mode for data transfer. That is, all data providers, such as sensor devices, upload the collected data to the agent in the form of topics, and all data consumers, such as analysis and processing applications of industrial production, subscribe to the specific data they need by browsing and querying the topic list on the agent. By the method, the decoupling of the space (namely connection) and the time (namely real-time receiving and sending) of the data provider and the data receiver is realized, and the data is split into units with a theme with finer granularity by taking the equipment and the server as the units, so that a more flexible data transmission mode is realized, and the method is suitable for a larger scene.
The selection and subscription of topics by existing publish-subscribe schemas is based on manual search and static rule matching, i.e., subscribers are manually used to access a list of topics on the broker and manually screen for appropriate and desired topics based on topic information. When the topics are more, the manual screening range is narrowed down by adopting a topic name retrieval mode. In the retrieval process, methods such as a hash table and the like are commonly used to reduce the matching cost and improve the searching efficiency, and meanwhile, the topics can be screened according to the relevant information of the topics, such as the frequency, the precision, the QoS and the like of the topic data. A more advanced method is MQTT +, which looks up and processes data in topics on the agent, thereby realizing topic-based content screening, such as subscribing only topics with changed content, or topics with magnitude greater than a threshold. In summary, existing solutions for determining subscription topics are based on manual search and static rule matching, including retrieval of topic names, conditional filtering of topic information, and static rule matching of topic content.
However, in the face of massive data topics in an intelligent factory, based on a publish-subscribe mode of search and matching, complex and constantly changing subscription requirements cannot be met through ways of topic name retrieval, topic information condition filtering, static rule matching of topic contents and the like, and especially many subscription requirements are difficult to define by simple names, conditions and rules. This can make topic subscription difficult and it is difficult to obtain the really needed data topics, thus wasting a lot of bandwidth and computing resources.
Secondly, in the existing topic subscription mode, all topics need to be actively searched and screened manually, and the process is complicated, slow, time-consuming and labor-consuming. This results in an inability to automate the data subscription process, increased labor and time costs, and inaccuracy and error susceptibility.
In the face of massive equipment, sensors and themes in an intelligent factory, a purely manual subscription mode can only use a rough mode to subscribe one type of theme in batch due to labor and time cost, so that the fine-grained advantage of the theme compared with the traditional server mode cannot be exerted, a large amount of redundant data and unneeded themes are subscribed, the data transmission amount is increased, and the later data screening and processing cost is increased.
Finally, through the existing subscription mode, whether a plurality of topics are needed by the subscriber is difficult to judge before subscription, and the subscriber can only analyze and judge after subscription, so that a plurality of data with low relevance are subscribed, the data transmission quantity is increased, and the calculation cost of the subscriber is also increased.
Therefore, those skilled in the art are dedicated to developing an intelligent factory data automatic subscription method based on collaborative filtering and MQTT, which can implement automatic subscription of data in an intelligent factory, reduce data transmission amount and query frequency, and improve automation degree and data transmission efficiency, thereby being better suitable for a large-scale dynamic scenario of intelligent factory data transmission.
Disclosure of Invention
In view of the above defects in the prior art, the technical problems to be solved by the present invention are that the existing manual search and data screening method has low automation degree and low accuracy, and cannot adapt to large-scale dynamic scenes.
In order to achieve the purpose, the invention provides an intelligent factory data automatic subscription method based on collaborative filtering and MQTT, which is characterized by comprising the following steps of constructing a data automatic subscription framework:
step 2, the sensor transmits the acquired original data to the edge gateway;
step 3, the edge gateway processes the original data and puts the processed data into the OPC UA information model;
step 4, the MQTT publisher acquires data and related information from the OPC UA information model and publishes the data and the related information to the MQTT agent in a theme mode;
step 5, storing the historical subscription relation on the MQTT agent in a database;
step 6, generating a data set by using the historical subscription relationship in the database, if a subscriber starts an automatic subscription function, generating an automatic subscription rule according to the data set by using an automatic subscription method, and modifying a configuration file by the MQTT agent according to the automatic subscription rule;
step 7, the MQTT agent sends data to a designated MQTT subscriber in a theme mode according to the modified configuration file;
and 8, the MQTT subscriber analyzes the received data and transmits the data to a required application.
Further, the step 6 comprises:
6.1, acquiring the data set from a database;
step 6.2, establishing a subscriber-topic matrix model;
6.3, calculating a similarity matrix;
6.4, generating a subscription relation;
step 6.5, configuring subscription frequency, precision and QoS;
and 6.6, the MQTT agent adjusts the configuration file according to the generated automatic subscription rule.
Further, the data set comprises a subscriber ID, a topic ID, whether to subscribe, subscription frequency, precision, QoS and whether to start an automatic subscription function.
Further, the step 6.2 comprises:
establishing the subscriber-topic Matrix model, namely an m x n subscription relation Matrix, by using the subscriber ID, the topic ID and the subscription in the data set; wherein m is the number of subscribers, n is the number of topics, Matrix [ ij ] represents whether the ith subscriber subscribes to the jth topic, if so, it is 1, otherwise, it is 0.
Further, the similarity matrix is used for calculating the similarity between the target subscriber who starts the automatic subscription function and other subscribers, and the similarity cosine is adopted for calculation:
wherein N (u) is the set of all the subscribed topics of the target subscriber u, N (v) is the set of all the subscribed topics of the subscriber v, WuvIs a similarity matrix of u and v.
Further, the step 6.4 further comprises the following steps:
step 6.4.1, finding K subscribers most similar to the target subscriber u from the similarity matrix, and expressing the K subscribers by a set S (u, K);
step 6.4.2, all the topics subscribed by the K subscribers in the S (u, K) are extracted, and the topics subscribed by the target subscriber u are removed to obtain candidate topics;
step 6.4.3, calculating the useful degree p of each candidate topic i to the target subscriber u;
and 6.4.4, sequencing all candidate topics according to the score p, taking all topics with scores larger than a threshold value as an automatic subscription set T (u) of a target subscriber u, and setting the subscription relationship between the target subscriber u and all topics in the automatic subscription set T (u) as subscription.
Further, the useful degree p in said step 6.4.3 is calculated by the following formula:
p(u,i)＝∑S(u,K)∩N(i)Wuv*Rvi
in the formula, RviRepresenting the grade of the subscriber v to the topic i, if the subscriber v subscribes to the topic i, RviIs 1, if there is no subscription, RviIs 0.
Further, if the subscriber v subscribes to the topic, R is set according to the subscription frequency, accuracy and QoS in the data setviSo that it is proportional to the above value.
Further, the step 6.5 comprises:
retrieving the subscription frequency, precision and QoS of all topics t in the automatic subscription set T (u) from the K subscribers in the step 6.4.1, and taking a mode as the automatically generated subscription frequency, precision and QoS;
and combining the subscription relationship generated in the step 6.4.4 to generate an automatic subscription rule, wherein the first column in the list of the automatic subscription rule is a subscriber ID, the second column is a subject ID, and the third, fourth and fifth columns respectively represent the subscription frequency, the subscription precision and the subscription QoS which are automatically generated, and the list of the automatic subscription rule is sent to the MQTT agent.
Further, the step 6.6 comprises:
the MQTT agent modifies the configuration file of the MQTT agent according to the received automatic subscription rule, namely the subscription relation between the subscribers in the list and the corresponding topics is set as subscription, and the frequency, the precision and the QoS of the subscription are the corresponding automatic subscription frequency, precision and QoS given in the list; the modification is completed and the MQTT agent sends topic data to the subscriber according to the new profile.
Compared with the prior art, the invention at least has the following beneficial technical effects:
1. an automatic subscription architecture is designed: the MQTT is deployed to the cloud, so that more devices can be connected, a wider range is spanned, more devices can be served, meanwhile, the automatic subscription method provided by the invention has a larger data set, the generation of an automatic subscription set is more accurate, and the cloud computing capability is stronger, so that a more complex algorithm can be supported, more functions are provided, and the automatic subscription rule is dynamically updated at a higher frequency;
2. combining OPC UA with MQTT: after the data is processed by protocol conversion and the like, the data is put into an OPC UA model, and then when an MQTT publisher publishes a theme, the data can be taken from the OPC UA model, the theme name, namely the name and the identification of the theme name in the OPC UA model, and the numerical value and the related information can also be directly obtained from the OPC UA model; through the combination of OPC UA and MQTT, a unified and common semantic information model can be established, meanwhile, the publishing and subscribing of MQTT are more automatic, the theme naming can be unified, and data and related information are standardized;
3. an automatic subscription scenario and method are proposed: the method is provided and designed, and a corresponding data set is constructed and obtained on the MQTT agent by combining industrial characteristics, so that automatic data subscription can be realized, the method is better suitable for large-scale dynamic scenes of intelligent factory data transmission, and the time cost, the labor cost, the transmission cost and the calculation cost of data transmission are reduced;
4. aiming at industrial characteristics, the traditional collaborative filtering is modified, the topic is automatically subscribed through the similarity, and simultaneously, the parameters of subscription frequency, precision, QoS and the like can be automatically set, so that the automation degree and efficiency of subscription are improved.
The conception, the specific structure and the technical effects of the present invention will be further described with reference to the accompanying drawings to fully understand the objects, the features and the effects of the present invention.
Drawings
FIG. 1 is a diagram of an auto-subscribe architecture in accordance with a preferred embodiment of the present invention;
FIG. 2 is a flow chart of an auto-subscribe method according to a preferred embodiment of the present invention;
FIG. 3 is a diagram illustrating a subscriber similarity matrix according to a preferred embodiment of the present invention.
Detailed Description
The technical contents of the preferred embodiments of the present invention will be more clearly and easily understood by referring to the drawings attached to the specification. The present invention may be embodied in many different forms of embodiments and the scope of the invention is not limited to the embodiments set forth herein.
In the drawings, structurally identical elements are represented by like reference numerals, and structurally or functionally similar elements are represented by like reference numerals throughout the several views. The size and thickness of each component shown in the drawings are arbitrarily illustrated, and the present invention is not limited to the size and thickness of each component. The thickness of the components may be exaggerated where appropriate in the figures to improve clarity.
Fig. 1 is a schematic diagram of an automatic subscription architecture according to a preferred embodiment of the present invention, in which OPC UA is used to establish a uniform information model, so that different protocols can communicate with each other, and different data can have a uniform organization; the MQTT communication protocol of the internet of things which is decoupled in time and space is used for reducing the data transmission quantity and meeting the requirement of large-scale data transmission in a more flexible mode. Meanwhile, the MQTT has centralized agents, so that all subscription relationships can be mastered, and by utilizing the historical subscription relationships, the relation between different requirements and data can be mined by an automatic subscription method, so that the data subscription rule is automatically generated. The MQTT agent is placed on the cloud so as to obtain more historical subscription relationships and strengthen higher calculation power, so that the automatic subscription rule is more accurate and more applications are served. The MQTT publisher and the MQTT subscriber are respectively arranged at a data provider and a data demand party and used as media for sending and receiving data to and from an MQTT agent, and the specific establishment of the automatic subscription architecture mainly comprises the following steps:
and S1, establishing an OPC UA information model on the edge gateway.
Specifically, according to the actual situation of the intelligent factory and the OPC UA modeling rule, the OPC UA information model is established and exported by using a visual OPC UA modeling tool and then is imported to a pure-code OPC UA server to run, and the pure-code mode is used because the internal interfaces and data can be better acquired so as to interact with the underlying equipment and the MQTT publisher.
S2, the sensor collects the data and transmits it to the edge gateway.
Specifically, the sensor acquires corresponding signals to generate analog quantity, the analog quantity is transmitted to the analog-to-digital conversion module after circuit conversion, the analog-to-digital conversion module converts the analog quantity into corresponding digital quantity in real time to be output, and a hardware interface and a transmission protocol which are in accordance with the edge gateway are used for transmitting the converted digital quantity to the edge gateway.
And S3, the edge gateway processes the original data and puts the processed data into an OPC UA information model.
Specifically, the edge gateway acquires the converted digital quantities by using a transmission protocol agreed with the analog-to-digital converter, then decodes and converts the digital quantities into values of physical quantities actually represented by the digital quantities, and the edge gateway can further perform operations such as down-sampling and abnormal value processing on the values, and then places the values into nodes corresponding to the OPC UA model.
And S4, the edge gateway acquires data and related information from the OPC UA by using the MQTT publisher and publishes the data and the related information to the MQTT proxy on the cloud in a theme mode.
Specifically, the edge gateway acquires needed data and relevant information thereof from an OPC UA server on line, combines names such as browsing names and the like in a corresponding data name space and identification information into a subject name, takes a value in the name space as subject content, and transmits the subject name and the subject content to an MQTT publisher operating on the edge gateway in a json format; and the MQTT publisher publishes the packaged data and information to an MQTT agent running on the cloud.
And S5, storing the historical subscription relation on the MQTT agent into a database.
Specifically, the MQTT proxy has information of publishers, topics and subscribers, and also has subscription relationships between subscribers and topics, including which topics the subscribers subscribe to, with what frequency, accuracy and QoS, these data are saved in a database to make a data set, and an automatic subscription method is used to mine association relationships therein, and generate automatic subscription rules.
And S6, generating a data set of the automatic subscription method by using the historical subscription relation in the database.
Specifically, the historical subscription relationships in the database may constitute a data set including all topics, subscribers, their subscription relationships, and the frequency, accuracy, and QoS with which the subscribers subscribe to the topics.
And S7, generating an automatic subscription rule according to the data set by the automatic subscription method.
Specifically, a data set generated according to the historical subscription relationship is input into an automatic subscription method, and if the subscriber starts an automatic subscription function, the automatic subscription method generates an automatic subscription rule for the data set, that is, automatically sets the subscription relationship, and the frequency, precision and QoS of the subscription.
And S8, the MQTT agent sends the data to the appointed MQTT subscriber in a topic form according to the generated automatic subscription rule.
Specifically, the MQTT proxy modifies its configuration file according to the automatic subscription rule generated by the automatic subscription method, thereby changing and setting the subscription relationship, subscription frequency, accuracy and QoS of the subscriber to the topic. The MQTT agent will send topic data to the designated MQTT subscriber according to the updated configuration file.
And S9, the application analyzes and obtains the needed data from the MQTT subscriber.
Specifically, on the edge device, the MQTT subscriber receives data sent by the MQTT proxy on the cloud, analyzes the data, and transmits the data to the corresponding application for application.
As shown in fig. 2, a flowchart of a data auto-subscription method of this embodiment mainly includes the following steps:
s1, acquiring a data set from the database;
s2, establishing a subscriber-topic matrix model;
s3, calculating a similarity matrix;
s4, generating a subscription relationship;
s5, configuring subscription frequency, precision and QoS.
And S6, the MQTT agent adjusts the configuration file according to the generated automatic subscription rule.
In particular, in conjunction with data features, automatic subscription is achieved using two collaborative filtering algorithms, topic-based and subscriber-based. Since the number of topics is much larger than the number of subscribers, the similarity matrix using subscriber-based collaborative filtering will be much smaller than topic-based collaborative filtering. Meanwhile, the ideas and methods based on the theme and the subscribers are the same, and only the theme in the method and the subscribers need to be correspondingly converted. The following description will only describe the automatic subscription method by using a subscriber-based collaborative filtering algorithm as an example. The individual steps are set forth in detail below.
S1, the database in the cloud stores the historical subscription information on the MQTT proxy, including information of all topics, information of subscribers and their subscription relationship, i.e. whether to subscribe, if so, subscription frequency, accuracy and QoS. Firstly, a data set can be made through the information, and the first column of the data set is the ID of the subscriber; the second column is a subject ID; the third column indicates whether the subscriber subscribes to the topic, if so, 1, and if not, 0; the fourth column is subscription frequency; the fifth column is subscription precision; the sixth column is QoS. If the subscription is made, the four columns, five columns and six columns are corresponding values, and if the subscription is not made, the number is null, namely None. The seventh column indicates whether the subscriber has the automatic subscription function, and 1 indicates on, and 0 indicates off.
S2, using the first 3 columns of the data set created in S1, a subscriber-topic Matrix model, i.e. an m × n subscription relationship Matrix, can be created. m is the number of subscribers, n is the number of topics, Matrix [ ij ] indicates whether the ith subscriber subscribes to the jth topic, if so, it is 1, otherwise, it is 0. The subscriber-topic matrix model is used as an input to an automatic subscription method.
And S3, the stage mainly calculates the similarity between the target subscriber who starts the self-subscription function and other subscribers. Let n (u) be the set of all the subscribed topics of the subscriber u, and n (v) be the set of all the subscribed topics of the subscriber v, the similarity between u and v can be calculated by the following method:
jaccard formula:
W_uv＝(|N(u)∩N(v)|)/(|N(u)∪N(v)|)
similarity cosine:
here, the similarity cosine is selected for calculation. The computational complexity of the method is O (| U | × | U |), which is proportional to the square of the number of subscribers, i.e., proportional to the size of the subscriber similarity matrix, and is very time-consuming due to the large scale of the matrix. However, since the subscriber only subscribes to a very small portion of the topics that the subscriber needs, it is observed that the subscriber-topic matrix is a sparse matrix, i.e., in many cases, | n (u) # n (v) | 0. So we can first compute | N (u) # N (v) | ≠ 0 and then compute
Therefore, an inverted list of 'theme-subscriber' can be established firstly, and the theme is stored for each themeTo a list of subscribers.
Let sparse matrix C [ u ]][v]Given that subscriber u and subscriber v belong to the subscriber list corresponding to K topics in the inverted index at the same time, there is C [ u (u), (n (v)) |, and][v]n (u) n (v) K. Then, the position which is not 0 is calculated
The cosine of the similarity can be obtained by dividing. The similarity matrix between subscribers is shown in fig. 3, and it is very intuitive to find out the subscribers similar to the target subscriber.
S4, after generating the similarity matrix among the subscribers through S3, K subscribers most similar to the target subscriber need to be found from the similarity matrix, and represented by the set S (u, K). Firstly, all the topics subscribed by the subscriber in S are extracted, and the topics subscribed by u are removed. For each candidate topic i, its usefulness to the subscriber is calculated using the following formula:
p(u,i)＝∑S(u,K)∩N(i)Wuv*Rvi
wherein R isviRepresents the rating of a subscriber v on a topic i, only the simplest case being considered here, i.e. if a subscriber v subscribes to a topic i, then RviIs 1, and if not subscribed, is 0. If the data set is subscribed, R can be set according to the subscription frequency, the precision and the QoS of four, five and six columns of the data setviSo that it is proportional to the above value. And finally, sorting all candidate topics according to the score p, taking all topics with scores larger than a threshold value as an automatic subscription set T (u) of a target subscriber u, and setting the subscription relationship between u and all topics in the set T (u) as subscription.
S5, for all topics t in T (u), retrieving the subscription frequency, precision and QoS of the K subscribers in S4, and taking the mode as the automatically generated subscription frequency, precision and QoS. And combining the subscription relationship generated by the S4 to generate an automatic subscription rule list, wherein the first column is a subscriber ID, the second column is a subject ID, and the third, fourth and fifth columns are respectively the automatically generated subscription frequency, precision and QoS and send the subscription rule list to the MQTT agent.
S6, the MQTT agent modifies the configuration file according to the received automatic subscription rule list, namely, the subscription relation between the subscriber in the list and the corresponding subject is set as subscription, and the frequency, the precision and the QoS of the subscription are the corresponding automatic subscription frequency, the precision and the QoS given in the list. Then, the MQTT agent may send the topic data to the subscriber according to the new profile.
In another embodiment, the automatic subscription set is generated without collaborative filtering, and other recommendation algorithms such as decision trees, random forests, AdaBoost, graph neural networks, and the like may be used.
The foregoing detailed description of the preferred embodiments of the invention has been presented. It should be understood that numerous modifications and variations could be devised by those skilled in the art in light of the present teachings without departing from the inventive concepts. Therefore, the technical solutions available to those skilled in the art through logic analysis, reasoning and limited experiments based on the prior art according to the concept of the present invention should be within the scope of protection defined by the claims.
Claims (10)
1. An intelligent factory data automatic subscription method based on collaborative filtering and MQTT is characterized by comprising the following steps of constructing a data automatic subscription framework:
step 1, establishing an OPC UA information model on an edge gateway, deploying an MQTT publisher and an MQTT subscriber, and deploying an MQTT agent at a cloud;
step 2, the sensor transmits the acquired original data to the edge gateway;
step 3, the edge gateway processes the original data and puts the processed data into the OPC UA information model;
step 4, the MQTT publisher acquires data and related information from the OPC UA information model and publishes the data and the related information to the MQTT agent in a theme mode;
step 5, storing the historical subscription relation on the MQTT agent in a database;
step 6, generating a data set by using the historical subscription relationship in the database, if a subscriber starts an automatic subscription function, generating an automatic subscription rule according to the data set by using an automatic subscription method, and modifying a configuration file by the MQTT agent according to the automatic subscription rule;
step 7, the MQTT agent sends data to the appointed MQTT subscriber in a theme mode according to the modified configuration file;
and 8, the MQTT subscriber analyzes the received data and transmits the data to a required application.
2. The intelligent plant data auto-subscription method based on collaborative filtering and MQTT according to claim 1, wherein the step 6 comprises:
6.1, acquiring the data set from a database;
step 6.2, establishing a subscriber-topic matrix model;
6.3, calculating a similarity matrix;
6.4, generating a subscription relation;
step 6.5, configuring subscription frequency, precision and QoS;
and 6.6, the MQTT agent adjusts the configuration file according to the generated automatic subscription rule.
3. The intelligent factory data auto-subscription method based on collaborative filtering and MQTT according to claim 2, wherein the data set comprises subscriber ID, topic ID, whether to subscribe, subscription frequency, accuracy, QoS, whether to turn on auto-subscription function.
4. The intelligent factory data auto-subscription method based on collaborative filtering and MQTT according to claim 3, wherein the subscriber-topic Matrix model uses the subscriber ID, the topic ID, the subscription establishment, i.e. an m x n subscription relationship Matrix, in the data set; wherein m is the number of subscribers, n is the number of topics, Matrix [ ij ] represents whether the ith subscriber subscribes to the jth topic, if so, it is 1, otherwise, it is 0.
5. The intelligent factory data automatic subscription method based on collaborative filtering and MQTT according to claim 4, wherein the similarity matrix is used for calculating the similarity between the target subscriber who starts the automatic subscription function and other subscribers, and the similarity cosine is adopted for calculation:
wherein N (u) is the set of all the subscribed topics of the target subscriber u, N (v) is the set of all the subscribed topics of the subscriber v, WuvIs a similarity matrix of u and v.
6. The intelligent plant data auto-subscription method based on collaborative filtering and MQTT according to claim 5, characterized in that the step 6.4 further comprises the steps of:
step 6.4.1, finding K subscribers most similar to the target subscriber u from the similarity matrix, and expressing the K subscribers by a set S (u, K);
step 6.4.2, all the topics subscribed by the K subscribers in the S (u, K) are extracted, and the topics subscribed by the target subscriber u are removed to obtain candidate topics;
step 6.4.3, calculating the useful degree p of each candidate topic i to the target subscriber u;
and 6.4.4, sequencing all candidate topics according to the score p, taking all topics with scores larger than a threshold value as an automatic subscription set T (u) of a target subscriber u, and setting the subscription relationship between the target subscriber u and all topics in the automatic subscription set T (u) as subscription.
7. The intelligent plant data auto-subscription method based on collaborative filtering and MQTT according to claim 6, characterized in that the degree of usefulness p in the step 6.4.3 is calculated by the following formula:
p(u,i)＝∑S(u,K)∩N(i)Wuv*Rvi
in the formula, RviRepresenting the grade of the subscriber v to the topic i, if the subscriber v subscribes to the topic i, RviIs 1, if there is no subscription, RviIs 0.
8. The intelligent factory data automatic subscription method based on collaborative filtering and MQTT according to the claim 6, wherein if a subscriber v subscribes to a topic, R is set according to the subscription frequency, precision and QoS in the data setviSo that it is proportional to the above value.
9. The intelligent plant data auto-subscription method based on collaborative filtering and MQTT according to claim 6, wherein the step 6.5 comprises:
retrieving the subscription frequency, precision and QoS of all topics t in the automatic subscription set T (u) from the K subscribers in the step 6.4.1, and taking a mode as the automatically generated subscription frequency, precision and QoS;
and combining the subscription relationship generated in the step 6.4.4 to generate an automatic subscription rule, wherein the first column in the list of the automatic subscription rule is a subscriber ID, the second column is a subject ID, and the third, fourth and fifth columns respectively represent the subscription frequency, the subscription precision and the subscription QoS which are automatically generated, and the list of the automatic subscription rule is sent to the MQTT agent.
10. The intelligent plant data auto-subscription method based on collaborative filtering and MQTT according to claim 9, characterized in that said step 6.6 comprises:
the MQTT agent modifies the configuration file of the MQTT agent according to the received automatic subscription rule, namely the subscription relation between the subscribers in the list and the corresponding topics is set as subscription, and the frequency, the precision and the QoS of the subscription are the corresponding automatic subscription frequency, precision and QoS given in the list; the modification is completed and the MQTT agent sends topic data to the subscriber according to the new profile.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111482972.5A CN114374708B (en) | 2021-12-07 | 2021-12-07 | Intelligent factory data automatic subscription method based on collaborative filtering and MQTT |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111482972.5A CN114374708B (en) | 2021-12-07 | 2021-12-07 | Intelligent factory data automatic subscription method based on collaborative filtering and MQTT |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN114374708A true CN114374708A (en) | 2022-04-19 |
| CN114374708B CN114374708B (en) | 2022-09-30 |
Family
ID=81140762
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111482972.5A Active CN114374708B (en) | 2021-12-07 | 2021-12-07 | Intelligent factory data automatic subscription method based on collaborative filtering and MQTT |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114374708B (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115065621A (en) * | 2022-06-10 | 2022-09-16 | 中国电信股份有限公司 | Data subscription method and device, electronic equipment and readable storage medium |
Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1953426A (en) * | 2005-10-19 | 2007-04-25 | 国际商业机器公司 | Publish/subscribe system and method for managing subscriptions |
| CN101848236A (en) * | 2010-05-06 | 2010-09-29 | 北京邮电大学 | Real-time data distribution system with distributed network architecture and working method thereof |
| CN102761581A (en) * | 2011-04-28 | 2012-10-31 | 国际商业机器公司 | Dynamic subscription method and device of management information, as well as dissemination method and system |
| CN103458033A (en) * | 2013-09-04 | 2013-12-18 | 北京邮电大学 | System for providing services of event-driven service-oriented internet of things and working method thereof |
| US20180167476A1 (en) * | 2016-12-12 | 2018-06-14 | Sap Se | Meta broker for publish-subscribe-based messaging |
| CN108183967A (en) * | 2018-01-16 | 2018-06-19 | 重庆邮电大学 | Suitable for the OPC UA publish/subscribe methods of IPv6 wireless sensor networks |
| CN110650084A (en) * | 2019-08-26 | 2020-01-03 | 山东省科学院自动化研究所 | Intelligent gateway, networking system and data processing method for industrial Internet of things |
| US20200145494A1 (en) * | 2017-05-02 | 2020-05-07 | Siemens Aktiengesellschaft | Method for Operating an Automation Network |
| US10812600B1 (en) * | 2019-11-28 | 2020-10-20 | International Business Machines Corporation | Enforcing session properties compliance for gateway connected publish-subscribe clients |
-
2021
- 2021-12-07 CN CN202111482972.5A patent/CN114374708B/en active Active
Patent Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1953426A (en) * | 2005-10-19 | 2007-04-25 | 国际商业机器公司 | Publish/subscribe system and method for managing subscriptions |
| CN101848236A (en) * | 2010-05-06 | 2010-09-29 | 北京邮电大学 | Real-time data distribution system with distributed network architecture and working method thereof |
| CN102761581A (en) * | 2011-04-28 | 2012-10-31 | 国际商业机器公司 | Dynamic subscription method and device of management information, as well as dissemination method and system |
| CN103458033A (en) * | 2013-09-04 | 2013-12-18 | 北京邮电大学 | System for providing services of event-driven service-oriented internet of things and working method thereof |
| US20180167476A1 (en) * | 2016-12-12 | 2018-06-14 | Sap Se | Meta broker for publish-subscribe-based messaging |
| US20200145494A1 (en) * | 2017-05-02 | 2020-05-07 | Siemens Aktiengesellschaft | Method for Operating an Automation Network |
| CN108183967A (en) * | 2018-01-16 | 2018-06-19 | 重庆邮电大学 | Suitable for the OPC UA publish/subscribe methods of IPv6 wireless sensor networks |
| CN110650084A (en) * | 2019-08-26 | 2020-01-03 | 山东省科学院自动化研究所 | Intelligent gateway, networking system and data processing method for industrial Internet of things |
| US10812600B1 (en) * | 2019-11-28 | 2020-10-20 | International Business Machines Corporation | Enforcing session properties compliance for gateway connected publish-subscribe clients |
Non-Patent Citations (5)
| Title |
|---|
| PETER PENIAK; EMÍLIA BUBENÍKOVÁ; JURAJ SPALEK: "Model of Integration Gateway for Communication of OPC/MQTT Devices", 《2020 CYBERNETICS & INFORMATICS (K&I)》 * |
| TORU MIZUYA; MAKOTO OKUDA; TATSUAKI NAGAO: "A case study of data acquisition from field devices using OPC UA and MQTT", 《2017 56TH ANNUAL CONFERENCE OF THE SOCIETY OF INSTRUMENT AND CONTROL ENGINEERS OF JAPAN (SICE)》 * |
| 于毅等: "基于消息队列遥测传输协议的切削数据主动推送系统设计", 《工具技术》 * |
| 姜妮等: "基于MQTT物联网消息推送系统", 《网络新媒体技术》 * |
| 杨坤等: "基于订阅/分发机制的军事信息系统知识库协同框架探讨", 《指挥控制与仿真》 * |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115065621A (en) * | 2022-06-10 | 2022-09-16 | 中国电信股份有限公司 | Data subscription method and device, electronic equipment and readable storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN114374708B (en) | 2022-09-30 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111435344B (en) | Big data-based drilling acceleration influence factor analysis model | |
| US6078918A (en) | Online predictive memory | |
| CN107766511A (en) | Intelligent answer method, terminal and storage medium | |
| JP2006164233A (en) | Directory structure in distributed data driven architecture environment | |
| CN107967518A (en) | A kind of knowledge auto-associating system and method based on product design | |
| CN103473128A (en) | Collaborative filtering method for mashup application recommendation | |
| CN114374708B (en) | Intelligent factory data automatic subscription method based on collaborative filtering and MQTT | |
| Prinz et al. | A novel I4. 0-enabled engineering method and its evaluation | |
| CN116128254A (en) | Embedded intelligent manufacturing resource configuration method and terminal system based on edge calculation | |
| CN105096023A (en) | System and method for pushing data relevant to working standard | |
| CN110086894A (en) | Personnel's related information method for digging, communication recommended method and relevant apparatus | |
| Schlemitz et al. | Approaches for data collection and process standardization in smart manufacturing: Systematic literature review | |
| CN114943284A (en) | Data processing system and method of behavior prediction model | |
| CN110505301A (en) | A kind of aeronautical manufacture workshop industry big data processing frame | |
| CN111368060B (en) | Self-learning method, device and system for conversation robot, electronic equipment and medium | |
| Ramis Ferrer et al. | Management of distributed knowledge encapsulated in embedded devices | |
| Qin et al. | A knowledge graph-based knowledge representation for adaptive manufacturing control under mass personalization | |
| CN108182600A (en) | A kind of method and system that extending user is determined according to weighted calculation | |
| Shen et al. | Blocking flow shop scheduling based on hybrid ant colony optimization | |
| Lu et al. | Framework of industrial networking sensing system based on edge computing and artificial intelligence | |
| CN114461626A (en) | Monitoring method and system for power grid terminal equipment | |
| Berezovskyi et al. | Efficient state update exchange in a CPS environment for linked data-based digital twins | |
| EP4439415A1 (en) | Plant infrastructure modelling update from engineering information | |
| CN108182235A (en) | A kind of method and system for being used to carry out user characteristics distributed coding | |
| CN112507127B (en) | Intelligent extraction system and method based on priori knowledge graph |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |