CN109104466B

CN109104466B - WoT resource management method based on P2P

Info

Publication number: CN109104466B
Application number: CN201810818741.9A
Authority: CN
Inventors: 李养群; 周梅
Original assignee: Nanjing University of Posts and Telecommunications
Current assignee: Nanjing University of Posts and Telecommunications
Priority date: 2018-07-24
Filing date: 2018-07-24
Publication date: 2021-01-26
Anticipated expiration: 2038-07-24
Also published as: CN109104466A

Abstract

The invention discloses a WoT resource management method based on P2P, which comprises the steps of firstly segmenting a text, extracting Top K most important topics in the text by adopting an algorithm, carrying out Hash calculation on the topics, constructing an ID of a resource by utilizing geographical location information of the resource, and realizing life cycle management of the resource such as storage, retrieval, updating, deletion and the like. Meanwhile, the node resources are uniformly distributed and stored on the whole of a plurality of nodes, and the availability of the system is improved. The invention applies the P2P technology to the distributed storage and management of resources in the WoT environment, and utilizes the Web technology to operate the resources, thereby realizing the access of large-scale heterogeneous devices and the management of resources/services in the environment of Internet of things.

Description

WoT resource management method based on P2P

Technical Field

The invention relates to the technical field of application of the Internet of things, in particular to a P2P-based method for managing resources of a world Wide Web (Web of things, WoT) of goods.

Background

The internet of things is defined in ITU-T y.2060 as a global infrastructure for information society that interconnects various devices with support of existing or developing interoperable information and communication technologies to provide advanced services. The concept reflects the main features of the internet of things including: global, interoperability, interconnectivity, device diversity, serviceability. Its essential attribute is to provide advanced services to the information society.

With the wide application of the internet of things, more and more sensing devices are deployed in real life. These numerous, functionally diverse terminals are distributed around the world, belonging to different business organizations and individuals. The internet of things technology connects the devices through various protocols to construct a network with various resources/services. For resources in the network, it is challenging to develop services that are more user-friendly by effectively using them. These challenges include: how to efficiently organize and manage these resources; how to efficiently discover resources; how efficiently these resources are used; how to effectively manage the life cycle of resources, provide robust and reliable services, and the like.

For example, when a person intends to visit a scenic spot for playing, the person needs to know some basic conditions of the scenic spot, such as the current approximate number of people in the scenic spot, how much air quality is in the scenic spot, the temperature and humidity of the scenic spot, the noise level of the scenic spot, and the like. This information can be monitored on the one hand by sensors of the scenic spot and on the other hand by means of services provided by the sensors in the scenic spot, for example demographics can be calculated by means of infrared sensors at the entrance to the scenic spot. In order to know the information, the user inquires whether corresponding resources or services exist in the internet of things or not according to needs. The internet of things needs to effectively manage the existing resources to perform effective resource search and service provision. When a node fails, the most suitable resource/service is searched for replacement, so that the availability of the service is ensured.

The integration between heterogeneous networks and various devices is realized by utilizing the existing mature technology of World Wide Web (WWW) on the basis of the Internet of things (Internet of things) technology by the World Wide Web (WoT), and the functions of acquisition, description, transmission, processing, application and development of data of the Internet of things can be conveniently and rapidly realized by utilizing the existing technology and resources of the WWW. It virtualizes the items as world wide web resources, and uses web2.0 and Restful techniques to achieve integration between items and virtual resources. The article world wide Web technology provides various Web means, so that various sensing devices establish connection on a Web layer, but the management problem of various resources in the internet of things is not solved, and new methods and mechanisms need to be adopted to meet the life cycle management of resources of the internet of things, namely the problems of resource connection, resource registration, resource storage, resource discovery, effective and safe access of resources, resource invalidation, resource availability and the like.

Of these problems, a considerable part of research work is currently focused mainly on the search of resources or the discovery of problems. At present, the quantity and variety of resources in the environment of the large-scale internet of things are huge, and how to manage the resources and accurately, quickly and effectively search the needed resources becomes a problem concerned by the academic and industrial fields at present.

The P2P (Peer-to-Peer) technology is an Overlay Network (Overlay Network) constructed by using dht (distributed Hash table) technology based on the existing Network. Compared with a Client/Server, the distributed computing technology has no special Server function role and is widely applied to the fields of resource sharing, virtual currency transaction and the like. It has the following characteristics:

1. the system has the advantages of decentralized structure, low system management and maintenance cost, automatic maintenance of the access and the leaving of the nodes by the system, no manual interference and excellent expandability.

2. The system availability is strong, and the problem of poor system availability caused by single point failure is solved;

3. the resource distribution and the multi-point storage improve the availability of the resource and the efficiency of the system.

At present, the patent with application number 201310356090.3, "a method for publishing and querying resources in a structured P2P network," proposes a method for publishing and querying resources in a structured P2P network, in which the method performs word segmentation on resource descriptions to obtain keywords, combines the keywords to form a plurality of keyword groups, and establishes an index for the resources through the keyword groups. The patent establishes a P2P node and an index node in a P2P network to realize resource search. The method needs to segment and combine the resource description texts and establish indexes, and more resources are consumed.

The patent with application number 201110213482.5, "mobile P2P network resource positioning and scheduling method", proposes a mobile P2P network resource positioning and scheduling method, which summarizes and centrally manages the mobile terminal resource information through ontology modeling according to the resource information resource types, and quickly positions the nodes in the mobile P2P network to other nodes in the sharing process, so as to implement a mechanism of nearby connection and sharing. The method obtains the nearest storage path by sensing the geographical position of the mobile terminal.

Disclosure of Invention

The invention aims to design a method for effectively managing goods world wide web resources under the condition of large-scale network environment by combining P2P technology and goods world wide web technology.

The invention discloses a WoT resource management method based on P2P, which comprises the following steps:

s100: establishing a P2P network based on geographic position, and taking gateway nodes with close physical positions as adjacent nodes;

s200: preprocessing resources given by a user to obtain word segmentation information of the resources;

s300: learning by using the existing corpus or data set through the hidden Dirichlet distribution algorithm to obtain the LDA learning model M_lda；

S400: : and (S200): the word segmentation information in (1) is input into the learning model M_ldaObtaining K themes and probability conditions of the themes described by the resources;

s500: selecting the theme with the highest probability as the service category, respectively performing Hash calculation on the geographic position identifier and the service category of the resource as the initial part of the resource ID, performing Hash calculation on the remaining themes as the remaining part of the resource ID, and expressing the resource ID as follows:

s600: : according to S500: the resource ID in (2) is compared with the gateway node ID of P2P to implement the life cycle management of the resource.

Lifecycle management for resources includes storage, retrieval, invocation, updating, and resource invalidation of resources.

When storing resources, searching for the P2P gateway node for storage according to the resource ID, and after finding the P2P gateway node, storing the < NodeID, theme, resource description > triple information of the resources in the P2P gateway node.

The establishment of the P2P network based on the geographic position is realized by adopting a Kademila protocol, and the method specifically comprises the following steps:

s101: the gateway node calculates to obtain the ID of the gateway node, and queries the node information closest to the gateway node by contacting the guide node;

s102: selecting a nearest gateway node as a neighbor node, constructing a route, wherein the XOR operation result of the front same bit of the nodes with the same geographic position is 0, and the distance is close;

wherein the gateway node calculation is obtained according to equation 3:

the geographical location identification of the resource is represented as:

the geolocation of Hash (country) + Hash (region) + Hash (zip code) + Hash (detailed address) (4).

The preprocessing in S200 includes text segmentation and stop word removal.

Resource retrieval, comprising the steps of:

s601: preprocessing retrieval contents input by a user to obtain word segmentation information;

s602: inputting word segmentation information into well-trained LDA learning model M_ldaSubject identification is carried out to obtain r_topic＝{topic₁,topic₂,topic₃......topic_k}；

S603: selecting the theme with the maximum probability, and performing Hash operation on the theme and the geographic position to obtain resources

Representing a join operation;

s604: searching the position P of the resource stored in the P2P network according to the resource ID_i；

S605: from position P_iIn the resource list stored by the gateway node, the stored resource theme s is carried out_topicAnd request resource topic r_topicAnd calculating a score according to the corresponding weight, wherein the similarity calculation method adopts cosine similarity calculation between vectors, and the formula is as follows:

wherein s is_topic＝{topic₁:w₁,topic₂:w₂,topic₂:w₂,topic₂:w₂......topic_k:w_k}，

S606: if Sim_topicIf the position P is greater than or equal to the specified threshold value, the position P is output_iStored by the gateway node<NodeID, topic, resource description>。

Has the advantages that: compared with the prior art, the invention realizes a distributed WoT resource management mechanism by utilizing the P2P technology, and has the following advantages compared with a centralized management mechanism:

1. the problem that the system cannot run due to single-point failure of the centralized resource management server is avoided, and therefore the robustness of the system is achieved; the performance influence of the performance reduction of the centralized resource management server on the application of the Internet of things is avoided, so that the high availability of the system is realized; the system has a non-centralized structure and good elastic expandability. The system can automatically adapt to the scale change of the WoT application system, the system management and maintenance cost is low, the addition, the separation, the resource storage and the like of the nodes are automatically maintained by the system, and manual interference is not needed.

2. By utilizing the geographic position information, the gateways with close geographic positions are adjacent or close in the P2P network, and meanwhile, WoT resources are stored in related nodes in the geographic position space, so that the resource storage/retrieval efficiency is improved.

3. And extracting the theme in the resource description, and performing Hash calculation on the theme to obtain the resource ID, so that the resources with similar themes can be automatically stored in the same node or adjacent nodes, and the retrieval and storage of the resources are facilitated.

Drawings

FIG. 1: an LDA model;

FIG. 2: a WoT resource management architecture based on P2P;

FIG. 3: network construction based on geographic location P2P;

FIG. 4: a WoT resource ID generation process;

FIG. 5: a storage schematic of the resource;

FIG. 6: a retrieval process of resources;

FIG. 7: and (5) deleting and updating the resource.

Detailed Description

The invention is further elucidated with reference to the drawing.

The invention provides a method for realizing effective management of goods world wide web resources under the condition of large-scale network environment by combining P2P technology and goods world wide web technology. The invention mainly comprises the following steps:

1. the P2P technology is used to realize resource distributed management in the commodity world wide web environment. On the P2P network, the resource management of ROA facing to the resource architecture style is realized by using the WoT technology and idea, and the GET/PUT/DELETE/UPDATE operation is carried out to the resource.

2. The physical location of resources in WoT applications is a very important factor. In the system, the nodes with close physical positions are used as adjacent nodes, so that the node positioning and resource searching speed is increased.

3. Resource description topic modeling. Extracting Top K most important subjects from the resource description, selecting the most important subjects as service categories, carrying out Hash calculation on the category identification, constructing a resource ID identifier by combining information such as resource geographic position and the like, and comparing the ID with a P2P node ID to realize the storage and retrieval of the resource.

4. When the resources are stored, the resources with similar functions and the same geographic position are uniformly distributed in each node with similar geographic position as much as possible, and the node load balance in the P2P network is realized.

5. And realizing the life cycle management of resources, including resource access, resource registration, resource search, resource calling, resource updating and resource invalidation.

The system comprises the following steps:

firstly, in the environment of the internet of things, the characteristics of diversity of resources, coexistence of multiple protocols, wide distribution, huge quantity and the like are required, and in practical application, access management of multiple resources needs to be realized through a gateway. The functions of the gateway comprise various protocol access functions, protocol/data format conversion functions, data caching and processing functions, safety mechanisms and the like, and the complex and tedious functions are realized through the gateway, so that the distributed management, data acquisition, application development and the like of the resources of the Internet of things are simplified.

Secondly, in the system, a plurality of goods world wide Web resources are connected with the internet of things gateway, and the internet of things gateway is accessed to an Overlay network (Overlay network) established by the P2P technology in a Web mode, so that distributed resource management is realized.

Then, the present invention proposes that in a network environment based on P2P, a gateway node close to the physical location is used as a neighboring node, and when storing resources, the resources related to the physical location are stored in the corresponding physical location node as much as possible, which has the following advantages:

1. resources with closer geographic positions can be stored in node spaces with closer geographic positions in a more centralized manner;

2. when a resource needs to be updated, the efficiency is higher, and only part of nodes in the geographic position area of the resource need to be influenced.

When computing gateway node IDs in a P2P network, the traditional computing method is:

NodeID＝Hash(IP address+UDP port) (5)

routes between the gateway nodes are then established by calculating the distance between the gateway node IDs. The distance between the nodes is obtained by calculating the exclusive or operation between the gateway node IDs based on the P2P technology of Kademila, and the gateway node IDs are calculated by formula 3 in order to realize that nodes with closer geographical locations can be aggregated in the network. Thus, the result of the exclusive or operation of the previous same bit bits of the nodes having the same geographical position is 0, thereby indicating that the distance is close.

And secondly, based on the resource text description information, obtaining TopK topic information and the weight of the topic information in the main text through text analysis and a topic modeling algorithm, sequencing the topics according to the importance degree, selecting the most important part of the topics to perform Hash calculation, and constructing a resource ID together with the information such as the geographical position of the resource. The calculation method is shown in formula 1.

Next, implementing lifecycle management for the resources, including: and managing processes of storing and retrieving resources, calling resource services, updating resources, invalidation of resources and the like.

Fig. 1 is a potential Dirichlet Allocation model (LDA) which can extract a hidden topic in a text, thereby reducing high-dimensional data of the text to low-dimensional data of the hidden topic, and further implementing applications such as text classification by using technologies such as text similarity calculation. The LDA topic model is a generating type three-layer Bayes probability topic model, and the combination of potential topics described by texts is realized by dividing text description, topics and words contained in the topics. The model has good prior probability hypothesis, fixed training parameters and high-efficiency reasoning algorithm. The LDA model calculates posterior probability by using condition distribution and observable variables on the basis of prior distribution hypothesis according to a Bayesian probability distribution principle, considers that the document contains different distribution subject words in a document set, and the subject words generate a plurality of words in the document according to distribution.

As shown in FIG. 1, in the model, K represents the number of topics in the training document set, and α and β are two important parameters of the system. The outer boxes in this model represent the documents or service description text in the dataset and N is the number of documents or service descriptions in the document set or refers to the number of times the process is repeated. The inner boxes represent the topics or concepts implied therein, obtained from the text or service description. This process is repeated M times, and M can also be considered to be the number of words in the document. W in the filled circles represents observable words and z represents the subject that needs to be estimated. Φ (K) is the distribution of the terms over the subject K, which is related to the parameter β. θ (d) is a multinomial distribution of the implied topic over the text d, which is related to the parameter α. α and β are the two dirichlet constant hyperparameters over the Φ (K) and θ (d) distributions, respectively. These two parameters are trained from existing text sets. According to the model, in the LDA model, the generation process of the implied topic can be represented by formula 6:

wherein P (W)_j) Representing the generation probability of the jth word; p (Z)_k) Is the sampling probability of the implied topic K in the text set; p (W)_j|Z_k) Indicating the distribution of the Wj words with underlying subject Zk distributions. Where P (Wj | Zk) is represented by Φ (K) in the figure, and P (Z) is represented by θ (d) in the figure. These two parameters, as well as the topic z, can be estimated by a variety of training and learning methods, including variational bayesian networks, expectation-diffusion, and gibbs sampling, among others.

Fig. 2 shows a WoT resource management architecture and framework based on P2P, and the gateways mainly function as two nodes in a P2P distributed network on one hand and as sensing resource access responsible for the local of the gateways on the other hand. As can be seen from fig. 2, in order to implement the distributed management function of resources, the system implements access to various external devices through the application gateway of the internet of things. The gateway shields the diversity of various network protocols at the bottom layer and provides device resources or functions to the outside through a high-level API or a RESTful style API. The gateway supports the current mainstream Internet of things application transmission protocol, such as application layer protocols like CoAP/HTTP/WebSocket/MQTT/AMQP.

The gateway is responsible for establishing a resource model of the local access device and realizing the functions of storing, accessing, safety controlling, updating and the like of resources, and the model is realized based on a tree-shaped hierarchical structure.

The existing P2P technology is utilized to establish a distributed overlay network, realize P2P network connection of each gateway node, and establish a route between the gateway nodes.

When a system user sends a resource query request to the P2P network, the query request contains related information such as keywords or text description of the found resource.

The modules 5-8 in the architecture in fig. 2 realize the application development related functions of the internet of things on the basis of the commodity world wide web resource management based on P2P. The module 5 realizes the resource discovery function and returns the resource discovery function to the user in a sequence from high to low according to the resource matching scores by inputting request information and executing a resource matching algorithm. The module 6 is a resource monitoring module, which implements caching of the found resource list and monitors the resources, and when the resources are invalid or the freshness of the resources is overtime, the resources are removed from the resource list cache, and when no resource list exists, the module 5 is called to search for new resources from the P2P storage network. Module 7 is a resource service function scheduling module. The module provides service for the module 8, when a certain resource used in the business process is invalid, the optimal resource meeting the requirement is selected from the resource standby list according to a scheduling algorithm for the business process, so that the normal execution of the business process is ensured, and the availability of the business process is ensured. The module 8 is a business process module, which faces to the user and meets the actual requirements of the user by combining multiple resource services. The business process module comprises functions of a service combination module, a business execution module and the like. The service combination module realizes the functions of service combination process description, modeling, verification and analysis. The service execution module is responsible for the functions of scheduling execution, monitoring and the like of the service process.

Fig. 3 depicts a network construction process and method based on geographic location P2P. The Kademila protocol is adopted for implementation, and in order to realize that nodes close to the geographical position can also be adjacent in the P2P network, the node ID calculation is carried out by adopting the formula (3). The ID is 160 bytes, i.e., 20 bytes, in the Kademila protocol. The geographical position identification in the formula (4) is carried out by adopting a region classification method which is adopted according to daily life and is divided into four parts:

geographical identification Hash (country) + Hash (region) + Hash (postal code) + Hash (detailed address) (4)

Wherein, for the convenience of Hash calculation, the Hash result of each part is 4 bytes. And the other 4 bytes are obtained by Hash calculation according to the IP address and the protocol port. According to the Kademila protocol and the identification calculation method adopted by the invention, a, b, c and d in fig. 3 show the process of establishing the P2P network based on the geographical location. The first node firstly calculates the ID of the first node, and then contacts the guide node to inquire the information of the node closest to the first node. When just started, there are few nodes in the P2P network, and therefore, the network connection is established as shown in fig. 3 (a). As shown in fig. 3 (b), when a newly added node calculates its ID and calculates the node distance from a node having the same ID as the previous node, the distance is closer according to the principle of exclusive or. Therefore, when constructing a route based on such an ID, the nearest node is selected as a neighbor node, and thus, the resulting topology between network nodes is similar to that in fig. 3 (c). When more nodes join, the resulting P2P network structure is shown in fig. 3 (d). In the same area, the nodes with close geographical positions are mutually used as neighbor nodes, so that the efficiency of resource management is improved.

FIG. 4 depicts the generation of a resource ID for the purpose of storing resources in a P2P network. First, a user or a resource provider gives resource description information. At the same time, the system selects different data sets and generates different topic models using a topic model algorithm. These data sets may be generic, such as wikipedia-based document sets, or sensor-domain oriented document sets. The text in the data set needs to be preprocessed, and the preprocessing comprises text word segmentation, stop word removal and the like. Learning by using the existing corpus or data set through the hidden Dirichlet distribution algorithm to obtain the LDA learning model M_lda. The number K of the topics in the algorithm model needs to be adjusted and optimized according to application scenes, a corpus, the number of texts and the like. Inputting the word segmentation information of the resource description provided by the user to the learning model M_ldaAnd obtaining K topics described by the resources and the probability condition of the K topics in the document.

s_topic＝{topic₁:w₁,topic₂:w₂,topic₂:w₂,topic₂:w₂......topic_k:w_kTherein of

In the text or service category, the topic in which the probability is the highest is selected as the service or text category. And selecting a theme with higher probability of service or text as a classification mark of the resource, and performing Hash calculation on the theme as a resource ID part so as to realize storage and retrieval of the resource.

And respectively carrying out Hash calculation on the geographical position identification and the service category of the resource as the most critical parts to be used as the initial part of the resource ID, and carrying out Hash calculation on the rest subjects to be used as the rest part of the resource ID. The resource storage in this way can make the resources with close geographic positions and similar functions stored on the nodes which are close as possible. During resource storage, the node P2P is searched for storage according to the resource ID, and after the node is found, the triple information of the < NodeID, theme, resource description > of the resource is stored in the node. The result of the resource storage is shown in fig. 5.

Fig. 6 shows a resource retrieval algorithm, comprising the following steps:

1. preprocessing retrieval contents input by a user to obtain word segmentation information;

2. inputting word segmentation information into well-trained LDA learning model M_ldaSubject identification is carried out to obtain r_topic＝{topic₁,topic₂,topic₃......topic_k}；

3. Selecting the theme with the maximum probability, and performing Hash operation on the theme and the geographic position to obtain resources

Representing a join operation;

4. searching for and storing the type of resource according to the resource ID at P2Location P in P-network_i；

5. From position P_iIn the resource list stored by the gateway node, the stored resource theme s is carried out_topicAnd request resource topic r_topicAnd calculating a score according to the corresponding weight, wherein the similarity calculation method adopts cosine similarity calculation between vectors, and the formula is as follows:

6. If Sim_topicIf the position P is greater than or equal to the specified threshold value, the position P is output_iStored by the gateway node<NodeID, topic, resource description>。

When the resource changes, the resource needs to be updated in the P2P network. When the resource is updated, because the resource description may change, the theme of the resource description may also change correspondingly according to the implementation mechanism of the previous system, and therefore, the resource storage may also change correspondingly, and therefore, the resource update needs to include two processes, one process is to delete the previous resource, and the other process is to re-execute the resource registration process by using the resource as a new resource. The specific implementation is shown in fig. 7.

Claims

1. A WoT resource management method based on P2P is characterized in that: the method comprises the following steps:

s100: establishing a P2P network based on geographic positions, taking gateway nodes with close physical positions as adjacent nodes, and obtaining the distance between the nodes through the XOR operation between the gateway node IDs;

S400: inputting the word segmentation information in S200 into the learning model M_ldaObtaining K themes and probability conditions of the themes described by the resources;

in the formula, t₁、t₂、t₃Respectively representing service categories corresponding to different topics obtained from the resource description;

s600: comparing the resource ID in the S500 with the gateway node ID of P2P to realize the life cycle management of the resource;

the establishing of the P2P network based on the geographic position is realized by adopting a Kademila protocol, and the establishing method specifically comprises the following steps:

s102: selecting a nearest gateway node as a neighbor node to construct a route;

wherein the gateway node calculation is obtained according to equation 3:

2. the method for managing WoT resources based on P2P, according to claim 1, wherein the method comprises the following steps: lifecycle management for resources includes storage, retrieval, invocation, updating, and resource invalidation of resources.

3. The method for managing WoT resources based on P2P, according to claim 2, wherein the method comprises the following steps: when storing resources, searching for the P2P gateway node for storage according to the resource ID, and after finding the P2P gateway node, storing the < NodeID, theme, resource description > triple information of the resources in the P2P gateway node.

4. The method for managing WoT resources based on P2P, according to claim 1, wherein the method comprises the following steps: the geographical location identification of the resource is represented as:

5. The method for managing WoT resources based on P2P, according to claim 1, wherein the method comprises the following steps: the preprocessing in S200 includes text segmentation and stop word removal.

6. The method for managing WoT resources based on P2P, according to claim 2, wherein the method comprises the following steps: resource retrieval, comprising the steps of:

Representing a join operation;

s604: looking up the resources storing this type in the P2P network based on the resource IDPosition P_i；

the probability of the ith subject extracted from the description information of the resource requested by the user;

a probability representing the ith topic obtained from the description information of the resource provided by the resource provider; topic_iName, w, representing the ith topic obtained in the resource description information_iRepresenting the ith topic_iA corresponding probability;