CN113077023A - Automatic matching Internet of things system for group strings - Google Patents

Abstract

The invention discloses an Internet of things system for automatic matching of strings, which has the main function of selecting strings with weight and length indexes meeting standards through an improved k-means clustering algorithm to form a group of products. The method provided by the invention comprises the following steps: the method comprises four key contents of the overall architecture design of the automatic string matching Internet of things system, a streaming network transmission encoder, a method for constructing a similar graph and a heap-based k-nearest neighbor fast search algorithm. The invention firstly designs an integral matching control system, can realize data acquisition and carries out matching on a group string according to a clustering result; secondly, the invention provides a reliable transmission streaming network encoder based on hierarchical data, which improves the reliability and stability of system data transmission; in order to accurately describe the actual distribution of the data, the invention constructs a similar graph by reducing the similar graph, thereby improving the clustering performance; the invention also provides a quick search algorithm based on the heaped K neighbors, and the speed of system matching is improved.

Description

Automatic matching Internet of things system for group strings

Technical Field

The invention relates to the field of statistical analysis for researching the matching problem and the classification problem of industrial parts, in particular to a clustering analysis method for iterative solution.

Background

The current automatic matching mainly relates to the measurement of small and medium-sized molds, accessories and the like of machinery, tool prototypes and the like, and mainly comprises the precise detection of the size, the shape and the like of parts, so that a large amount of measurement data is generated. Although it becomes easy and inexpensive to collect stored data due to the development of computers and cloud platforms, large-scale data analysis remains a formidable challenge. The manual analysis of mass measurement data is unrealistic, so a data analysis technology for extracting useful information to help production is developed.

The invention optimizes the traditional classification analysis method and reduces the solving space by using a heuristic algorithm, thereby providing a reasonable feasible solution under the acceptable calculating time and calculating the optimal solution of the string grouping.

Clustering is a main method under an unsupervised learning category, and the task of the method is to group object sets into clusters according to similarity defined by a user, under an ideal condition, the clusters can express natural grouping of the objects, the natural grouping is objective and does not depend on external factors such as objects, representations and the like, similar samples are classified into the same cluster, and the similarity of sample points of different clusters is far smaller than the similarity between points of the same cluster. The cluster structure obtained at the end is determined by the definition of the similarity, generalOften defined on a distance function. Mathematically, from the N string of groups, K qualifying groups are found, the solution space for the problem is 2^NThe problem belongs to a combinatorial optimization problem, is NP-hard, and does not have a polynomial time solving algorithm.

The construction of the similar graph is the first and most important step in the process of the spectral clustering algorithm, and whether the similar graph can accurately reflect the internal relation of the data and accurately describe the actual distribution of the data plays a crucial role in the final clustering result. The proper similarity graph is beneficial for the whole algorithm to return better clustering results.

Disclosure of Invention

The invention aims to solve the technical problem of designing a cluster matching Internet of things system aiming at the defects involved in the background technology, wherein the system is improved based on the selection of an initial clustering center of a K-means algorithm, so that the distribution characteristics of a data set can be well depicted, the spectral clustering algorithm is improved in two aspects, firstly, unreliable similar connections are removed by using additional neighborhood accumulated information to improve the reliability and the authenticity of clustering results, secondly, a heap data structure is used for quickly searching out a K-of-sight neighborhood of each sample point, and the execution efficiency which is far higher than that of the original sequencing-based classical algorithm is obtained; and secondly, removing unreliable similar connections by using additional neighborhood accumulated information to improve the reliability and the authenticity of the clustering result.

The invention adopts the following technical scheme for solving the technical problems:

the utility model provides a cluster automatic selection thing networking control system, this system overall implementation frame is: the Internet of things integrates coordination control and intelligent management and control of online detection, a network transmission system, a matching system and the like, wherein the online detection system comprises a string device information acquisition module, an RFID read-write storage module and a string positioning module; the network transmission system comprises network transparent transmission equipment, a network convergence module and a storage database; the matching system comprises a server cluster and a mechanical control module. The system realizes the matching function flow as follows:

step 1: placing the strings on a stepping assembly line through a manipulator;

step 2: gap detection, which is to measure the length and the gap through a sensor group string;

and step 3: NG elimination, wherein unqualified NG strings are transmitted out of the production line;

and 4, step 4: length calibration, writing the length of the string measured in step 2 into the RFID label of the string;

and 5: static weighing, wherein the string group is statically weighed, and the string group is weighed to obtain the weight to be written into the RFID label;

step 6: n group strings are pushed into a sorting disc through an air cylinder, a background program controls a manipulator to carry out matching, and a background program algorithm carries out matching on the group strings on the sorting disc;

and 7: and the manipulator classifies the matched group strings.

A kind of stream network encoder based on reliable transmission of Internet of things data of hierarchical data in real time, the encoder encoding strategy is: the coding coefficient generation mode of the encoder avoids frequent random coefficient generation work, reduces coding time delay, enables the coding coefficient to be transmitted without being attached in a data packet through negotiation between a sender and a receiver, reduces invalid load and waste of bandwidth, and improves transmission friendliness. When the encoding operation is executed, following the progressive strategy, after the original data packets are grouped through the data packet grouping module, the encoder starts the encoding operation as long as the original data packets are received, and the encoder does not need to wait for all the original data packets in the encoding group to arrive.

Meanwhile, the stream type network encoder introduces a stream type calculation mechanism into multi-path transmission, solves the problem of disorder of data packets of a receiving party in the multi-path transmission and simultaneously enables a solution based on network encoding to be firmer in real time and more reliable. The embedded coding coefficient selection and generation mechanism reduces the complexity of coding and decoding and reduces the invalid load in multi-path transmission.

A similarity graph is constructed based on neighborhood coexistence similarity graph reduction, and the similarity graph construction method comprises the following steps:

defining a cumulative matrix C according to the method of reducing the similarity graph by the neighborhood coexistence information:

since each piece of data in the data set corresponds to a point, each point has a k neighborhood which belongs to the point independently, and the total number of n neighborhoods is accumulated. The number of times that any two points appear together in the neighborhood is necessarily less than the total number of neighborhoods, i.e. t_ijN is less than or equal to n. And since coexistence is mutual, the calculated coexistence accumulation matrix is symmetric along the diagonal, i.e., t_ij＝t_ji. When a pair of points co-occur in the k neighborhood, the side indicates that they are close together, the more common the number of co-occurrences. Each point of the pair is accumulated (x)_i，x_j) The times T (T < n) of common occurrence in the same neighborhood are set as a threshold value T, and T (x) is truncated_i，x_j) < T Point pairs (x)_i，x_j) And unreliable edges are arranged between the two adjacent images, so that the connection of similar images is more stable and reliable.

In a symmetric kNN diagram, each pair of points (x)_i，x_j) But not in the neighborhood of each other or each other. Point-to-point pair (x)_i，x_j) Co-occurrence at a third party point x_kK neighborhood kNN (x)_k) In (x)_i，x_j) Is part of the same neighborhood, implying x_iAnd x_jMay also be similar to each other. If (x)_i，x_j) It is more likely that they will be similar to each other in the k neighborhood of the points. The neighborhood co-existence matrix is able to capture more such point pairs (x)_i，x_j) The phases of the graph nodes are related to information, so that the similarity or the difference between the graph nodes can be determined more.

Establishing a similarity matrix common Gaussian kernel to convert the sparse distance matrix into an affinity matrix A, wherein only non-zero elements are operated:

distinds is a non-zero element value of a distance matrix, δ is a manually set scale parameter, good δ can gather points of the same cluster closer, and points of different clusters farther, and the method uses radius information of a neighborhood as the assistance of the scale, and is expressed as follows:

δ＝mean(Dist(i，j))，P(i，j)＞τ

the scale selection fully utilizes neighborhood information, can ensure that similar nodes are connected more closely, and different nodes are isolated farther.

A quick search algorithm based on heaped K neighbors comprises the following search strategies:

the k neighbors are neighbors in the vicinity of an object, i.e. k sets of objects S are located before the nearest object in a given distance space. The purpose of the neighbor search of each object is to find the k objects, which constitute point x_iK of (a) is adjacent to the cluster S.

The strategy used here is to divide the point x in the sample space U by_iThe first k points, outside itself, initialize a set S, iteratively comparing and replacing the remaining points with the farthest points in S. Let p_kIs in S from x_iThe element with the greatest distance traverses the point in U-S, if closer to x_iReplace p in S with a new point_kAnd re-searching for the farthest point p in S_k. The new farthest point may or may not be the newly added element. After the traversal compares all the points in the surplus, S is the nearest k neighbors. S can be abstracted as a binary heap (heap) that can speed up the search for the farthest point p_kThe method can be obtained by directly acquiring the heap top, and the whole process can achieve the purpose of accelerating the search of k nearest neighbors of all points. For each sample point x_iApplying a large top pile with the size of k for storing k nearest points, initializing the pile by using the marks of k nodes before the distance array of the ith row of the distance matrix, iteratively comparing the distances of the rest sample points with the distance of the top of the pile, replacing the top of the pile and sinking to adjust the pile if the distance is less than the top of the pile, and otherwise, discarding the pile.

Traversing, comparing and replacing x_iAfter the distance to all sample points, the element in the stack is x_iThe nearest k neighbors of.

By utilizing the proposed quick search based on the heap K neighbors, the strategy is realized by the following steps:

step 1: selecting a first cluster center c by using a cluster center automatic selection strategy;

step 2: the master broadcasts a cluster center c to all workers;

and step 3: each worker finds a point farthest from the cluster center cluster c in the local sample points of the worker;

and 4, step 4: selecting one from the master as a new cluster center c;

and 5: each worker divides a local sample point into each cluster and calculates a new family center;

step 6: circulating the step for 1-5k times;

and 7: and obtaining the final k cluster centers and the cluster to which all the sample points belong.

Compared with the prior art, the invention adopting the technical scheme has the following technical effects:

1. the invention designs and realizes an automatic string-grouping matching Internet of things system, which solves the problems of time and labor consumption in string-grouping matching and improves the production efficiency.

2. The invention utilizes the stream type network encoder to complete the design of a novel transmission mechanism, and avoids the waste of invalid load to bandwidth. The problem of data packet disorder of a receiver is solved, and the purposes of effectiveness and high efficiency are achieved.

3. The method generates a better clustering result by utilizing the similar graph pruned by the assistance of the neighborhood information, and the similar graph cut by the threshold value is more sparse, so that the decomposition of the spectral clustering characteristic quantity can be accelerated.

4. The invention is based on the heap K nearest neighbor fast search, the time complexity of the K nearest neighbor of each point is O (n log K), and the overall worst complexity of the K nearest neighbor search of all points is O (n log K)²log k), the theoretical efficiency is higher than that of conventional spectral clustering.

Drawings

FIG. 1 is an architecture of an automated configuration Internet of things of the present invention;

FIG. 2 is a schematic flow chart of a streaming network encoder

FIG. 3 is a sample similarity calculation implementation based on Spark

FIG. 4 is a schematic diagram of the RDD Cartesian product

FIG. 5 is a schematic diagram of distributed similarity matrix calculation and storage;

FIG. 6 is a flow chart of an implementation of the fitting algorithm.

Detailed Description

The technical scheme of the invention is further explained in detail by combining the attached drawings:

1) as shown in fig. 1, the system architecture includes 3 parts, the data acquisition control layer also has three parts, the device and detection service is responsible for collecting the length and weight of the group string, the warehousing and RFID service reads and writes the record information of the group string through RFID, the mobile robot service locates the position of the group string on the production line through wireless devices, so as to control the device to classify or reject the group string; the data transmission layer is provided with an edge server at each acquisition layer device to store and control the acquisition devices, and then the data transmission layer is converged to the switch to be uploaded to a big data center; the storage computing layer comprises an online service cluster and a storage cluster, and the main task is to compute the matching clustering algorithm of the string and store the data of the string, and then return the result after matching.

Fig. 2 is a schematic diagram of the flow of the streaming network encoder, including the generation and selection of coefficients, the generation of encoded packets and redundant encoded packets. The data packet grouping module provides the original data packet, the coding group size N estimated in real time and the number R of the redundant coding packets to the streaming network coder, so that the streaming network coder can execute the streaming network coding operation. By using a high-efficiency coding coefficient generation and selection mechanism, according to N, R and information carried by an original data packet, a coding packet or a coefficient vector corresponding to a redundant coding packet is selected from coefficient matrixes (a coding packet coefficient matrix and a redundant coding packet coefficient matrix), and then a streaming network coding operation is performed.

The key stage of the operation of the streaming network encoder is to find the coefficient vectors corresponding to the coding packets and the redundant coding packets through the efficient coding coefficient generation and selection rules according to N, R and the information carried by the data packets.

2) As shown in fig. 3, the sample similarity calculation input is a matrix of n × d, where n represents the number of sample points and d represents the dimension of the sample points, each point being assigned to a different node. And after the Cartesian product is calculated, more partitions are generated, the point pairs are dispersed into each partition, and the distance and similarity between the point pairs are calculated on each partition. In the figure, Similarity (4, i) represents the Similarity of the 4 th sample point and the ith sample point, and Similarity (i +1, n) represents the Similarity of the i +1 th point and the nth point, which are calculated in parallel on different partitions and different machine nodes. Wherein the Euclidean distance is expanded as follows:

the two-normal form of all points can be pre-computed

To save time. And after the distance of each partition is calculated, introducing a thermonuclear function to map the thermonuclear function into similarity, searching local k neighbors of each partition by using a fast k neighbor search algorithm, and finally, reducing by a master host to find a global k neighborhood. And finally, further thinning the distributed similarity matrix S by using neighborhood accumulated information.

3) Distributed similarity matrix calculation and storage: based on the initial data set data in the memory, after a Cartesian method of RDD is used for carrying out Cartesian product on the data, the method can generate new RDD of RDD [ (Int, Vector), (Int, Vector) ] type, and sample point pairs are hashed to a plurality of nodes of the cluster and distributed on more partitions. As shown in FIG. 4, the large rectangle on the left side of the graph represents the RDD, and the small rectangle nested inside represents a partition, which contains the initial sample point. The cartesian product is shown on the right, and the generated sample point pairs are distributed in the cluster, with the generated point pairs between the old RDDs being stored in one partition.

Then, taking the index pair (i, j) as a key, calculating the distance between every two Vector components in parallel by using the fastSquare distance and the Vector sqdist Euclidean distance of MLUtil on the two Vector components of each row as a value by a reduceByKey method, and storing the corresponding index and the distance (r, c, v) to form a new RDD (matrix entry)]. Passing the result RDD into the fabricThe manufactured function may be converted to CoordinateMatrix and the distributed similarity matrix partitions and converts the matrix into IndexRowMatrix type using a toindedexRowMatrix function, the memory structure being shown in FIG. 5. Then, another map converts each element into similarity by using a Gaussian thermal kernel function, the k neighbor of each sample point is efficiently found out in each partition by using an improved k neighbor fast search algorithm, and finally, v is obtained through reduction_iK for a point is nearest neighbor. And finally, according to the neighborhood accumulated information, the unreliable similar connections are further cleared to be zero so as to sparsify the similar matrix, and the sparser similar matrix has higher efficiency for solving the Laplace matrix and decomposing the features.

4) K-means clustering is performed in the transformed easily separable data space, the overall process is shown in fig. 6, the previous step is transformed into a new feature space, each sample point has a lower dimensionality and stronger separability. The parallel k-means extracts k sample points from it as the initialization center points for the k clusters. Broadcasting cluster centers to all worker nodes, calculating Euclidean distances between local data samples and k cluster centers in parallel by each node, and dividing each sample into a cluster closest to the cluster center. And calculating the mean value of the cluster in each iteration process to update the cluster heart variable. And iterating until the error estimation of the clustering is smaller than a set threshold value or the iteration times exceed a set iteration time limit, stopping circulation and outputting a final clustering analysis result.

The invention creates the following main protection technical points:

(1) the system is suitable for the Internet of things control, transmission, storage and calculation system for automatic group string matching.

(2) And a hierarchical data stream encoder is adopted for data transmission, so that the reliability of data transmission is ensured.

(3) The data processing adopts the similar image reduction structure similar image based on neighborhood coexistence, and the original k neighbor neighborhood with fixed size is replaced by the neighborhood capable of reflecting the local structure, thereby improving the reliability and stability of the similar image.

(4) Compared with the standard spectral clustering, the fast k-nearest neighbor search based on the heap has greatly improved execution efficiency.

It will be understood by those skilled in the art that, unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs. It will be further understood that terms, such as those defined in commonly used dictionaries, should be interpreted as having a meaning that is consistent with their meaning in the context of the prior art and will not be interpreted in an idealized or overly formal sense unless expressly so defined herein.

The above-mentioned embodiments, objects, technical solutions and advantages of the present invention are further described in detail, it should be understood that the above-mentioned embodiments are only illustrative of the present invention and are not intended to limit the present invention, and any modifications, equivalents, improvements and the like made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims (6)

1. The whole architecture of the networking system for automatically selecting and matching the group strings is shown in figure 1, and the networking system comprises terminal data acquisition and control design, data transmission and storage design and algorithm realization and optimization design.

2. As in claim 1, the data transmission process is as shown in fig. 2, the streaming network encoder introduces a streaming computation mechanism into the multi-path transmission, wherein the embedded coding coefficient selection and generation mechanism reduces the coding and decoding complexity and reduces the invalid load in the multi-path transmission.

3. The data packet grouping module of claim 2, wherein the data packet grouping module provides the original data packet, the real-time estimated coding group size N and the number R of redundant coding packets to the streaming network encoder so that the streaming network encoder can perform the streaming network encoding operation.

4. The method of claim 1, wherein the algorithm optimization comprises similarity map reduction and optimization improvement based on a clustering algorithm, and the optimization is calculated by a parallel calculation and a storage optimization algorithm.

5. The method of claim 4, wherein the cumulative matrix C is:

each piece of data of the data set corresponds to one point of the matrix, each point is provided with one independent k neighborhood, and therefore the total number of accumulated n neighborhoods is total. The number of times that any two points appear together in the neighborhood is necessarily less than the total number of neighborhoods, i.e. t_ij≤n。

6. The method of claim 4, wherein the heap-based clustering optimization algorithm uses a division point x in the sample space U_iThe first k points, outside itself, initialize a set S, iteratively comparing and replacing the remaining points with the farthest points in S.

Images