CN117640695A - Internet of things communication platform and Internet of things communication method based on communication identification information - Google Patents
Internet of things communication platform and Internet of things communication method based on communication identification information Download PDFInfo
- Publication number
- CN117640695A CN117640695A CN202311710789.5A CN202311710789A CN117640695A CN 117640695 A CN117640695 A CN 117640695A CN 202311710789 A CN202311710789 A CN 202311710789A CN 117640695 A CN117640695 A CN 117640695A
- Authority
- CN
- China
- Prior art keywords
- module
- internet
- voice
- things
- communication
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000004891 communication Methods 0.000 title claims abstract description 127
- 238000000034 method Methods 0.000 title claims abstract description 29
- 238000012545 processing Methods 0.000 claims abstract description 23
- 239000011159 matrix material Substances 0.000 claims description 47
- 239000013598 vector Substances 0.000 claims description 46
- 230000004927 fusion Effects 0.000 claims description 33
- 238000000605 extraction Methods 0.000 claims description 31
- 230000008909 emotion recognition Effects 0.000 claims description 25
- 238000010276 construction Methods 0.000 claims description 21
- 230000008451 emotion Effects 0.000 claims description 19
- 238000013136 deep learning model Methods 0.000 claims description 17
- 238000010586 diagram Methods 0.000 claims description 16
- 238000005516 engineering process Methods 0.000 claims description 13
- 238000013527 convolutional neural network Methods 0.000 claims description 12
- 238000001228 spectrum Methods 0.000 claims description 12
- 238000005457 optimization Methods 0.000 claims description 10
- 230000011218 segmentation Effects 0.000 claims description 9
- 230000006835 compression Effects 0.000 claims description 8
- 238000007906 compression Methods 0.000 claims description 8
- 238000002156 mixing Methods 0.000 claims description 8
- 238000001914 filtration Methods 0.000 claims description 6
- 230000006870 function Effects 0.000 claims description 6
- 238000007781 pre-processing Methods 0.000 claims description 6
- 230000002708 enhancing effect Effects 0.000 claims description 5
- 238000004364 calculation method Methods 0.000 claims description 4
- 238000013507 mapping Methods 0.000 claims description 3
- 238000005070 sampling Methods 0.000 claims description 3
- 230000003993 interaction Effects 0.000 description 4
- 230000008569 process Effects 0.000 description 4
- 238000007418 data mining Methods 0.000 description 2
- 238000000354 decomposition reaction Methods 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000005540 biological transmission Effects 0.000 description 1
- 238000013079 data visualisation Methods 0.000 description 1
- 238000005286 illumination Methods 0.000 description 1
- 238000010801 machine learning Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000010606 normalization Methods 0.000 description 1
- 230000009467 reduction Effects 0.000 description 1
Classifications
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D30/00—Reducing energy consumption in communication networks
- Y02D30/70—Reducing energy consumption in communication networks in wireless communication networks
Landscapes
- Telephonic Communication Services (AREA)
Abstract
The invention relates to the field of Internet of things and discloses an Internet of things communication platform and an Internet of things communication method based on communication identification information, wherein the platform comprises Internet of things equipment, a cloud server, an intelligent equipment management module, an application server, a network server and an archive information manager; the Internet of things communication method based on the communication identification information comprises the following steps: collecting various types of data and converting the data into digital signals; storing, analyzing, processing and displaying the data converted into the digital signal, and providing services and applications; intelligent management is carried out on the equipment of the Internet of things; exchanging communication data with the internet of things device in a plurality of ways; receiving, processing and forwarding communication data sent by an application server; and the database is used for storing and managing preset communication identification information, and distributing communication data to corresponding network servers according to different communication identification information. Personalized and intelligent service for users is realized, and user experience and satisfaction are improved.
Description
Technical Field
The invention relates to the field of the Internet of things, in particular to an Internet of things communication platform and an Internet of things communication method based on communication identification information.
Background
The internet of things (Internet ofThings, ioT) refers to a technology that connects various intelligent devices and services through the internet to enable exchange and sharing of information. The communication of the Internet of things refers to the process of data transmission and interaction between equipment and services in the Internet of things, and is the core and foundation of the Internet of things. The Internet of things communication platform is a platform for providing Internet of things communication services, and can realize functions of discovering, connecting, authenticating, authorizing, managing, optimizing and the like of Internet of things equipment and services, and ensure the safety, reliability, efficiency and flexibility of Internet of things communication.
The communication identification information is information for identifying the equipment and the service of the Internet of things, and comprises equipment identification, service identification, position identification and the like. The communication identification information can be used for realizing the positioning and identification of the equipment and the service of the Internet of things, and improving the accuracy and individuation of the communication of the Internet of things.
Chinese patent CN103685467B discloses an interconnection and interworking platform of internet of things and a communication method thereof, which accesses an intelligent device located in each global area to the platform through a portal server, communicates with a proxy device on the platform or other intelligent devices accessing the platform, and can uniformly access an infinite number of intelligent devices to the internet through the platform. However, the above platform has the following disadvantages: the increasing number and types of the internet of things equipment and services lead to the increasing of the complexity and difficulty of the internet of things communication, and a more efficient and intelligent communication management and optimization method is needed.
For the problems in the related art, no effective solution has been proposed at present.
Disclosure of Invention
In view of this, an object of the embodiments of the present application is to provide an internet of things communication platform and an internet of things communication method based on communication identification information, so as to overcome the above technical problems existing in the related art.
For this purpose, the invention adopts the following specific technical scheme:
according to one aspect of the invention, an Internet of things communication platform based on communication identification information is provided, and comprises Internet of things equipment, a cloud server, an intelligent equipment management module, an application server, a network server and an archive information manager.
The internet of things device is used for collecting various types of data and converting the data into digital signals.
The cloud server is used for storing, analyzing, processing and displaying the data converted into the digital signals and providing services and applications.
And the equipment intelligent management module is used for intelligently managing the equipment of the Internet of things.
And the application server is used for exchanging communication data with the Internet of things equipment in a plurality of modes.
And the network server is used for receiving, processing and forwarding the communication data sent by the application server.
And the archive information manager is used for storing and managing a database of preset communication identification information and distributing communication data to the corresponding network server according to different communication identification information.
Optionally, the intelligent equipment management module comprises an instruction acquisition module, a word meaning extraction module, a voice feature extraction module, a classification module and an emotion recognition module;
the instruction acquisition module is used for acquiring a plurality of rounds of dialogue information of the user and the Internet of things equipment and extracting n dialogue instructions corresponding to the rounds of dialogue information, wherein the dialogue instructions refer to sentences or commands of the user which express requirements through voice and text input;
the word sense extraction module is used for extracting text content from the dialogue instruction and converting the text content into word vectors by utilizing a word embedding technology;
the voice feature extraction module is used for extracting voice features from the dialogue instruction by utilizing the compression delay network and combining the voice features with corresponding word vectors to form mixed features;
the classification module is used for positioning the key points of the voice and giving out a requirement classification result according to a preset key word library through a convolutional neural network;
and the emotion recognition module is used for recognizing the emotion of the mixed characteristic.
Optionally, the word sense extraction module comprises a text extraction module, a word embedding module and a semantic splicing module;
the text extraction module is used for segmenting the text content into a plurality of single words by using a word segmentation technology;
the word embedding module is used for mapping each word into a high-dimensional dense vector by using a word embedding technology and representing semantic information;
and the semantic splicing module is used for splicing all word vectors representing semantic information into vectors with fixed length and used as vector representations of text contents.
Optionally, the voice feature extraction module comprises a voice feature extraction module, an energy spectrum acquisition module, a filtering module, a voice splicing module and a mixing module;
the voice feature extraction module is used for extracting voice features from the dialogue instruction, dividing the voice features into frames with fixed lengths, and each frame comprises a plurality of sampling points;
the energy spectrum acquisition module is used for carrying out fast Fourier transform on each frame of voice characteristics to obtain an energy spectrum on a frequency domain;
the filtering module is used for carrying out filter bank processing on the energy spectrum to obtain a Mel frequency cepstrum coefficient as a characteristic vector of the voice signal;
the voice splicing module is used for splicing a plurality of adjacent mel frequency cepstrum coefficient characteristic vectors into a high-dimensional vector and inputting the high-dimensional vector into the compression delay network;
and the mixing module is used for taking the output of the compressed time delay network as a voice characteristic and splicing the voice characteristic with the vector representation of the text content to obtain a mixed characteristic.
Optionally, in the classification module, the convolutional neural network is used for processing the mixed features, the mixed features are learned through a convolutional layer, and the mixed features are mapped into the requirement classification result through a fully connected layer.
Optionally, the emotion recognition module comprises a graph construction module, an optimization module and an emotion prediction module;
the diagram construction module is used for constructing two diagram structures by utilizing voice characteristics and text contents and respectively representing an acoustic mode and a text mode;
the optimizing module is used for optimizing, reconstructing and fusing the text content and the acoustic features by utilizing the GCN deep learning model to obtain a final fusion graph;
and the emotion prediction module is used for predicting node labels of the fused graphs by utilizing the graph enhancement and graph convolution network to realize voice emotion recognition.
Optionally, the graph construction module comprises a voice feature graph construction module and a text content graph construction module;
the voice feature diagram construction module is used for constructing a voice feature diagram structure through voice features, wherein the voice feature diagram structure G= (V, A), V represents a node set of the voice features, A represents an adjacent matrix of the voice features, and the element of the adjacent matrix is the similarity between two nodes;
the similarity calculation formula is:
wherein a is i A j Respectively representing the ith and the jth voice feature vectors;
||a i -a j || μ representing the euclidean distance, μ being the average of the euclidean distances for all speech features, d representing a scale parameter, exp representing an exponential function.
The text content graph construction module is used for constructing a text content graph structure through text content and defining a node set of the text content and an adjacency matrix of the text content.
Optionally, the optimization module comprises a preprocessing module, a reconstruction module and a multi-mode feature fusion module;
the preprocessing module is used for optimizing node characteristics of a graph constructed by text contents by utilizing a GCN deep learning model, normalizing an adjacent matrix of the text contents and obtaining optimized text content representation;
the reconstruction module is used for reconstructing the adjacent matrix based on the text content and the acoustic feature by utilizing the node feature of the second hidden layer of the GCN deep learning model to obtain a reconstructed text content adjacent matrix and an acoustic feature adjacent matrix, and fusing the text content adjacent matrix and the acoustic feature adjacent matrix to obtain a fused adjacent matrix;
and the multi-mode feature fusion module is used for fusing the text content after reconstruction optimization with the non-optimized acoustic features to obtain final fusion features, and constructing a final fusion graph through the adjacent matrix after fusion and the final fusion features.
Optionally, the emotion prediction module comprises a graph enhancement module and a label prediction module;
the image enhancement module is used for enhancing the node feature matrix of the final fusion image for a plurality of times through random propagation and carrying out feature propagation to obtain a plurality of enhanced feature matrices;
the label prediction module is used for inputting the feature matrix of each enhancement chart into the GCN deep learning model to perform node label prediction to obtain a prediction label matrix of each enhancement chart, and calculating the average value of the prediction labels of each enhancement chart.
According to another aspect of the present invention, there is provided an internet of things communication method based on communication identification information, the internet of things communication method based on communication identification information including the steps of:
various types of data are collected and converted into digital signals.
And storing, analyzing, processing and displaying the data converted into the digital signal, and providing services and applications.
And intelligent management is carried out on the Internet of things equipment.
Communication data is exchanged with the internet of things device in several ways.
And receiving, processing and forwarding the communication data sent by the application server.
And the database is used for storing and managing preset communication identification information, and distributing communication data to corresponding network servers according to different communication identification information.
Embodiments of the present invention include the following beneficial effects:
(1) According to the Internet of things communication platform and the Internet of things communication method based on the communication identification information, data of the Internet of things communication platform are collected and processed, so that data mining and utilization of Internet of things equipment and services are realized, and rich and various functions and services are provided for users; communication data are exchanged with the Internet of things equipment in a plurality of modes, so that the Internet of things equipment and services can be interconnected and intercommunicated, various communication protocols and network technologies are supported, and the safety and reliability of the Internet of things communication are ensured; communication data are distributed to corresponding network servers according to different communication identification information through a database for storing and managing preset communication identification information, so that communication positioning and identification of the Internet of things equipment and services can be realized, and the accuracy and individuation of Internet of things communication are improved.
(2) According to the invention, the intelligent management is carried out on the Internet of things equipment through the equipment intelligent management module, the interaction instruction of the user and the Internet of things equipment is obtained, the requirements of the user using the Internet of things equipment can be intelligently identified, personalized and intelligent service to the user is realized, the user experience and satisfaction are improved, the characteristics are mapped into the requirement classification result through the full connection layer of the convolutional neural network through the mixing of voice characteristics and text contents and the learning characteristics of the convolutional neural network, and the accuracy of identification is improved.
(3) According to the emotion recognition method and device, through the emotion recognition module, the voice feature map is built, the text content map is built, the map enhancement and the map fusion are carried out, the emotion attribute of the user using the Internet of things equipment is obtained, the emotion recognition accuracy is improved, and the complexity of the emotion recognition module is reduced. The method comprises the steps of constructing two graph structures by utilizing voice features and text contents, respectively representing acoustic modes and text modes, realizing multi-mode representation of voice and text information, enhancing the expressive power and richness of information, optimizing, reconstructing and fusing the text contents and the acoustic features by utilizing a GCN deep learning model to obtain a final fusion graph, realizing multi-mode fusion of the voice and the text information, improving the consistency and complementarity of information, and realizing voice emotion recognition by carrying out node label prediction on the fused graph by utilizing a graph enhancement and graph convolution network, so that classification or regression of emotion or emotion in voice can be realized, and the accuracy and robustness of emotion recognition are improved.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings that are needed in the embodiments will be briefly described below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.
Fig. 1 is a schematic block diagram of an internet of things communication platform based on communication identification information according to an embodiment of the present invention.
In the figure:
1. the Internet of things equipment; 2. the cloud server; 3. an intelligent equipment management module; 4. an application server; 5. a network server; 6. and a archive information manager.
Detailed Description
In order to make the objects, technical solutions and advantages of the present application more apparent, the present application will be further described in detail with reference to the accompanying drawings and examples. It should be understood that the specific embodiments described herein are for purposes of illustration only and are not intended to limit the present application.
For the purpose of further illustrating the various embodiments, the present invention provides the accompanying drawings, which are a part of the disclosure of the present invention, and which are mainly used to illustrate the embodiments and, together with the description, serve to explain the principles of the embodiments, and with reference to these descriptions, one skilled in the art will recognize other possible implementations and advantages of the present invention, wherein elements are not drawn to scale, and like reference numerals are generally used to designate like elements.
Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing embodiments of the present application only and is not intended to be limiting of the present application.
According to the embodiment of the invention, an Internet of things communication platform and an Internet of things communication method based on communication identification information are provided.
Referring to the drawings and the detailed description, as shown in fig. 1, according to an embodiment of the present invention, an internet of things communication platform based on communication identification information is provided, where the internet of things communication platform based on communication identification information includes an internet of things device 1, a cloud server 2, a device intelligent management module 3, an application server 4, a network server 5 and an archive information manager 6.
The internet of things device 1 is configured to collect various types of data, such as temperature, humidity, illumination, sound, position, and the like, and convert the data into digital signals.
The cloud server 2 is used for storing, analyzing, processing and displaying the data converted into the digital signals, and providing services and applications, such as data visualization, remote control, intelligent decision making and the like.
The device intelligent management module 3 is used for intelligently managing the Internet of things device;
in a further embodiment, the device intelligent management module 3 includes an instruction acquisition module, a word sense extraction module, a voice feature extraction module, a classification module and an emotion recognition module;
the instruction acquisition module is used for acquiring a plurality of rounds of dialogue information of the user and the Internet of things equipment, and extracting n dialogue instructions corresponding to the rounds of dialogue information, wherein the dialogue instructions refer to sentences or commands which express requirements and are input by the user through voice and text.
And the word sense extraction module is used for extracting text contents from the dialogue instruction and converting the text contents into word vectors by utilizing a word embedding technology, wherein the word embedding technology reflects semantic relations and similarity among words by converting each word in the text contents into numerical vector representations.
The voice feature extraction module is used for extracting voice features from the dialogue instruction by utilizing the compression delay network, can use the full-connection layer to reduce the dimension of the voice features, and is combined with corresponding word vectors (word vectors indicating text content in the word sense extraction module) to form mixed features, wherein the compression delay network is a network capable of compressing voice signals into low-dimension vector representation without losing voice quality, and improves the efficiency and accuracy of voice processing; and simultaneously, a word embedding or sentence embedding method is used for encoding the text content to obtain a text feature vector with the same dimension as the voice feature vector. And then splicing the two feature vectors by using a splicing method to obtain a high-dimensional mixed feature vector, and then performing dimension reduction processing on the high-dimensional mixed feature vector by using a full-connection layer.
And the classification module is used for positioning the voice key points according to a preset key word library through the convolutional neural network, giving out a requirement classification result, finding out the key words most similar to the key word library, and taking the key words as the voice key points. And judging the requirements of the participants in the interaction scene according to the difference of the voice key points, and giving corresponding feedback. The convolutional neural network is a network capable of extracting local features from input data and carrying out hierarchical combination, and can realize efficient classification or regression of data such as voice or images.
And the emotion recognition module is used for recognizing the emotion of the mixed characteristic. In a further embodiment, the word sense extraction module comprises a text extraction module, a word embedding module and a semantic stitching module;
the text extraction module is used for segmenting the text content into a plurality of single words by using word segmentation technology such as barker word segmentation (Jieba); for example, clause segmentation is performed on the input text content by using barker words, and the text content is segmented into a plurality of clauses by using a regular expression with punctuation marks or non-Chinese characters as boundaries. Then, the resultant word segmentation scans the word graph of each clause, and a Directed Acyclic Graph (DAG) is generated by using a prefix dictionary to represent the situation that all Chinese characters in the clause are possible to form words. And the barking word segmentation uses a dynamic programming algorithm to search the maximum probability path on the word graph, calculates the probability of each segmentation scheme according to the occurrence frequency of the words, and selects the scheme with the maximum probability as the final word segmentation result.
The Word embedding module is used for mapping each Word into a high-dimensional dense vector by using Word embedding technology such as Word2Vec or GloVe and representing semantic information;
and the semantic splicing module is used for splicing all word vectors representing semantic information into vectors with fixed length and used as vector representations of text contents.
In a further embodiment, the voice feature extraction module includes a voice feature extraction module, an energy spectrum acquisition module, a filtering module, a voice splicing module, and a mixing module;
the voice feature extraction module is used for extracting voice features from the dialogue instruction, dividing the voice features into frames with fixed lengths, and each frame comprises a plurality of sampling points;
the energy spectrum acquisition module is used for carrying out fast Fourier transform on each frame of voice characteristics to obtain an energy spectrum on a frequency domain;
the filtering module is used for carrying out filter bank processing on the energy spectrum to obtain a Mel frequency cepstrum coefficient as a characteristic vector of the voice signal;
and the voice splicing module is used for splicing a plurality of adjacent mel frequency cepstrum coefficient characteristic vectors into a high-dimensional vector and inputting the high-dimensional vector into the compression delay network. The compression time delay network consists of a plurality of time delay units and a convolution layer, and compresses a parameter matrix through singular value decomposition (Singular Value Decomposition, SVD) to reduce the calculation complexity;
and the mixing module is used for taking the output of the compressed time delay network as a voice characteristic and splicing the voice characteristic with the vector representation of the text content to obtain a mixed characteristic.
In a further embodiment, in the classification module, the convolutional neural network is used to process the hybrid features, and the hybrid features are learned by a convolutional layer and mapped to the demand classification result by a fully connected layer.
In a further embodiment, the emotion recognition module comprises a graph construction module, an optimization module and an emotion prediction module;
the diagram construction module is used for constructing two diagram structures by utilizing voice characteristics and text contents and respectively representing an acoustic mode and a text mode;
the optimizing module is used for optimizing, reconstructing and fusing the text content and the acoustic features by utilizing the GCN deep learning model to obtain a final fusion graph, namely realizing multi-mode feature fusion;
and the emotion prediction module is used for predicting node labels of the fused graphs by utilizing the graph enhancement and graph convolution network to realize voice emotion recognition.
In a further embodiment, the graph construction module includes a speech feature graph construction module and a text content graph construction module;
the voice feature diagram construction module is used for constructing a voice feature diagram structure through voice features, wherein the voice feature diagram structure G= (V, A), V represents a node set of the voice features, A represents an adjacent matrix of the voice features, and the element of the adjacent matrix is the similarity between two nodes;
the similarity calculation formula is:
wherein a is i A j Respectively representing the ith and the jth voice feature vectors;
||a i -a j || μ representing the euclidean distance, μ being the average of the euclidean distances for all speech features, d representing a scale parameter, exp representing an exponential function.
The text content graph construction module is used for constructing a text content graph structure through text content and defining a node set of the text content and an adjacency matrix of the text content.
In a further embodiment, the optimization module comprises a preprocessing module, a reconstruction module and a multi-mode feature fusion module;
the preprocessing module is used for optimizing node characteristics of a graph constructed by text contents by utilizing a GCN deep learning model, normalizing an adjacent matrix of the text contents and obtaining optimized text content representation;
the reconstruction module is used for reconstructing the adjacent matrix based on the text content and the acoustic feature by utilizing the node feature of the second hidden layer of the GCN deep learning model to obtain a reconstructed text content adjacent matrix and an acoustic feature adjacent matrix, and fusing the text content adjacent matrix and the acoustic feature adjacent matrix to obtain a fused adjacent matrix;
it should be noted that GCN is a deep learning model based on graph structure, and is used for machine learning task of processing graph data. It performs a convolution operation on the map data similar to a convolutional neural network performing a convolution operation on the image data.
And the multi-mode feature fusion module is used for fusing the text content after reconstruction optimization with the non-optimized acoustic features to obtain final fusion features, and constructing a final fusion graph through the adjacent matrix after fusion and the final fusion features.
In a further embodiment, the emotion prediction module includes a graph enhancement module and a label prediction module;
the image enhancement module is used for enhancing the node feature matrix of the final fusion image for a plurality of times through random propagation and carrying out feature propagation to obtain a plurality of enhanced feature matrices;
the label prediction module is used for inputting the feature matrix of each enhancement chart into the GCN deep learning model to perform node label prediction to obtain a prediction label matrix of each enhancement chart, and calculating the average value of the prediction labels of each enhancement chart. The GCN predicts the label for each node in each enhancement map by learning the relationship between the topology of the map and the node characteristics. Each row in the predictive tag matrix represents a node and each column represents a possible tag. The mean of the predicted labels represents the average predicted probability for all nodes in the enhancement map for each label.
It should be noted that the hybrid features of the present invention include word vectors of speech features and text content. And carrying out node characteristic optimization on the graph constructed by the text content by utilizing the GCN deep learning model, and carrying out normalization processing on an adjacent matrix of the text content to obtain an optimized text content representation which is used for reconstructing and fusing with acoustic characteristics to obtain a final fusion graph, namely realizing multi-mode characteristic fusion. And simultaneously, carrying out a plurality of times of enhancement on the node feature matrix of the final fusion graph, carrying out feature propagation to obtain a plurality of enhanced feature matrices, inputting the feature matrix of each enhancement graph into a GCN deep learning model for node label prediction, and predicting one or more labels for each node by the GCN through the relationship between the topological structure of the learning graph and the node features so as to judge the emotion type of the voice. The text content and the voice features are mutually supplemented, so that the accuracy and the robustness of emotion recognition are improved.
The application server 4 is configured to exchange communication data with the internet of things device in several manners (e.g. short message, network, bluetooth, etc.).
The network server 5 is used for receiving, processing and forwarding the communication data sent by the application server. It is an intermediary between the application server and the profile information manager.
And the archive information manager 6 is used for storing and managing a database of preset communication identification information and distributing communication data to corresponding network servers according to different communication identification information.
According to another embodiment of the present invention, there is provided an internet of things communication method based on communication identification information, including the steps of:
various types of data are collected and converted into digital signals.
And storing, analyzing, processing and displaying the data converted into the digital signal, and providing services and applications.
And intelligent management is carried out on the Internet of things equipment.
Communication data is exchanged with the internet of things device in several ways.
And receiving, processing and forwarding the communication data sent by the application server.
And the database is used for storing and managing preset communication identification information, and distributing communication data to corresponding network servers according to different communication identification information.
In summary, according to the communication identification information-based internet of things communication platform and the internet of things communication method provided by the invention, the data of the internet of things communication platform is collected and processed, so that the data mining and utilization of internet of things equipment and services are realized, and rich and various functions and services are provided for users; communication data are exchanged with the Internet of things equipment in a plurality of modes, so that the Internet of things equipment and services can be interconnected and intercommunicated, various communication protocols and network technologies are supported, and the safety and reliability of the Internet of things communication are ensured; communication data are distributed to corresponding network servers according to different communication identification information through a database for storing and managing preset communication identification information, so that communication positioning and identification of the Internet of things equipment and services can be realized, and the accuracy and individuation of Internet of things communication are improved. According to the invention, the intelligent management is carried out on the Internet of things equipment through the equipment intelligent management module, the interaction instruction of the user and the Internet of things equipment is obtained, the requirements of the user using the Internet of things equipment can be intelligently identified, personalized and intelligent service to the user is realized, the user experience and satisfaction are improved, the characteristics are mapped into the requirement classification result through the full connection layer of the convolutional neural network through the mixing of voice characteristics and text contents and the learning characteristics of the convolutional neural network, and the accuracy of identification is improved. According to the emotion recognition method and device, through the emotion recognition module, the voice feature map is built, the text content map is built, the map enhancement and the map fusion are carried out, the emotion attribute of the user using the Internet of things equipment is obtained, the emotion recognition accuracy is improved, and the complexity of the emotion recognition module is reduced. The method comprises the steps of constructing two graph structures by utilizing voice features and text contents, respectively representing acoustic modes and text modes, realizing multi-mode representation of voice and text information, enhancing the expressive power and richness of information, optimizing, reconstructing and fusing the text contents and the acoustic features by utilizing a GCN deep learning model to obtain a final fusion graph, realizing multi-mode fusion of the voice and the text information, improving the consistency and complementarity of information, and carrying out node label prediction on the fused graph by utilizing a graph enhancement and graph convolution network to realize voice emotion recognition, so that classification or regression of emotion or emotion in voice can be realized, and the accuracy and robustness of emotion recognition are improved.
Those of ordinary skill in the art will appreciate that all or some of the steps of the methods, systems, functional charging modules/units in the devices disclosed above may be implemented as software, firmware, hardware, and suitable combinations thereof.
The terms "first," "second," "third," "fourth," and the like in the description of the present application and in the above-described figures, if any, are used for distinguishing between similar objects and not necessarily for describing a particular sequential or chronological order. It is to be understood that the data so used may be interchanged where appropriate such that embodiments of the present application described herein may be implemented in sequences other than those illustrated or otherwise described herein. Furthermore, the terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, system, article, or apparatus that comprises a list of steps or elements is not necessarily limited to those steps or elements expressly listed but may include other steps or elements not expressly listed or inherent to such process, method, article, or apparatus.
The foregoing description of the preferred embodiments of the invention is not intended to be limiting, but rather is intended to cover all modifications, equivalents, alternatives, and improvements that fall within the spirit and scope of the invention.
Claims (10)
1. The Internet of things communication platform based on the communication identification information is characterized by comprising Internet of things equipment, a cloud server, an intelligent equipment management module, an application server, a network server and an archive information manager;
the Internet of things equipment is used for collecting various types of data and converting the data into digital signals;
the cloud server is used for storing, analyzing, processing and displaying the data converted into the digital signals and providing services and applications;
the equipment intelligent management module is used for intelligently managing the equipment of the Internet of things;
the application server is used for exchanging communication data with the Internet of things equipment in a plurality of modes;
the network server is used for receiving, processing and forwarding the communication data sent by the application server;
the archive information manager is used for storing and managing a database of preset communication identification information, and distributing communication data to corresponding network servers according to different communication identification information.
2. The internet of things communication platform based on communication identification information according to claim 1, wherein the device intelligent management module comprises an instruction acquisition module, a word sense extraction module, a voice feature extraction module, a classification module and an emotion recognition module;
the instruction acquisition module is used for acquiring a plurality of rounds of dialogue information of the user and the Internet of things equipment, and extracting n dialogue instructions corresponding to the rounds of dialogue information, wherein the dialogue instructions refer to sentences or commands which express requirements and are input by the user through voice and text; the word sense extraction module is used for extracting text content from the dialogue instruction and converting the text content into word vectors by utilizing a word embedding technology;
the voice feature extraction module is used for extracting voice features from the dialogue instruction by utilizing the compression delay network and combining the voice features with corresponding word vectors to form mixed features;
the classification module is used for positioning the key points of the voice and giving out a requirement classification result according to a preset key word library through a convolutional neural network;
and the emotion recognition module is used for recognizing emotion of the mixed features.
3. The internet of things communication platform based on communication identification information according to claim 2, wherein the word sense extraction module comprises a text extraction module, a word embedding module and a semantic splicing module;
the text extraction module is used for segmenting the text content into a plurality of single words by using a word segmentation technology;
the word embedding module is used for mapping each word into a high-dimensional dense vector by using a word embedding technology and representing semantic information;
the semantic stitching module is used for stitching all word vectors representing semantic information into vectors with fixed lengths, and the vectors are used as vector representations of text contents.
4. The internet of things communication platform based on communication identification information according to claim 3, wherein the voice feature extraction module comprises a voice feature extraction module, an energy spectrum acquisition module, a filtering module, a voice splicing module and a mixing module;
the voice feature extraction module is used for extracting voice features from the dialogue instruction, dividing the voice features into frames with fixed lengths, and each frame comprises a plurality of sampling points;
the energy spectrum acquisition module is used for carrying out fast Fourier transform on each frame of voice characteristics to obtain an energy spectrum on a frequency domain;
the filtering module is used for carrying out filter bank processing on the energy spectrum to obtain a Mel frequency cepstrum coefficient as a characteristic vector of the voice signal;
the voice splicing module is used for splicing a plurality of adjacent mel frequency cepstrum coefficient characteristic vectors into a high-dimensional vector and inputting the high-dimensional vector into the compression delay network;
and the mixing module is used for taking the output of the compressed time delay network as a voice characteristic and splicing the voice characteristic with the vector representation of the text content to obtain a mixed characteristic.
5. The internet of things communication platform based on communication identification information according to claim 4, wherein in the classification module, a convolutional neural network is used for processing the hybrid features, the hybrid features are learned through a convolutional layer, and the hybrid features are mapped into a requirement classification result through a full connection layer.
6. The internet of things communication platform based on communication identification information according to claim 5, wherein the emotion recognition module comprises a graph construction module, an optimization module and an emotion prediction module;
the diagram construction module is used for constructing two diagram structures by utilizing voice characteristics and text contents and respectively representing an acoustic mode and a text mode;
the optimizing module is used for optimizing, reconstructing and fusing the text content and the acoustic characteristics by utilizing the GCN deep learning model to obtain a final fusion graph;
the emotion prediction module is used for predicting node labels of the fused graphs by utilizing the graph enhancement and graph convolution network to realize voice emotion recognition.
7. The internet of things communication platform based on communication identification information according to claim 6, wherein the graph construction module comprises a voice feature graph construction module and a text content graph construction module;
the voice feature diagram construction module is used for constructing a voice feature diagram structure through voice features, wherein the voice feature diagram structure G= (V, A), V represents a node set of the voice features, A represents an adjacency matrix of the voice features, and the element of the adjacency matrix is the similarity between two nodes;
the similarity calculation formula is as follows:
wherein a is i A j Respectively representing the ith and the jth voice feature vectors;
||a i -a j || μ representing euclidean distance, μ being the average of euclidean distances for all speech features, d representing a scale parameter, exp representing an exponential function;
the text content graph construction module is used for constructing a text content graph structure through text content and defining a node set of the text content and an adjacency matrix of the text content.
8. The internet of things communication platform based on communication identification information according to claim 7, wherein the optimization module comprises a preprocessing module, a reconstruction module and a multi-mode feature fusion module;
the preprocessing module is used for optimizing node characteristics of a graph structure constructed by text contents by utilizing a GCN deep learning model, normalizing an adjacent matrix of the text contents and obtaining optimized text content representation;
the reconstruction module is used for reconstructing the adjacent matrix based on the text content and the acoustic feature by utilizing the node feature of the second hidden layer of the GCN deep learning model to obtain a reconstructed text content adjacent matrix and an acoustic feature adjacent matrix, and fusing the text content adjacent matrix and the acoustic feature adjacent matrix to obtain a fused adjacent matrix;
the multi-mode feature fusion module is used for fusing the optimized text content and the non-optimized acoustic features to obtain final fusion features, and constructing a final fusion graph through the fused adjacency matrix and the final fusion features.
9. The internet of things communication platform based on communication identification information according to claim 8, wherein the emotion prediction module comprises a graph enhancement module and a tag prediction module;
the image enhancement module is used for enhancing the node feature matrix of the final fusion image for a plurality of times through random propagation and carrying out feature propagation to obtain a plurality of enhanced feature matrices;
the label prediction module is used for inputting the feature matrix of each enhancement chart into the GCN deep learning model to perform node label prediction, obtaining a prediction label matrix of each enhancement chart, and calculating the average value of the prediction labels of each enhancement chart.
10. The communication identification information-based internet of things communication method is applied to the communication identification information-based internet of things communication platform according to any one of claims 1 to 9, and is characterized in that the communication identification information-based internet of things communication method comprises the following steps:
collecting various types of data and converting the data into digital signals;
storing, analyzing, processing and displaying the data converted into the digital signal, and providing services and applications;
intelligent management is carried out on the equipment of the Internet of things;
exchanging communication data with the internet of things device in a plurality of ways;
receiving, processing and forwarding communication data sent by an application server;
and the database is used for storing and managing preset communication identification information, and distributing communication data to corresponding network servers according to different communication identification information.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311710789.5A CN117640695A (en) | 2023-12-13 | 2023-12-13 | Internet of things communication platform and Internet of things communication method based on communication identification information |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311710789.5A CN117640695A (en) | 2023-12-13 | 2023-12-13 | Internet of things communication platform and Internet of things communication method based on communication identification information |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN117640695A true CN117640695A (en) | 2024-03-01 |
Family
ID=90023373
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202311710789.5A Pending CN117640695A (en) | 2023-12-13 | 2023-12-13 | Internet of things communication platform and Internet of things communication method based on communication identification information |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117640695A (en) |
-
2023
- 2023-12-13 CN CN202311710789.5A patent/CN117640695A/en active Pending
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN113762322B (en) | Video classification method, device and equipment based on multi-modal representation and storage medium | |
| CN110647632B (en) | Image and text mapping technology based on machine learning | |
| CN112818861A (en) | Emotion classification method and system based on multi-mode context semantic features | |
| CN112989212B (en) | Media content recommendation method, device and equipment and computer storage medium | |
| CN112632244A (en) | Man-machine conversation optimization method and device, computer equipment and storage medium | |
| CN113806588A (en) | Method and device for searching video | |
| CN111538818A (en) | Data query method and device, electronic equipment and storage medium | |
| CN114329051B (en) | Data information identification method, device, apparatus, storage medium and program product | |
| CN116610818A (en) | Construction method and system of power transmission and transformation project knowledge base | |
| CN113051384B (en) | User portrait extraction method based on dialogue and related device | |
| CN110991279A (en) | Document image analysis and recognition method and system | |
| CN115374765B (en) | Computing power network 5G data analysis system and method based on natural language processing | |
| CN116977701A (en) | Video classification model training method, video classification method and device | |
| CN117640695A (en) | Internet of things communication platform and Internet of things communication method based on communication identification information | |
| CN113297367A (en) | Method for generating user conversation linking language and related equipment | |
| CN114970494A (en) | Comment generation method and device, electronic equipment and storage medium | |
| CN117221839B (en) | 5G signaling identification method and system thereof | |
| Huang et al. | Video Recommendation Method Based on Deep Learning of Group Evaluation Behavior Sequences | |
| CN116561350B (en) | Resource generation method and related device | |
| CN117876941B (en) | Target multi-mode model system, construction method, video processing model training method and video processing method | |
| CN117875266B (en) | Training method and device for text coding model, electronic equipment and storage medium | |
| CN116975219A (en) | Data processing method, device, computer equipment and storage medium | |
| CN113792163B (en) | Multimedia recommendation method and device, electronic equipment and storage medium | |
| CN113821739B (en) | Local event detection method, device, equipment and storage medium | |
| CN111460169B (en) | Semantic expression generation method, device and equipment |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |