CN112528873A - Signal semantic recognition method based on multi-stage semantic representation and semantic calculation - Google Patents

Signal semantic recognition method based on multi-stage semantic representation and semantic calculation Download PDF

Info

Publication number
CN112528873A
CN112528873A CN202011476389.9A CN202011476389A CN112528873A CN 112528873 A CN112528873 A CN 112528873A CN 202011476389 A CN202011476389 A CN 202011476389A CN 112528873 A CN112528873 A CN 112528873A
Authority
CN
China
Prior art keywords
semantic
network
signal
equal
representation
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN202011476389.9A
Other languages
Chinese (zh)
Other versions
CN112528873B (en
Inventor
石光明
杨旻曦
高大华
谢雪梅
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Xidian University
Original Assignee
Xidian University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Xidian University filed Critical Xidian University
Priority to CN202011476389.9A priority Critical patent/CN112528873B/en
Publication of CN112528873A publication Critical patent/CN112528873A/en
Application granted granted Critical
Publication of CN112528873B publication Critical patent/CN112528873B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F2218/00—Aspects of pattern recognition specially adapted for signal processing
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G06N3/088—Non-supervised learning, e.g. competitive learning

Abstract

The invention provides a semantic identification method based on multilevel semantic representation and semantic calculation, which mainly solves the problems of poor interpretability and low generalization capability of signal semantic identification in the prior art. The implementation scheme is as follows: acquiring a training set and a test set; constructing a signal semantic identification network consisting of a cascade multistage semantic representation network and a semantic calculation network so as to carry out learnable multistage semantic representation on the signal and calculate the semantic category of the signal according to the semantic representation; setting a semantic representation loss function and a cross entropy loss function to train a multi-stage semantic representation network and a semantic calculation network in sequence to obtain a trained signal semantic recognition network; and acquiring a semantic recognition result of the signal to be recognized based on the trained signal semantic recognition network. The invention effectively improves the interpretability and generalization capability of signal semantic recognition. The method can be used for man-machine interaction and semantic information retrieval.

Description

Signal semantic recognition method based on multi-stage semantic representation and semantic calculation
Technical Field
The invention belongs to the technical field of artificial intelligence, and particularly relates to a signal semantic recognition method which can be used for man-machine interaction and semantic information retrieval.
Background
Signal semantic recognition refers to determining the semantic category to which a signal belongs according to the characteristics of the signal.
Before the deep learning technology is applied, researchers generally use methods such as bag-of-words feature matching or random forests to perform semantic recognition on signals. The method based on bag-of-words feature matching comprises the steps of firstly representing each semantic category by a plurality of feature sets obtained through feature engineering, and then judging the semantic category by calculating the overall similarity degree of a sample signal and the feature sets. And the random forest firstly predicts the semantic categories of the sample signals independently by a plurality of decision trees, then votes all prediction categories, and elects the category with the highest vote number as a final prediction result. However, both the bag-of-words feature matching and the random forest method require a user to design a large number of manual features according to professional knowledge in the field of target tasks, which is time-consuming, labor-consuming and difficult to implement. In addition, since the manual features are fixed after being designed, the manual features can only be used in specific scenes, and the generalization performance is poor.
In recent years, the accuracy of recognition methods of signals such as images and voices based on deep learning exceeds the human level in general data sets, and have been put into practical use in applications such as face recognition. Firstly, establishing a data set with semantic labels based on a deep learning method, then designing a model and a loss function, then performing end-to-end training on the model on the data set, and finally inputting a sample signal into the trained model to obtain a recognition result. However, artificial intelligence technology is expected to provide better services to people in more fields, and what is needed is more than higher accuracy, such as: in human-computer interaction, auxiliary medical diagnosis and automatic driving, the interpretability of an artificial intelligent model is highly required. The current deep learning method uses a deep neural network as a model, but the parameter quantity of the neural network is huge, the neural network lacks of theoretical or visual explanation, and the vulnerability problem of the attack is resisted, so that the learned characteristics are difficult to understand. In addition, because the current deep learning method generally adopts an end-to-end training method, when different problems are faced, different data sets need to be constructed, and a large amount of time and calculation power are spent on carrying out iterative training on the model, so that the generalization capability of the model is poor. In a patent application with the application publication number of CN110059741A and the name of image identification method based on semantic capsule fusion network, in 7/25/2019, a method for performing semantic identification on image signals by fusing a semantic capsule network and a convolutional neural network is disclosed. The method firstly extracts image features with specific semantic meanings through manually designed semantic primitives, then further extracts the semantic features through a plurality of parallel semantic capsules, and finally identifies images according to distinguishing features obtained by directly adding the semantic features and the image features extracted by the traditional convolutional neural network, although the interpretability and the generalization capability of the network are improved, the method has the following two disadvantages:
firstly, the distinguishing features on which the recognition is based are obtained by directly adding the semantic features extracted by the semantic capsule network and the image features which are extracted by the convolutional neural network and have poor interpretability, so the interpretability is extremely poor;
secondly, because the semantic capsule network adopts fixed and unchangeable semantic elements, semantic feature extraction can be carried out only after different semantic elements are designed for different problems, and the whole network adopts an end-to-end training method, the features of all semantic capsules are not independent, so that the semantic capsules are difficult to migrate to other problems, and the generalization capability of the method is insufficient.
Disclosure of Invention
The invention aims to overcome the defects of the prior art and provides a semantic identification method based on multi-level semantic representation and calculation so as to improve the interpretability and generalization capability of signal semantic identification.
In order to achieve the above purpose, the implementation scheme of the invention comprises the following steps:
(1) m signals with labels randomly selected from the signal semantic recognition data set form a training sample set SaThe residual signal constitutes a test sample set SbWherein M is more than or equal to 100;
(2) constructing a signal semantic recognition network H:
(2a) set up by NlSemantic representation of sub-networks Wr (l)Composed semantically-characterized networks Wr＝{Wr (l)For characterizing semantic features in the signal, where NlSemantic characterization of sub-network W for ≧ 2r (l)Sequence number of (1) l is less than or equal to Nl；
(2b) Establishing a semantic computation network W consisting of a plurality of stacked graph convolution layers and a global graph average pooling layercFor computing semantic categories of the signal from the semantic features;
(2c) characterizing semantics into a network WrAnd semantic computing network WcCascading to form a signal semantic recognition network H;
(3) training a signal semantic recognition network H:
(3a) will train the sample set SaInput to a semantic representation network WrIn (1), and let l be 1, and SaCharacterizing a subnetwork W as a semanticr (l)Input training sample set Sa (l)It is iteratively trained as follows:
(3a1) setting a semantic representation subnetwork Wr (l)Has a loss function of Lr＝Lr1+λLr2The maximum iteration time T is more than or equal to 10, and the initial iteration time T is 0, wherein Lr1Characterizing the independence loss function for semantics, Lr2For semantic characterisation of the intensity loss function, λ is Lf1And Lf2λ > 0;
(3a2) will Sa (l)Input into a semantic representation subnetwork Wr (l)In (1) obtaining Wr (l)Output of (2) O(l)According to O(l)Calculating LrAnd using a gradient descent method for Wr (l)Updating is carried out;
(3a3) judging whether T is more than or equal to T, if so, obtaining a trained semantic tableToken network Wr (l)', go to (3a4), otherwise, let t be t +1, return to (3a 2);
(3a4) judging that l is more than or equal to NlIf yes, obtaining a well-trained characterization network Wr'and its output O', perform (3b), otherwise, let l ═ l +1, and let Sa (l)Is O(l-1)', return (3a 1);
(3b) for semantic computation network WcThe following iterative training is performed:
(3b1) setting a semantic computation network WcIs a cross entropy loss function LcThe maximum iteration frequency Q is more than or equal to 100, and the initial iteration frequency Q is 0;
(3b2) the well-trained characterization network Wr' output O ' of the ' is input to the semantic computation network WcIn (1) obtaining WcOutput of (2) OcAnd according to OcCalculating LcBy gradient descent of WcUpdating is carried out;
(3b3) judging whether Q is more than or equal to Q, if so, obtaining a trained semantic computation sub-network WcIf not, returning to (3b2) by making q + 1;
(3c) the well-trained characterization network Wr' AND trained semantic computation subnetwork WcCarrying out cascade connection to form a trained signal semantic recognition network H';
(4) set of test samples SbAnd inputting the signal into a trained signal semantic recognition network H' to obtain a signal semantic recognition result.
Compared with the prior art, the invention has the following beneficial effects:
firstly, in the semantic identification model based on the multilevel semantic features and calculation, the multilevel semantic feature network is established to carry out semantic representation on the signals, and the semantic calculation network based on graph convolution is established to carry out semantic identification on the semantic features of the signals, so that the defect that the interpretability is reduced because the interpretable semantic features and the signal features extracted by a neural network with poor interpretability are directly added when distinguishing features are extracted in the prior art is avoided, and the interpretability of the signal semantic identification method is effectively improved;
secondly, the semantic representation loss function is set in the constructed semantic identification model based on the multilevel semantic representation and calculation to perform unsupervised training on the semantic representation sub-network, so that learnable semantic representation is realized, and the constructed multilevel semantic representation network and the calculation network are trained in sequence, thereby avoiding the defect of insufficient generalization capability caused by using fixed and unchangeable semantic elements and an end-to-end training method in the prior art, and effectively improving the generalization capability of signal semantic identification.
Drawings
FIG. 1 is a general flow chart of an implementation of the present invention;
FIG. 2 is a diagram of a signal semantic recognition network according to the present invention;
FIG. 3 is a sub-flowchart of training a signal semantic recognition network according to the present invention.
Detailed Description
The invention is described in further detail below with reference to the figures and specific examples.
Referring to fig. 1, the implementation steps of this example are as follows:
step 1, a training sample set and a testing sample set are obtained.
And acquiring a training sample set and a testing sample set from the signal semantic recognition data set. Existing signal semantic recognition datasets include an MNIST handwritten digit dataset, a CIFAR dataset and an ImageNet dataset, and the embodiment preferably uses, but not limited to, the MNIST handwritten digit recognition dataset as the signal semantic recognition dataset. The MNIST handwritten digit recognition data set comprises 70000 single-channel handwritten digital image samples with the size of 28 multiplied by 28 and 70000 one-hot label vectors with the length of 10;
randomly selecting M60000 image signals with labels from MNIST hand-written digit recognition data set to form training sample set SaAnd the remaining 10000 image signals with labels form a test sample set Sb。
And 2, constructing a signal semantic recognition network H.
Referring to FIG. 2, this exampleConstructed signal semantic recognition network H is represented by semanticrAnd semantic computing network WcThe cascade connection is composed of the following construction steps:
2.1) building a semantic representation network Wr：
2.1.1) creation of semantic transformation parameter Generator Gj (l)The method is used for extracting features from an input sample to obtain semantic transformation parameters:
Gj (l)the structure of (1) is as follows in sequence: the first convolution layer → the second convolution layer → the third convolution layer → the global average pooling layer, the number of convolution kernels of the three convolution layers is 4, the number of convolution kernels is 3 multiplied by 3, the step length is 1, and the filling number is 1;
2.1.2) building semantic converter Tj (l)For transforming the parameter generator G according to semanticsj (l)Output semantic transformation parameters vs. semantic primitives pj (l)The conversion is carried out, so that the semantic elements can be dynamically adjusted according to the input samples, and the generalization performance of signal semantic recognition is effectively improved;
the existing semantic transformation methods include inversion transformation, gray-scale gamma transformation, affine transformation and rotation scaling transformation. Because the handwritten digits are all composed of arc segments with different sizes and angles, the embodiment preferably but not limited to selects and selects the rotation expansion transformation to process the semantic elements to match the arc segments with different sizes and angles, and the generalization capability of signal semantic identification is improved, namely the semantic converter Tj (l)According to Gj (l)Generated semantic transformation parameter thetai,j (l)For semantic primitive pj (l)Semantic transformation is performed by the following formula:
Figure BDA0002835628860000041
wherein the content of the first and second substances,
Figure BDA0002835628860000051
for semantic elements pj (l)Dimension of the nth element is NmCoordinate vector of an(m) is
Figure BDA0002835628860000052
M is not less than 1 and not more than Nm，
Figure BDA0002835628860000053
For transformed semantic elements pi,j (l)' dimension of the N-th element is NmCoordinate vector of bn(m) is
Figure BDA0002835628860000054
The m-th-dimensional coordinate of (a),
Figure BDA0002835628860000055
for semantic elements pj (l)The center coordinate is
Figure BDA0002835628860000056
The elements of (a) and (b),
Figure BDA0002835628860000057
for transformed semantic elements pi,j (l)' center coordinate is
Figure BDA0002835628860000058
The elements of (a) and (b),
Figure BDA0002835628860000059
θi,j (l)is Gj (l)The semantic transformation parameters generated from the ith input sample, i ≦ 1 ≦ M, N in this examplem＝2；
2.1.3) semantic conversion parameter generator G built by 2.1.1)j (l)Semantic converter T established with 2.1.2)j (l)Cascading, composing semantic representation modules Rj (l)Wherein j is a semantic representation module Rj (l)J is not less than 1 and not more than NpIn this example, Np＝4；
2.1.4) reacting NpSemantic representation modelBlock Rj (l)Parallel, constituent semantic representation subnetworks Wr (l)For characterizing semantic features of the same hierarchy as NpA separate semantic representation module Rj (l)Because the habit that the knowledge is expressed as the combination of a plurality of knowledge points in the knowledge cognition process of the human is met, the human can conveniently understand the semantic information represented in the knowledge, and the independent semantic representation modules can be conveniently recombined to obviously improve the generalization capability;
2.1.5) reacting NlSemantic representation of a subnetwork Wr (l)Cascading, composing a semantic representation network WrFor semantically characterizing a semantic in an image signal as a structure of multiple levels, wherein l is a semantic characterizing subnetwork Wr (l)Sequence number of (1) l is less than or equal to Nl. In this example, NlThis semantic representation of the network W2rBecause the method conforms to the habit that the knowledge is divided into a plurality of hierarchical representations in the knowledge cognition process of human beings, the method is not only convenient for the human beings to understand the semantic information represented in the knowledge, but also convenient for the trained network structure to be migrated to a new problem, and the interpretability and generalization capability of signal semantic recognition are improved.
2.2) building a semantic computation network Wc：
Semantic computing network WcFor characterizing the network W according to semanticsrThe extracted semantic features calculate semantic classes of the signal. The structure is as follows in sequence: first map convolutional layer → second map convolutional layer → global map average pooling layer, the parameter matrix sizes of these two map convolutional layers are 4 × 8 and 8 × 10, respectively.
2.3) characterizing the semantics of the Web WrAnd semantic computing network WcAnd carrying out cascade connection to form a signal semantic recognition network H.
And 3, training the signal semantic recognition network H.
Referring to fig. 3, the steps of training the signal semantic recognition network H in this example are as follows:
3.1) will train the sample set SaInput to a semantic representation network WrIn (1), initializing the semantic tableToken network Wr (l)The serial number l of (1);
3.2) characterizing the subnetwork W for semantic purposesr (l)Performing iterative training:
3.2.1) setting up the semantic representation sub-network Wr (l)Is Sa (l)：
Judging whether l is 1, if so, judging SaCharacterizing subnetworks W as first layer semanticsr (1)Input training sample set Sa (1)Otherwise, the last layer of the trained semantic representation sub-network W is used for representingr (l-1)Output of O of `(l-1)' characterizing a sub-network W as a semanticr (l)Input training set Sa (l)；
3.2.2) setting up the semantic representation sub-network Wr (l)Has a loss function of Lr＝Lr1+λLr2Wherein λ > 0 is Lf1And Lf2Balanced weight of, Lr1And Lr2The semantic representation independence loss function and the semantic representation strength loss function are respectively. In this example, λ ═ 1;
existing semantic representation independence loss function Lr1Comprises a variance function, an average Euclidean distance function and an existing semantic representation intensity loss function Lr2Including a minimum function and a mean function. The embodiment preferably but not limited to, the mean Euclidean distance function and the mean function are respectively taken as the semantic representation independence loss function Lr1And semantically characterizing the intensity loss function Lr2Which are respectively represented as follows:
Figure BDA0002835628860000061
Figure BDA0002835628860000062
wherein G isa (l)And Gb (l)Are each Ra (l)And Rb (l)Semantic transformation parameter generator of (1), Ta (l)And Tb (l)Are each Ra (l)And Rb (l)Semantic converter of (1), pa (l)And pb (l)Are each Ra (l)And Rb (l)Corresponding semantic element, Ra (l)And Rb (l)Respectively semantically characterizing sub-networks Wr (l)The a and b semantic representation modules in the system are that a is more than or equal to 1 and less than or equal to Np，1≤b≤Np，a≠b，si (l)Characterizing a subnetwork W for semanticsr (l)Input training sample set Sa (l)In the ith training sample, i is more than or equal to 1 and less than or equal to M,
Ga (l)(si (l)) Is Ga (l)To si (l)Semantic transformation parameters, G, obtained by extracting featuresb (l)(si (l)) Is Gb (l)To si (l)Semantic transformation parameters, v, obtained by extracting featuresi,a (l)For the ith signal si (l)With the a-th transformed semantic primitive pa (l)' feature map obtained by convolution, vi,b (l)For the ith signal si (l)With the b-th transformed semantic primitive pb (l)' feature map obtained by convolution, vi,a (l)(n) and vi,b (l)(n) are each vi,a (l)And vi,b (l)N is not less than 1 and not more than Nv，NvIs v isi,a (l)Or vi,b (l)The total number of the elements in the Chinese character,
Figure BDA0002835628860000063
to be composed of
Figure BDA0002835628860000064
Transforming parameter pairs p for semanticsa (l)Obtained by performing semantic conversionThe semantic elements after the transformation are processed,
Figure BDA0002835628860000065
to be composed of
Figure BDA0002835628860000071
Transforming parameter pairs p for semanticsb (l)Performing semantic transformation to obtain transformed semantic elements;
3.2.3) setting the semantic-characterization sub-network Wr (l)The maximum iteration number T of the training is more than 10, and the initial iteration number T is 0. In this example, T ═ 15;
3.2.4) setting semantic primitives P(l)：
The existing setting method of semantic elements comprises the steps of randomly clipping signals to obtain the semantic elements, manually designing the semantic elements according to common knowledge or domain professional knowledge, and selecting classical kernel functions, such as: a Gaussian kernel function, a Laplace kernel function, a wavelet kernel function and the like are used as semantic primitives;
this example preferably, but not exclusively, involves the common knowledge that handwritten numbers are formed from strokes, and that 4 arcs of different lengths are manually designed and recorded in matrices of 11 × 11 size and 1 number of channels, respectively, as a first-level semantic representation subnetwork Wr (1)Semantic primitives of (1)
Figure BDA0002835628860000072
In this embodiment, preferably, but not limited to, according to the extraction mode of the human retinal neurons on the image features, the combination of the zeroth order to third order derivatives of the gaussian kernel function can be used to approximate the prior knowledge in the neuroscience professional field, and the zeroth order to third order derivatives of the gaussian kernel function are selected and recorded in matrices with the size of 11 × 11 and the number of channels of 4 respectively as the second-order semantic representation subnetwork Wr (2)Semantic primitives of (1)
Figure BDA0002835628860000073
3.2.5) calculating semantic transformation parameters: i.e. training sample si (l)Input to semantic transformation parameter generator Gj (l)And transforming the data into a matrix to obtain a semantic transformation parameter thetai,j (l). In this example, the semantic transformation parameter θi,j (l)The size is 2 x 2;
3.2.6) semantic conversion of semantic primitives:
semantic converter Tj (l)According to Gj (l)Generated semantic transformation parameter thetai,j (l)Semantic primitive p by the transformation formula in 2.1.2)j (l)Performing semantic transformation to obtain transformed semantic primitive pi,j (l)′；
3.2.7) convolving the transformed semantic elements with the input samples:
transforming the semantic elements pi,j (l)' AND input samples si (l)Performing convolution to obtain a characteristic diagram vi,j (l)And i is the same vi,j (l)Splicing along the channel dimension to obtain Wr (l)Output of (2)
Figure BDA0002835628860000074
3.2.8) updating the semantic representation sub-network W by adopting a gradient descent methodr (l)：
Calculating a loss function LrAnd a gradient descent method is adopted to semantically characterize the subnetwork Wr (l)Updating is carried out;
3.2.9) will characterize the subnetwork W for the semanticr (l)Comparing the current iteration time T with the maximum iteration time T:
if T is more than or equal to T, obtaining a trained semantic representation sub-network Wr (l)' and its output O(l)', perform 3.3),
otherwise, let t be t +1, go back to 3.2.5);
3.3) semantically characterizing the sub-network Wr (l)Number of (1) and total number of (N)lAnd (3) comparison:
if l is not less than NlThen all the trained semantics are characterized into sub-networks
Figure BDA0002835628860000081
Cascading to form a trained semantic representation network Wr', and the last layer of trained semantic characterization sub-network
Figure BDA0002835628860000082
Output of (2)
Figure BDA0002835628860000083
As WrOutput O 'of' performs 3.4), otherwise, let l ═ l +1, return 3.2).
3.4) semantic computation network WcPerforming iterative training:
3.4.1) setting the semantic computation network WcLoss function and maximum number of iterations:
setting a semantic computation network WcIs a cross entropy loss function LcThe maximum iteration number Q is more than or equal to 100, and the initial iteration number Q is 0, wherein the cross entropy loss function LcExpressed as follows:
Figure BDA0002835628860000084
wherein, yiFor training sample set SaThe label of the ith training sample in (1),
Figure BDA0002835628860000085
computing a network W for semanticscFor y i1 ≦ i ≦ M, in this example, Q200;
3.4.2) will train the well-characterized network Wr' the ith sample O in the output Oi'the pixel points in the' are used as vertexes, and the adjacent vertexes are connected by an edge with the weight of 1 to obtain a sample g shown in the figurei；
3.4.3) sample g of the graphiInput to a semanticizerComputing network WcTo obtain a predicted result
Figure BDA0002835628860000086
3.4.4) based on the prediction
Figure BDA0002835628860000087
Calculating a loss function LcAnd computing the network W for semantic by using a gradient descent methodcUpdating is carried out;
3.4.5) compute the network W for semanticcComparing the current iteration number Q of training with the maximum iteration number Q:
if Q is more than or equal to Q, obtaining a trained semantic computation sub-network Wc', execute 3.5), otherwise, let q ═ q +1, return 3.4.3);
3.5) the well-trained characterizing network Wr' AND trained semantic computation subnetwork Wc'cascading to form a trained signal semantic recognition network H'.
Step 4, testing sample set SbAnd performing signal semantic recognition.
Set of test samples SbAnd inputting the signal into a trained signal semantic recognition network H' to obtain a signal semantic recognition result.
The foregoing description is only an example of the present invention and is not intended to limit the invention, so that it will be apparent to those skilled in the art that various changes and modifications in form and detail may be made therein without departing from the spirit and scope of the invention.

Claims (7)

1. A signal semantic identification method based on multilevel semantic representation and semantic computation is characterized by comprising the following steps:
(1) m signals with labels randomly selected from the signal semantic recognition data set form a training sample set SaResidual signal composition testSample set SbWherein M is more than or equal to 100;
(2) constructing a signal semantic recognition network H:
(2a) set up by NlSemantic representation of sub-networks Wr (l)Composed semantically-characterized networks Wr＝{Wr (l)For characterizing semantic features in the signal, where NlSemantic characterization of sub-network W for ≧ 2r (l)Sequence number of (1) l is less than or equal to Nl；
(2b) Establishing a semantic computation network W consisting of a plurality of stacked graph convolution layers and a global graph average pooling layercFor computing semantic categories of the signal from the semantic features;
(2c) characterizing semantics into a network WrAnd semantic computing network WcCascading to form a signal semantic recognition network H;
(3) training a signal semantic recognition network H:
(3a) will train the sample set SaInput to a semantic representation network WrIn (1), and let l be 1, and SaCharacterizing a subnetwork W as a semanticr (l)Input training sample set Sa (l)It is iteratively trained as follows:
(3a1) setting a semantic representation subnetwork Wr (l)Has a loss function of Lr＝Lr1+λLr2The maximum iteration time T is more than or equal to 10, and the initial iteration time T is 0, wherein Lr1Characterizing the independence loss function for semantics, Lr2For semantic characterisation of the intensity loss function, λ is Lf1And Lf2λ > 0;
(3a2) will Sa (l)Input into a semantic representation subnetwork Wr (l)In (1) obtaining Wr (l)Output of (2) O(l)According to O(l)Calculating LrAnd using a gradient descent method for Wr (l)Updating is carried out;
(3a3) judging whether T is more than or equal to T, if so, obtaining a trained semantic representation sub-network Wr (l)′，Executing (3a4), otherwise, making t equal to t +1, and returning to (3a 2);
(3a4) judging that l is more than or equal to NlIf yes, obtaining a well-trained characterization network Wr'and its output O', perform (3b), otherwise, let l ═ l +1, and let Sa (l)Is O(l-1)', return (3a 1);
(3b) for semantic computation network WcThe following iterative training is performed:
(3b1) setting a semantic computation network WcIs a cross entropy loss function LcThe maximum iteration frequency Q is more than or equal to 100, and the initial iteration frequency Q is 0;
(3b2) the well-trained characterization network Wr' output O ' of the ' is input to the semantic computation network WcIn (1) obtaining WcOutput of (2) OcAnd according to OcCalculating LcBy gradient descent of WcUpdating is carried out;
(3b3) judging whether Q is more than or equal to Q, if so, obtaining a trained semantic computation sub-network WcIf not, returning to (3b2) by making q + 1;
(3c) the well-trained characterization network Wr' AND trained semantic computation subnetwork WcCarrying out cascade connection to form a trained signal semantic recognition network H';
(4) set of test samples SbAnd inputting the signal into a trained signal semantic recognition network H' to obtain a signal semantic recognition result.
2. The method of claim 1, wherein the semantic representation in (2a) characterizes subnetwork Wr (l)Is composed of NpA parallel semantic representation module Rj (l)，1≤j≤Np，NpNot less than 2, each semantic representation module Rj (l)Semantic transformation parameter generator G comprising a cascadej (l)And semantic converter Tj (l)，Gj (l)Comprising a plurality of stacked convolutional layers and a global mean pooling layer, Tj (l)For according to Gj (l)Generated semantic transformation parameter thetai,j (l)For semantic primitive pj (l)Performing a semantic transformation of thetai,j (l)Is Gj (l)And i is more than or equal to 1 and less than or equal to M according to the semantic transformation parameters generated by the ith input sample.
3. The method of claim 2, wherein the semantic transformation parameter generator Gj (l)The structure is as follows in sequence:
first convolution layer → second convolution layer → third convolution layer → global average pooling layer.
4. Method according to claim 2, characterized in that the semantic converter Tj (l)According to Gj (l)Generated semantic transformation parameter thetai,j (l)For semantic primitive pj (l)Performing semantic transformation by the following formula:
Figure FDA0002835628850000021
wherein the content of the first and second substances,
Figure FDA0002835628850000022
for semantic elements pj (l)Dimension of the nth element is NmCoordinate vector of an(m) is
Figure FDA0002835628850000023
M is not less than 1 and not more than Nm，
Figure FDA0002835628850000024
For transformed semantic elements pi,j (l)' dimension of the N-th element is NmCoordinate vector of bn(m) is
Figure FDA0002835628850000025
The m-th-dimensional coordinate of (a),
Figure FDA0002835628850000026
for semantic elements pj (l)The center coordinate is
Figure FDA0002835628850000027
The elements of (a) and (b),
Figure FDA0002835628850000028
for transformed semantic elements pi,j (l)' center coordinate is
Figure FDA0002835628850000029
The elements of (a) and (b),
Figure FDA00028356288500000210
5. the method of claim 1, wherein the semantic computation network W in (2b)cThe structure is as follows: first graph volume layer → second graph volume layer → global graph average pooling layer.
6. The method of claim 1, wherein the semantic representation independence loss function L in (3a1)r1And semantically characterizing the intensity loss function Lr2Which are respectively represented as follows:
Figure FDA0002835628850000031
Figure FDA0002835628850000032
wherein G isa (l)And Gb (l)Are each Ra (l)And Rb (l)Semantic transformation parameter generator of (1), Ta (l)And Tb (l)Are each Ra (l)And Rb (l)Semantic converter of (1), pa (l)And pb (l)Are each Ra (l)And Rb (l)Corresponding semantic element, Ra (l)And Rb (l)Respectively semantically characterizing sub-networks Wr (l)The a and b semantic representation modules in the system are that a is more than or equal to 1 and less than or equal to Np，1≤b≤Np，a≠b，Np≥2，si (l)Characterizing a subnetwork W for semanticsr (l)Input training sample set Sa (l)I is more than or equal to 1 and less than or equal to M and G of the ith training samplea (l)(si (l)) Is Ga (l)To si (l)Semantic transformation parameters, G, obtained by extracting featuresb (l)(si (l)) Is Gb (l)To si (l)Semantic transformation parameters, v, obtained by extracting featuresi,a (l)For the ith signal si (l)With the a-th transformed semantic primitive pa (l)' feature map obtained by convolution, vi,b (l)For the ith signal si (l)With the b-th transformed semantic primitive pb (l)' feature map obtained by convolution, vi,a (l)(n) and vi,b (l)(n) are each vi,a (l)And vi,b (l)N is not less than 1 and not more than Nv，NvIs v isi,a (l)Or vi,b (l)Total number of middle elements.
7. The method of claim 1, wherein the cross-entropy loss function L in (3b1)cExpressed as follows:
Figure FDA0002835628850000033
wherein, yiFor training sample set SaThe label of the ith training sample in (1),
Figure FDA0002835628850000034
computing a network W for semanticscFor yiFor the prediction, i is more than or equal to 1 and less than or equal to M.
CN202011476389.9A 2020-12-15 2020-12-15 Signal semantic recognition method based on multi-stage semantic representation and semantic calculation Active CN112528873B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202011476389.9A CN112528873B (en) 2020-12-15 2020-12-15 Signal semantic recognition method based on multi-stage semantic representation and semantic calculation

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202011476389.9A CN112528873B (en) 2020-12-15 2020-12-15 Signal semantic recognition method based on multi-stage semantic representation and semantic calculation

Publications (2)

Publication Number Publication Date
CN112528873A true CN112528873A (en) 2021-03-19
CN112528873B CN112528873B (en) 2022-03-22

Family

ID=75000115

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202011476389.9A Active CN112528873B (en) 2020-12-15 2020-12-15 Signal semantic recognition method based on multi-stage semantic representation and semantic calculation

Country Status (1)

Country Link
CN (1) CN112528873B (en)

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113315972A (en) * 2021-05-19 2021-08-27 西安电子科技大学 Video semantic communication method and system based on hierarchical knowledge expression
CN113705245A (en) * 2021-09-01 2021-11-26 北京邮电大学 Semantic communication method, device, system, computer equipment and storage medium
CN114224343A (en) * 2022-01-13 2022-03-25 平安科技（深圳）有限公司 Cognitive disorder detection method, device, equipment and storage medium

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110119736A (en) * 2018-02-07 2019-08-13 浙江宇视科技有限公司 License plate location recognition method, device and electronic equipment
US20190294661A1 (en) * 2018-03-21 2019-09-26 Adobe Inc. Performing semantic segmentation of form images using deep learning
CN110309861A (en) * 2019-06-10 2019-10-08 浙江大学 A kind of multi-modal mankind's activity recognition methods based on generation confrontation network
CN110349215A (en) * 2019-07-10 2019-10-18 北京悉见科技有限公司 A kind of camera position and orientation estimation method and device
US20200020102A1 (en) * 2017-04-14 2020-01-16 Tusimple, Inc. Method and device for semantic segmentation of image

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20200020102A1 (en) * 2017-04-14 2020-01-16 Tusimple, Inc. Method and device for semantic segmentation of image
CN110119736A (en) * 2018-02-07 2019-08-13 浙江宇视科技有限公司 License plate location recognition method, device and electronic equipment
US20190294661A1 (en) * 2018-03-21 2019-09-26 Adobe Inc. Performing semantic segmentation of form images using deep learning
CN110309861A (en) * 2019-06-10 2019-10-08 浙江大学 A kind of multi-modal mankind's activity recognition methods based on generation confrontation network
CN110349215A (en) * 2019-07-10 2019-10-18 北京悉见科技有限公司 A kind of camera position and orientation estimation method and device

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
石光明，李莹玉，谢雪梅: "语义通讯———智能时代的产物", 《模式识别与人工智能》 *

Cited By (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113315972A (en) * 2021-05-19 2021-08-27 西安电子科技大学 Video semantic communication method and system based on hierarchical knowledge expression
CN113315972B (en) * 2021-05-19 2022-04-19 西安电子科技大学 Video semantic communication method and system based on hierarchical knowledge expression
CN113705245A (en) * 2021-09-01 2021-11-26 北京邮电大学 Semantic communication method, device, system, computer equipment and storage medium
CN114224343A (en) * 2022-01-13 2022-03-25 平安科技（深圳）有限公司 Cognitive disorder detection method, device, equipment and storage medium
CN114224343B (en) * 2022-01-13 2023-10-20 平安科技（深圳）有限公司 Cognitive disorder detection method, device, equipment and storage medium

Also Published As

Publication number Publication date
CN112528873B (en) 2022-03-22

Similar Documents

Publication Publication Date Title
CN112308158B (en) Multi-source field self-adaptive model and method based on partial feature alignment
CN112528873B (en) Signal semantic recognition method based on multi-stage semantic representation and semantic calculation
US11256960B2 (en) Panoptic segmentation
CN110020682B (en) Attention mechanism relation comparison network model method based on small sample learning
Bruggemann et al. Automated search for resource-efficient branched multi-task networks
CN109934261B (en) Knowledge-driven parameter propagation model and few-sample learning method thereof
CN107526785A (en) File classification method and device
CN111291556A (en) Chinese entity relation extraction method based on character and word feature fusion of entity meaning item
CN109508686B (en) Human behavior recognition method based on hierarchical feature subspace learning
CN112651940B (en) Collaborative visual saliency detection method based on dual-encoder generation type countermeasure network
CN111639719A (en) Footprint image retrieval method based on space-time motion and feature fusion
CN113378938B (en) Edge transform graph neural network-based small sample image classification method and system
CN113486190A (en) Multi-mode knowledge representation method integrating entity image information and entity category information
CN115995293A (en) Circular RNA and disease association prediction method
Liu et al. Image retrieval using CNN and low-level feature fusion for crime scene investigation image database
CN108805280A (en) A kind of method and apparatus of image retrieval
Elthakeb et al. Divide and conquer: Leveraging intermediate feature representations for quantized training of neural networks
CN114492581A (en) Method for classifying small sample pictures based on transfer learning and attention mechanism element learning application
Hou et al. A face detection algorithm based on two information flow block and retinal receptive field block
CN111695436B (en) High spatial resolution remote sensing image scene classification method based on target enhancement
Yildirim et al. REGP: A NEW POOLING ALGORITHM FOR DEEP CONVOLUTIONAL NEURAL NETWORKS.
CN112686306B (en) ICD operation classification automatic matching method and system based on graph neural network
Bodenhausen et al. Connectionist architectural learning for high performance character and speech recognition
Ilie-Ablachim et al. Sparse representations with cone atoms
Ko et al. Deep model compression and inference speedup of sum–product networks on tensor trains

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant