CN113255819B - Method and device for identifying information - Google Patents
Method and device for identifying information Download PDFInfo
- Publication number
- CN113255819B CN113255819B CN202110650616.3A CN202110650616A CN113255819B CN 113255819 B CN113255819 B CN 113255819B CN 202110650616 A CN202110650616 A CN 202110650616A CN 113255819 B CN113255819 B CN 113255819B
- Authority
- CN
- China
- Prior art keywords
- information
- category
- feature
- attribute
- neural network
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 64
- 238000013528 artificial neural network Methods 0.000 claims description 91
- 238000003860 storage Methods 0.000 claims description 17
- 230000000007 visual effect Effects 0.000 claims description 11
- 238000004590 computer program Methods 0.000 claims description 9
- 238000012512 characterization method Methods 0.000 abstract description 26
- 230000009286 beneficial effect Effects 0.000 abstract 1
- 230000006870 function Effects 0.000 description 31
- 238000013527 convolutional neural network Methods 0.000 description 10
- 230000008569 process Effects 0.000 description 9
- 238000012549 training Methods 0.000 description 9
- 238000010586 diagram Methods 0.000 description 8
- 238000012545 processing Methods 0.000 description 8
- 238000004891 communication Methods 0.000 description 5
- 238000002372 labelling Methods 0.000 description 5
- 230000004927 fusion Effects 0.000 description 4
- 238000011176 pooling Methods 0.000 description 4
- 238000000605 extraction Methods 0.000 description 3
- 230000003287 optical effect Effects 0.000 description 3
- 238000007500 overflow downdraw method Methods 0.000 description 3
- 238000005096 rolling process Methods 0.000 description 3
- 230000000694 effects Effects 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 239000000835 fiber Substances 0.000 description 2
- 238000005457 optimization Methods 0.000 description 2
- 230000000644 propagated effect Effects 0.000 description 2
- 239000013598 vector Substances 0.000 description 2
- 230000004913 activation Effects 0.000 description 1
- 238000013473 artificial intelligence Methods 0.000 description 1
- 230000003190 augmentative effect Effects 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 238000013135 deep learning Methods 0.000 description 1
- 238000009826 distribution Methods 0.000 description 1
- 230000010365 information processing Effects 0.000 description 1
- 239000004973 liquid crystal related substance Substances 0.000 description 1
- 238000010801 machine learning Methods 0.000 description 1
- 239000011159 matrix material Substances 0.000 description 1
- 238000005259 measurement Methods 0.000 description 1
- 230000007246 mechanism Effects 0.000 description 1
- 239000013307 optical fiber Substances 0.000 description 1
- 239000004065 semiconductor Substances 0.000 description 1
- 238000012360 testing method Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/22—Matching criteria, e.g. proximity measures
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/25—Fusion techniques
- G06F18/253—Fusion techniques of extracted features
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N20/00—Machine learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- Artificial Intelligence (AREA)
- General Physics & Mathematics (AREA)
- Evolutionary Computation (AREA)
- Life Sciences & Earth Sciences (AREA)
- Software Systems (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Computing Systems (AREA)
- Mathematical Physics (AREA)
- Biomedical Technology (AREA)
- Evolutionary Biology (AREA)
- Biophysics (AREA)
- Computational Linguistics (AREA)
- General Health & Medical Sciences (AREA)
- Molecular Biology (AREA)
- Health & Medical Sciences (AREA)
- Bioinformatics & Computational Biology (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Medical Informatics (AREA)
- Image Analysis (AREA)
Abstract
Embodiments of the present disclosure disclose methods and apparatus for identifying information. One embodiment of the method comprises the following steps: acquiring a first attribute characteristic of information to be identified; acquiring a category feature set, wherein each category feature in the category feature set is used for representing different information categories, and each category feature is obtained by fusing a first attribute feature and a second attribute feature of information in a target information set of a corresponding information category; determining the similarity between the first attribute features of the information to be identified and each feature in the category feature set to obtain a similarity set; and determining the information category to which the information to be identified belongs according to the similarity set. The embodiment realizes that the corresponding category characterization with higher confidence coefficient is generated by utilizing various attribute characteristics of the information under the information category, thereby being beneficial to realizing the robust and credible identification of the information category under the multi-attribute condition.
Description
Technical Field
Embodiments of the present disclosure relate to the field of computer technology, and in particular, to a method and apparatus for identifying information.
Background
Class learning is one of the main application scenarios of artificial intelligence. For example, the corresponding plant category is identified from the plant image. For another example, the item category is identified from the item image. As another example, a category to which the video content belongs, and so on. At present, classification learning is mainly realized by using a network model based on deep learning. In order to make a network model have better classification capability, a large number of labeling samples are usually required to train the network model.
In some application scenarios, obtaining a large number of labeling samples requires high labor or time costs, or only a small number of labeling samples can be obtained. In these cases, how to implement classification learning with fewer labeling samples, i.e., small sample learning, is a considerable problem.
Currently, the main method of small sample Learning is Meta-Learning (Meta-Learning). The meta learning for the small sample mainly comprises two kinds of meta learning methods based on optimization and meta learning methods based on measurement. The meta-learning method based on optimization is usually to train a network model for classification by using a small amount of labeling samples, and then optimize the network model by using new tasks in a test stage. Metric-based meta-learning methods typically learn class characterizations using a small number of labeled samples, and then determine the class of the object to be classified based on the relationship between the characteristic characterizations and the class characterizations of the object to be classified.
Disclosure of Invention
Embodiments of the present disclosure propose methods and apparatus for identifying information.
In a first aspect, embodiments of the present disclosure provide a method for identifying information, the method comprising: acquiring a first attribute characteristic of information to be identified; acquiring a category feature set, wherein each category feature in the category feature set is used for representing different information categories, and each category feature is obtained by fusing a first attribute feature and a second attribute feature of information in a target information set of a corresponding information category; determining the similarity between the first attribute features of the information to be identified and each feature in the category feature set to obtain a similarity set; and determining the information category to which the information to be identified belongs according to the similarity set.
In a second aspect, embodiments of the present disclosure provide an apparatus for identifying information, the apparatus comprising: an attribute feature acquisition unit configured to acquire a first attribute feature of information to be identified; the information processing device comprises a category feature acquisition unit, a processing unit and a processing unit, wherein the category feature acquisition unit is configured to acquire a category feature set, wherein each category feature in the category feature set is used for representing different information categories, and each category feature is obtained by fusing a first attribute feature and a second attribute feature of information in a target information set of a corresponding information category; the similarity determining unit is configured to determine the similarity between the first attribute features of the information to be identified and each category feature in the category feature set respectively to obtain a similarity set; and the information category determining unit is configured to determine the information category to which the information to be identified belongs according to the similarity set.
In a third aspect, embodiments of the present disclosure provide an electronic device comprising: one or more processors; a storage means for storing one or more programs; the one or more programs, when executed by the one or more processors, cause the one or more processors to implement the method as described in any of the implementations of the first aspect.
In a fourth aspect, embodiments of the present disclosure provide a computer readable medium having stored thereon a computer program which, when executed by a processor, implements a method as described in any of the implementations of the first aspect.
The method and the device for identifying information provided by the embodiment of the disclosure determine the category characteristics corresponding to each information category in advance by fusing at least two kinds of attribute characteristics of information in the target information set corresponding to each information category, and then can determine the information category of the information to be identified according to the similarity between one kind of attribute characteristics of the information to be identified and the category characteristics corresponding to various information categories respectively. By utilizing the multiple attribute characteristics of the information under each information category, the method can generate more accurate category characterization, so that the accuracy, the robustness and the credibility of the subsequent category identification for the information to be identified are improved.
Drawings
Other features, objects and advantages of the present disclosure will become more apparent upon reading of the detailed description of non-limiting embodiments, made with reference to the following drawings:
FIG. 1 is an exemplary system architecture diagram in which an embodiment of the present disclosure may be applied;
FIG. 2 is a flow chart of one embodiment of a method for identifying information according to the present disclosure;
FIG. 3 is a flow chart of one embodiment of a method of determining a category characterization in a method for identifying information according to the present disclosure;
FIG. 4 is a schematic diagram of one application scenario of a method for identifying information according to an embodiment of the present disclosure;
FIG. 5 is a schematic structural view of one embodiment of an apparatus for identifying information according to the present disclosure;
fig. 6 is a schematic structural diagram of an electronic device suitable for use in implementing embodiments of the present disclosure.
Detailed Description
The present disclosure is described in further detail below with reference to the drawings and examples. It is to be understood that the specific embodiments described herein are merely illustrative of the invention and are not limiting of the invention. It should be noted that, for convenience of description, only the portions related to the present invention are shown in the drawings.
It should be noted that, without conflict, the embodiments of the present disclosure and features of the embodiments may be combined with each other. The present disclosure will be described in detail below with reference to the accompanying drawings in conjunction with embodiments.
Fig. 1 illustrates an exemplary architecture 100 to which embodiments of the methods for identifying information or the apparatus for identifying information of the present disclosure may be applied.
As shown in fig. 1, a system architecture 100 may include terminal devices 101, 102, 103, a network 104, and a server 105. The network 104 is used as a medium to provide communication links between the terminal devices 101, 102, 103 and the server 105. The network 104 may include various connection types, such as wired, wireless communication links, or fiber optic cables, among others.
The terminal devices 101, 102, 103 interact with the server 105 via the network 104 to receive or send messages or the like. Various client applications can be installed on the terminal devices 101, 102, 103. Such as a search class application, a browser class application, an image processing class application, a text processing class application, a shopping class application, and the like.
The terminal devices 101, 102, 103 may be hardware or software. When the terminal devices 101, 102, 103 are hardware, they may be various electronic devices including, but not limited to, smartphones, tablet computers, electronic book readers, laptop and desktop computers, and the like. When the terminal devices 101, 102, 103 are software, they can be installed in the above-listed electronic devices. Which may be implemented as multiple software or software modules (e.g., multiple software or software modules for providing distributed services) or as a single software or software module. The present invention is not particularly limited herein.
The server 105 may be a server providing various services, such as a server providing back-end support for client applications installed on the terminal devices 101, 102, 103. The server 105 may be hardware or software. When the server 105 is hardware, it may be implemented as a distributed server cluster formed by a plurality of servers, or as a single server. When server 105 is software, it may be implemented as multiple software or software modules (e.g., multiple software or software modules for providing distributed services), or as a single software or software module. The present invention is not particularly limited herein.
It should be noted that, the method for identifying information provided by the embodiments of the present disclosure is generally performed by the server 105, and accordingly, the device for identifying information is generally disposed in the server 105.
It should also be noted that the terminal devices 101, 102, 103 may also have an information identification class application installed therein, and the terminal devices 101, 102, 103 may also identify information to be identified based on the information identification class application. At this time, the method for identifying information may also be performed by the terminal devices 101, 102, 103, and correspondingly, the means for identifying information may also be provided in the terminal devices 101, 102, 103. At this point, the exemplary system architecture 100 may not have the server 105 and network 104 present.
It should be understood that the number of terminal devices, networks and servers in fig. 1 is merely illustrative. There may be any number of terminal devices, networks, and servers, as desired for implementation.
With continued reference to fig. 2, a flow 200 of one embodiment of a method for identifying information according to the present disclosure is shown. The method for identifying information includes the steps of:
Step 201, a first attribute feature of information to be identified is obtained.
In the present embodiment, the information to be identified may be various types of information. For example, the types of information to be identified include, but are not limited to: images, text, video, audio, etc.
The first attribute feature may be used to characterize an attribute value of a first attribute of the information. Wherein the first attribute may be any attribute of the information. For example, for an image, the first attribute may be a visual feature of the image (e.g., texture feature, shape feature, spatial relationship feature, shape feature, etc.).
In this embodiment, the execution subject of the method for identifying information (e.g., the server 105 shown in fig. 1, etc.) may acquire the first attribute characteristics of the information to be identified from a local or other storage device (e.g., a connected database, the terminal devices 101, 102, 103 shown in fig. 1, etc.).
Step 202, a category feature set is obtained.
In this embodiment, the set of category characteristics may consist of several category characteristics. Category features may be used to characterize the category of information to which the information belongs. Each of the category characteristics in the category characteristic set may correspond to a different category of information, respectively.
The information category dividing mode can be preset according to the actual application scene and the application requirement. For example, in an application scenario for item identification, information categories may include various item categories. For another example, in the context of plant identification applications, information categories may include various plant categories.
For each category feature in the set of category features, the category feature may be derived by fusing the first attribute feature and the second attribute feature of the information in the set of target information for the corresponding category of information. The target information set may refer to a target information set corresponding to an information category characterized by the category characteristics. The target information set corresponding to an information category may consist of several pieces of information belonging to the information category.
The second attribute feature may be used to characterize an attribute value of a second attribute of the information. Wherein the second attribute may be any attribute of the information. Generally, the first attribute is different from the second attribute. At this time, the first attribute feature and the second attribute feature are used to describe different attribute features, respectively.
Specifically, various fusion methods can be flexibly adopted to fuse the first attribute features and the second attribute features of the information in the target information set to obtain category characterization. For example, the first attribute features corresponding to each information in the target information set may be averaged to obtain a fused first attribute feature, then the second attribute features corresponding to each information may be averaged to obtain a fused second attribute feature, and then the first attribute feature and the second attribute feature (such as adding or multiplying the first attribute feature and the second attribute feature) may be fused to obtain a category representation corresponding to the information category to which the corresponding target information set belongs.
In this embodiment, the executing body may obtain the category feature set from a local or other storage device. The category feature in the category feature set may be obtained by the execution subject by fusing the first attribute feature and the second attribute feature of the information in the target information set corresponding to the information category, or may be obtained by other electronic devices or the like by fusing the first attribute feature and the second attribute feature of the information in the target information set corresponding to the information category.
Step 203, determining the similarity between the first attribute features of the information to be identified and each category feature in the category feature set, and obtaining a similarity set.
In this embodiment, the first attribute feature and the category feature may be generally represented using vectors. The similarity between the first attribute feature and the category feature may be determined using various existing similarity calculation methods. For example, the similarity between the first attribute feature and the category feature may be determined based on a euclidean distance and/or cosine similarity between the first attribute feature and the category feature.
Step 204, determining the information category to which the information to be identified belongs according to the similarity set.
In this embodiment, after the similarity set corresponding to the information to be identified is obtained, various methods may be adopted to determine the information category to which the information to be identified belongs according to each similarity in the similarity set and the actual application scenario.
For example, the maximum similarity may be selected from the similarity set, and then the information category corresponding to the maximum similarity may be determined as the information category to which the information to be identified belongs. For example, a plurality of similarities can be selected according to the order from large to small, the information categories corresponding to the selected similarities are used as candidate information categories of the information to be identified, and then the candidate information categories of the information to be identified are comprehensively determined by combining the identification results of other information category identification methods.
In some optional implementations of the present embodiment, the features represented by the first attribute feature and the features represented by the second attribute feature may be features of different modalities. For example, the features represented by the first attribute feature and the features represented by the second attribute feature may include visual features and semantic features. At this time, the visual features and semantic features of the information can be integrated to accurately represent the category features.
As an example, the first attribute feature represents a visual feature of the information and the second attribute feature represents a semantic feature of the information. Or the first attribute features represent semantic features of the information and the second attribute features represent visual features of the information.
The multi-mode characteristics of the information are utilized to improve the richness of the information attribute characteristics, so that relatively accurate category characterization can be obtained by utilizing the richer attribute characteristics, and further, the accuracy of information identification based on the category characterization can be improved.
In some optional implementations of this embodiment, the number of the second attribute features may be one or more than two, and may specifically be flexibly set according to different application scenarios or actual requirements, so as to accurately represent the category features in combination with various effective attribute features.
In some optional implementations of this embodiment, for each of the set of category characteristics, the number of information included in the set of target information for deriving that category characteristic is no greater than a preset threshold. The preset threshold may be preset by a technician according to an actual application scenario.
In some cases, the number of information of each information category that can be acquired is limited. At this time, it is difficult to learn the category criteria of the corresponding information category by using only less information. For this case, the accuracy of category characterization can be improved from another perspective by augmenting the attribute features of the information.
It should be noted that, in this disclosure, for convenience in describing different objects (e.g., different attributes, different attribute features, different neural networks, etc.), first and second are used to distinguish between the two objects, and those skilled in the art should understand that the first or second does not constitute a particular limitation on the related object.
According to the method provided by the embodiment of the invention, the types of the attribute characteristics of the information are expanded when the category characterization of each information category is learned, so that the category characterization of each information category is learned by combining the multiple attribute characteristics of the information, and therefore, the problem that the learned category characterization is unreliable due to the fact that the number of information of each information category which can be used is small or the information represented by the designated attribute characteristics of the information is insufficient can be solved to a certain extent, and the reliability of the category characterization is improved.
Referring further to FIG. 3, a flow 300 of one embodiment of a method for determining a category characterization in a method for identifying information is shown. As shown in fig. 3, for each category feature in the set of category features, it may be obtained by:
step 301, obtaining a first attribute feature and a second attribute feature of information in a target information set corresponding to the category feature.
The execution subject for determining the category characterization may be the execution subject of the method for identifying information of the present disclosure, or may be any other electronic device or the like.
In this embodiment, the executing body for determining the category characterization may acquire, from a local or other storage device, a first attribute feature and a second attribute feature corresponding to each piece of information in the target information set, respectively.
Step 302, for information in the target information set corresponding to the category characteristic, fusing the first attribute characteristic and the second attribute characteristic of the information to obtain a fused characteristic corresponding to the information.
In this embodiment, for each piece of information in the target information set, the first attribute feature and the second attribute feature of the piece of information may be fused by using various fusion methods, to obtain a fused feature corresponding to the piece of information.
For example, the first attribute feature and the second attribute feature of the information may be directly spliced as the corresponding fused feature of the information. For another example, the first attribute information and the second attribute information of the information may be multiplied to obtain the fused feature corresponding to the information.
Alternatively, the first attribute feature and the second attribute feature of the information may be fused using a nonlinear fusion method. For example, by calculating a weighted sum of the first attribute feature and the second attribute feature of the information as the corresponding fused feature of the information. Wherein the weights respectively corresponding to the first attribute feature and the second attribute feature are between 0 and 1, and the sum is 1. Thus, when the first attribute features and the second attribute features are fused, an attention mechanism can be introduced to distinguish the roles of the different attribute features, so that more accurate category characterization is facilitated.
As an example, the first attribute feature and the second attribute feature of the information may be fused by the following formula:
P′m＝aPm+(1-a)Sm.
Wherein Pm represents a first attribute feature, sm represents a second attribute feature, P'm represents a fused feature, and a represents a weight.
Step 303, determining the category characteristics according to the fused characteristics respectively corresponding to the information in the target information set corresponding to the category characteristics.
In this embodiment, after the fused features corresponding to each information in the target information set are obtained, various methods may be used to determine the category characterization of the corresponding information category. For example, the fused features corresponding to each information may be averaged or weighted to obtain a fusion result as a category characterization of the corresponding information category.
When the target information set includes only one piece of information, the fused feature corresponding to the piece of information may be directly used as a category representation of the information category to which the piece of information belongs.
In some alternative implementations of the present embodiments, for information in the target information, a first attribute feature of the information may be acquired using a first neural network that is pre-trained, and a second attribute feature of the information may be acquired using a second neural network that is pre-trained.
Wherein the first neural network and the second neural network may be designed based on the structure of the existing various feature extraction networks. The first neural network and the second neural network may have the same structure or different structures. For example, the first neural network may be a convolutional neural network, a pooled convolutional neural network, a graph convolution network, a language model, or the like.
As an example, the first or second neural network may be a multi-layer convolutional neural network to enable feature extraction of an image using the following formula:
Where x represents the input of the convolutional neural network, The output of the convolutional neural network is represented by R, h represents the height of the input image, w represents the width of the input image, c represents the number of channels, d represents the dimension, and θ is a parameter of the convolutional neural network.
As yet another example, the convolutional neural network may include a pooling layer to average the feature map of the network layer output in the convolutional neural network using the following formula:
Where Sm denotes a feature map, pm denotes a pooling result, (xi, yi) denotes a pixel point in the feature map, θ is a parameter of the pooling layer, and f (x i, θ) denotes a representation function of the pooling layer.
As yet another example, the first neural network or the second neural network may be a graph convolution network to implement feature extraction using the following formula:
where Z represents the output of the graph rolling network, reLU is a nonlinear activation function, Representing a laplace matrix, X representing the input to the graph rolling network, W1 and W2 being parameters of the graph rolling network.
The first neural network and the second neural network can be trained in advance by adopting various existing training methods based on machine learning. The first neural network and the second neural network may be trained separately.
In some alternative implementations of the present embodiment, the first neural network and the second neural network may be derived through co-training. Specifically, the first neural network and the second neural network can be trained by using various existing co-training methods so that the first neural network and the second neural network can learn each other in the training process, thereby improving the characteristic characterization capability of the first neural network and the second neural network.
Alternatively, the first neural network and the second neural network co-trained penalty function may include one or more of a first penalty function, a second penalty function, and a third penalty function.
Wherein the first loss function may be used to characterize a difference between the output of the first neural network and the corresponding desired output, and/or to characterize a difference between the output of the second neural network and the corresponding desired output. The output of the neural network may refer to the output result of the neural network on the input sample in the training process, and the corresponding expected output may refer to the output result of the neural network labeled in advance for the input sample. As an example, the first loss function may be designed based on the output of the neural network and a 2-norm corresponding to the desired output.
The second loss function may be used to represent a difference between the information category determined from the output of the first and second neural networks and the annotated information category. Specifically, in the training process, output results of the first neural network and the second neural network for the same input sample may be fused, then information types of the input sample are predicted according to the fusion results, and then a second loss function is calculated according to differences between the predicted information and the types of the input sample labeled in advance.
A third loss function may be used to align the first neural network and the second neural network. Specifically, in the training process, the third loss function can be designed by using various existing network alignment methods to align the feature distribution of the vector spaces corresponding to the first neural network and the second neural network respectively, so as to improve the robustness of the overall recognition effect.
As an example, the third loss function may be expressed as the following formula:
Where L MA represents a third loss function and E [ ] represents a mathematical expectation. P represents a first neural network. S denotes a second neural network. D (P) represents an output of the discrimination network corresponding to the first neural network. D (S) represents the output of the discrimination network corresponding to the second neural network. The values of D (P) and D (S) are all 0 to 1.
At this time, the first neural network and the second neural network may be aligned based on the generated countermeasure network. Specifically, the first neural network and the second neural network may be used as generation models, and discrimination models corresponding to the first neural network and the second neural network may be respectively constructed.
In an actual training process, parameters of the first neural network and the second neural network may be adjusted using one or more of the first loss function, the second loss function, and the third loss function to complete the training of the first neural network and the second neural network.
With continued reference to fig. 4, fig. 4 is an exemplary application scenario 400 of a method for identifying information according to the present embodiment. In the application scenario of fig. 4, an image 401 to be identified may be acquired, and then visual features 403 of the image 401 to be identified may be extracted using a pre-trained convolutional neural network 402. And obtaining category characterizations 404 corresponding to each pre-divided information category respectively, calculating the similarity between the visual features 403 of the image 401 to be identified and each category feature to obtain a similarity set 405, and selecting the information category corresponding to the maximum similarity from the similarity set as the information category to which the image to be identified belongs.
The category characterizations 404 corresponding to the pre-divided information categories, respectively, may be obtained in advance through a process shown as reference numeral 407 in the figure. Specifically, taking an information category as an example, a sample image set belonging to the information category may be acquired first, and then a knowledge graph corresponding to a sample image in the sample image set may be acquired.
For sample images in a sample image set, the sample images can be input into a convolutional neural network to extract visual features of the sample images, and meanwhile, a knowledge graph corresponding to the sample image set is processed by using a graph convolution network to extract semantic features of nodes representing the sample images in the knowledge graph. And then, weighting and summing the visual characteristics and the semantic characteristics of the sample image to obtain the fusion characteristics corresponding to the sample image. And then, the fusion characteristics respectively corresponding to the sample images in the sample image set can be averaged to be used as the category characterization of the information category corresponding to the sample image set.
According to the method provided by the embodiment of the invention, for each information category corresponding target information set, multiple attribute characteristics of target information are obtained, multiple attribute characteristics of each information are fused, and the fused characteristics corresponding to each information are synthesized to represent the information category, so that the multiple attribute characteristics of the information can be fully mined to represent the information more accurately, and the characteristic representation of each information is assisted to represent the category representation of the corresponding information category more accurately. Furthermore, the generated category characterization with higher confidence coefficient can better identify the information category for the information to be identified, so that the robust and reliable identification of the information category under the multi-attribute condition is realized.
With further reference to fig. 5, as an implementation of the method shown in the above figures, the present disclosure provides an embodiment of an apparatus for identifying information, which corresponds to the method embodiment shown in fig. 2, and which is particularly applicable in various electronic devices.
As shown in fig. 5, the apparatus 500 for identifying information provided in the present embodiment includes an attribute feature acquisition unit 501, a category feature acquisition unit 502, a similarity determination unit 503, and an information category determination unit 504. Wherein the attribute feature acquisition unit 501 is configured to acquire a first attribute feature of information to be identified; the category feature acquisition unit 502 is configured to acquire a category feature set, wherein each category feature in the category feature set is used for characterizing a different information category, and each category feature is obtained by fusing a first attribute feature and a second attribute feature of information in a target information set of a corresponding information category; the similarity determining unit 503 is configured to determine the similarity between the first attribute features of the information to be identified and each of the category features in the category feature set, so as to obtain a similarity set; the information category determination unit 504 is configured to determine, from the similarity set, an information category to which the information to be identified belongs.
In the present embodiment, in the apparatus 500 for identifying information: specific processes of the attribute feature acquiring unit 501, the category feature acquiring unit 502, the similarity determining unit 503, and the information category determining unit 504 and technical effects thereof may refer to the relevant descriptions of step 201, step 202, step 203, and step 204 in the corresponding embodiment of fig. 2, respectively, and are not described herein again.
In some optional implementations of this embodiment, the target information set includes no more than a preset threshold of information.
In some optional implementations of this embodiment, each category feature in the set of category features is obtained by: acquiring a first attribute feature and a second attribute feature of information in a target information set corresponding to the category feature; for information in the target information set corresponding to the category characteristics, fusing the first attribute characteristics and the second attribute characteristics of the information to obtain fused characteristics corresponding to the information; and determining the category characteristics according to the fused characteristics respectively corresponding to the information in the target information set corresponding to the category characteristics.
In some optional implementations of this embodiment, obtaining the first attribute feature and the second attribute feature of the information in the target information set corresponding to the category feature includes: acquiring a first attribute characteristic of information in a target information set corresponding to the category characteristic by using a first neural network trained in advance; acquiring second attribute characteristics of information in the target information set corresponding to the category characteristics by using a pre-trained second neural network; wherein the first neural network and the second neural network are co-trained.
In some optional implementations of the present embodiment, the first neural network and the second neural network co-trained penalty functions include a first penalty function, wherein the first penalty function is used to characterize a difference between an output of the first neural network and a corresponding desired output, and/or a difference between an output of the second neural network and a corresponding desired output.
In some alternative implementations of the present embodiment, the first neural network and the second neural network co-trained penalty functions include a second penalty function, wherein the second penalty function is used to represent the difference between the information category determined from the output of the first neural network and the second neural network and the annotated information category.
In some optional implementations of the present embodiment, the first neural network and the second neural network co-trained penalty function includes a third penalty function, wherein the third penalty function is used to align the first neural network and the second neural network.
In some alternative implementations of the present embodiment, the features represented by the first attribute feature and the features represented by the second attribute feature include visual features and semantic features.
The device provided by the embodiment of the present disclosure acquires the first attribute feature of the information to be identified through the attribute feature acquisition unit; the method comprises the steps that a category characteristic acquisition unit acquires a category characteristic set, wherein each category characteristic in the category characteristic set is used for representing different information categories respectively, and each category characteristic is obtained by fusing a first attribute characteristic and a second attribute characteristic of information in a target information set of a corresponding information category; the similarity determining unit determines the similarity between the first attribute features of the information to be identified and each category feature in the category feature set respectively to obtain a similarity set; and the information category determining unit determines the information category to which the information to be identified belongs according to the similarity set. By utilizing the multiple attribute characteristics of the information under each information category, the method can generate more accurate category characterization, so that the accuracy, the robustness and the credibility of the subsequent category identification for the information to be identified are improved.
Referring now to fig. 6, a schematic diagram of an electronic device (e.g., server in fig. 1) 600 suitable for use in implementing embodiments of the present disclosure is shown. The server illustrated in fig. 6 is merely an example, and should not be construed as limiting the functionality and scope of use of the embodiments of the present disclosure in any way.
As shown in fig. 6, the electronic device 600 may include a processing means (e.g., a central processing unit, a graphics processor, etc.) 601, which may perform various appropriate actions and processes according to a program stored in a Read Only Memory (ROM) 602 or a program loaded from a storage means 608 into a Random Access Memory (RAM) 603. In the RAM603, various programs and data required for the operation of the electronic apparatus 600 are also stored. The processing device 601, the ROM 602, and the RAM603 are connected to each other through a bus 604. An input/output (I/O) interface 605 is also connected to bus 604.
In general, the following devices may be connected to the I/O interface 605: input devices 606 including, for example, a touch screen, touchpad, keyboard, mouse, camera, microphone, accelerometer, gyroscope, and the like; an output device 607 including, for example, a Liquid Crystal Display (LCD), a speaker, a vibrator, and the like; storage 608 including, for example, magnetic tape, hard disk, etc.; and a communication device 609. The communication means 609 may allow the electronic device 600 to communicate with other devices wirelessly or by wire to exchange data. While fig. 6 shows an electronic device 600 having various means, it is to be understood that not all of the illustrated means are required to be implemented or provided. More or fewer devices may be implemented or provided instead. Each block shown in fig. 6 may represent one device or a plurality of devices as needed.
In particular, according to embodiments of the present disclosure, the processes described above with reference to flowcharts may be implemented as computer software programs. For example, embodiments of the present disclosure include a computer program product comprising a computer program embodied on a computer readable medium, the computer program comprising program code for performing the method shown in the flowcharts. In such an embodiment, the computer program may be downloaded and installed from a network via communication means 609, or from storage means 608, or from ROM 602. The above-described functions defined in the methods of the embodiments of the present disclosure are performed when the computer program is executed by the processing means 601.
It should be noted that, the computer readable medium according to the embodiments of the present disclosure may be a computer readable signal medium or a computer readable storage medium, or any combination of the two. The computer readable storage medium can be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or a combination of any of the foregoing. More specific examples of the computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In an embodiment of the present disclosure, a computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. Whereas in embodiments of the present disclosure, the computer-readable signal medium may comprise a data signal propagated in baseband or as part of a carrier wave, with computer-readable program code embodied therein. Such a propagated data signal may take any of a variety of forms, including, but not limited to, electro-magnetic, optical, or any suitable combination of the foregoing. A computer readable signal medium may also be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device. Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to: electrical wires, fiber optic cables, RF (radio frequency), and the like, or any suitable combination of the foregoing.
The computer readable medium may be contained in the electronic device; or may exist alone without being incorporated into the electronic device. The computer readable medium carries one or more programs which, when executed by the electronic device, cause the electronic device to: acquiring a first attribute characteristic of information to be identified; acquiring a category feature set, wherein each category feature in the category feature set is used for representing different information categories, and each category feature is obtained by fusing a first attribute feature and a second attribute feature of information in a target information set of a corresponding information category; determining the similarity between the first attribute features of the information to be identified and each feature in the category feature set to obtain a similarity set; and determining the information category to which the information to be identified belongs according to the similarity set.
Computer program code for carrying out operations of embodiments of the present disclosure may be written in one or more programming languages, including an object oriented programming language such as Java, smalltalk, C ++ and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any kind of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or may be connected to an external computer (for example, through the Internet using an Internet service provider).
The flowcharts and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.
The units involved in the embodiments described in the present disclosure may be implemented by means of software, or may be implemented by means of hardware. The described units may also be provided in a processor, for example, described as: a processor includes an attribute feature acquisition unit, a category feature acquisition unit, a similarity determination unit, and an information category determination unit. The names of these units do not constitute a limitation of the unit itself in some cases, and for example, the attribute feature acquisition unit may also be described as "a unit that acquires the first attribute feature of the information to be identified".
The foregoing description is only of the preferred embodiments of the present disclosure and description of the principles of the technology being employed. It will be appreciated by those skilled in the art that the scope of the invention in the embodiments of the present disclosure is not limited to the specific combination of the above technical features, but encompasses other technical features formed by any combination of the above technical features or their equivalents without departing from the spirit of the invention. Such as the above-described features, are mutually substituted with (but not limited to) the features having similar functions disclosed in the embodiments of the present disclosure.
Claims (10)
1. A method for identifying information, comprising:
Acquiring a first attribute characteristic of information to be identified, wherein the information to be identified is an image;
Obtaining a category feature set, wherein each category feature in the category feature set is used for representing different information categories, each category feature is obtained by fusing a first attribute feature and a second attribute feature of information in a target information set corresponding to the information category, the first attribute feature of the information in the target information set is obtained by using a pre-trained first neural network, the second attribute feature of the information in the target information set is obtained by using a pre-trained second neural network, the first neural network and the second neural network are cooperatively trained, a loss function of the first neural network and the second neural network are cooperatively trained comprises a third loss function, and the third loss function is used for aligning the first neural network and the second neural network;
Determining the similarity of the first attribute features of the information to be identified and each category feature in the category feature set respectively to obtain a similarity set;
And determining the information category to which the information to be identified belongs according to the similarity set.
2. The method of claim 1, wherein the target information set includes no more than a preset threshold of information.
3. The method according to claim 1 or 2, wherein each category feature in the set of category features is obtained by:
Acquiring a first attribute feature and a second attribute feature of information in a target information set corresponding to the category feature;
For information in the target information set corresponding to the category characteristics, fusing the first attribute characteristics and the second attribute characteristics of the information to obtain fused characteristics corresponding to the information;
and determining the category characteristics according to the fused characteristics respectively corresponding to the information in the target information set corresponding to the category characteristics.
4. A method according to claim 3, wherein the obtaining the first attribute feature and the second attribute feature of the information in the target information set corresponding to the category feature includes:
acquiring a first attribute characteristic of information in a target information set corresponding to the category characteristic by using a first neural network trained in advance;
and acquiring a second attribute characteristic of the information in the target information set corresponding to the category characteristic by using a pre-trained second neural network.
5. The method of claim 4, wherein the first and second neural networks co-trained loss function comprises a first loss function, wherein the first loss function is used to characterize a difference between an output of the first neural network and a corresponding desired output, and/or a difference between an output of the second neural network and a corresponding desired output.
6. The method of claim 4, wherein the first and second neural networks co-trained loss function comprises a second loss function, wherein the second loss function is used to represent a difference between an information category determined from the output of the first and second neural networks and a labeled information category.
7. The method of claim 4, wherein the features represented by the first attribute feature and the features represented by the second attribute feature comprise visual features and semantic features.
8. An apparatus for identifying information, wherein the apparatus comprises:
The device comprises an attribute characteristic acquisition unit, a storage unit and a storage unit, wherein the attribute characteristic acquisition unit is configured to acquire a first attribute characteristic of information to be identified, and the information to be identified is an image;
A category feature acquisition unit configured to acquire a category feature set, wherein each category feature in the category feature set is used for characterizing different information categories, each category feature is obtained by fusing a first attribute feature and a second attribute feature of information in a target information set corresponding to the information category, the first attribute feature of the information in the target information set is obtained by using a pre-trained first neural network, the second attribute feature of the information in the target information set is obtained by using a pre-trained second neural network, the first neural network and the second neural network are cooperatively trained, and a loss function of the first neural network and the second neural network are cooperatively trained includes a third loss function, and the third loss function is used for aligning the first neural network and the second neural network;
The similarity determining unit is configured to determine the similarity between the first attribute features of the information to be identified and each type of features in the category feature set respectively to obtain a similarity set;
and the information category determining unit is configured to determine the information category to which the information to be identified belongs according to the similarity set.
9. An electronic device, comprising:
one or more processors;
A storage device having one or more programs stored thereon;
When executed by the one or more processors, causes the one or more processors to implement the method of any of claims 1-7.
10. A computer readable medium having stored thereon a computer program, wherein the program when executed by a processor implements the method of any of claims 1-7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110650616.3A CN113255819B (en) | 2021-06-11 | 2021-06-11 | Method and device for identifying information |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110650616.3A CN113255819B (en) | 2021-06-11 | 2021-06-11 | Method and device for identifying information |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113255819A CN113255819A (en) | 2021-08-13 |
| CN113255819B true CN113255819B (en) | 2024-04-19 |
Family
ID=77187507
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110650616.3A Active CN113255819B (en) | 2021-06-11 | 2021-06-11 | Method and device for identifying information |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113255819B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114003188B (en) * | 2021-10-09 | 2024-02-20 | 北京用友薪畴数字科技有限公司 | Information encryption display method, device, electronic equipment and computer readable medium |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108830334A (en) * | 2018-06-25 | 2018-11-16 | 江西师范大学 | A kind of fine granularity target-recognition method based on confrontation type transfer learning |
| CN111523574A (en) * | 2020-04-13 | 2020-08-11 | 云南大学 | Image emotion recognition method and system based on multi-mode data |
| CN111598164A (en) * | 2020-05-15 | 2020-08-28 | 北京百度网讯科技有限公司 | Method and device for identifying attribute of target object, electronic equipment and storage medium |
| CN112016601A (en) * | 2020-08-17 | 2020-12-01 | 华东师范大学 | Network model construction method based on knowledge graph enhanced small sample visual classification |
| CN112149699A (en) * | 2019-06-28 | 2020-12-29 | 北京京东尚科信息技术有限公司 | Method and device for generating model and method and device for recognizing image |
| CN112434628A (en) * | 2020-11-30 | 2021-03-02 | 西安理工大学 | Small sample polarization SAR image classification method based on active learning and collaborative representation |
| CN112528058A (en) * | 2020-11-23 | 2021-03-19 | 西北工业大学 | Fine-grained image classification method based on image attribute active learning |
Family Cites Families (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20100161524A1 (en) * | 2008-12-19 | 2010-06-24 | International Business Machines Corporation | Method and system for identifying graphical model semantics |
| US10747795B2 (en) * | 2018-01-11 | 2020-08-18 | International Business Machines Corporation | Cognitive retrieve and rank search improvements using natural language for product attributes |
-
2021
- 2021-06-11 CN CN202110650616.3A patent/CN113255819B/en active Active
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108830334A (en) * | 2018-06-25 | 2018-11-16 | 江西师范大学 | A kind of fine granularity target-recognition method based on confrontation type transfer learning |
| CN112149699A (en) * | 2019-06-28 | 2020-12-29 | 北京京东尚科信息技术有限公司 | Method and device for generating model and method and device for recognizing image |
| CN111523574A (en) * | 2020-04-13 | 2020-08-11 | 云南大学 | Image emotion recognition method and system based on multi-mode data |
| CN111598164A (en) * | 2020-05-15 | 2020-08-28 | 北京百度网讯科技有限公司 | Method and device for identifying attribute of target object, electronic equipment and storage medium |
| CN112016601A (en) * | 2020-08-17 | 2020-12-01 | 华东师范大学 | Network model construction method based on knowledge graph enhanced small sample visual classification |
| CN112528058A (en) * | 2020-11-23 | 2021-03-19 | 西北工业大学 | Fine-grained image classification method based on image attribute active learning |
| CN112434628A (en) * | 2020-11-30 | 2021-03-02 | 西安理工大学 | Small sample polarization SAR image classification method based on active learning and collaborative representation |
Non-Patent Citations (1)
| Title |
|---|
| 基于松散条件下协同学习的中文微博情感分析;孙念;李玉强;刘爱华;刘春;黎威威;;浙江大学学报(工学版)(08);全文 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113255819A (en) | 2021-08-13 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111860573B (en) | Model training method, image category detection method and device and electronic equipment | |
| CN109858445B (en) | Method and apparatus for generating a model | |
| CN108520220B (en) | Model generation method and device | |
| CN108898186B (en) | Method and device for extracting image | |
| CN108427939B (en) | Model generation method and device | |
| CN108280477B (en) | Method and apparatus for clustering images | |
| CN110288049B (en) | Method and apparatus for generating image recognition model | |
| CN109740018B (en) | Method and device for generating video label model | |
| CN108229419B (en) | Method and apparatus for clustering images | |
| CN109492128B (en) | Method and apparatus for generating a model | |
| CN109145828B (en) | Method and apparatus for generating video category detection model | |
| CN109376267B (en) | Method and apparatus for generating a model | |
| CN111666416B (en) | Method and device for generating semantic matching model | |
| CN109800730B (en) | Method and device for generating head portrait generation model | |
| CN110009059B (en) | Method and apparatus for generating a model | |
| CN109961032B (en) | Method and apparatus for generating classification model | |
| CN112149699B (en) | Method and device for generating model and method and device for identifying image | |
| CN112766284B (en) | Image recognition method and device, storage medium and electronic equipment | |
| CN111738010A (en) | Method and apparatus for generating semantic matching model | |
| CN111340015A (en) | Positioning method and device | |
| CN114898154A (en) | Incremental target detection method, device, equipment and medium | |
| CN113256339B (en) | Resource release method and device, storage medium and electronic equipment | |
| CN113255819B (en) | Method and device for identifying information | |
| CN112241761B (en) | Model training method and device and electronic equipment | |
| CN113592033A (en) | Oil tank image recognition model training method, oil tank image recognition method and oil tank image recognition device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| CB02 | Change of applicant information |
Address after: 100176 601, 6th floor, building 2, No. 18, Kechuang 11th Street, Daxing Economic and Technological Development Zone, Beijing Applicant after: Jingdong Technology Information Technology Co.,Ltd. Address before: 100176 601, 6th floor, building 2, No. 18, Kechuang 11th Street, Daxing Economic and Technological Development Zone, Beijing Applicant before: Jingdong Shuke Haiyi Information Technology Co.,Ltd. |
|
| CB02 | Change of applicant information | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |