CN114626480A - Multi-source heterogeneous data feature extraction device and method, storage medium and electronic equipment - Google Patents
Multi-source heterogeneous data feature extraction device and method, storage medium and electronic equipment Download PDFInfo
- Publication number
- CN114626480A CN114626480A CN202210291613.XA CN202210291613A CN114626480A CN 114626480 A CN114626480 A CN 114626480A CN 202210291613 A CN202210291613 A CN 202210291613A CN 114626480 A CN114626480 A CN 114626480A
- Authority
- CN
- China
- Prior art keywords
- features
- source
- encoder
- specific
- data
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/213—Feature extraction, e.g. by transforming the feature space; Summarisation; Mappings, e.g. subspace methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/40—Information retrieval; Database structures therefor; File system structures therefor of multimedia data, e.g. slideshows comprising image and additional audio data
- G06F16/43—Querying
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/40—Information retrieval; Database structures therefor; File system structures therefor of multimedia data, e.g. slideshows comprising image and additional audio data
- G06F16/45—Clustering; Classification
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- General Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- Life Sciences & Earth Sciences (AREA)
- Artificial Intelligence (AREA)
- Evolutionary Computation (AREA)
- Molecular Biology (AREA)
- Computational Linguistics (AREA)
- Databases & Information Systems (AREA)
- Multimedia (AREA)
- Health & Medical Sciences (AREA)
- Biomedical Technology (AREA)
- Biophysics (AREA)
- Software Systems (AREA)
- General Health & Medical Sciences (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Computing Systems (AREA)
- Mathematical Physics (AREA)
- Evolutionary Biology (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Bioinformatics & Computational Biology (AREA)
- Image Analysis (AREA)
Abstract
The invention discloses a multi-source heterogeneous data feature extraction device, a method, a storage medium and electronic equipment, wherein the device comprises a corresponding specific self-encoder, a data processing module and a data processing module, wherein the corresponding specific self-encoder is used for extracting high-level features of source data based on low-level features of the source data; the system comprises a specific part encoder, a corresponding part encoder, a prediction decoder and a reconstruction decoder, wherein the specific part encoder converts low-level features of corresponding source data into specific features, the corresponding part encoder converts the low-level features of the corresponding source data into corresponding features, the prediction encoder predicts the low-level features of other source data through the corresponding features, and the reconstruction decoder reconstructs the low-level features of the corresponding source data through the corresponding features and the corresponding features. By implementing the method, the corresponding specific self-encoder is arranged, and the corresponding de-entangled characteristics and the specific characteristics of the multi-source heterogeneous data are obtained through characteristic extraction. The method solves the problems that only corresponding characteristics or specific characteristics can be learned singly and the learned characteristics are inaccurate.
Description
Technical Field
The invention relates to the technical field of data processing, in particular to a multi-source heterogeneous data feature extraction device and method, a storage medium and electronic equipment.
Background
Currently, many tasks rely on information in a variety of forms for support. For example, the diagnosis of the insulation performance of a transformer in an electric power system requires information such as documents, pictures, videos and the like of comprehensive technical standards, test reports, papers and other data; whether the monitoring system gives an alarm needs to synthesize information such as pedestrians, infrared rays, vibration, smoke concentration, biological fingerprints and the like in the video; the diagnosis of cognitive impairment in the medical field requires the integration of daily behaviors such as gait, sleep quality, outdoor walking distance and the like of a human and brain function imaging information. The first task of constructing an effective retrieval and classification decision model is to explore a multi-source heterogeneous data feature extraction algorithm.
In general, data from different sources includes both common portions and portions that are unique to each source. Dual-source heterogeneous data as shown in fig. 1 (a): a picture and its text description. Obviously, the two concepts of "sunset" and "sea" can be seen in images and also appear in text description, and are common parts contained in data in both image and text forms; the 'Changbai island', 'iPhone 8' and 'good mood' in the text description can not be directly obtained from the image and are the information special for the text form; the "person" and "sailboat" are only visible in the image and cannot obtain such information from the text, which are parts that are characteristic of the form of the image. FIG. 1(b) gives a set-based description of the above description using the Venturi map. The left circle represents the set of image information and the right circle represents the set of text information, and the intersection of the two represents the common part of the two source information.
Although there are various methods for extracting features of multi-source heterogeneous data in the prior art, most of the existing methods respectively perform learning of a specific part or a common part, so that it cannot be clearly shown that the learned corresponding features only include the common part of the multi-source data, and a specific part of each source data is excluded.
Disclosure of Invention
In view of this, embodiments of the present invention provide a device, a method, a storage medium, and an electronic device for extracting multi-source heterogeneous data features, so as to solve a technical problem that a common extraction of a common part and a specific part cannot be accurately achieved by a method for extracting multi-source heterogeneous data features in the prior art.
The technical scheme provided by the invention is as follows:
a first aspect of an embodiment of the present invention provides a multi-source heterogeneous data feature extraction device, including: the number of the corresponding specific self-encoders is the same as that of the data sources; the corresponding specific self-encoder is used for extracting high-level features of each source data based on the low-level features of each source data, and the high-level features comprise specific features of each source data and corresponding features of a plurality of source data; the corresponding specific self-encoder comprises a specific part encoder, a corresponding part encoder, a prediction decoder and a reconstruction decoder, wherein the specific part encoder converts the low-level features of the corresponding source data into specific features, the corresponding part encoder converts the low-level features of the corresponding source data into corresponding features, the prediction encoder predicts the low-level features of other source data through the corresponding features, and the reconstruction decoder reconstructs the low-level features of the corresponding source data through the corresponding features and the specific features.
Optionally, the corresponding specific self-encoder comprises: the method comprises the steps of inputting a layer, a corresponding characteristic layer, a specific characteristic layer, a prediction output layer and a reconstruction output layer, wherein each corresponding specific characteristic layer of a specific self-encoder has a correlation constraint.
Optionally, the configuration parameters corresponding to the specific self-encoder include: the connection weight and the offset from the input layer to the corresponding characteristic layer, the connection weight and the offset from the input layer to the specific characteristic layer, the connection weight and the offset from the corresponding characteristic layer to the prediction output layer, the connection weight and the offset from the specific characteristic layer to the reconstruction output layer, and the connection weight from the corresponding characteristic layer to the prediction output layer in each corresponding specific self-encoder are consistent.
Optionally, the corresponding specific self-encoder is obtained by training through a gradient descent algorithm; and the configuration parameters of the corresponding specific self-encoder are finely adjusted by adopting a gradient descent algorithm.
Optionally, the loss function corresponding to a particular self-encoder is determined from a two-norm mapping between each of the source input layer characteristics, the predicted output layer characteristics, and the reconstructed output layer characteristics, and a two-norm mapping between corresponding partial encoders.
Optionally, the corresponding specific self-encoder further comprises: gates of a predictive decoder, the reconstruction decoder and the gates of the predictive decoder reconstructing low-level features of the respective source data from the corresponding features and the particular features.
A second aspect of an embodiment of the present invention provides a multi-source heterogeneous data feature extraction method, which is applied to the multi-source heterogeneous data feature extraction device described in any one of the first aspect and the first aspect of the embodiment of the present invention, and the method includes: extracting low-level features of the source data by using a feature extraction algorithm; based on the low-level features of each source data, the high-level features of each source data are extracted using a corresponding specific self-encoder.
Optionally, the multi-source heterogeneous data feature extraction method further includes: calculating the distance of different source data according to the corresponding feature in the high-level features of the source data; searching the matched object according to the distance, and executing a multi-source heterogeneous data searching task; and training a multi-source data fusion classification model according to specific characteristics in the high-level characteristics of each source data and corresponding characteristics of the source data, and executing a multi-source heterogeneous data classification task.
A third aspect of the embodiments of the present invention provides a computer-readable storage medium, where the computer-readable storage medium stores computer instructions for causing the computer to execute the multi-source heterogeneous data feature extraction method according to any one of the second aspect and the second aspect of the embodiments of the present invention.
A fourth aspect of an embodiment of the present invention provides an electronic device, including: the multi-source heterogeneous data feature extraction method comprises a memory and a processor, wherein the memory and the processor are mutually connected in a communication mode, the memory stores computer instructions, and the processor executes the computer instructions so as to execute the multi-source heterogeneous data feature extraction method according to the second aspect of the embodiment of the invention.
The technical scheme provided by the invention has the following effects:
according to the multi-source heterogeneous data feature extraction device, method, storage medium and electronic equipment provided by the embodiment of the invention, the corresponding specific self-encoder is arranged, so that the low-level features of the acquired source data are subjected to feature extraction, and the corresponding de-entangled features and the specific features of the multi-source heterogeneous data are obtained. Because the specific part encoder, the corresponding part encoder, the prediction decoder and the reconstruction decoder are arranged in the corresponding specific self-encoder, the specific features and the corresponding features can be simultaneously extracted by processing the corresponding features through the encoders and the decoders; moreover, conversion, prediction and reconstruction in the method ensure that the corresponding features only contain common parts of multi-source data, and the specific features only contain specific parts of the multi-source data, so that the problems that the corresponding features or the specific features can only be singly learned and the learned features are inaccurate in the processing method in the prior art are solved.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the embodiments or the prior art descriptions will be briefly described below, and it is obvious that the drawings in the following description are some embodiments of the present invention, and other drawings can be obtained by those skilled in the art without creative efforts.
FIG. 1(a) is a schematic diagram of dual-source heterogeneous data including images and text;
FIG. 1(b) is a schematic diagram of a set-based description given for dual-source heterogeneous data using a Venturi map;
FIG. 1(c) is a characteristic diagram of data using a learning common part;
FIG. 2 is a block diagram of a multi-source heterogeneous data feature extraction apparatus according to an embodiment of the present invention;
fig. 3 is a block diagram of a multi-source heterogeneous data feature extraction apparatus according to another embodiment of the present invention;
FIG. 4 is a flow chart of a multi-source heterogeneous data feature extraction method according to an embodiment of the invention;
FIG. 5 is a flow chart of a multi-source heterogeneous data feature extraction method according to another embodiment of the invention;
FIG. 6 is a schematic structural diagram of a computer-readable storage medium provided according to an embodiment of the present invention;
fig. 7 is a schematic structural diagram of an electronic device according to an embodiment of the present invention.
Detailed Description
In order to make those skilled in the art better understand the technical solutions of the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
The terms "first," "second," "third," "fourth," and the like in the description and in the claims, as well as in the drawings, are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order. It will be appreciated that the data so used may be interchanged under appropriate circumstances such that the embodiments described herein may be practiced otherwise than as specifically illustrated or described herein. Furthermore, the terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, system, article, or apparatus that comprises a list of steps or elements is not necessarily limited to those steps or elements expressly listed, but may include other steps or elements not expressly listed or inherent to such process, method, article, or apparatus.
As described in the background art, most of the current processing methods respectively perform learning on a specific part or a common part according to the characteristics of the specific part and the common part in multi-source data. For example, one approach is to learn a shared feature for data from multiple sources, and the data from multiple sources are merged to obtain a shared feature layer, which may be referred to as a shared feature policy. The goal of the model adopting the strategy is to learn a feature layer which can correspond to the Venturi map in FIG. 1(b), namely, part of dimensions of the feature layer only contain information specific to an image, part of dimensions only contain information specific to a text, and the other part of dimensions only contain common part information. However, models that employ this strategy do not specify which dimensions of the feature layer represent common portions and which dimensions represent specific portions. It is only assumed that the feature layer has learned such features.
Another processing method is to learn the corresponding feature for each source data separately, but add similarity constraints in the feature space of the data from different sources to establish the corresponding association between the multi-source data, which may be referred to as a corresponding feature policy. The goal of the model using this strategy is to learn a feature layer that can correspond to that shown in FIG. 1(c), i.e., the feature layer contains only common portions of image and text data. The optimization goal of a model employing this strategy is to minimize the distance between matching (matching refers to image-source data and text-source data being matched, e.g., { (fig. 1, text 1), (fig. 2, text 2) }) multi-source data features, while maximizing the distance between non-matching (non-matching refers to image-source data and text-source data not being matched, e.g., (fig. 1, text 2)) multi-source data features.
These models can be divided into two categories, depending on the method of maximizing the distance between unmatched multi-source data features: firstly, unmatched multi-source data pairs are constructed, and then the distance between the data pairs in the feature layer is maximized in an optimization target; and secondly, the optimization target of the feature learning model is directly utilized, so that the distance between similar input in the feature space is smaller, the distance between dissimilar input in the feature space is larger, and the optimization target of minimizing the distance between matched multi-source data features is matched, thereby achieving the aim of enabling unmatched multi-source data pairs to be far in the feature space.
Although the processing mode has excellent performance on the task of fusing and aligning multi-source heterogeneous data, it cannot be clearly shown until now that the learned corresponding features only contain common parts of the multi-source data, and specific parts of each source data are excluded. Moreover, the features extracted by a better learning multi-source heterogeneous feature extraction model simultaneously have the capability of reserving and distinguishing the common part and the specific part, so that the features can better finish the alignment retrieval task of multi-source heterogeneous data and can also better finish the multi-source classification decision task.
In view of this, an embodiment of the present invention provides a multi-source heterogeneous data feature extraction apparatus, which detangles features of each source data into corresponding features and specific features by setting a corresponding specific self-encoder, where the corresponding features are suitable for mining a relationship of multi-source data and executing a search task, and the specific features are suitable for executing a multi-source data classification task together with the corresponding features.
An embodiment of the present invention provides a multi-source heterogeneous data feature extraction device, as shown in fig. 2, the device includes: the number of the corresponding specific self-encoders is the same as that of the data sources; the corresponding specific self-encoder is used for extracting high-level features of each source data based on the low-level features of each source data, and the high-level features comprise specific features of each source data and corresponding features of a plurality of source data; the corresponding specific self-encoder comprises a specific part encoder, a corresponding part encoder, a prediction decoder and a reconstruction decoder, wherein the specific part encoder converts the low-level features of the corresponding source data into specific features, the corresponding part encoder converts the low-level features of the corresponding source data into corresponding features, the prediction encoder predicts the low-level features of other source data through the corresponding features, and the reconstruction decoder reconstructs the low-level features of the corresponding source data through the corresponding features and the specific features.
According to the multi-source heterogeneous data feature extraction device provided by the embodiment of the invention, the corresponding specific self-encoder is arranged, so that the low-level features of the acquired source data are subjected to feature extraction, and the corresponding de-entangled features and the specific features of the multi-source heterogeneous data are obtained. Because the specific part encoder, the corresponding part encoder, the prediction decoder and the reconstruction decoder are arranged in the corresponding specific self-encoder, the specific features and the corresponding features can be simultaneously extracted by processing the corresponding features through the encoders and the decoders; moreover, conversion, prediction and reconstruction in the method ensure that the corresponding features only contain common parts of multi-source data, and the specific features only contain specific parts of the multi-source data, so that the problems that the corresponding features or the specific features can only be singly learned and the learned features are inaccurate in the processing method in the prior art are solved.
In one embodiment, the low-level features of the source data may be extracted in advance by using a feature extraction method. The specific feature extraction method may be any existing method capable of realizing feature extraction. The low-level features may be edge information, surface or apparent information in the data, such as features like texture. Advanced features are some higher level semantic feature information.
Wherein, for a specific data source, the low-level characteristic of the source data is x(k)Then the specific partial encoder is denoted as fI→SThe corresponding partial encoder is denoted as fI→CThe predictive decoder is denoted as gC→IThe reconstruction decoder is denoted as { g }S→I,gC→I}. Whereby the corresponding feature is denoted fI→C(x(k)) With a particular feature denoted as fI→S(x(k)) The feature obtained by reconstructing the low-level feature of the corresponding source data through the corresponding feature and the specific feature is expressed as
If the low-level features of the data from other sources are denoted as y(k)The corresponding partial encoders in the other sources are denoted as fT→CThen, the low-level features of the data from other sources are predicted to obtain features represented as
For the corresponding Specific auto-Encoder (COSE), it has both the corresponding feature and the Specific feature extraction capability. Wherein the corresponding features correspond to a common portion of the different source data by which mutual prediction of the different source data can be achieved. That is, the corresponding specific self-encoder corresponding to each source can predict the data of other sources through the corresponding features. Such as inputting data x(k)By means of an encoder fI→CGet the corresponding feature and then pass through the decoder gC→ITo predict data y of a second source(k)(ii) a By which corresponding particular autocoder of other sources inputs data y(k)To predict the data x of the source(k). If the mutually predicted part is lacked, the situation that only a specific feature participates in the reconstruction input and the corresponding feature does not participate in the reconstruction input occurs, namely, the corresponding features of the two sources are the same fixed value, the associated error is 0, and the corresponding specific self-encoders of the two sources respectively reconstruct the data of the self-sources, which is equivalent to two independent self-encoders, and the total error is minimum at the moment. It is clear that the corresponding features of the corresponding specific self-encoders, which lack the mutual prediction part, do not contain any information, and that the specific features contain the full information of each source data. Therefore, this portion is indispensable to the COSE.
Secondly, the specific features correspond to the personality parts of the corresponding source data, and although the personality parts of the data cannot be directly obtained as reconstruction targets, the COSE reconstructs information of a single source by using the specific features and the corresponding features together. Thus, since a corresponding particular autoencoder is able to predict data from another source via a corresponding feature, which, when reconstructing data from a current source together with the particular feature, serves to reconstruct a common portion of the multi-source data, the particular feature can focus on reconstructing the remaining portion, i.e., the personalized portion of the data from its corresponding source. For example, in the pair x(k)When reconstruction is performed, the input source comprises two parts: one is that the specific feature is mapped by a function gS→IThe decoding result of (2), the corresponding feature passes through the mapping function gC→IAnd (5) decoding results. Therefore, only let function gC→IThe result of the decoding is a common part of the two source data, enabling the function g to be implementedS→IIs the input data x(k)The personality part of (1). Only then can reconstruction of the input data x be accomplished(k)The task of (2). After the model is trained, the corresponding features of the two aligned source data are close to each other, and the two aligned source data are subjected to the mutual prediction task, that is, the mapping function gC→IThe object of decoding is the corresponding feature of the two source data, becauseThus, the decoding result tends to restore the common part of the two source data.
In one embodiment, the corresponding specific self-encoder comprises: the method comprises the steps of inputting a layer, a corresponding characteristic layer, a specific characteristic layer, a prediction output layer and a reconstruction output layer, wherein each corresponding specific characteristic layer of a specific self-encoder has a correlation constraint. The configuration parameters of the corresponding specific self-encoder comprise: the connection weight and the offset from the input layer to the corresponding characteristic layer, the connection weight and the offset from the input layer to the specific characteristic layer, the connection weight and the offset from the corresponding characteristic layer to the prediction output layer, the connection weight and the offset from the specific characteristic layer to the reconstruction output layer, and the connection weight from the corresponding characteristic layer to the prediction output layer in each corresponding specific self-encoder are consistent.
Specifically, in order to make the corresponding features of the aligned multi-source heterogeneous data similar, a scheme of explicitly increasing the corresponding feature similarity constraint in the optimization target may be adopted, or a scheme of explicitly increasing the corresponding feature similarity constraint in the optimization target while sharing the decoder weight of the corresponding feature may also be adopted. Since the corresponding feature layers of the data from the two sources are similar, it decodes the reconstructed data x(k)And y(k)Should be consistent, i.e. the connection weight from the corresponding feature layer to the prediction output layer in each corresponding specific self-encoder is consistent.
In one embodiment, the corresponding specific self-encoder is obtained by training with a gradient descent algorithm; and the configuration parameters of the corresponding specific self-encoder are finely adjusted by adopting a gradient descent algorithm. Specifically, the structure of the specific part encoder, the corresponding part encoder, the prediction decoder and the reconstruction decoder in the specific self-encoder may adopt the structure of an existing encoder or decoder, for example, a neural network including a layer of hidden nodes, and having the same number of nodes as the input and the output. Corresponding to the above structure, the corresponding feature layer and the specific feature layer are hidden layers in the corresponding specific self-encoder.
In one embodiment, the loss function corresponding to a particular self-encoder is determined based on a two-norm mapping between each of the source input layer characteristics, the predicted output layer characteristics, and the reconstructed output layer characteristics, and a two-norm mapping between corresponding partial encoders. The corresponding specific auto-encoder further comprises: gates of a predictive decoder, the reconstruction decoder and the gates of the predictive decoder reconstructing low-level features of the respective source data from the corresponding features and the particular features.
For the predictive decoder in the corresponding specific self-encoder, the input information only contains the information capable of decoding and outputting the common part, so the decoding result contains the noise of the predictive specific part. Since the connection weights of the corresponding feature layer to the prediction output layer are consistent, noise also affects the reconstructed part of another corresponding specific self-encoder. Therefore, when reconstructing the self-source data, a gate structure is added to filter out the noise.
In one embodiment, assume that there are N pieces of data that make up a multi-source heterogeneous dataset, denoted as D ═ { z ═ z1,z2,…,zk,…,zNIn which z isk＝{x(k),y(k)Is the kth data, x(k),y(k)The low-level characteristics of the first source data and the low-level characteristics of the second source data of the piece of data, respectively. As shown in fig. 3, represented as f by the corresponding special auto-encoder (including the special section encoder) in the first sourceI→sThe corresponding partial encoder is denoted as fI→CThe predictive decoder is denoted as gC→IThe reconstruction decoder is denoted as { g }S→I,gC→I}) and a corresponding particular auto-encoder of the second source (including a particular partial encoder denoted fT→SThe corresponding partial encoder is denoted as fT→CThe predictive decoder is denoted as gC→TThe reconstruction decoder is denoted as { g }S→T,gC→T}) to predict and reconstruct, the first source corresponds to a particular autoencoder reconstruction output
And predicted output
The second source corresponds to a particular self-encoder prediction output
And reconstructing the output
The specific expression of (A) is as follows:
the configuration parameters of the corresponding specific self-encoder in the first source include: first source input layer to hidden layer common portion (corresponding feature layer) connection weight WI→CAnd bias bI→CFirst source input layer to hidden layer specific portion (specific feature layer) connection weight WI→SAnd bias bI→SConnecting the public part of the first source hidden layer to the predicted second source output layer with a weight WI→TAnd bias bI→TConnecting the first source hidden layer to the reconstructed first source output layer with a weight WI→IAnd bias bI→I. The configuration parameters of the corresponding specific self-encoder in the first source include: second source input layer to hidden layer common portion connection weight WT→CAnd bias bT→CSecond source input layer to hidden layer specific portion connection weight WT→SAnd bias bT→SConnecting weights of the common part of the second source hidden layer to the predicted output layer of the first sourceWT→IAnd bias bT→IConnecting the second source hidden layer to the reconstructed second source output layer by the weight WT→TAnd bias bT→T。
Meanwhile, based on the above, the loss function corresponding to a specific self-encoder is expressed as:
wherein the content of the first and second substances,
α represents a constant, α ∈ (0, 1);
is a two-norm mapping.
When the corresponding specific self-encoder is trained and the configuration parameters are finely adjusted, the final configuration parameters are obtained by training and adjusting with the minimum loss function as a target.
After the gate structure of the predictive decoder is added to the corresponding specific self-encoder, the specific calculation method of the reconstructed corresponding source characteristics is as follows:
wherein σC→IAnd σC→TAre respectively predictive decoders gC→IAnd gC→TThe door of (2).
An embodiment of the present invention further provides a multi-source heterogeneous data feature extraction method, as shown in fig. 4, where the method is applied to the multi-source heterogeneous data feature extraction device described in the above embodiment, and the method includes the following steps:
step S101: and extracting low-level features of the source data by using a feature extraction algorithm. Specifically, the low-level features of the source data may be extracted in advance by means of feature extraction. The specific feature extraction method may be any existing method capable of realizing feature extraction. The low-level features may be edge information, surface or apparent information in the data, such as features such as texture.
Step S102: based on the low-level features of each source data, the high-level features of each source data are extracted using a corresponding specific self-encoder. The structure corresponding to the specific self-encoder may adopt the structure in the above device embodiment, and is not described herein again.
According to the multi-source heterogeneous data feature extraction method provided by the embodiment of the invention, the corresponding specific self-encoder is used for extracting the features of the low-level features of the acquired source data to obtain the corresponding de-entangled features and the specific features of the multi-source heterogeneous data. Because the specific part encoder, the corresponding part encoder, the prediction decoder and the reconstruction decoder are arranged in the corresponding specific self-encoder, the specific features and the corresponding features can be simultaneously extracted by processing the corresponding features through the encoders and the decoders; moreover, conversion, prediction and reconstruction in the method ensure that the corresponding features only contain common parts of multi-source data, and the specific features only contain specific parts of the multi-source data, so that the problems that the corresponding features or the specific features can only be singly learned and the learned features are inaccurate in the processing method in the prior art are solved.
In an embodiment, as shown in fig. 5, the multi-source heterogeneous data feature extraction method further includes: calculating the distance of different source data according to the corresponding feature in the high-level features of the source data; searching the matched object according to the distance, and executing a multi-source heterogeneous data searching task; and training a multi-source data fusion classification model according to specific characteristics in the high-level characteristics of each source data and corresponding characteristics of the source data, and executing a multi-source heterogeneous data classification task.
The retrieval task refers to cross-source retrieval of multi-source heterogeneous data, namely source 1 retrieving source 2 or source 2 retrieving source 1. Because both the query and candidate sets (corresponding features) have corresponding representations after step 102, calculating the corresponding distances between the query and candidate sets enables the candidate sets to be sorted by distance, with the distances near top.
If two sources are included, the classification task refers to training a classification model by combining all the features (corresponding features and specific features) of the source 1 and the source 2, and executing the classification task. The classification model may be any classification model and the classification tasks may be any classification tasks related to sources 1 and 2. For example, assuming the classification task of the product as an example, the source 1 is a product picture, the source 2 is a product title, and the classification task is a product category. Then the method of the invention can extract features from both sources and then perform the classification task on all features using any classification model.
The functional description of the multi-source heterogeneous data feature extraction method provided by the embodiment of the invention refers to the description of the multi-source heterogeneous data feature extraction device in the embodiment in detail.
An embodiment of the present invention further provides a storage medium, as shown in fig. 6, on which a computer program 601 is stored, where the instructions, when executed by a processor, implement the steps of the multi-source heterogeneous data feature extraction method in the foregoing embodiment. The storage medium is also stored with audio and video stream data, characteristic frame data, an interactive request signaling, encrypted data, preset data size and the like. The storage medium may be a magnetic Disk, an optical Disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a Flash Memory (Flash Memory), a Hard Disk (Hard Disk Drive, abbreviated as HDD) or a Solid State Drive (SSD), etc.; the storage medium may also comprise a combination of memories of the kind described above.
It will be understood by those skilled in the art that all or part of the processes of the methods of the embodiments described above can be implemented by a computer program, which can be stored in a computer-readable storage medium, and when executed, can include the processes of the embodiments of the methods described above. The storage medium may be a magnetic Disk, an optical Disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a Flash Memory (Flash Memory), a Hard Disk (Hard Disk Drive, abbreviated as HDD), a Solid State Drive (SSD), or the like; the storage medium may also comprise a combination of memories of the kind described above.
An embodiment of the present invention further provides an electronic device, as shown in fig. 7, the electronic device may include a processor 51 and a memory 52, where the processor 51 and the memory 52 may be connected by a bus or in another manner, and fig. 7 takes the connection by the bus as an example.
The processor 51 may be a Central Processing Unit (CPU). The Processor 51 may also be other general purpose processors, Digital Signal Processors (DSPs), Application Specific Integrated Circuits (ASICs), Field Programmable Gate Arrays (FPGAs) or other Programmable logic devices, discrete Gate or transistor logic devices, discrete hardware components, or combinations thereof.
The memory 52, which is a non-transitory computer readable storage medium, may be used to store non-transitory software programs, non-transitory computer executable programs, and modules, such as the corresponding program instructions/modules in the embodiments of the present invention. The processor 51 executes various functional applications and data processing of the processor by running non-transitory software programs, instructions and modules stored in the memory 52, that is, implements the multi-source heterogeneous data feature extraction method in the above method embodiment.
The memory 52 may include a storage program area and a storage data area, wherein the storage program area may store an operating device, an application program required for at least one function; the storage data area may store data created by the processor 51, and the like. Further, the memory 52 may include high speed random access memory, and may also include non-transitory memory, such as at least one magnetic disk storage device, flash memory device, or other non-transitory solid state storage device. In some embodiments, the memory 52 may optionally include memory located remotely from the processor 51, and these remote memories may be connected to the processor 51 via a network. Examples of such networks include, but are not limited to, the internet, intranets, local area networks, mobile communication networks, and combinations thereof.
The one or more modules are stored in the memory 52 and when executed by the processor 51, perform the multi-source heterogeneous data feature extraction method in the embodiment shown in fig. 4-5.
The details of the electronic device may be understood by referring to the corresponding descriptions and effects in the embodiments shown in fig. 4 to fig. 5, which are not described herein again.
Although the embodiments of the present invention have been described in conjunction with the accompanying drawings, those skilled in the art may make various modifications and variations without departing from the spirit and scope of the invention, and such modifications and variations fall within the scope defined by the appended claims.
Claims (10)
1. A multi-source heterogeneous data feature extraction device is characterized by comprising: the number of the corresponding specific self-encoders is the same as that of the data sources;
the corresponding specific self-encoder is used for extracting high-level features of each source data based on the low-level features of each source data, and the high-level features comprise specific features of each source data and corresponding features of a plurality of source data;
the corresponding specific self-encoder comprises a specific part encoder, a corresponding part encoder, a prediction decoder and a reconstruction decoder, wherein the specific part encoder converts the low-level features of the corresponding source data into specific features, the corresponding part encoder converts the low-level features of the corresponding source data into corresponding features, the prediction encoder predicts the low-level features of other source data through the corresponding features, and the reconstruction decoder reconstructs the low-level features of the corresponding source data through the corresponding features and the specific features.
2. The multi-source heterogeneous data feature extraction apparatus according to claim 1, wherein the corresponding specific self-encoder comprises: the method comprises the steps of inputting a layer, a corresponding characteristic layer, a specific characteristic layer, a prediction output layer and a reconstruction output layer, wherein each corresponding specific characteristic layer of a specific self-encoder has a correlation constraint.
3. The apparatus according to claim 2, wherein the configuration parameters corresponding to the specific self-encoder comprise: the connection weight and the offset from the input layer to the corresponding characteristic layer, the connection weight and the offset from the input layer to the specific characteristic layer, the connection weight and the offset from the corresponding characteristic layer to the prediction output layer, the connection weight and the offset from the specific characteristic layer to the reconstruction output layer, and the connection weight from the corresponding characteristic layer to the prediction output layer in each corresponding specific self-encoder are consistent.
4. The multi-source heterogeneous data feature extraction apparatus according to claim 1,
the corresponding specific self-encoder is obtained by adopting gradient descent algorithm training;
and the configuration parameters of the corresponding specific self-encoder are finely adjusted by adopting a gradient descent algorithm.
5. The apparatus of claim 3, wherein the penalty function for a particular respective encoder is determined based on a two-norm mapping between each of the source input layer features, the predicted output layer features, and the reconstructed output layer features, and a two-norm mapping between respective portions of the encoders.
6. The apparatus according to claim 1, wherein the corresponding specific self-encoder further comprises: gates of a predictive decoder, the reconstruction decoder and the gates of the predictive decoder reconstructing low-level features of the respective source data from the corresponding features and the particular features.
7. A multi-source heterogeneous data feature extraction method applied to the multi-source heterogeneous data feature extraction device of any one of claims 1 to 6, the method comprising:
extracting low-level features of the source data by using a feature extraction algorithm;
based on the low-level features of each source data, the high-level features of each source data are extracted using a corresponding specific self-encoder.
8. The multi-source heterogeneous data feature extraction method according to claim 7, further comprising:
calculating the distance of different source data according to the corresponding characteristics in the high-level characteristics of each source data;
searching the matched object according to the distance, and executing a multi-source heterogeneous data searching task;
and training a multi-source data fusion classification model according to specific characteristics in the high-level characteristics of each source data and corresponding characteristics of the source data, and executing a multi-source heterogeneous data classification task.
9. A computer-readable storage medium storing computer instructions for causing a computer to perform the multi-source heterogeneous data feature extraction method according to claim 7 or 8.
10. An electronic device, comprising: a memory and a processor, the memory and the processor being communicatively connected to each other, the memory storing computer instructions, and the processor executing the computer instructions to perform the multi-source heterogeneous data feature extraction method according to claim 7 or 8.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210291613.XA CN114626480A (en) | 2022-03-22 | 2022-03-22 | Multi-source heterogeneous data feature extraction device and method, storage medium and electronic equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210291613.XA CN114626480A (en) | 2022-03-22 | 2022-03-22 | Multi-source heterogeneous data feature extraction device and method, storage medium and electronic equipment |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN114626480A true CN114626480A (en) | 2022-06-14 |
Family
ID=81904692
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210291613.XA Pending CN114626480A (en) | 2022-03-22 | 2022-03-22 | Multi-source heterogeneous data feature extraction device and method, storage medium and electronic equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114626480A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116502092A (en) * | 2023-06-26 | 2023-07-28 | 国网智能电网研究院有限公司 | Semantic alignment method, device, equipment and storage medium for multi-source heterogeneous data |
-
2022
- 2022-03-22 CN CN202210291613.XA patent/CN114626480A/en active Pending
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116502092A (en) * | 2023-06-26 | 2023-07-28 | 国网智能电网研究院有限公司 | Semantic alignment method, device, equipment and storage medium for multi-source heterogeneous data |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111930992B (en) | Neural network training method and device and electronic equipment | |
| CN108986186B (en) | Method and system for converting text into video | |
| CN112131350B (en) | Text label determining method, device, terminal and readable storage medium | |
| WO2021139191A1 (en) | Method for data labeling and apparatus for data labeling | |
| CN113762322A (en) | Video classification method, device and equipment based on multi-modal representation and storage medium | |
| CN112989212B (en) | Media content recommendation method, device and equipment and computer storage medium | |
| CN114359775A (en) | Key frame detection method, device, equipment, storage medium and program product | |
| CN115114395A (en) | Content retrieval and model training method and device, electronic equipment and storage medium | |
| CN114329051B (en) | Data information identification method, device, apparatus, storage medium and program product | |
| CN116304745A (en) | Text topic matching method and system based on deep semantic information | |
| CN115238126A (en) | Method, device and equipment for reordering search results and computer storage medium | |
| CN114661951A (en) | Video processing method and device, computer equipment and storage medium | |
| CN114626480A (en) | Multi-source heterogeneous data feature extraction device and method, storage medium and electronic equipment | |
| CN117437317A (en) | Image generation method, apparatus, electronic device, storage medium, and program product | |
| CN117217368A (en) | Training method, device, equipment, medium and program product of prediction model | |
| CN116977701A (en) | Video classification model training method, video classification method and device | |
| WO2023155546A1 (en) | Structure data generation method and apparatus, device, medium, and program product | |
| CN116304104A (en) | Knowledge graph construction method, knowledge graph construction device, medium and electronic equipment | |
| CN116663523A (en) | Semantic text similarity calculation method for multi-angle enhanced network | |
| CN117011745A (en) | Data processing method, device, computer equipment and readable storage medium | |
| CN112861474B (en) | Information labeling method, device, equipment and computer readable storage medium | |
| CN113239184B (en) | Knowledge base acquisition method and device, computer equipment and storage medium | |
| CN115186085A (en) | Reply content processing method and interaction method of media content interaction content | |
| CN116756676A (en) | Abstract generation method and related device | |
| CN115878839A (en) | Video recommendation method and device, computer equipment and computer program product |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |