WO2024001653A1 - 特征提取方法、装置、存储介质及电子设备 - Google Patents

特征提取方法、装置、存储介质及电子设备 Download PDF

Info

Publication number
WO2024001653A1
WO2024001653A1 PCT/CN2023/097373 CN2023097373W WO2024001653A1 WO 2024001653 A1 WO2024001653 A1 WO 2024001653A1 CN 2023097373 W CN2023097373 W CN 2023097373W WO 2024001653 A1 WO2024001653 A1 WO 2024001653A1
Authority
WO
WIPO (PCT)
Prior art keywords
data
network
feature extraction
local
feature
Prior art date
Application number
PCT/CN2023/097373
Other languages
English (en)
French (fr)
Other versions
WO2024001653A9 (zh
Inventor
魏书琪
哈谦
耿凯
Original Assignee
京东方科技集团股份有限公司
北京京东方技术开发有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 京东方科技集团股份有限公司, 北京京东方技术开发有限公司 filed Critical 京东方科技集团股份有限公司
Publication of WO2024001653A1 publication Critical patent/WO2024001653A1/zh
Publication of WO2024001653A9 publication Critical patent/WO2024001653A9/zh

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/77—Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
    • G06V10/7715—Feature extraction, e.g. by transforming the feature space, e.g. multi-dimensional scaling [MDS]; Mappings, e.g. subspace methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/40—Extraction of image or video features
    • G06V10/44—Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components
    • G06V10/443—Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components by matching or filtering
    • G06V10/449—Biologically inspired filters, e.g. difference of Gaussians [DoG] or Gabor filters
    • G06V10/451—Biologically inspired filters, e.g. difference of Gaussians [DoG] or Gabor filters with interaction between the filter responses, e.g. cortical complex cells
    • G06V10/454—Integrating the filters into a hierarchical structure, e.g. convolutional neural networks [CNN]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/77—Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
    • G06V10/80—Fusion, i.e. combining data from various sources at the sensor level, preprocessing level, feature extraction level or classification level
    • G06V10/806—Fusion, i.e. combining data from various sources at the sensor level, preprocessing level, feature extraction level or classification level of extracted features
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks

Definitions

  • the present disclosure relates to the field of artificial intelligence technology, and specifically, to a feature extraction method, a feature extraction device, a computer-readable storage medium, and an electronic device.
  • neural networks have been applied to various fields of artificial intelligence, such as image recognition, driverless cars and other fields.
  • image features extracted by neural networks can be used to complete specific tasks in subsequent neural networks, such as face recognition, image segmentation, etc.
  • face recognition image segmentation
  • image feature extraction has always been a technical issue that has attracted much attention.
  • the present disclosure provides a feature extraction method, feature extraction device, computer-readable storage medium and electronic equipment.
  • the present disclosure provides a feature extraction method, including:
  • the preset neural network is used to perform fusion feature extraction on the input data to obtain the target features of the data to be identified.
  • the preset neural network at least includes an encoder network and a decoder network.
  • the encoder network includes a plurality of downsampling layers, and each downsampling layer includes at least a plurality of convolutions.
  • a product layer and a pooling layer the decoder network includes multiple upsampling layers, and each upsampling layer includes at least one deconvolution layer and multiple convolutional layers.
  • the preset neural network also includes a local feature extraction network, and the network structure of the local feature extraction network is the same as that of the encoder network;
  • the input data of the preset neural network is obtained from the local data in the data and the data to be identified, including:
  • the local data of the data to be recognized is used as the input data of the local feature extraction network.
  • using the preset neural network to perform fusion feature extraction on the input data to obtain the target features of the data to be identified includes:
  • the first output feature of the encoder network is obtained;
  • the decoder network is used to perform feature extraction on the first input feature to obtain the target feature of the data to be identified.
  • the method before using the preset neural network to perform fusion feature extraction on the input data, the method further includes:
  • the encoder network and the local feature extraction network in the preset neural network are trained; wherein the parameter initial values of the encoder network are the same as the parameter initial values of the local feature extraction network.
  • training the encoder network and the local feature extraction network in the preset neural network includes:
  • the parameter mapping relationship is used to determine the parameters of the iterated encoder network and the corresponding parameters of the local feature extraction network.
  • obtaining input data of a preset neural network based on the data to be recognized and the local data in the data to be recognized includes:
  • the combined data is used as the input data of the encoder network.
  • combining the data to be identified and corresponding elements in the local data matrix to obtain combined data includes:
  • the element value at the corresponding spatial position represent the image data X and the local data matrix Y respectively. Row, No.
  • the preset neural network is used to fuse the input data.
  • Combined feature extraction is used to obtain the target features of the data to be identified, including:
  • the encoder network is used to perform feature extraction on the combined data to obtain the second output feature of the encoder network.
  • the first downsampling layer of the encoder network includes multiple atrous convolution layers and a pooling layer. layer;
  • the decoder network is used to perform feature extraction on the second input feature to obtain the target feature of the data to be identified.
  • the preset neural network also includes a local feature fusion network, and the local feature fusion network includes a plurality of atrous convolution layers and a pooling layer; the use of the preset neural network Assume that the neural network performs fusion feature extraction on the input data to obtain the target features of the data to be identified, including:
  • the encoder network and decoder network in the preset neural network are used to perform feature extraction on the fused data to obtain the target features of the data to be identified.
  • the preset neural network further includes a classifier; after obtaining the target features of the data to be identified, the method further includes:
  • the target features of the data to be identified are classified and predicted by the classifier to obtain a classification result of the data to be identified.
  • the present disclosure provides a feature extraction device, including:
  • An input data generation module configured to obtain the input data of the preset neural network based on the data to be recognized and the local data in the data to be recognized;
  • a target feature extraction module is configured to use the preset neural network to perform fusion feature extraction on the input data to obtain the target features of the data to be identified.
  • the present disclosure provides a computer-readable storage medium on which a computer program is stored.
  • a computer program is stored on which a computer program is stored.
  • the computer program is executed by a processor, any one of the methods described above is implemented.
  • the present disclosure provides an electronic device, including: a processor; and a memory for storing executable instructions of the processor; wherein the processor is configured to perform any of the above by executing the executable instructions. method described.
  • Figure 1 shows a schematic diagram of an exemplary system architecture in which a feature extraction method and device according to embodiments of the present disclosure can be applied;
  • Figure 2 schematically shows a schematic diagram of image segmentation according to an embodiment of the present disclosure
  • Figure 3 schematically shows a flow chart of a feature extraction method according to an embodiment of the present disclosure
  • Figure 4 schematically shows a schematic diagram of the improved network structure of the U-Net network according to one embodiment of the present disclosure
  • Figure 5 schematically shows a schematic diagram of the improved network structure of the U-Net network according to another embodiment of the present disclosure
  • Figure 6 schematically shows a structural diagram of a dilated convolution layer according to an embodiment of the present disclosure
  • Figure 7 schematically shows a schematic diagram of convolution operation between matrices according to an embodiment of the present disclosure
  • Figure 8 schematically shows a flow chart of fusion feature extraction according to one embodiment of the present disclosure
  • Figure 9 schematically shows a flow chart of fusion feature extraction according to another embodiment of the present disclosure.
  • Figure 10 schematically shows a schematic diagram of the improved network structure of the U-Net network according to yet another embodiment of the present disclosure
  • Figure 11 schematically shows a flow chart of fusion feature extraction according to yet another embodiment of the present disclosure
  • Figure 12 schematically shows a block diagram of a feature extraction device according to an embodiment of the present disclosure
  • FIG. 13 schematically shows a structural diagram of a computer system suitable for implementing an electronic device according to an embodiment of the present disclosure.
  • Example embodiments will now be described more fully with reference to the accompanying drawings.
  • Example embodiments may, however, be embodied in various forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the concepts of the example embodiments.
  • the described features, structures or characteristics may be combined in any suitable manner in one or more embodiments.
  • numerous specific details are provided to provide a thorough understanding of embodiments of the disclosure.
  • those skilled in the art will appreciate that the technical solutions of the present disclosure may be practiced without one or more of the specific details described, or other methods, components, devices, steps, etc. may be adopted.
  • well-known technical solutions have not been shown or described in detail to avoid obscuring aspects of the disclosure.
  • FIG. 1 shows a schematic diagram of the system architecture of an exemplary application environment in which a feature extraction method and device according to embodiments of the present disclosure can be applied.
  • the system architecture 100 may include one or more of terminal devices 101, 102, 103, a network 104 and a server 105.
  • Network 104 is used to provide communication links between terminal devices 101, 102, 103 and server 105 The medium of the road.
  • Network 104 may include various connection types, such as wired, wireless communication links, or fiber optic cables, among others.
  • the terminal devices 101, 102, and 103 may be various electronic devices, including but not limited to desktop computers, portable computers, smart phones, tablet computers, etc. It should be understood that the number of terminal devices, networks and servers in Figure 1 is only illustrative. Depending on implementation needs, there can be any number of end devices, networks, and servers.
  • the server 105 may be one server, a server cluster composed of multiple servers, or a cloud computing platform or a virtualization center. Specifically, the server 105 may be used to: obtain the input data of a preset neural network based on the data to be recognized and the local data in the data to be recognized; and use the preset neural network to perform fusion feature extraction on the input data, Obtain the target characteristics of the data to be identified.
  • the feature extraction method provided by the embodiment of the present disclosure is generally executed by the server 105. Accordingly, the feature extraction device is generally provided on the server 105.
  • the server 105 can send the target features of the data to be recognized output by the preset neural network model to the terminal device, and display them to the user through the terminal device.
  • the feature extraction method provided by the embodiments of the present disclosure can also be executed by one or more of the terminal devices 101, 102, and 103.
  • the feature extraction device can also be provided on the terminal device. 101, 102, 103.
  • the extracted target features of the data to be identified can be directly displayed on the display screen of the terminal device, which is not particularly limited in this exemplary embodiment.
  • Figure (A) is an image to be recognized, and the animals and plants in the image need to be identified.
  • the image can be set as three-dimensional data and used as input data of the neural network to obtain output data of the same dimension.
  • the input data can be: the number of image rows (h) * the number of image columns (w) * the number of image channels (channels)
  • the output data can be: the number of image rows (h) * the number of image columns (w) * the number of image categories (classes).
  • Figures (B) and (C) are schematic diagrams of the classification results under accurate identification.
  • Figure (B) is the leopard classification result
  • figure (C) is the tree classification result.
  • the neural network easily recognizes the leopard ears in Figure (A) as the background, thus reducing the accuracy of the neural network recognition.
  • this example implementation provides a feature extraction method, which can be applied to the above-mentioned server 105 or one or more of the above-mentioned terminal devices 101, 102, 103.
  • This example There is no special limitation on this in the specific embodiment. Referring to Figure 3, the feature extraction method may include step S310 and step S320:
  • Step S310 Obtain the input data of the preset neural network according to the data to be recognized and the local data in the data to be recognized;
  • Step S320 Use the preset neural network to perform fusion feature extraction on the input data to obtain the target features of the data to be identified.
  • the input data of the preset neural network is obtained according to the data to be recognized and the local data in the data to be recognized; and the input data is processed using the preset neural network. Fusion feature extraction is used to obtain the target features of the data to be identified.
  • This disclosure introduces local data into the neural network, And the neural network is used to fuse the global data and the local data to extract features, and the enhanced global features can be obtained, which improves the feature accuracy of the neural network recognition, thereby improving the accuracy of the neural network recognition.
  • step S310 the input data of the preset neural network is obtained according to the data to be recognized and the local data in the data to be recognized.
  • the data to be recognized is image data
  • the local data in the data to be recognized is part of the data contained in the image data, and this part of the data has features with a high degree of differentiation.
  • the classification result corresponding to the local data in the data to be identified can be determined based on prior knowledge.
  • prior knowledge can refer to the special properties of the research object. For example, it can be determined based on prior knowledge that the part of the image data corresponding to Figure 2(A) with larger pixel values belongs to a certain category, such as the tree category.
  • the data to be recognized when the data to be recognized is input into a preset neural network for feature extraction, the data to be recognized can be represented by a three-dimensional matrix, such as h*w*channels, to represent that the data to be recognized is image data.
  • the local data in the data to be recognized when used as the input data of the preset neural network, the local data in the data to be recognized can be initialized to a three-dimensional matrix of all 0s, which is recorded as the local data matrix. Its size is the same as the output The target features are of the same size, h*w*classes.
  • the elements of the classification channel corresponding to this part of the pixels can be assigned in the local data matrix, such as assigning this element to the positive category for which the part of the pixels is identified.
  • Probability value the interval of probability value can be [0, 1].
  • the corresponding element in the local data matrix can be assigned a probability value of identifying the part of the pixels as the leopard category, for example, the probability value is 0.8.
  • the preset neural network may include at least an encoder network and a decoder network.
  • the encoder network may include multiple downsampling layers. Each downsampling layer may include at least multiple convolutional layers and a pooling layer.
  • the decoder network may include It includes multiple upsampling layers, and each upsampling layer includes at least one deconvolution layer and multiple convolutional layers. It can be understood that, according to implementation needs, the number of downsampling layers and upsampling layers can be arbitrary, and the number of convolutional layers included in the downsampling layer is also arbitrary, and this disclosure does not specifically limit this.
  • the preset neural network may be a U-Net network, a Transformer network, etc., or other network models including an encoder network and a decoder network, which is not specifically limited in this disclosure.
  • the preset neural network may also include a local feature extraction network.
  • a local feature extraction network can be constructed based on the encoder network, where the network structure of the local feature extraction network is the same as that of the encoder network, and the data to be identified is used as the input data of the encoder network, and the data to be identified is The local data is used as the input data of the local feature extraction network.
  • the preset neural network can be an improved U-Net network for illustration.
  • the U-Net network contains a decoder network 202 and a dual-stream encoder, namely the encoder network 201 and Encoder network 203.
  • Encoder network 203 is a local feature extraction network constructed based on encoder network 201. It can be seen that the network structures of encoder network 201 and encoder network 203 are exactly the same. Among them, the encoder network 201 includes three downsampling layers (2011, 2012 and 2013), Decoder network 202 includes three upsampling layers (2022, 2023 and 2024).
  • this disclosure does not specifically limit the number of down-sampling layers and the number of up-sampling layers, but the number of down-sampling layers and the number of up-sampling layers are the same.
  • Three feature propagation layers (2031, 2032 and 2033) are established between the encoder network 201 and the decoder network 202.
  • features of the data to be identified can be extracted through three down-sampling layers, and the extracted feature maps are transmitted to the decoder network 202 through three feature propagation layers.
  • feature extraction of the data to be identified can be performed through three upsampling layers, and feature fusion of the data to be identified can also be performed.
  • the downsampling layer 2011 includes two convolutional layers and a pooling layer.
  • the convolutional layer may be a shallow convolutional layer. , the receptive field of the shallow convolutional layer is smaller, and the overlapping area of the receptive fields is also smaller, which can ensure that the neural network captures more details.
  • the convolutional layer has learnable parameters, and the parameters in the convolutional layer can be fitted by training the preset neural network.
  • the pooling layer can be a maximum pooling layer or an average pooling layer.
  • the horizontal hollow arrow represents convolution processing by the convolution layer
  • the downward solid arrow represents the max pooling downsampling process by the max pooling layer, or represents the average pooling downsampling by the average pooling layer. Sampling processing.
  • the upsampling layer 2022 is taken as an example.
  • the upsampling layer 2022 includes a deconvolution layer and two convolution layers. Both the convolution layer and the deconvolution layer can have The learned parameters can also be fitted by training the preset neural network.
  • horizontal hollow arrows indicate convolution processing by the convolution layer
  • upward solid arrows indicate deconvolution upsampling processing by the deconvolution layer.
  • the data to be recognized can be used as the input data of the encoder network 201 to obtain the global characteristics of the data to be recognized.
  • the local data of the data to be recognized can be used as input data of the local feature extraction network (encoder network 203) to obtain the local features of the data to be recognized, and the local features are propagated to the encoder network 201 to extract the global features of the data to be recognized. Spliced with the channel dimensions of local features, the feature map 2014 shown in Figure 4 is obtained. Perform a convolution operation on the spliced feature map 2014 to obtain the input of the decoder network 202, that is, the feature map 2021 shown in Figure 4.
  • the global features and local features of the data to be recognized can also be propagated to the feature map parameters by element-by-element summation, that is, the local features of the data to be recognized are transmitted to the encoder network 201.
  • the network structure shown in Figure 4 is scalable and fault-tolerant. This network structure can be extended to all neural networks that include encoding and decoding network structures. It can extract features from the data to be identified and the local data in the data to be identified. At the same time, The local data of the data to be identified is used as the input of the neural network, so that the feature map output by the neural network can more accurately describe the data to be identified, thereby improving the feature accuracy of the neural network.
  • the newly added network branch, the encoder network 203 is used to extract features of the local data of the data to be recognized at the same time, and the extracted local features are integrated into the encoder network 201, so that the decoder network 202 can obtain the global features and local features of the data to be recognized. Features are fused to extract features, which improves the operating efficiency of the neural network.
  • the first downsampling layer of the encoder network in the preset neural network may include multiple atrous convolutional layers and a pooling layer.
  • the default neural network is still the improved U-Net network for explanation.
  • FIG. 5 a schematic diagram of the network structure of another improved U-Net network is schematically provided.
  • the U-Net network contains an encoder network 201 and a decoder network 202 . It should be noted that the code shown in Figure 5
  • the downsampling layer 2011 of the encoder network 201 consists of two atrous convolutional layers (2041 and 2042) and a pooling layer (2043).
  • Figure 6 a schematic structural diagram of the atrous convolution layer is given.
  • the atrous convolution layer can increase the receptive field of the convolution operation by injecting holes into the convolution kernel of the standard convolution, so that the convolution output contains A wider range of information.
  • the expansion rate is used to define the distance between the elements of the convolution kernel. It can be understood that this disclosure does not limit the specific values of the expansion rate and step size of the dilated convolution layer.
  • the pooling layer 2043 in the downsampling layer 2011 can be an average pooling layer or a maximum pooling layer. , this disclosure does not limit this.
  • a matrix representation of the local data in the data to be identified can be constructed to obtain a local data matrix; the data to be identified and the corresponding elements in the local data matrix can be combined to obtain The data are combined and used as input data to the encoder network 201 shown in FIG. 5 .
  • the data to be recognized when the data to be recognized is image data, the data to be recognized can be expressed in the form of a three-dimensional matrix to obtain the image data X, which is h*w*channels.
  • the prior knowledge can be digitized according to the matrix size of the image data, that is, the local data in the data to be recognized is expressed in the form of a three-dimensional matrix to obtain the local data matrix Y, such as h*w*channels, two The matrices are of the same size.
  • Each corresponding element in the image data X and the local data matrix Y can be combined according to formula (1). Specifically, it can be based on:
  • the combined data Z is obtained.
  • the matrix size of the combined data Z is (2h)*(2w)*channels.
  • Z(i,j,k) represents the element value at the spatial position corresponding to the i-th row, j-th column, and k-th channel in the combined data Z, represent the image data X and the local data matrix Y respectively.
  • the element value at the spatial position corresponding to the column and k-th channel Represents the upward rounding operation, is the combination coefficient.
  • the element value at the spatial position and the combined data Z obtained after assignment are shown in Figure 7.
  • the network structure shown in Figure 5 does not add a new network branch. Instead, the shallow convolution layer in the original encoder network is replaced by a dilated convolution layer. Combined with the pooling layer, the data to be identified and the local data to be identified can be realized. Parallel fusion of data. Compared with the network structure shown in Figure 4, it can not only improve the training efficiency and operating efficiency of the neural network, but also keep the neural network lightweight.
  • step S320 the preset neural network is used to perform fusion feature extraction on the input data to obtain the target features of the data to be identified.
  • the data to be recognized can be used as the input data of the encoder network 201, and the local data of the data to be recognized can be used as the encoder network.
  • the encoder network 201 is used to extract features of the data to be identified to obtain the global features of the data to be identified
  • the encoder network 203 is used to extract features of the local data of the data to be identified to obtain the local features of the data to be identified.
  • the decoder network 202 is used to perform fusion feature extraction on the global features and local features of the data to be identified, to obtain the target features of the data to be identified.
  • the input data can be fused and feature extracted through the network structure of the U-Net network shown in FIG. 4 according to steps S810 to S840 to obtain the target features of the data to be identified.
  • Step S810 Use the encoder network to perform feature extraction on the data to be identified, and obtain the first output feature of the encoder network.
  • the image data can be input to the downsampling layer 2011 of the encoder network 201, and the input image data can be convolved twice through the two convolutional layers in the downsampling layer 2011. product processing to obtain a first feature map, and transmit the first feature map to the upsampling layer 2024 in the decoder network 202 through the feature propagation layer 2031.
  • the first feature map can be down-sampled through the pooling layer in the down-sampling layer 2011.
  • the first feature map can be down-sampled by max pooling. Then, the down-sampled first feature map is input into the down-sampling layer 2012.
  • the processing performed by the down-sampling layer 2012 and the down-sampling layer 2013 on the input feature map is the same as the processing performed by the down-sampling layer 2011 on the image data.
  • the processing methods are the same and will not be repeated here.
  • the second feature map can be obtained through the downsampling layer 2012, and the second feature map can be transmitted to the upsampling layer 2023 in the decoder network 202 through the feature propagation layer 2032.
  • the third feature map can be obtained through the downsampling layer 2013, and the third feature map is transmitted to the upsampling layer 2022 in the decoder network 202 through the feature propagation layer 2033.
  • the third feature map is the third feature map of the encoder network 201. an output feature.
  • the encoder network layer deepens, the feature extraction of image data gradually abstracts from local description to global description, which can describe image data more accurately, thus helping to ensure the accuracy of image segmentation.
  • Step S820 Use the local feature extraction network to perform feature extraction on the local data in the data to be identified, and obtain a first feature representation of the local data.
  • the local data in the data to be recognized is the partial data in the image data.
  • the partial data can be represented using a local data matrix.
  • the local feature extraction network is the encoder network 203, and its network structure is the same as the encoder network 201.
  • the encoder network 203 can be used to perform feature extraction on the local data matrix to obtain the first feature representation of the local data matrix.
  • step S810 For the specific feature extraction process, refer to step S810, which will not be described in detail here.
  • Step S830 Splice the first output feature of the encoder network and the first feature representation of the local data to obtain the first input feature of the decoder network.
  • the first feature representation of the local data can be spliced with the first output feature of the encoder network.
  • the channel dimensions of the two can be spliced, or the two can be spliced by element-wise summation to achieve feature spread.
  • a convolution operation can be performed on the feature map 2014 to obtain the feature map 2021 as shown in Figure 4 , and use the feature map 2021 as the first input feature of the decoder network 202 to use the decoder network 202 to perform feature fusion on the first input feature.
  • Step S840 Use the decoder network to perform feature extraction on the first input feature to obtain the target feature of the data to be identified.
  • the feature map 2021 can be input into the upsampling layer 2022 of the decoder network 202 as the first feature map to be fused, and the feature map 2021 can be deconvolved and upsampled through the deconvolution layer in the upsampling layer 2022, that is, The first output feature of the encoder network contained in the feature map 2021 and the first feature representation of the local data are fused to obtain a first fused feature map.
  • the first fused feature map is spliced with the third feature map transmitted through the feature propagation layer, and the feature map after splicing the first fused feature map and the third feature map is passed through the two convolutional layers in the upsampling layer 2022 Perform two convolution processes in sequence to obtain the second feature map to be fused.
  • the second feature map to be fused is input into the upsampling layer 2023, and the second feature map to be fused is deconvolved and upsampled through the deconvolution layer in the upsampling layer 2023.
  • the upsampling layer 2024 processes the input feature map to be fused in the same manner as the upsampling layer 2022 processes the feature map 2021, which will not be described in detail here.
  • the third feature map to be fused can be obtained through the upsampling layer 2023, and the third feature map to be fused is input into the upsampling layer 2024, and deconvolution upsampling and two convolutions are performed sequentially.
  • the upsampling layer 2024 The output data is the target feature of the image data.
  • the local data of the data to be identified is used as part of the input data of the preset neural network containing a dual-stream encoder.
  • the global features and local features of the data to be identified are extracted through the dual-stream encoder and passed through the preset neural network.
  • the decoder network fuses the global features and local features of the data to be recognized, the feature enhancement of the global features can be achieved, which improves the feature accuracy of the preset neural network recognition.
  • the accuracy of the preset neural network recognition can be improved.
  • the network structure Before fusion feature extraction is performed on the input data through the network structure of the U-Net network shown in Figure 4, the network structure can be trained. For example, a data set composed of input data and output data can be used to fit the parameters. After parameter fitting is completed, feature extraction, image segmentation, etc. can be performed through the network structure.
  • the U-Net network includes three parts: an encoder network (201), a local feature extraction network (ie, the encoder network 203) and a decoder network (202).
  • an encoder network 201
  • a local feature extraction network ie, the encoder network 203
  • a decoder network 202
  • This section mainly explains the training process of the encoder network and local feature extraction network. It can be understood that while training the encoder network and local feature extraction network, the parameters of the decoder network are also continuously iterated to obtain better parameters by fitting.
  • a parameter interconnection mechanism can be added to the training process of this part of the network structure, that is, a mapping relationship between the parameters of the dual-stream encoder can be established to improve the Training efficiency of network structure.
  • the initial values of parameters of the encoder network are the same as those of the local feature extraction network.
  • the parameter set of the encoder network 201 is surjective to the parameter set of the local feature extraction network 203, that is, each parameter in the local feature extraction network 203 has a corresponding parameter in the encoder network 201.
  • X represents the parameter set of the encoder network 201
  • Y represents the parameter set of the local feature extraction network 203
  • the parameter mapping relationship can be used to determine the parameters of the iterated encoder network and the corresponding parameters of the local feature extraction network.
  • the parameter mapping relationship between the encoder network 201 and the local feature extraction network 203 can be preset as:
  • X 1 (i) represents the i-th parameter of the encoder network 201 after the current iteration
  • Y 1 (i) represents the i-th parameter of the local feature extraction network 203 after the current iteration
  • the i-th parameter of the encoder network 201 for the next iteration Y 1 )i) represents the i-th parameter of the local feature extraction network 203 for the next iteration.
  • the initial values of the parameters of the encoder network are the same as those of the local feature extraction network.
  • the parameter mapping relationship can be used to make the parameters of the encoder network used for each iteration and the parameters of the corresponding local feature extraction network the same.
  • the average value of X 1 (i) and Y 1 (i) can be used as the i-th parameter of the encoder network 201 and the local feature extraction network 203 of the next iteration.
  • the initial values of the parameters of the encoder network need to be preset to be the same as the initial values of the parameters of the local feature extraction network. Use the same initial parameter values for the encoder network and the local feature extraction network.
  • the parameters will continue to change during training.
  • the encoder The network and the local feature extraction network can maintain a certain mapping relationship during the parameter fitting process, which facilitates fitting to obtain better parameters.
  • the parameters of the encoder network and the parameters of the local feature extraction network can be iterated according to a preset parameter mapping relationship.
  • the iteration termination condition is met, the training of the encoder network and the local feature extraction network is completed.
  • an objective function can be constructed based on the input data and output data.
  • the stochastic gradient descent algorithm can be used to iteratively update the parameters of the encoder network and the parameters of the local feature extraction network.
  • the iteration termination conditions are met, the algorithm is completed. Training of encoder network and local feature extraction network.
  • the iteration termination condition can be that the training of all parameters is completed when the objective function converges, or the parameters can be updated through reverse iteration.
  • the preset number of iterations is met, the training of all parameters is completed.
  • the parameter interconnection mechanism can realize feature sharing between various network structures and facilitate fitting to obtain better parameters.
  • the parameters of the encoder network and the parameters of the local feature extraction network have a certain parameter mapping relationship.
  • the parameters of the encoder network can be calculated from the parameters of the local feature extraction network.
  • the preset neural network is the network structure of the U-Net network shown in Figure 5
  • a matrix representation of the local data in the data to be identified can be constructed to obtain a local data matrix, and the data to be identified and the local data matrix
  • Each corresponding element in is combined to obtain combined data, and the combined data is used as the input data of the encoder network 201 shown in FIG. 5 .
  • the encoder network 201 and the decoder network 202 are used to perform fusion feature extraction on the combined data to obtain the target features of the data to be identified.
  • the input data can be fused and feature extracted through the network structure of the U-Net network shown in FIG. 5 according to steps S910 to S930 to obtain the target features of the data to be identified.
  • Step S910 Use the encoder network to perform feature extraction on the combined data to obtain the second output feature of the encoder network.
  • the first downsampling layer of the encoder network includes multiple atrous convolution layers and A pooling layer.
  • the data to be recognized is image data
  • the local data in the data to be recognized is partial data in the image data.
  • the combined data is obtained from the image data and the local data matrix.
  • the combined data can be input into the first downsampling layer 2011 of the encoder network 201, which consists of two atrous convolutional layers (2041 and 2042) and a pooling layer (2043).
  • the input image data is convolved twice through the two atrous convolution layers in the downsampling layer 2011 to obtain a fourth feature map, and the fourth feature map is transmitted to the decoder network 202 through the feature propagation layer 2031 In the upsampling layer 2024.
  • the fourth feature map can be down-sampled through the pooling layer in the down-sampling layer 2011.
  • the fourth feature map can be down-sampled by average pooling. It should be noted that after using the dilated convolution layer to perform convolution operations on the combined data, the image data part and the local data matrix part in the combined data still maintain a relatively independent spatial distribution state until the pooling layer combines the image data and the local data. Matrix fusion.
  • the down-sampled fourth feature map is input into the down-sampling layer 2012.
  • the processing performed by the down-sampling layer 2012 and the down-sampling layer 2013 on the input feature map is the same as the processing performed by the down-sampling layer 2011 on the image data.
  • the processing methods are the same and will not be repeated here.
  • the fifth feature map can be obtained through the downsampling layer 2012, and the fifth feature map can be transmitted to the upsampling layer 2023 in the decoder network 202 through the feature propagation layer 2032.
  • the sixth feature map can be obtained through the downsampling layer 2013, and the third feature map is transmitted to the upsampling layer 2022 in the decoder network 202 through the feature propagation layer 2033.
  • the sixth feature map is the third feature map of the encoder network 201. 2. Output features.
  • Step S920 Perform a convolution operation on the second output feature of the encoder network to obtain the second input feature of the decoder network.
  • the feature map 2015 shown in Figure 5 is the second output feature of the encoder network 201.
  • a convolution operation can be performed on the feature map 2015 to obtain the feature map 2020 shown in Figure 5, and the feature map 2020 is used as the decoder network 202
  • the second input feature is used to perform feature fusion on the second input feature using the decoder network 202.
  • Step S930 Use the decoder network to perform feature extraction on the second input feature to obtain the target feature of the data to be identified.
  • the feature map 2020 can be input into the upsampling layer 2022 of the decoder network 202 as the fourth feature map to be fused, and the feature map 2020 can be deconvolved and upsampled through the deconvolution layer in the upsampling layer 2022, that is, the features
  • the image data contained in graph 2020 and the local data matrix are fused to obtain a fourth fusion feature map.
  • the fourth fused feature map is spliced with the third feature map transmitted through the feature propagation layer, and the feature map after splicing the fourth fused feature map and the third feature map is passed through the two convolution layers in the upsampling layer 2022 Perform two convolution processes in sequence to obtain the fifth feature map to be fused.
  • the fifth feature map to be fused is input into the upsampling layer 2023, and the fifth feature map to be fused is deconvolved and upsampled through the deconvolution layer in the upsampling layer 2023.
  • the upsampling layer 2023 processes the input feature map to be fused in the same manner as the upsampling layer 2022 processes the feature map 2020, and will not be described in detail here.
  • the sixth feature map to be fused can be obtained through the upsampling layer 2023, and the sixth feature map to be fused is input into the upsampling layer 2024, and deconvolution upsampling and two convolutions are performed sequentially.
  • the upsampling layer 2024 The output data is the target feature of the image data.
  • the data to be recognized and the local data of the data to be recognized are input into the encoder network of the preset neural network.
  • the downsampling layer of the encoder network that contains two atrous convolution layers and a pooling layer, we can
  • the data to be identified and the local data of the data to be identified can also be fused through the decoder network, thereby realizing the parallel fusion of the local data of the data to be identified and improving the overall situation.
  • the features are further enhanced, improving the feature accuracy of preset neural network recognition.
  • the preset neural network may include an encoder network and a decoder network, and may also include a local feature fusion network.
  • the local feature fusion network includes multiple dilated convolutional layers and a pooling layer.
  • the local feature fusion network may include the first downsampling layer 2011 of the encoder network 201 as shown in Figure 5, where the downsampling layer 2011 consists of two atrous convolutional layers (2041 and 2042) and a pooling layer.
  • the pooling layer 2043 may be an average pooling layer or a maximum pooling layer, and the present disclosure does not limit this.
  • the local feature fusion network can be connected in series with the encoder network and decoder network to fuse the local data of the data to be recognized into the preset neural network.
  • a matrix representation of the local data in the data to be identified can be constructed to obtain a local data matrix, and the data to be identified and corresponding elements in the local data matrix can be combined to obtain combined data.
  • the combined data can be used as the input data of the local feature fusion network, that is, the downsampling layer 2011.
  • the pooling layer 2043 in the downsampling layer 2011 can fuse the data part to be identified and the local data part in the combined data to obtain fused data. Then, the fused data is passed through the encoder network and decoder network for feature extraction and further feature fusion to obtain the output data.
  • the input data can be fused and feature extracted through the network structure of the preset neural network shown in FIG. 10 according to steps S1110 to S1120 to obtain the target features of the data to be identified.
  • Step S1110 Input the combined data into the local feature fusion network to obtain fused data.
  • the data to be recognized is image data
  • the local data in the data to be recognized is partial data in the image data.
  • the combined data is obtained from the image data and the local data matrix.
  • the local feature fusion network is the downsampling layer 2011 shown in Figure 10.
  • the combined data can be input into the downsampling layer 2011, and the input image data is processed through two atrous convolution layers in the downsampling layer 2011. Secondary convolution processing, the fourth feature map is obtained.
  • the fourth feature map may be down-sampled through the pooling layer in the down-sampling layer 2011.
  • the fourth feature map may be subjected to average pooling down-sampling to obtain fusion data.
  • Step S1120 Use the encoder network and decoder network in the preset neural network to perform feature fusion on the fused data to obtain the target features of the data to be identified.
  • the fused data is used as the input data of the encoder network, feature extraction and further feature fusion are performed on the fused data through the encoder network and decoder network, and finally the target features of the image data are obtained.
  • the encoder network and the decoder network as the encoder network 201 and the decoder network 202 shown in Figure 4 as an example, after using the fused data as the input data of the encoder network 201, the encoder network 201 and the decoder network 202 pair fusion
  • the feature extraction process and feature fusion process of the data please refer to step 810 and step 840, which will not be described again here.
  • the local feature fusion network serves as the input layer of the preset neural network, and the original network structure composed of the encoder network and the decoder network becomes the middle layer and output layer of the preset neural network.
  • the improved neural network the data to be identified and the local data of the data to be identified are further integrated.
  • the original calculation amount remains basically unchanged, it provides a new way for the local data of the data to be identified to be integrated into the encoding and decoding neural network. ways to improve.
  • the preset neural network may also include a classifier, and the classifier may be arranged after the decoder network.
  • the classifier can be a Softmax classifier, a sigmoid classifier, etc.
  • the classifier can be used to classify and predict the target features of the data to be identified, and the classification results of the data to be identified can be obtained.
  • the classifier can be used to calculate the probability that the pixels in the target features belong to different categories, so as to realize the classification of the pixels in the target features. Prediction to perform pixel classification, that is, image segmentation.
  • the feature extraction method of the present disclosure can be applied to a variety of scenarios that require feature extraction.
  • the present disclosure only illustrates feature extraction in an image segmentation scenario.
  • the input data of the preset neural network is obtained according to the data to be recognized and the local data in the data to be recognized; and the input data is processed using the preset neural network. Fusion feature extraction is used to obtain the target features of the data to be identified.
  • the present disclosure can obtain feature-enhanced global features, improve the feature accuracy of neural network recognition, and thereby improve the accuracy of neural network recognition. accuracy.
  • a feature extraction device is also provided.
  • the device can be used in a final end device or server.
  • the feature extraction device 1200 may include an input data generation module 1210 and a target feature extraction module 1220, where:
  • the input data generation module 1210 is used to obtain the input data of the preset neural network based on the data to be recognized and the local data in the data to be recognized;
  • the target feature extraction module 1220 is configured to use the preset neural network to perform fusion feature extraction on the input data to obtain the target features of the data to be identified.
  • the preset neural network in the feature extraction device 1200 includes at least an encoder network and a decoder network.
  • the encoder network includes multiple down-sampling layers. Each down-sampling layer At least it includes a plurality of convolutional layers and a pooling layer, the decoder network includes a plurality of upsampling layers, and each upsampling layer includes at least a deconvolution layer and a plurality of convolutional layers.
  • the preset neural network also includes a local feature extraction network, and the network structure of the local feature extraction network is the same as that of the encoder network;
  • the input data generation module 1210 includes:
  • Extraction network construction sub-module used to construct the local feature extraction network according to the encoder network, the network structure of the local feature extraction network is the same as the network structure of the encoder network;
  • the first input data generation submodule is used to use the data to be recognized as the input data of the encoder network; and to use the local data of the data to be recognized as the input data of the local feature extraction network.
  • the target feature extraction module 1220 includes:
  • a first feature extraction submodule configured to use the encoder network to perform feature extraction on the data to be identified, and obtain the first output feature of the encoder network
  • the second feature extraction submodule is used to use the local feature extraction network to perform feature extraction on the local data in the data to be identified, and obtain the first feature representation of the local data;
  • the first input feature generation sub-module is used to splice the first output feature of the encoder network and the first feature representation of the local data to obtain the first input feature of the decoder network;
  • the first target feature generation submodule is used to use the decoder network to perform feature extraction on the first input feature to obtain the target feature of the data to be identified.
  • the feature extraction device 1200 further includes:
  • a network training module used to train the encoder network and the local feature extraction network in the preset neural network; wherein the parameter initial values of the encoder network and the parameters of the local feature extraction network The initial values are the same.
  • the network training module is configured to iterate the parameters of the encoder network and the parameters of the local feature extraction network according to a preset parameter mapping relationship.
  • the iteration termination condition is met,
  • the parameter mapping relationship is used to determine the parameters of the iterated encoder network and the corresponding parameters of the local feature extraction network.
  • the input data generation module 1210 includes:
  • the combined data generation submodule is used to construct the partial data in the data to be identified based on the data to be identified. Matrix representation to obtain a local data matrix; combine the data to be identified and corresponding elements in the local data matrix to obtain combined data;
  • the second input data generation submodule is used to use the combined data as input data of the encoder network.
  • the combined data generation sub-module is configured to be based on:
  • the element value at the corresponding spatial position represent the image data X and the local data matrix Y respectively. Row, No.
  • the target feature extraction module 1220 includes:
  • the third feature extraction submodule is used to use the encoder network to perform feature extraction on the combined data to obtain the second output features of the encoder network.
  • the first downsampling layer of the encoder network includes multiple A dilated convolutional layer and a pooling layer;
  • the first input feature generation submodule is used to perform a convolution operation on the second output feature of the encoder network to obtain the second input feature of the decoder network;
  • the second target feature generation submodule is used to use the decoder network to perform feature extraction on the second input feature to obtain the target feature of the data to be identified.
  • the preset neural network also includes a local feature fusion network, which includes multiple dilated convolutional layers and a pooling layer;
  • the target feature extraction module 1220 includes:
  • the fusion data generation submodule is used to input the combined data into the local feature fusion network to obtain fusion data;
  • the third target feature generation submodule is used to use the encoder network and decoder network in the preset neural network to perform feature fusion on the fused data to obtain the target features of the data to be identified.
  • the preset neural network further includes a classifier; the feature extraction device 1200 further includes:
  • a data identification module is used to perform classification prediction on the target features of the data to be identified through the classifier, and obtain a classification result of the data to be identified.
  • Each module in the above device can be a general-purpose processor, including a central processing unit, a network processor, etc.; it can also be a digital signal processor, an application-specific integrated circuit, a field programmable gate array or other programmable logic devices, discrete gates or transistors Logic devices, discrete hardware components. Each module can also be implemented by software, firmware, etc. Each processor in the above device can be an independent processor or can be integrated together.
  • Exemplary embodiments of the present disclosure also provide a computer-readable storage medium on which a program product capable of implementing the method described above in this specification is stored.
  • various aspects of the present disclosure can also be implemented in the form of a program product, which includes program code.
  • the program product When the program product is run on an electronic device, the program code is used to cause the electronic device to execute the above-mentioned instructions in this specification.
  • the steps according to various exemplary embodiments of the present disclosure are described in the "Exemplary Methods" section.
  • the program product may take the form of a portable compact disk read-only memory (CD-ROM) and include the program code, and may be run on an electronic device, such as a personal computer.
  • CD-ROM portable compact disk read-only memory
  • the program product of the present disclosure is not limited thereto.
  • a readable storage medium may be any tangible medium containing or storing a program that may be used by or in conjunction with an instruction execution system, apparatus, or device.
  • the Program Product may take the form of one or more readable media in any combination.
  • the readable medium may be a readable signal medium or a readable storage medium.
  • the readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination thereof. More specific examples (non-exhaustive list) of readable storage media include: electrical connection with one or more conductors, portable disk, hard disk, random access memory (RAM), read only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination of the above.
  • a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave carrying readable program code therein. Such propagated data signals may take many forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above.
  • a readable signal medium may also be any readable medium other than a readable storage medium that can send, propagate, or transport the program for use by or in connection with an instruction execution system, apparatus, or device.
  • Program code embodied on a readable medium may be transmitted using any suitable medium, including but not limited to wireless, wireline, optical cable, RF, etc., or any suitable combination of the foregoing.
  • Program code for performing the operations of the present disclosure may be written in any combination of one or more programming languages, including object-oriented programming languages such as Java, C++, etc., as well as conventional procedural programming. Language—such as "C” or a similar programming language.
  • the program code may execute entirely on the user's computing device, partly on the user's device, as a stand-alone software package, partly on the user's computing device and partly on a remote computing device, or entirely on the remote computing device or server execute on.
  • the remote computing device may be connected to the user computing device through any kind of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computing device, such as provided by an Internet service. (business comes via Internet connection).
  • LAN local area network
  • WAN wide area network
  • Exemplary embodiments of the present disclosure also provide an electronic device capable of implementing the above method.
  • An electronic device 1300 according to such an exemplary embodiment of the present disclosure is described below with reference to FIG. 13 .
  • the electronic device 1300 shown in FIG. 13 is only an example and should not bring any limitations to the functions and scope of use of the embodiments of the present disclosure.
  • electronic device 1300 may take the form of a general-purpose computing device.
  • the components of the electronic device 1300 may include, but are not limited to: at least one processing unit 1310, at least one storage unit 1320, a bus 1330 connecting different system components (including the storage unit 1320 and the processing unit 1310), and a display unit 1340.
  • the storage unit 1320 stores program code, which can be executed by the processing unit 1310, so that the processing unit 1310 performs the steps according to various exemplary embodiments of the present disclosure described in the "Exemplary Method" section above.
  • the processing unit 1310 may perform any one or more method steps in FIG. 3, FIG. 8, FIG. 9, and FIG. 11.
  • the storage unit 1320 may include a readable medium in the form of a volatile storage unit, such as a random access storage unit (RAM) 1321 and/or a cache storage unit 1322, and may further include a read-only storage unit (ROM) 1323.
  • RAM random access storage unit
  • ROM read-only storage unit
  • Storage unit 1320 may also include a program/utility 1324 having a set of (at least one) program modules 1325 including, but not limited to: an operating system, one or more application programs, other program modules, and program data, Each of these examples, or some combination, may include the implementation of a network environment.
  • program/utility 1324 having a set of (at least one) program modules 1325 including, but not limited to: an operating system, one or more application programs, other program modules, and program data, Each of these examples, or some combination, may include the implementation of a network environment.
  • Bus 1330 may be a local area representing one or more of several types of bus structures, including a memory unit bus or memory unit controller, a peripheral bus, a graphics acceleration port, a processing unit, or using any of a variety of bus structures. bus.
  • Electronic device 1300 may also communicate with one or more external devices 1400 (e.g., keyboard, pointing device, Bluetooth device, etc.), may also communicate with one or more devices that enable a user to interact with electronic device 1300, and/or with Any device that enables the electronic device 1300 to communicate with one or more other computing devices (eg, router, modem, etc.). This communication may occur through an input/output (I/O) interface 1350.
  • the electronic device 1300 may also communicate with one or more networks (eg, a local area network (LAN), a wide area network (WAN), and/or a public network, such as the Internet) through the network adapter 1360. As shown in Figure 13, network adapter 1360 communicates with other modules of electronic device 1300 through bus 1330.
  • network adapter 1360 communicates with other modules of electronic device 1300 through bus 1330.
  • the example embodiments described here can be implemented by software, or can be implemented by software combined with necessary hardware. Therefore, the technical solution according to the embodiment of the present disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, U disk, mobile hard disk, etc.) or on the network , including several instructions to cause a computing device (which may be a personal computer, a server, a terminal device, a network device, etc.) to execute a method according to an exemplary embodiment of the present disclosure.
  • a computing device which may be a personal computer, a server, a terminal device, a network device, etc.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Evolutionary Computation (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • General Health & Medical Sciences (AREA)
  • Artificial Intelligence (AREA)
  • Health & Medical Sciences (AREA)
  • Software Systems (AREA)
  • Computing Systems (AREA)
  • Multimedia (AREA)
  • Databases & Information Systems (AREA)
  • Medical Informatics (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Biomedical Technology (AREA)
  • Molecular Biology (AREA)
  • Biodiversity & Conservation Biology (AREA)
  • Biophysics (AREA)
  • Computational Linguistics (AREA)
  • Data Mining & Analysis (AREA)
  • General Engineering & Computer Science (AREA)
  • Mathematical Physics (AREA)
  • Image Analysis (AREA)

Abstract

一种特征提取方法、装置、存储介质及电子设备；涉及人工智能技术领域。方法包括：根据待识别数据和待识别数据中的局部数据得到预设神经网络的输入数据（S310）；利用预设神经网络对输入数据进行融合特征提取，得到待识别数据的目标特征（S320）。利用神经网络对全局数据和局部数据进行融合特征提取，可以提高神经网络识别的特征精度。

Description

特征提取方法、装置、存储介质及电子设备
交叉引用
本申请要求于2022年06月30日提交的申请号为202210770670.6、名称为“特征提取方法、装置、存储介质及电子设备”的中国专利申请的优先权，该中国专利申请的全部内容通过引用全部并入全文。
技术领域
本公开涉及人工智能技术领域，具体而言，涉及一种特征提取方法、特征提取装置、计算机可读存储介质以及电子设备。
背景技术
随着大数据时代的来临，神经网络被应用到人工智能的各个领域当中，如图像识别、无人汽车等领域。
例如，在图像识别领域中，利用神经网络提取的图像特征可以在后续的神经网络中完成特定的任务，如人脸识别，图像分割等。其中，如何提高图像特征提取的准确性，一直是备受关注的技术问题。
需要说明的是，在上述背景技术部分公开的信息仅用于加强对本公开的背景的理解，因此可以包括不构成对本领域普通技术人员已知的现有技术的信息。
发明内容
本公开提供一种特征提取方法、特征提取装置、计算机可读存储介质以及电子设备。
本公开提供一种特征提取方法，包括：
根据待识别数据和所述待识别数据中的局部数据得到预设神经网络的输入数据；
利用所述预设神经网络对所述输入数据进行融合特征提取，得到所述待识别数据的目标特征。
在本公开的一种示例性实施例中，所述预设神经网络至少包括编码器网络和解码器网络，所述编码器网络包括多个下采样层，每个下采样层至少包括多个卷积层和一个池化层，所述解码器网络包括多个上采样层，每个上采样层至少包括一个反卷积层和多个卷积层。
在本公开的一种示例性实施例中，所述预设神经网络还包括局部特征提取网络，所述局部特征提取网络的网络结构与所述编码器网络的网络结构相同；所述根据待识别数据和所述待识别数据中的局部数据得到预设神经网络的输入数据，包括：
将所述待识别数据作为所述编码器网络的输入数据；
将所述待识别数据的局部数据作为所述局部特征提取网络的输入数据。
在本公开的一种示例性实施例中，所述利用所述预设神经网络对所述输入数据进行融合特征提取，得到所述待识别数据的目标特征，包括：
利用所述编码器网络对所述待识别数据进行特征提取，得到所述编码器网络的第一输出特征；
利用所述局部特征提取网络对所述待识别数据中的局部数据进行特征提取，得到所述局部数据的第一特征表示；
将所述编码器网络的第一输出特征和所述局部数据的第一特征表示进行拼接，得到所述解码器网络的第一输入特征；
利用所述解码器网络对所述第一输入特征进行特征提取，得到所述待识别数据的目标特征。
在本公开的一种示例性实施例中，利用所述预设神经网络对所述输入数据进行融合特征提取前，所述方法还包括：
对所述预设神经网络中的所述编码器网络和所述局部特征提取网络进行训练；其中，所述编码器网络的参数初始值与所述局部特征提取网络的参数初始值相同。
在本公开的一种示例性实施例中，所述对所述预设神经网络中的所述编码器网络和所述局部特征提取网络进行训练，包括：
根据预设的参数映射关系对所述编码器网络的参数和所述局部特征提取网络的参数进行迭代，当满足迭代终止条件时，完成对所述编码器网络和所述局部特征提取网络的训练；
其中，所述参数映射关系用于确定迭代后的编码器网络的参数和对应的局部特征提取网络的参数。
在本公开的一种示例性实施例中，所述根据待识别数据和所述待识别数据中的局部数据得到预设神经网络的输入数据，包括：
基于所述待识别数据，构建所述待识别数据中的局部数据的矩阵表示，得到局部数据矩阵；
将所述待识别数据和所述局部数据矩阵中的各对应元素进行组合，得到组合数据；
将所述组合数据作为所述编码器网络的输入数据。
在本公开的一种示例性实施例中，所述将所述待识别数据和所述局部数据矩阵中的各对应元素进行组合，得到组合数据，包括：
根据：
将待识别数据X和局部数据矩阵Y中的各对应元素进行组合，得到组合数据Z；其中，Z(i,j,k)表示组合数据Z中第i行、第j列、第k个通道对应的空间位置上的元素值，分别表示图像数据X、局部数据矩阵Y中第行、第列、第k个通道对应的空间位置上的元素值，表示向上取整运算，为组合系数。
在本公开的一种示例性实施例中，所述利用所述预设神经网络对所述输入数据进行融 合特征提取，得到所述待识别数据的目标特征，包括：
利用所述编码器网络对所述组合数据进行特征提取，得到所述编码器网络的第二输出特征，所述编码器网络的第一个下采样层包括多个空洞卷积层和一个池化层；
对所述编码器网络的第二输出特征进行卷积运算，得到所述解码器网络的第二输入特征；
利用所述解码器网络对所述第二输入特征进行特征提取，得到所述待识别数据的目标特征。
在本公开的一种示例性实施例中，所述预设神经网络还包括局部特征融合网络，所述局部特征融合网络包括多个空洞卷积层和一个池化层；所述利用所述预设神经网络对所述输入数据进行融合特征提取，得到所述待识别数据的目标特征，包括：
将所述组合数据输入所述局部特征融合网络中，得到融合数据；
利用所述预设神经网络中的编码器网络和解码器网络对所述融合数据进行特征提取，得到所述待识别数据的目标特征。
在本公开的一种示例性实施例中，所述预设神经网络还包括分类器；得到所述待识别数据的目标特征后，所述方法还包括：
通过所述分类器对所述待识别数据的目标特征进行分类预测，得到所述待识别数据的分类结果。
本公开提供一种特征提取装置，包括：
输入数据生成模块，用于根据待识别数据和所述待识别数据中的局部数据得到预设神经网络的输入数据；
目标特征提取模块，用于利用所述预设神经网络对所述输入数据进行融合特征提取，得到所述待识别数据的目标特征。
本公开提供一种计算机可读存储介质，其上存储有计算机程序，所述计算机程序被处理器执行时实现上述任意一项所述的方法。
本公开提供一种电子设备，包括：处理器；以及存储器，用于存储所述处理器的可执行指令；其中，所述处理器配置为经由执行所述可执行指令来执行上述任意一项所述的方法。
应当理解的是，以上的一般描述和后文的细节描述仅是示例性和解释性的，并不能限制本公开。
附图说明
此处的附图被并入说明书中并构成本说明书的一部分，示出了符合本公开的实施例，并与说明书一起用于解释本公开的原理。显而易见地，下面描述中的附图仅仅是本公开的一些实施例，对于本领域普通技术人员来讲，在不付出创造性劳动的前提下，还可以根据这些附图获得其它的附图。
图1示出了可以应用本公开实施例的一种特征提取方法及装置的示例性系统架构的示意图；
图2示意性示出了根据本公开的一个实施例的图像分割的示意图；
图3示意性示出了根据本公开的一个实施例的特征提取方法的流程图；
图4示意性示出了根据本公开的一个实施例的U-Net网络改进后的网络结构示意图；
图5示意性示出了根据本公开的另一个实施例的U-Net网络改进后的网络结构示意图；
图6示意性示出了根据本公开的一个实施例的空洞卷积层的结构示意图；
图7示意性示出了根据本公开的一个实施例的矩阵间进行卷积运算的示意图；
图8示意性示出了根据本公开的一个实施例的融合特征提取的流程图；
图9示意性示出了根据本公开的另一个实施例的融合特征提取的流程图；
图10示意性示出了根据本公开的又一个实施例的U-Net网络改进后的网络结构示意图；
图11示意性示出了根据本公开的又一个实施例的融合特征提取的流程图；
图12示意性示出了根据本公开的一个实施例的特征提取装置的框图；
图13示意性示出了适于用来实现本公开实施例的电子设备的计算机系统的结构示意图。
具体实施方式
现在将参考附图更全面地描述示例实施方式。然而，示例实施方式能够以多种形式实施，且不应被理解为限于在此阐述的范例；相反，提供这些实施方式使得本公开将更加全面和完整，并将示例实施方式的构思全面地传达给本领域的技术人员。所描述的特征、结构或特性可以以任何合适的方式结合在一个或更多实施方式中。在下面的描述中，提供许多具体细节从而给出对本公开的实施方式的充分理解。然而，本领域技术人员将意识到，可以实践本公开的技术方案而省略所述特定细节中的一个或更多，或者可以采用其它的方法、组元、装置、步骤等。在其它情况下，不详细示出或描述公知技术方案以避免喧宾夺主而使得本公开的各方面变得模糊。
此外，附图仅为本公开的示意性图解，并非一定是按比例绘制。图中相同的附图标记表示相同或类似的部分，因而将省略对它们的重复描述。附图中所示的一些方框图是功能实体，不一定必须与物理或逻辑上独立的实体相对应。可以采用软件形式来实现这些功能实体，或在一个或多个硬件模块或集成电路中实现这些功能实体，或在不同网络和/或处理器装置和/或微控制器装置中实现这些功能实体。
图1示出了可以应用本公开实施例的一种特征提取方法及装置的示例性应用环境的系统架构的示意图。
如图1所示，系统架构100可以包括终端设备101、102、103中的一个或多个，网络104和服务器105。网络104用以在终端设备101、102、103和服务器105之间提供通信链 路的介质。网络104可以包括各种连接类型，例如有线、无线通信链路或者光纤电缆等等。终端设备101、102、103可以是各种电子设备，包括但不限于台式计算机、便携式计算机、智能手机和平板电脑等。应该理解，图1中的终端设备、网络和服务器的数目仅仅是示意性的。根据实现需要，可以具有任意数目的终端设备、网络和服务器。例如，服务器105可以是一个服务器，也可以是多个服务器组成的服务器集群，还可以是云计算平台或者虚拟化中心。具体地，服务器105可以用于执行：根据待识别数据和所述待识别数据中的局部数据得到预设神经网络的输入数据；利用所述预设神经网络对所述输入数据进行融合特征提取，得到所述待识别数据的目标特征。
本公开实施例所提供的特征提取方法一般由服务器105执行，相应的，特征提取装置一般设置于服务器105。例如，服务器105可以将预设神经网络模型输出的待识别数据的目标特征发送至终端设备，并通过终端设备向用户进行展示。但本领域技术人员容易理解的是，本公开实施例所提供的特征提取方法也可以由终端设备101、102、103中的一个或多个执行，相应地，特征提取装置也可以设置于终端设备101、102、103中。例如，由终端设备101执行特征提取方法后，可以将提取到的待识别数据的目标特征直接显示在终端设备的显示屏上，本示例性实施例中对此不做特殊限定。
以下对本公开实施例的技术方案进行详细阐述：
本公开示例实施方式中，可以以利用神经网络进行图像分割的场景为例进行说明。如图2所示，图(A)为待识别的图像，需要识别该图像中的动植物。具体地，可以将该图像设置为三维数据，并将其作为神经网络的输入数据，得到同样维度大小的输出数据。例如，输入数据可以是：图像行数(h)*图像列数(w)*图像通道数(channels)，输出数据可以是：图像行数(h)*图像列数(w)*图像分类数(classes)。图(B)和图(C)为准确识别下的分类结果示意图，其中，图(B)为豹类分类结果，图(C)为树木类分类结果。
但是，在实际应用中，提取图像特征的特征精度存在一定的局限性。例如，神经网络容易将图(A)中的豹耳识别为背景，从而降低了神经网络识别的准确性。
基于上述一个或多个问题，本示例实施方式提供了一种特征提取方法，该方法可以应用于上述服务器105，也可以应用于上述终端设备101、102、103中的一个或多个，本示例性实施例中对此不做特殊限定。参考图3所示，该特征提取方法可以包括步骤S310和步骤S320：
步骤S310.根据待识别数据和所述待识别数据中的局部数据得到预设神经网络的输入数据；
步骤S320.利用所述预设神经网络对所述输入数据进行融合特征提取，得到所述待识别数据的目标特征。
在本公开示例实施方式所提供的特征提取方法中，根据待识别数据和所述待识别数据中的局部数据得到预设神经网络的输入数据；利用所述预设神经网络对所述输入数据进行融合特征提取，得到所述待识别数据的目标特征。本公开通过将局部数据引入神经网络中， 并利用神经网络对全局数据和局部数据进行融合特征提取，可以得到特征增强后的全局特征，提高了神经网络识别的特征精度，进而提高了神经网络识别的准确性。
下面，对于本示例实施方式的上述步骤进行更加详细的说明。
在步骤S310中，根据待识别数据和所述待识别数据中的局部数据得到预设神经网络的输入数据。
仍以图像分割场景为例，对应的，待识别数据为图像数据，待识别数据中的局部数据为图像数据中包含的部分数据，且该部分数据具有可区分度较高的特征。本公开示例实施方式中，可以根据先验知识确定待识别数据中的局部数据对应的分类结果。其中，先验知识可以是指研究对象具有的特殊性质。例如，可以根据先验知识确定图2(A)对应的图像数据中像素值较大的部分数据属于一个确定的类别，如为树木类别。
示例性的，将待识别数据输入预设神经网络进行特征提取时，可以将待识别数据用一个三维矩阵表示，如h*w*channels，用来表示待识别数据为图像数据。类似的，将待识别数据中的局部数据作为预设神经网络的输入数据时，可以将待识别数据中的局部数据初始化为一个全0的三维矩阵，记为局部数据矩阵，其大小与输出的目标特征大小相同，为h*w*classes。若根据先验知识可以确定待识别数据中部分像素的分类，则可以在局部数据矩阵中对该部分像素对应的分类通道的元素进行赋值，如将该元素赋值为该部分像素识别为正类别的概率值，概率值的区间可以为[0，1]。例如，若该部分像素的正确分类类别为豹子类别，可以将局部数据矩阵中对应的元素赋值为该部分像素识别为豹子类别的概率值，如概率值为0.8。
其中，预设神经网络可以至少包括编码器网络和解码器网络，编码器网络可以包括多个下采样层，每个下采样层至少包括多个卷积层和一个池化层，解码器网络可以包括多个上采样层，每个上采样层至少包括一个反卷积层和多个卷积层。可以理解的是，根据实现需要，下采样层和上采样层的数目可以是任意的，以及下采样层中包含的卷积层的数目也是任意的，本公开对此不做具体限定。示例性的，预设神经网络可以是U-Net网络、Transformer网络等，也可以是包含编码器网络和解码器网络的其他网络模型，本公开对此不做具体限定。
一种示例实施方式中，预设神经网络还可以包括局部特征提取网络。具体地，可以根据编码器网络构建局部特征提取网络，其中，局部特征提取网络的网络结构与编码器网络的网络结构相同，并将待识别数据作为编码器网络的输入数据，将待识别数据的局部数据作为局部特征提取网络的输入数据。
该示例中，可以以预设神经网络为改进后的U-Net网络为例进行说明。参考图4所示，示意性的给出了一种改进后的U-Net网络的网络结构的示意图，该U-Net网络中含有解码器网络202和双流编码器，分别为编码器网络201和编码器网络203，编码器网络203为根据编码器网络201构建的局部特征提取网络，可以看出，编码器网络201和编码器网络203的网络结构完全相同。其中，编码器网络201包括三个下采样层(2011、2012和2013)， 解码器网络202包括三个上采样层(2022、2023和2024)。需要说明的是，本公开对下采样层的数目和上采样层的数目不做具体限定，但下采样层的数目和上采样层的数目是相同的。在编码器网络201与解码器网络202之间建立了三个特征传播层(2031、2032和2033)。对于编码器网络201，可以通过三个下采样层对待识别数据进行特征提取，并由三个特征传播层将提取到的特征图传输至解码器网络202。对于解码器网络202，可以通过三个上采样层对待识别数据进行特征提取，同时还对待识别数据进行特征融合。
具体地，对于编码器网络201中各个下采样层的网络结构，以下采样层2011为例，下采样层2011包括两个卷积层和一个池化层，卷积层可以是浅层卷积层，浅层卷积层的感受野较小，感受野重叠区域也较小，可以保证神经网络捕获更多细节。卷积层中具有可学习的参数，可以通过对预设神经网络进行训练以对卷积层中的参数进行拟合。池化层可以为最大池化层，也可以为平均池化层。在编码器网络201中，横向空心箭头表示由卷积层进行卷积处理，向下实心箭头表示由最大池化层进行最大池化下采样处理，或表示由平均池化层进行平均池化下采样处理。对于解码器网络202中各个上采样层的网络结构，以上采样层2022为例，上采样层2022包括一个反卷积层和两个卷积层，卷积层与反卷积层中均有可学习的参数，同样可以通过对预设神经网络进行训练以对该参数进行拟合。在解码器网络202中，横向空心箭头表示由卷积层进行卷积处理，向上实心箭头表示由反卷积层进行反卷积上采样处理。
可以将待识别数据作为编码器网络201的输入数据，得到待识别数据的全局特征。可以将待识别数据的局部数据作为局部特征提取网络(编码器网络203)的输入数据，得到待识别数据的局部特征，将该局部特征传播至编码器网络201，以将待识别数据的全局特征和局部特征的通道维度进行拼接，得到图4中所示的特征图2014。对拼接得到的特征图2014进行卷积运算，得到解码器网络202的输入，即图4中所示的特征图2021。其他示例中，也可以将待识别数据的全局特征和局部特征通过逐元素求和的方式对特征图参数进行传播，也就是将待识别数据的局部特征传输至编码器网络201，本公开对此不做限定。
图4所示的网络结构具备拓展性与容错性，该网络结构可以扩展到所有包含编解码网络结构的神经网络中，均可以对待识别数据和待识别数据中的局部数据进行特征提取，同时将待识别数据的局部数据作为神经网络的输入，使得神经网络输出的特征图可以更准确的描述待识别数据，从而提高了神经网络的特征精度。利用新增的网络分支即编码器网络203同时对待识别数据的局部数据进行特征提取，并将提取到的局部特征融入编码器网络201中，以便于解码器网络202对待识别数据的全局特征和局部特征进行融合特征提取，提高了神经网络的运行效率。
另一种示例实施方式中，预设神经网络中编码器网络的第一个下采样层可以包括多个空洞卷积层和一个池化层。该示例中，仍以预设神经网络为改进后的U-Net网络为例进行说明。参考图5所示，示意性的给出了另一种改进后的U-Net网络的网络结构的示意图，该U-Net网络中含有编码器网络201和解码器网络202。需要说明的是，图5中所示的编 码器网络201的下采样层2011由两个空洞卷积层(2041和2042)和一个池化层(2043)组成。如图6所示，给出了空洞卷积层的结构示意图，其中，空洞卷积层通过在标准卷积的卷积核上注入空洞，可以增加卷积运算的感受野，使得卷积输出包含较大范围的信息。该示例中，可以初始化空洞卷积层的超参数，如预设空洞卷积层的膨胀率'＝2，步长(＝2，膨胀率用来定义卷积核各元素间的距离，可以理解的是，本公开对空洞卷积层的膨胀率和步长的具体取值不做限定。另外，下采样层2011中的池化层2043可以为平均池化层，也可以为最大池化层，本公开对此也不做限定。
该示例中，可以基于待识别数据，构建所述待识别数据中的局部数据的矩阵表示，得到局部数据矩阵；将所述待识别数据和所述局部数据矩阵中的各对应元素进行组合，得到组合数据，并将该组合数据作为图5中所示的编码器网络201的输入数据。具体地，参考图7所示，待识别数据为图像数据时，可以将待识别数据以三维矩阵的形式表示，得到图像数据X，为h*w*channels。同时，可以根据图像数据的矩阵大小，将先验知识数据化，也就是将待识别数据中的局部数据以三维矩阵的形式表示，得到局部数据矩阵Y，如也为h*w*channels，二者矩阵大小相同。
可以根据公式(1)对图像数据X和局部数据矩阵Y中的各对应元素进行组合。具体地，可以根据：
得到组合数据Z，组合数据Z的矩阵大小为(2h)*(2w)*channels。其中，Z(i,j,k)表示组合数据Z中第i行、第j列、第k个通道对应的空间位置上的元素值， 分别表示图像数据X、局部数据矩阵Y的第行、第列、第k个通道对应的空间位置上的元素值，表示向上取整运算，为组合系数。
举例而言，对于组合数据Z中第1行、第1列、第k个通道对应的空间位置上的元素值，即i＝1，j＝1时，组合系数此时，可以将图像数据X中第1行、第1列、第k个通道对应的空间位置上的元素值赋值为组合数据Z中第1行、第1列、第k个通道对应的空间位置上的元素值。类似的，对于组合数据Z中第1行、第2列、第k个通道对应的空间位置上的元素值，即i＝1，j＝2时，组合系数此时，可以将局部数据矩阵Y中第1行、第2列、第k个通道对应的空间位置上的元素值赋值为组合数据Z中第1行、第2列、第k个通道对应的空间位置上的元素值，赋值后得到的组合数据Z如图7所示。
图5所示的网络结构未新增网络分支，而是将原编码器网络中的浅层卷积层替换为空洞卷积层，结合池化层即可实现待识别数据和待识别数据的局部数据之间的并流融合。相比于图4所示的网络结构，不仅可以提高神经网络的训练效率和运行效率，同时还可以保持神经网络的轻量化。
在步骤S320中，利用所述预设神经网络对所述输入数据进行融合特征提取，得到所述待识别数据的目标特征。
预设神经网络为图4所示的U-Net网络的网络结构时，可以将待识别数据作为其中的编码器网络201的输入数据，同时可以将待识别数据的局部数据作为其中的编码器网络203的输入数据。利用编码器网络201对待识别数据进行特征提取，得到待识别数据的全局特征，利用编码器网络203对待识别数据的局部数据进行特征提取，得到待识别数据的局部特征。然后，再利用解码器网络202对待识别数据的全局特征和局部特征进行融合特征提取，得到该待识别数据的目标特征。
一种示例实施方式中，参考图8所示，可以根据步骤S810至步骤S840通过图4所示的U-Net网络的网络结构对输入数据进行融合特征提取，得到待识别数据的目标特征。
步骤S810.利用所述编码器网络对所述待识别数据进行特征提取，得到所述编码器网络的第一输出特征。
示例性的，待识别数据为图像数据时，可以将图像数据输入至编码器网络201的下采样层2011中，通过下采样层2011中的两个卷积层对输入的图像数据进行两次卷积处理，得到第一特征图，并将第一特征图通过特征传播层2031传输至解码器网络202中的上采样层2024中。同时，在编码器网络201中，可以通过下采样层2011中的池化层对第一特征图进行下采样处理，例如，可以对第一特征图进行最大池化下采样处理。然后，将下采样处理后的第一特征图输入下采样层2012中，可以理解的是，下采样层2012、下采样层2013对输入的特征图所做的处理与下采样层2011对图像数据的处理方式相同，此处不再依次赘述。类似的，可以通过下采样层2012得到第二特征图，将第二特征图通过特征传播层2032传输至解码器网络202中的上采样层2023中。最后，可以通过下采样层2013得到第三特征图，将第三特征图通过特征传播层2033传输至解码器网络202中的上采样层2022中，第三特征图即为编码器网络201的第一输出特征。
其中，随着编码器网络层次加深，对图像数据的特征提取逐渐由局部描述抽象为全局描述，可以更加准确地描述图像数据，从而有利于保证图像分割的精度。
步骤S820.利用所述局部特征提取网络对所述待识别数据中的局部数据进行特征提取，得到所述局部数据的第一特征表示。
与步骤S810对应，待识别数据中的局部数据为图像数据中的部分数据，例如，可以使用局部数据矩阵表示该部分数据。局部特征提取网络为编码器网络203，其网络结构与编码器网络201相同。可以利用编码器网络203对局部数据矩阵进行特征提取，得到局部数据矩阵的第一特征表示，具体的特征提取过程可以参考步骤S810，此处不再详细赘述。
步骤S830.将所述编码器网络的第一输出特征和所述局部数据的第一特征表示进行拼接，得到所述解码器网络的第一输入特征。
可以将局部数据的第一特征表示与编码器网络的第一输出特征进行拼接，例如，可以将二者的通道维度进行拼接，也可以通过逐元素求和的方式将二者进行拼接，以实现特征 传播。将局部数据的第一特征表示与编码器网络的第一输出特征拼接得到如图4所示的特征图2014后，可以对特征图2014进行卷积运算，得到如图4所示的特征图2021，并将特征图2021作为解码器网络202的第一输入特征，以利用解码器网络202对该第一输入特征进行特征融合。
步骤S840.利用所述解码器网络对所述第一输入特征进行特征提取，得到所述待识别数据的目标特征。
首先，可以将特征图2021作为第一待融合特征图输入解码器网络202的上采样层2022，通过上采样层2022中的反卷积层对特征图2021进行反卷积上采样处理，也就是对特征图2021中包含的编码器网络的第一输出特征和局部数据的第一特征表示进行融合，得到第一融合特征图。将第一融合特征图与通过特征传播层传输来的第三特征图进行拼接，并通过上采样层2022中的两个卷积层对第一融合特征图和第三特征图拼接后的特征图依次进行两次卷积处理，得到第二待融合特征图。将第二待融合特征图输入上采样层2023中，通过上采样层2023中的反卷积层对第二待融合特征图进行反卷积上采样处理，可以理解的是，上采样层2023、上采样层2024对输入的待融合特征图所做的处理与上采样层2022对特征图2021的处理方式相同，此处不再依次赘述。类似的，可以通过上采样层2023得到第三待融合特征图，将第三待融合特征图输入上采样层2024中，依次进行反卷积上采样处理和两次卷积处理，上采样层2024的输出数据即为图像数据的目标特征。
该示例中，将待识别数据的局部数据作为含有双流编码器的预设神经网络的输入数据的一部分，通过双流编码器提取到待识别数据的全局特征和局部特征，并通过预设神经网络中的解码器网络融合待识别数据的全局特征和局部特征后，可以实现全局特征的特征增强，提高了预设神经网络识别的特征精度。进一步的，利用含有双流编码器的预设神经网络进行相关领域的识别应用时，如进行图像识别、人脸识别等，可以提高预设神经网络识别的准确性。
通过图4所示的U-Net网络的网络结构对输入数据进行融合特征提取前，可以对该网络结构进行训练。示例性，可以利用由输入数据和输出数据组成的数据集，对参数进行拟合。参数拟合完成后，可以通过该网络结构进行特征提取、图像分割等。
一种示例实施方式中，如图4所示，该U-Net网络中包括编码器网络(201)、局部特征提取网络(即编码器网络203)和解码器网络(202)三部分，本示例中主要对编码器网络和局部特征提取网络的训练过程进行说明。可以理解的是，训练编码器网络和局部特征提取网络的同时，解码器网络的参数也在不断地进行迭代，以拟合得到较优的参数。示例性的，在编码器网络和局部特征提取网络的训练过程中，可以在该部分网络结构的训练过程中加入参数互联机制，也就是建立双流编码器的参数之间的映射关系，以提高该网络结构的训练效率。其中，编码器网络的参数初始值与局部特征提取网络的参数初始值相同。
具体地，编码器网络中有可学习的卷积层参数，编码器网络201与局部特征提取网络203具有相同的网络结构，因此，二者在对应卷积层、池化层的参数规模也是相同的，对 应的，编码器网络201的参数集合到局部特征提取网络203的参数集合是满射的，即局部特征提取网络203中的每个参数在编码器网络201中都有对应的参数。例如，X表示编码器网络201的参数集合，Y表示局部特征提取网络203的参数集合，编码器网络201与局部特征提取网络203之间的参数映射关系，表示为：
Y(i)＝)[X(i)]      (2)
其中，参数映射关系可以用于确定迭代后的编码器网络的参数和对应的局部特征提取网络的参数。
举例而言，可以预设编码器网络201与局部特征提取网络203之间的参数映射关系为：
其中，X1(i)表示当前迭代后的编码器网络201的第i个参数，Y1(i)表示当前迭代后的局部特征提取网络203的第i个参数，X2(i)表示用于下一次迭代的编码器网络201的第i个参数，Y1)i)表示用于下一次迭代的局部特征提取网络203的第i个参数。需要说明的是，编码器网络的参数初始值与局部特征提取网络的参数初始值相同，当局部特征提取网络203的参数和编码器网络201的参数进行梯度更新后，两个网络中的参数会发生变化。本示例中，可以利用参数映射关系使得每次用于迭代的编码器网络的参数和对应的局部特征提取网络的参数相同。具体地，可以将X1(i)和Y1(i)的平均值作为下一次迭代的编码器网络201和局部特征提取网络203的第i个参数。
需要说明的是，训练过程中，需要预设编码器网络的参数初始值与局部特征提取网络的参数初始值相同。对编码器网络与局部特征提取网络使用相同的参数初始值，参数会在训练中不断变化，但是，通过使用参数映射关系调整编码器网络中的参数与局部特征提取网络中的参数，使得编码器网络与局部特征提取网络在参数拟合的过程中可以保持确定的映射关系，便于拟合得到较优的参数。
示例性的，可以根据预设的参数映射关系对编码器网络的参数和局部特征提取网络的参数进行迭代，当满足迭代终止条件时，完成对编码器网络和局部特征提取网络的训练。例如，可以根据输入数据和输出数据构建目标函数，基于该目标函数，可以利用随机梯度下降算法对编码器网络的参数和局部特征提取网络的参数进行迭代更新，当满足迭代终止条件时，完成对编码器网络和局部特征提取网络的训练。其中，迭代终止条件可以是目标函数收敛时，完成对所有参数的训练，也可以通过反向迭代式更新参数，当满足预设的迭代次数时，完成对所有参数的训练，本公开对此不做限定。
该示例中，编码器网络与局部特征提取网络的各个层级之间无特征传播层，仅是将编码器网络的输出特征与局部特征提取网络的输出特征进行了拼接操作，因此，在前向推理过程中，编码器网络与局部特征提取网络的各层推理结果相对独立。而且，若编码器网络与局部特征提取网络具有不同的初始参数值，二者提取到的特征也不同。以及，若编码器网络的参数与局部特征提取网络的参数之间无映射关系，导致信息不能相互传递。基于此进行参数拟合时，参数的梯度更新仅受到所在网络结构的影响，使得参数拟合的效果较差。 可以看出，参数互联机制可以实现各个网络结构间的特征共享，便于拟合得到较优的参数。另外，参数互联机制中，编码器网络的参数与局部特征提取网络的参数有确定的参数映射关系，例如，编码器网络的参数可以由局部特征提取网络的参数计算得到，参数储存时，只需存储编码器网络的参数或局部特征提取网络的参数，节约了储存空间，使得神经网络的结构更加轻量化。
预设神经网络为图5所示的U-Net网络的网络结构时，可以基于待识别数据，构建待识别数据中的局部数据的矩阵表示，得到局部数据矩阵，将待识别数据和局部数据矩阵中的各对应元素进行组合，得到组合数据，并将该组合数据作为图5中所示的编码器网络201的输入数据。利用编码器网络201和解码器网络202对组合数据进行融合特征提取，得到该待识别数据的目标特征。
另一种示例实施方式中，参考图9所示，可以根据步骤S910至步骤S930通过图5所示的U-Net网络的网络结构对输入数据进行融合特征提取，得到待识别数据的目标特征。
步骤S910.利用所述编码器网络对所述组合数据进行特征提取，得到所述编码器网络的第二输出特征，所述编码器网络的第一个下采样层包括多个空洞卷积层和一个池化层。
示例性的，待识别数据为图像数据，待识别数据中的局部数据为图像数据中的部分数据，且用局部数据矩阵表示时，由图像数据和局部数据矩阵得到组合数据。可以将组合数据输入编码器网络201的第一个下采样层2011中，下采样层2011由两个空洞卷积层(2041和2042)和一个池化层(2043)组成。通过下采样层2011中的两个空洞卷积层对输入的图像数据进行两次卷积处理，得到第四特征图，并将第四特征图通过特征传播层2031传输至解码器网络202中的上采样层2024中。同时，在编码器网络201中，可以通过下采样层2011中的池化层对第四特征图进行下采样处理，例如，可以对第四特征图进行平均池化下采样处理。需要说明的是，利用空洞卷积层对组合数据进行卷积运算后，组合数据中的图像数据部分与局部数据矩阵部分仍保持相对独立的空间分布状态，直至池化层将图像数据与局部数据矩阵融合。
然后，将下采样处理后的第四特征图输入下采样层2012中，可以理解的是，下采样层2012、下采样层2013对输入的特征图所做的处理与下采样层2011对图像数据的处理方式相同，此处不再依次赘述。类似的，可以通过下采样层2012得到第五特征图，将第五特征图通过特征传播层2032传输至解码器网络202中的上采样层2023中。最后，可以通过下采样层2013得到第六特征图，将第三特征图通过特征传播层2033传输至解码器网络202中的上采样层2022中，第六特征图即为编码器网络201的第二输出特征。
步骤S920.对所述编码器网络的第二输出特征进行卷积运算，得到所述解码器网络的第二输入特征。
图5所示的特征图2015为编码器网络201的第二输出特征，可以对特征图2015进行卷积运算，得到如图5所示的特征图2020，并将特征图2020作为解码器网络202的第二输入特征，以利用解码器网络202对该第二输入特征进行特征融合。
步骤S930.利用所述解码器网络对所述第二输入特征进行特征提取，得到所述待识别数据的目标特征。
可以将特征图2020作为第四待融合特征图输入解码器网络202的上采样层2022，通过上采样层2022中的反卷积层对特征图2020进行反卷积上采样处理，也就是对特征图2020中包含的图像数据和局部数据矩阵进行融合，得到第四融合特征图。将第四融合特征图与通过特征传播层传输来的第三特征图进行拼接，并通过上采样层2022中的两个卷积层对第四融合特征图和第三特征图拼接后的特征图依次进行两次卷积处理，得到第五待融合特征图。将第五待融合特征图输入上采样层2023中，通过上采样层2023中的反卷积层对第五待融合特征图进行反卷积上采样处理，可以理解的是，上采样层2023、上采样层2024对输入的待融合特征图所做的处理与上采样层2022对特征图2020的处理方式相同，此处不再依次赘述。类似的，可以通过上采样层2023得到第六待融合特征图，将第六待融合特征图输入上采样层2024中，依次进行反卷积上采样处理和两次卷积处理，上采样层2024的输出数据即为图像数据的目标特征。
该示例中，将待识别数据和待识别数据的局部数据一起输入预设神经网络的编码器网络中，通过编码器网络中包含两个空洞卷积层和一个池化层的下采样层，可以将待识别数据和待识别数据的局部数据进行融合，通过解码器网络也可以将待识别数据和待识别数据的局部数据进行融合，从而实现了待识别数据的局部数据的并流融合，对全局特征进一步增强，提高了预设神经网络识别的特征精度。
由于模型重用或结构冲突等原因，无法对神经网络的网络结构进行如图4和图5所示的改进时。参考图10所示，示意性的给出了又一种改进后的预设神经网络的网络结构的示意图，该预设神经网络可以包括编码器网络和解码器网络，还可以包括局部特征融合网络，所述局部特征融合网络包括多个空洞卷积层和一个池化层。示例性的，局部特征融合网络可以包括如图5所示的编码器网络201的第一个下采样层2011，其中，下采样层2011由两个空洞卷积层(2041和2042)和一个池化层(2043)组成，池化层2043可以为平均池化层，也可以为最大池化层，本公开对此也不做限定。进一步的，可以将局部特征融合网络与编码器网络、解码器网络串联，以将待识别数据的局部数据融合到预设神经网络中。
可以基于待识别数据，构建待识别数据中的局部数据的矩阵表示，得到局部数据矩阵，并将待识别数据和局部数据矩阵中的各对应元素进行组合，得到组合数据。可以将组合数据作为局部特征融合网络即下采样层2011的输入数据，通过下采样层2011中的池化层2043将组合数据中的待识别数据部分和局部数据部分进行融合，得到融合数据。然后，将融合数据通过编码器网络和解码器网络进行特征提取和进一步的特征融合，得到输出数据。
又一种示例实施方式中，参考图11所示，可以根据步骤S1110至步骤S1120通过图10所示的预设神经网络的网络结构对输入数据进行融合特征提取，得到待识别数据的目标特征。
步骤S1110.将所述组合数据输入所述局部特征融合网络中，得到融合数据。
示例性的，待识别数据为图像数据，待识别数据中的局部数据为图像数据中的部分数据，且用局部数据矩阵表示时，由图像数据和局部数据矩阵得到组合数据。本示例中，局部特征融合网络为图10所示的下采样层2011，可以将组合数据输入下采样层2011中，通过下采样层2011中的两个空洞卷积层对输入的图像数据进行两次卷积处理，得到第四特征图。可以通过下采样层2011中的池化层对第四特征图进行下采样处理，例如，可以对第四特征图进行平均池化下采样处理，得到融合数据。
步骤S1120.利用所述预设神经网络中的编码器网络和解码器网络对所述融合数据进行特征融合，得到所述待识别数据的目标特征。
将融合数据作为编码器网络的输入数据，通过编码器网络和解码器网络对该融合数据进行特征提取和进一步的特征融合，最后得到图像数据的目标特征。以编码器网络和解码器网络为图4所示的编码器网络201和解码器网络202为例，将融合数据作为编码器网络201的输入数据后，编码器网络201和解码器网络202对融合数据的特征提取过程和特征融合过程可以参考步骤810和步骤840，此处不再赘述。
该示例中，局部特征融合网络作为预设神经网络的输入层，由编码器网络和解码器网络构成的原始网络结构成为了预设神经网络的中间层与输出层。通过改进后的神经网络，进一步融合了待识别数据和待识别数据的局部数据，在原有的计算量基本保持不变的基础上，为待识别数据的局部数据融入编解码神经网络提供了新的改进方法。
一种示例实施方式中，预设神经网络中还可以包括分类器，分类器可以设置于解码器网络之后。例如，分类器可以是Softmax分类器、sigmoid分类器等。可以通过分类器对待识别数据的目标特征进行分类预测，得到待识别数据的分类结果。示例性的，待识别数据为图像数据时，解码器网络输出图像数据的目标特征后，可以利用分类器计算目标特征中的像素点属于不同类别的概率，实现对目标特征中的像素点的类别预测，从而进行像素点分类，即图像分割。
需要说明的是，本公开的特征提取方法可以适用于有特征提取需求的多种场景，本公开仅是以图像分割场景中的特征提取进行说明。
在本公开示例实施方式所提供的特征提取方法中，根据待识别数据和所述待识别数据中的局部数据得到预设神经网络的输入数据；利用所述预设神经网络对所述输入数据进行融合特征提取，得到所述待识别数据的目标特征。本公开通过将局部数据引入神经网络中，并利用神经网络对全局数据和局部数据进行融合特征提取，可以得到特征增强后的全局特征，提高了神经网络识别的特征精度，进而提高了神经网络识别的准确性。
应当注意，尽管在附图中以特定顺序描述了本公开中方法的各个步骤，但是，这并非要求或者暗示必须按照该特定顺序来执行这些步骤，或是必须执行全部所示的步骤才能实现期望的结果。附加的或备选的，可以省略某些步骤，将多个步骤合并为一个步骤执行，以及/或者将一个步骤分解为多个步骤执行等。
进一步的，本示例实施方式中，还提供了一种特征提取装置。该装置可以应用于一终 端设备或服务器。参考图12所示，该特征提取装置1200可以包括输入数据生成模块1210和目标特征提取模块1220，其中：
输入数据生成模块1210，用于根据待识别数据和所述待识别数据中的局部数据得到预设神经网络的输入数据；
目标特征提取模块1220，用于利用所述预设神经网络对所述输入数据进行融合特征提取，得到所述待识别数据的目标特征。
在一种可选的实施方式中，该特征提取装置1200中的所述预设神经网络至少包括编码器网络和解码器网络，所述编码器网络包括多个下采样层，每个下采样层至少包括多个卷积层和一个池化层，所述解码器网络包括多个上采样层，每个上采样层至少包括一个反卷积层和多个卷积层。
在一种可选的实施方式中，所述预设神经网络还包括局部特征提取网络，所述局部特征提取网络的网络结构与所述编码器网络的网络结构相同；输入数据生成模块1210包括：
提取网络构建子模块，用于根据所述编码器网络构建所述局部特征提取网络，所述局部特征提取网络的网络结构与所述编码器网络的网络结构相同；
第一输入数据生成子模块，用于将所述待识别数据作为所述编码器网络的输入数据；以及，将所述待识别数据的局部数据作为所述局部特征提取网络的输入数据。
在一种可选的实施方式中，目标特征提取模块1220包括：
第一特征提取子模块，用于利用所述编码器网络对所述待识别数据进行特征提取，得到所述编码器网络的第一输出特征；
第二特征提取子模块，用于利用所述局部特征提取网络对所述待识别数据中的局部数据进行特征提取，得到所述局部数据的第一特征表示；
第一输入特征生成子模块，用于将所述编码器网络的第一输出特征和所述局部数据的第一特征表示进行拼接，得到所述解码器网络的第一输入特征；
第一目标特征生成子模块，用于利用所述解码器网络对所述第一输入特征进行特征提取，得到所述待识别数据的目标特征。
在一种可选的实施方式中，特征提取装置1200还包括：
网络训练模块，用于对所述预设神经网络中的所述编码器网络和所述局部特征提取网络进行训练；其中，所述编码器网络的参数初始值与所述局部特征提取网络的参数初始值相同。
在一种可选的实施方式中，网络训练模块被配置为用于根据预设的参数映射关系对所述编码器网络的参数和所述局部特征提取网络的参数进行迭代，当满足迭代终止条件时，完成对所述编码器网络和所述局部特征提取网络的训练；其中，所述参数映射关系用于确定迭代后的编码器网络的参数和对应的局部特征提取网络的参数。
在一种可选的实施方式中，输入数据生成模块1210包括：
组合数据生成子模块，用于基于所述待识别数据，构建所述待识别数据中的局部数据 的矩阵表示，得到局部数据矩阵；将所述待识别数据和所述局部数据矩阵中的各对应元素进行组合，得到组合数据；
第二输入数据生成子模块，用于将所述组合数据作为所述编码器网络的输入数据。
在一种可选的实施方式中，组合数据生成子模块被配置为用于根据：
将待识别数据X和局部数据矩阵Y中的各对应元素进行组合，得到组合数据Z；其中，Z(i,j,k)表示组合数据Z中第i行、第j列、第k个通道对应的空间位置上的元素值，分别表示图像数据X、局部数据矩阵Y中第行、第列、第k个通道对应的空间位置上的元素值，表示向上取整运算，为组合系数。
在一种可选的实施方式中，目标特征提取模块1220包括：
第三特征提取子模块，用于利用所述编码器网络对所述组合数据进行特征提取，得到所述编码器网络的第二输出特征，所述编码器网络的第一个下采样层包括多个空洞卷积层和一个池化层；
第一输入特征生成子模块，用于对所述编码器网络的第二输出特征进行卷积运算，得到所述解码器网络的第二输入特征；
第二目标特征生成子模块，用于利用所述解码器网络对所述第二输入特征进行特征提取，得到所述待识别数据的目标特征。
在一种可选的实施方式中，所述预设神经网络还包括局部特征融合网络，所述局部特征融合网络包括多个空洞卷积层和一个池化层；目标特征提取模块1220包括：
融合数据生成子模块，用于将所述组合数据输入所述局部特征融合网络中，得到融合数据；
第三目标特征生成子模块，用于利用所述预设神经网络中的编码器网络和解码器网络对所述融合数据进行特征融合，得到所述待识别数据的目标特征。
在一种可选的实施方式中，所述预设神经网络还包括分类器；特征提取装置1200还包括：
数据识别模块，用于通过所述分类器对所述待识别数据的目标特征进行分类预测，得到所述待识别数据的分类结果。
上述特征提取装置中各模块的具体细节已经在对应的特征提取方法中进行了详细的描述，因此此处不再赘述。
上述装置中各模块可以是通用处理器，包括：中央处理器、网络处理器等；还可以是数字信号处理器、专用集成电路、现场可编程门阵列或者其他可编程逻辑器件、分立门或者晶体管逻辑器件、分立硬件组件。各模块也可以由软件、固件等形式来实现。上述装置中的各处理器可以是独立的处理器，也可以集成在一起。
本公开的示例性实施方式还提供了一种计算机可读存储介质，其上存储有能够实现本说明书上述方法的程序产品。在一些可能的实施方式中，本公开的各个方面还可以实现为一种程序产品的形式，其包括程序代码，当程序产品在电子设备上运行时，程序代码用于使电子设备执行本说明书上述“示例性方法”部分中描述的根据本公开各种示例性实施方式的步骤。该程序产品可以采用便携式紧凑盘只读存储器(CD-ROM)并包括程序代码，并可以在电子设备，例如个人电脑上运行。然而，本公开的程序产品不限于此，在本文件中，可读存储介质可以是任何包含或存储程序的有形介质，该程序可以被指令执行系统、装置或者器件使用或者与其结合使用。
程序产品可以采用一个或多个可读介质的任意组合。可读介质可以是可读信号介质或者可读存储介质。可读存储介质例如可以为但不限于电、磁、光、电磁、红外线、或半导体的系统、装置或器件，或者任意以上的组合。可读存储介质的更具体的例子(非穷举的列表)包括：具有一个或多个导线的电连接、便携式盘、硬盘、随机存取存储器(RAM)、只读存储器(ROM)、可擦式可编程只读存储器(EPROM或闪存)、光纤、便携式紧凑盘只读存储器(CD-ROM)、光存储器件、磁存储器件、或者上述的任意合适的组合。
计算机可读信号介质可以包括在基带中或者作为载波一部分传播的数据信号，其中承载了可读程序代码。这种传播的数据信号可以采用多种形式，包括但不限于电磁信号、光信号或上述的任意合适的组合。可读信号介质还可以是可读存储介质以外的任何可读介质，该可读介质可以发送、传播或者传输用于由指令执行系统、装置或者器件使用或者与其结合使用的程序。
可读介质上包含的程序代码可以用任何适当的介质传输，包括但不限于无线、有线、光缆、RF等等，或者上述的任意合适的组合。
可以以一种或多种程序设计语言的任意组合来编写用于执行本公开操作的程序代码，程序设计语言包括面向对象的程序设计语言—诸如Java、C++等，还包括常规的过程式程序设计语言—诸如“C”语言或类似的程序设计语言。程序代码可以完全地在用户计算设备上执行、部分地在用户设备上执行、作为一个独立的软件包执行、部分在用户计算设备上部分在远程计算设备上执行、或者完全在远程计算设备或服务器上执行。在涉及远程计算设备的情形中，远程计算设备可以通过任意种类的网络，包括局域网(LAN)或广域网(WAN)，连接到用户计算设备，或者，可以连接到外部计算设备(例如利用因特网服务提供商来通过因特网连接)。
本公开的示例性实施方式还提供了一种能够实现上述方法的电子设备。下面参照图13来描述根据本公开的这种示例性实施方式的电子设备1300。图13显示的电子设备1300仅仅是一个示例，不应对本公开实施方式的功能和使用范围带来任何限制。
如图13所示，电子设备1300可以以通用计算设备的形式表现。电子设备1300的组件可以包括但不限于：至少一个处理单元1310、至少一个存储单元1320、连接不同系统组件(包括存储单元1320和处理单元1310)的总线1330和显示单元1340。
存储单元1320存储有程序代码，程序代码可以被处理单元1310执行，使得处理单元1310执行本说明书上述“示例性方法”部分中描述的根据本公开各种示例性实施方式的步骤。例如，处理单元1310可以执行图3、图8、图9和图11中任意一个或多个方法步骤。
存储单元1320可以包括易失性存储单元形式的可读介质，例如随机存取存储单元(RAM)1321和/或高速缓存存储单元1322，还可以进一步包括只读存储单元(ROM)1323。
存储单元1320还可以包括具有一组(至少一个)程序模块1325的程序/实用工具1324，这样的程序模块1325包括但不限于：操作系统、一个或者多个应用程序、其它程序模块以及程序数据，这些示例中的每一个或某种组合中可能包括网络环境的实现。
总线1330可以为表示几类总线结构中的一种或多种，包括存储单元总线或者存储单元控制器、外围总线、图形加速端口、处理单元或者使用多种总线结构中的任意总线结构的局域总线。
电子设备1300也可以与一个或多个外部设备1400(例如键盘、指向设备、蓝牙设备等)通信，还可与一个或者多个使得用户能与该电子设备1300交互的设备通信，和/或与使得该电子设备1300能与一个或多个其它计算设备进行通信的任何设备(例如路由器、调制解调器等等)通信。这种通信可以通过输入/输出(I/O)接口1350进行。并且，电子设备1300还可以通过网络适配器1360与一个或者多个网络(例如局域网(LAN)，广域网(WAN)和/或公共网络，例如因特网)通信。如图13所示，网络适配器1360通过总线1330与电子设备1300的其它模块通信。应当明白，尽管图13中未示出，可以结合电子设备1300使用其它硬件和/或软件模块，包括但不限于：微代码、设备驱动器、冗余处理单元、外部磁盘驱动阵列、RAID系统、磁带驱动器以及数据备份存储系统等。
通过以上的实施方式的描述，本领域的技术人员易于理解，这里描述的示例实施方式可以通过软件实现，也可以通过软件结合必要的硬件的方式来实现。因此，根据本公开实施方式的技术方案可以以软件产品的形式体现出来，该软件产品可以存储在一个非易失性存储介质(可以是CD-ROM，U盘，移动硬盘等)中或网络上，包括若干指令以使得一台计算设备(可以是个人计算机、服务器、终端装置、或者网络设备等)执行根据本公开示例性实施方式的方法。
此外，上述附图仅是根据本公开示例性实施方式的方法所包括的处理的示意性说明，而不是限制目的。易于理解，上述附图所示的处理并不表明或限制这些处理的时间顺序。另外，也易于理解，这些处理可以是例如在多个模块中同步或异步执行的。
应当注意，尽管在上文详细描述中提及了用于动作执行的设备的若干模块或者单元，但是这种划分并非强制性的。实际上，根据本公开的实施方式，上文描述的两个或更多模块或者单元的特征和功能可以在一个模块或者单元中具体化。反之，上文描述的一个模块或者单元的特征和功能可以进一步划分为由多个模块或者单元来具体化。
应当理解的是，本公开并不局限于上面已经描述并在附图中示出的精确结构，并且可以在不脱离其范围进行各种修改和改变。本公开的范围仅由所附的权利要求来限制。

Claims (14)

  1. 一种特征提取方法，其特征在于，包括：
    根据待识别数据和所述待识别数据中的局部数据得到预设神经网络的输入数据；
    利用所述预设神经网络对所述输入数据进行融合特征提取，得到所述待识别数据的目标特征。
  2. 根据权利要求1所述的特征提取方法，其特征在于，所述预设神经网络至少包括编码器网络和解码器网络，所述编码器网络包括多个下采样层，每个下采样层至少包括多个卷积层和一个池化层，所述解码器网络包括多个上采样层，每个上采样层至少包括一个反卷积层和多个卷积层。
  3. 根据权利要求2所述的特征提取方法，其特征在于，所述预设神经网络还包括局部特征提取网络，所述局部特征提取网络的网络结构与所述编码器网络的网络结构相同；所述根据待识别数据和所述待识别数据中的局部数据得到预设神经网络的输入数据，包括：
    将所述待识别数据作为所述编码器网络的输入数据；
    将所述待识别数据的局部数据作为所述局部特征提取网络的输入数据。
  4. 根据权利要求3所述的特征提取方法，其特征在于，所述利用所述预设神经网络对所述输入数据进行融合特征提取，得到所述待识别数据的目标特征，包括：
    利用所述编码器网络对所述待识别数据进行特征提取，得到所述编码器网络的第一输出特征；
    利用所述局部特征提取网络对所述待识别数据中的局部数据进行特征提取，得到所述局部数据的第一特征表示；
    将所述编码器网络的第一输出特征和所述局部数据的第一特征表示进行拼接，得到所述解码器网络的第一输入特征；
    利用所述解码器网络对所述第一输入特征进行特征提取，得到所述待识别数据的目标特征。
  5. 根据权利要求1-4任一项所述的特征提取方法，其特征在于，利用所述预设神经网络对所述输入数据进行融合特征提取前，所述方法还包括：
    对所述预设神经网络中的所述编码器网络和所述局部特征提取网络进行训练；其中，所述编码器网络的参数初始值与所述局部特征提取网络的参数初始值相同。
  6. 根据权利要求5所述的特征提取方法，其特征在于，所述对所述预设神经网络中的所述编码器网络和所述局部特征提取网络进行训练，包括：
    根据预设的参数映射关系对所述编码器网络的参数和所述局部特征提取网 络的参数进行迭代，当满足迭代终止条件时，完成对所述编码器网络和所述局部特征提取网络的训练；
    其中，所述参数映射关系用于确定迭代后的编码器网络的参数和对应的局部特征提取网络的参数。
  7. 根据权利要求2所述的特征提取方法，其特征在于，所述根据待识别数据和所述待识别数据中的局部数据得到预设神经网络的输入数据，包括：
    基于所述待识别数据，构建所述待识别数据中的局部数据的矩阵表示，得到局部数据矩阵；
    将所述待识别数据和所述局部数据矩阵中的各对应元素进行组合，得到组合数据；
    将所述组合数据作为所述编码器网络的输入数据。
  8. 根据权利要求7所述的特征提取方法，其特征在于，所述将所述待识别数据和所述局部数据矩阵中的各对应元素进行组合，得到组合数据，包括：
    根据：
    将待识别数据X和局部数据矩阵Y中的各对应元素进行组合，得到组合数据Z；其中，Z(i,j,k)表示组合数据Z中第i行、第j列、第k个通道对应的空间位置上的元素值，分别表示图像数据X、局部数据矩阵Y中第行、第列、第k个通道对应的空间位置上的元素值，表示向上取整运算，为组合系数。
  9. 根据权利要求7所述的特征提取方法，其特征在于，所述利用所述预设神经网络对所述输入数据进行融合特征提取，得到所述待识别数据的目标特征，包括：
    利用所述编码器网络对所述组合数据进行特征提取，得到所述编码器网络的第二输出特征，所述编码器网络的第一个下采样层包括多个空洞卷积层和一个池化层；
    对所述编码器网络的第二输出特征进行卷积运算，得到所述解码器网络的第二输入特征；
    利用所述解码器网络对所述第二输入特征进行特征提取，得到所述待识别数据的目标特征。
  10. 根据权利要求7所述的特征提取方法，其特征在于，所述预设神经网络还包括局部特征融合网络，所述局部特征融合网络包括多个空洞卷积层和一个池化层；所述利用所述预设神经网络对所述输入数据进行融合特征提取，得 到所述待识别数据的目标特征，包括：
    将所述组合数据输入所述局部特征融合网络中，得到融合数据；
    利用所述预设神经网络中的编码器网络和解码器网络对所述融合数据进行特征提取，得到所述待识别数据的目标特征。
  11. 根据权利要求1所述的特征提取方法，其特征在于，所述预设神经网络还包括分类器；得到所述待识别数据的目标特征后，所述方法还包括：
    通过所述分类器对所述待识别数据的目标特征进行分类预测，得到所述待识别数据的分类结果。
  12. 一种特征提取装置，其特征在于，包括：
    输入数据生成模块，用于根据待识别数据和所述待识别数据中的局部数据得到预设神经网络的输入数据；
    目标特征提取模块，用于利用所述预设神经网络对所述输入数据进行融合特征提取，得到所述待识别数据的目标特征。
  13. 一种计算机可读存储介质，其上存储有计算机程序，其特征在于，所述计算机程序被处理器执行时实现权利要求1-11任一项所述的方法。
  14. 一种电子设备，其特征在于，包括：
    处理器；以及
    存储器，用于存储所述处理器的可执行指令；
    其中，所述处理器配置为经由执行所述可执行指令来执行权利要求1-11任一项所述的方法。
PCT/CN2023/097373 2022-06-30 2023-05-31 特征提取方法、装置、存储介质及电子设备 WO2024001653A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202210770670.6A CN114998668A (zh) 2022-06-30 2022-06-30 特征提取方法、装置、存储介质及电子设备
CN202210770670.6 2022-06-30

Publications (2)

Publication Number Publication Date
WO2024001653A1 true WO2024001653A1 (zh) 2024-01-04
WO2024001653A9 WO2024001653A9 (zh) 2024-04-25

Family

ID=83020262

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2023/097373 WO2024001653A1 (zh) 2022-06-30 2023-05-31 特征提取方法、装置、存储介质及电子设备

Country Status (2)

Country Link
CN (1) CN114998668A (zh)
WO (1) WO2024001653A1 (zh)

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114998668A (zh) * 2022-06-30 2022-09-02 北京京东方技术开发有限公司 特征提取方法、装置、存储介质及电子设备

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109359696A (zh) * 2018-10-29 2019-02-19 重庆中科云丛科技有限公司 一种车款识别方法、系统及存储介质
US20210201499A1 (en) * 2019-12-30 2021-07-01 Medo Dx Pte. Ltd Apparatus and method for image segmentation using a deep convolutional neural network with a nested u-structure
CN113936256A (zh) * 2021-10-15 2022-01-14 北京百度网讯科技有限公司 一种图像目标检测方法、装置、设备以及存储介质
CN114581462A (zh) * 2022-03-01 2022-06-03 京东科技信息技术有限公司 一种图像分割方法、装置、设备和存储介质
CN114998668A (zh) * 2022-06-30 2022-09-02 北京京东方技术开发有限公司 特征提取方法、装置、存储介质及电子设备

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109359696A (zh) * 2018-10-29 2019-02-19 重庆中科云丛科技有限公司 一种车款识别方法、系统及存储介质
US20210201499A1 (en) * 2019-12-30 2021-07-01 Medo Dx Pte. Ltd Apparatus and method for image segmentation using a deep convolutional neural network with a nested u-structure
CN113936256A (zh) * 2021-10-15 2022-01-14 北京百度网讯科技有限公司 一种图像目标检测方法、装置、设备以及存储介质
CN114581462A (zh) * 2022-03-01 2022-06-03 京东科技信息技术有限公司 一种图像分割方法、装置、设备和存储介质
CN114998668A (zh) * 2022-06-30 2022-09-02 北京京东方技术开发有限公司 特征提取方法、装置、存储介质及电子设备

Also Published As

Publication number Publication date
CN114998668A (zh) 2022-09-02
WO2024001653A9 (zh) 2024-04-25

Similar Documents

Publication Publication Date Title
US11775574B2 (en) Method and apparatus for visual question answering, computer device and medium
EP3940591A1 (en) Image generating method, neural network compression method, and related apparatus and device
JP7559263B2 (ja) テキストを認識するための方法および装置
WO2022156561A1 (zh) 一种自然语言处理方法以及装置
CN112258512B (zh) 点云分割方法、装置、设备和存储介质
WO2022001805A1 (zh) 一种神经网络蒸馏方法及装置
JP2022006174A (ja) モデルをトレーニングするための方法、装置、デバイス、媒体、およびプログラム製品
CN111932546A (zh) 图像分割模型训练方法、图像分割方法、装置、设备及介质
JP2022078310A (ja) 画像分類モデル生成方法、装置、電子機器、記憶媒体、コンピュータプログラム、路側装置およびクラウド制御プラットフォーム
WO2024001653A1 (zh) 特征提取方法、装置、存储介质及电子设备
CN114926636A (zh) 一种点云语义分割方法、装置、设备及存储介质
CN115578570A (zh) 图像处理方法、装置、可读介质及电子设备
CN111444335B (zh) 中心词的提取方法及装置
CN118397147A (zh) 一种基于深度学习的图像文本生成方法及装置
WO2024160219A1 (zh) 一种模型量化方法及其装置
CN118229632A (zh) 显示屏缺陷检测方法、模型训练方法、装置、设备及介质
US11948090B2 (en) Method and apparatus for video coding
CN114065915A (zh) 网络模型的构建方法、数据处理方法、装置、介质及设备
CN113360683A (zh) 训练跨模态检索模型的方法以及跨模态检索方法和装置
CN113191364B (zh) 车辆外观部件识别方法、装置、电子设备和介质
CN113139490B (zh) 一种图像特征匹配方法、装置、计算机设备及存储介质
CN114792388A (zh) 图像描述文字生成方法、装置及计算机可读存储介质
CN111898658A (zh) 图像分类方法、装置和电子设备
CN111950572A (zh) 训练分类器的方法、装置、电子设备和计算机可读存储介质
CN112417260A (zh) 本地化推荐方法、装置及存储介质

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 23829838

Country of ref document: EP

Kind code of ref document: A1