WO2024078404A1 - 特征图处理方法、装置及设备 - Google Patents

特征图处理方法、装置及设备 Download PDF

Info

Publication number
WO2024078404A1
WO2024078404A1 PCT/CN2023/123323 CN2023123323W WO2024078404A1 WO 2024078404 A1 WO2024078404 A1 WO 2024078404A1 CN 2023123323 W CN2023123323 W CN 2023123323W WO 2024078404 A1 WO2024078404 A1 WO 2024078404A1
Authority
WO
WIPO (PCT)
Prior art keywords
feature map
channels
original
processing
residual
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/CN2023/123323
Other languages
English (en)
French (fr)
Inventor
李胜曦
陈超然
刘铁
张子夫
徐迈
吕卓逸
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Vivo Mobile Communication Co Ltd
Original Assignee
Vivo Mobile Communication Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Vivo Mobile Communication Co Ltd filed Critical Vivo Mobile Communication Co Ltd
Publication of WO2024078404A1 publication Critical patent/WO2024078404A1/zh
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/70—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals characterised by syntax aspects related to video coding, e.g. related to compression standards
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/134—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or criterion affecting or controlling the adaptive coding
    • H04N19/146—Data rate or code amount at the encoder output
    • H04N19/149—Data rate or code amount at the encoder output by estimating the code amount by means of a model, e.g. mathematical model or statistical model
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/134—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or criterion affecting or controlling the adaptive coding
    • H04N19/157—Assigned coding mode, i.e. the coding mode being predefined or preselected to be further used for selection of another element or parameter
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/42—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals characterised by implementation details or hardware specially adapted for video compression or decompression, e.g. dedicated software implementation

Definitions

  • the present application belongs to the field of image compression technology, and specifically relates to a feature map processing method, device and equipment.
  • the related compression method is to compress the image by using spatial correlation and temporal correlation to reduce the spatiotemporal pixel-level redundancy of the image.
  • This compression method cannot achieve effective compression of the feature map, thereby reducing the compression efficiency of the feature map.
  • the embodiments of the present application provide a feature map processing method, device and equipment, which can solve the problem that existing solutions cannot achieve effective compression of feature maps, thereby reducing the efficiency of feature map compression.
  • a feature map processing method comprising:
  • the encoding end compresses the original feature map to obtain a first feature map; the number of channels of the first feature map is smaller than the number of channels of the original feature map, and/or the spatial size of the first feature map is smaller than the spatial size of the original feature map;
  • the encoding end encodes the first feature map to obtain a first code stream and a second feature map
  • the encoding end performs enhancement processing on the second feature map to determine a third feature map; the number of channels of the third feature map is equal to the number of channels of the original feature map, and the spatial size of the third feature map is equal to the spatial size of the original feature map;
  • the encoding end encodes the residual feature map to obtain a second bitstream; the residual feature map is determined based on the original feature map and the third feature map.
  • a feature map processing method comprising:
  • the decoding end decodes the target bitstream to obtain a second feature map and a residual feature map, wherein the target bitstream includes the first bitstream and a second code stream, wherein the second feature map is determined based on the first code stream, and the residual feature map is determined based on the second code stream;
  • the decoding end performs enhancement processing on the second feature map to determine a third feature map; the number of channels of the third feature map is greater than the number of channels of the second feature map, and/or the spatial size of the third feature map is greater than the spatial size of the second feature map;
  • the decoding end determines a reconstructed feature map based on the third feature map and the residual feature map.
  • a feature map processing device which is applied to an encoding end, and the device includes:
  • a first processing module is used to compress the original feature map to obtain a first feature map; the number of channels of the first feature map is smaller than the number of channels of the original feature map, and/or the spatial size of the first feature map is smaller than the spatial size of the original feature map;
  • a first encoding module used for encoding the first feature map to obtain a first code stream and a second feature map
  • a second processing module is used to perform enhancement processing on the second feature map to determine a third feature map; the number of channels of the third feature map is equal to the number of channels of the original feature map, and the spatial size of the third feature map is equal to the spatial size of the original feature map;
  • the second encoding module is used to encode the residual feature map to obtain a second code stream; the residual feature map is determined based on the original feature map and the third feature map.
  • an image processing device which is applied to a decoding end, and the device includes:
  • a decoding module configured to decode a target bitstream to obtain a second feature map and a residual feature map, wherein the target bitstream includes a first bitstream and a second bitstream, wherein the second feature map is determined based on the first bitstream, and the residual feature map is determined based on the second bitstream;
  • a third processing module configured to perform enhancement processing on the second feature map to determine a third feature map; the number of channels of the third feature map is greater than the number of channels of the second feature map, and/or the spatial size of the third feature map is greater than the spatial size of the second feature map;
  • a determination module is used to determine a reconstructed feature map based on the third feature map and the residual feature map.
  • a terminal which includes a processor and a memory, wherein the memory stores a program or instruction that can be run on the processor, and when the program or instruction is executed by the processor, the steps of the method described in the first aspect are implemented, or the steps of the method described in the second aspect are implemented.
  • a readable storage medium on which a program or instruction is stored.
  • the program or instruction is executed by a processor, the steps of the method described in the first aspect are implemented, or the steps of the method described in the second aspect are implemented.
  • a chip comprising a processor and a communication interface, wherein the communication interface is coupled to the processor, and the processor is used to run a program or instruction to implement the method described in the first aspect, or to implement the method described in the second aspect.
  • a computer program/program product is provided, wherein the computer program/program product is stored in a storage medium, and the computer program/program product is executed by at least one processor to implement the method as described in the first aspect.
  • a system comprising an encoding end and a decoding end, wherein the encoding end performs the steps of the method described in the first aspect, and the decoding end performs the steps of the method described in the second aspect.
  • the original feature map is compressed to obtain a first feature map; the first feature map is encoded to obtain a first code stream and a second feature map; the second feature map is enhanced to determine a third feature map; the residual feature map is encoded to obtain a second code stream.
  • the second code stream is obtained by encoding the residual generated by the original feature map during the compression process, thereby ensuring that in subsequent decoding operations, a reconstructed feature map with a smaller difference from the original feature map can be decoded.
  • the original feature map is compressed to obtain a first feature map with a smaller number of channels and/or spatial size, thereby effectively reducing the channel redundancy and spatial redundancy of the feature map, rather than reducing the pixel-level redundancy of the image as in the related art, thereby achieving effective compression of the feature map, thereby improving the compression efficiency of the feature map.
  • FIG1 is a schematic diagram of a flow chart of a feature graph processing method according to an embodiment of the present application.
  • FIG2 is a schematic diagram of a framework of a feature graph processing method provided in an embodiment of the present application.
  • FIG3 is one of the schematic diagrams of the framework of the first neural network provided in an embodiment of the present application.
  • FIG4 is a second schematic diagram of the framework of the first neural network provided in an embodiment of the present application.
  • FIG5 is one of the schematic diagrams of the framework of the second neural network provided in the embodiment of the present application.
  • FIG6 is a second schematic diagram of the framework of the second neural network provided in an embodiment of the present application.
  • FIG7 is a second flow chart of a feature graph processing method provided in an embodiment of the present application.
  • FIG8 is a structural diagram of an image processing device provided in an embodiment of the present application.
  • FIG9 is a second structural diagram of the image processing device provided in an embodiment of the present application.
  • FIG10 is a structural diagram of a communication device provided in an embodiment of the present application.
  • FIG. 11 is a schematic diagram of the hardware structure of the terminal provided in an embodiment of the present application.
  • first, second, etc. in the specification and claims of the present application are used to distinguish similar objects, and are not used to describe a specific order or sequence. It should be understood that the terms used in this way are interchangeable under appropriate circumstances, so that the embodiments of the present application can be implemented in an order other than those illustrated or described here, and the objects distinguished by “first” and “second” are generally of the same type, and the number of objects is not limited.
  • the first object can be one or more.
  • “and/or” in the specification and claims represents at least one of the connected objects, and the character “/" generally represents that the objects associated with each other are in an "or” relationship.
  • FIG1 is a flow chart of a feature graph processing method provided in an embodiment of the present application.
  • the feature graph processing method provided in this embodiment includes the following steps:
  • the encoding end compresses the original feature map to obtain a first feature map.
  • an optional implementation method is that the encoding end directly compresses the original feature map and extracts the feature map from the image, and the process of obtaining the original feature map is not performed at the encoding end.
  • the encoder uses a neural network model to extract feature maps from the image to obtain the original feature maps.
  • the above neural networks include but are not limited to convolutional neural networks (Feature Pyramid Networks, FPN), Fast Regional Convolutional Neural Networks (Region Convolutional Neural Networks, R-CNN), Mask R-CNN or YOLO model.
  • the image in Example 1 consists of 3 color channels and has a resolution of W ⁇ H, where W is the image width and H is the image height.
  • 4 original feature maps are obtained, namely P2, P3, P4, and P5.
  • the resolution corresponding to P2 is
  • the resolution corresponding to P3 is
  • the resolution corresponding to P4 is
  • the resolution corresponding to P5 is
  • the number of channels corresponding to the original feature maps is 256.
  • the OpenImage V6 dataset can be used as a training dataset for FPN.
  • the training method of the above FPN is end-to-end training, and the above FPN adopts the following loss function:
  • L represents the loss function corresponding to FPN
  • L cls represents the logarithmic loss
  • L reg represents the regression loss
  • N cls represents the number of calculation points of the logarithmic loss
  • N reg represents the number of calculation points of the regression loss
  • ⁇ represents the balance parameter
  • the value of the balance parameter is set to 10.
  • Example 2 uses the following loss function:
  • L represents the loss function corresponding to FPN
  • L cls represents the logarithmic loss
  • L reg represents the regression loss
  • N cls represents the number of calculation points of the logarithmic loss
  • N reg represents the number of calculation points of the regression loss
  • ⁇ represents the balance parameter
  • ⁇ represents the weighting parameter
  • L mse represents the root mean square error loss.
  • the value of the balance parameter is set to 10 and the weighting parameter is set to 1/50.
  • Example 2 by introducing the root mean square error loss, the loss in the encoding process can be reduced, and the code rate in the encoding process can be reduced.
  • the encoder compresses the original feature map to obtain a compact feature representation map, namely, the first feature map.
  • the number of channels of the first feature map is smaller than the number of channels of the original feature map, and/or the spatial size of the first feature map is smaller than the spatial size of the original feature map.
  • the encoding end encodes the first feature map to obtain a first code stream and a second feature map.
  • the first feature map can be encoded by a codec to obtain a first code stream and a reconstructed first feature map (ie, a second feature map).
  • codecs include but are not limited to the codecs involved in the Versatile Video Coding (VVC) standard, the codecs involved in the High Efficiency Video Coding (HEVC) standard or the codecs involved in the H.264 standard.
  • VVC Versatile Video Coding
  • HEVC High Efficiency Video Coding
  • S103 The encoding end performs enhancement processing on the second feature map to determine a third feature map.
  • the number of channels of the third feature map is equal to the number of channels of the original feature map
  • the spatial size of the third feature map is equal to the spatial size of the original feature map.
  • S104 The encoding end encodes the residual feature map to obtain a second bit stream.
  • the encoding end may perform a difference between the third feature map and the original feature map to obtain a residual feature map, and encode the residual feature map to obtain a second bitstream.
  • the feature maps may be normalized before being encoded using the codec.
  • val new represents the normalized value of the sample point
  • val ori represents the value of the sample point before normalization
  • the encoder will also encode the above maximum value (norm max ) and minimum value (norm min ) during the encoding process and transmit them to the decoder.
  • the encoded data is dispersed as much as possible, reducing the loss in the encoding process, thereby improving the encoding compression effect.
  • An optional implementation is that when FPN is used to extract feature maps of an image and the codec is a codec involved in the VVC standard, the codec uses different quantization parameters to encode the residual feature map and the first feature map. When encoding the first feature map, the quantization parameter is set to 22 to reduce the encoding loss at a low bit rate.
  • Another optional implementation is that when FPN is used to extract feature maps from an image and the codec is a codec involved in the VVC standard, the codec uses the same quantization parameter to encode the residual feature map and the first feature map.
  • the solid line in Figure 2 represents the data path of the feature map processing at the encoder.
  • the encoder compresses the original feature map through the compression network to obtain the first feature map; inputs the first feature map into the codec for encoding to generate the second feature map; and enhances the second feature map through the enhancement network. processing to generate residual information; generating a third feature map based on the residual information and the second feature map; subtracting the third feature map from the original feature map to obtain a residual feature map, and encoding the residual feature map to obtain a second bitstream.
  • the original feature map is compressed to obtain a first feature map; the first feature map is encoded to obtain a first code stream and a second feature map; the second feature map is enhanced to determine a third feature map; the residual feature map is encoded to obtain a second code stream.
  • the second code stream is obtained by encoding the residual generated by the original feature map during the compression process, thereby ensuring that in subsequent decoding operations, a reconstructed feature map with a smaller difference from the original feature map can be decoded.
  • the original feature map is compressed to obtain a first feature map with a smaller number of channels and/or spatial size, thereby effectively reducing the channel redundancy and spatial redundancy of the feature map, rather than reducing the pixel-level redundancy of the image as in the related art, thereby achieving effective compression of the feature map, thereby improving the compression efficiency of the feature map.
  • compressing the original feature map to obtain the first feature map includes:
  • the encoding end uses a first neural network to enhance the original feature map, and downsamples the original feature map to obtain a first feature map.
  • the first neural network can be used to enhance the original feature map, and the original feature map can be downsampled to obtain the first feature map.
  • the first neural network is also called a compression network.
  • first neural network downsamples the number of channels of the original feature map to reduce the number of channels of the original feature map to obtain the first feature map.
  • the first neural network downsamples the feature map width and feature map height of the original feature map to reduce the spatial size of the original feature map to obtain the first feature map.
  • the first neural network downsamples the number of channels, the feature map width, and the feature map height of the original feature map to reduce the number of channels and the spatial size of the original feature map to obtain a first feature map.
  • the original feature map is input into the first neural network, and the original feature map is transformed and enhanced through the reflection filling layer, convolution layer, residual module and ReLU activation layer in the first neural network to reduce the loss in the subsequent encoding process.
  • the number of the residual modules is set to 6, and the kernel size of the convolution layer is set to 7.
  • the enhanced original feature map is input to a downsampling module consisting of two convolutional layers to perform downsampling on the original feature map.
  • a downsampling module consisting of two convolutional layers to perform downsampling on the original feature map.
  • the width and height of the original feature map are downsampled to half of the original through the convolutional layer 1 in FIG3 ; and the number of channels of the original feature map is downsampled to half of the original through the convolutional layer 2 in FIG3 .
  • the downsampled original feature map is enhanced through the ReLU activation layer, residual module, reflection filling layer and convolution layer 3 in the first neural network, and the number of channels of the downsampled original feature map is reduced to 8 through convolution layer 3 to obtain the first feature map.
  • the original feature map is input into the first neural network, and the original feature map is converted and enhanced through the reflection filling layer, convolution layer, residual module and ReLU activation layer in the first neural network.
  • the enhanced original feature map is input into the downsampling module composed of two convolutional layers to downsample the original feature map. Sampling process.
  • the width and height of the original feature map are downsampled to half of the original width and height through the convolution layer 1 in FIG3 .
  • the downsampled original feature map is enhanced through the ReLU activation layer, residual module, reflection filling layer and convolution layer in the first neural network to obtain the first feature map.
  • the original feature map is compressed to obtain a first feature map with a smaller number of channels and/or spatial size, thereby effectively reducing the channel redundancy and spatial redundancy of the feature map, thereby achieving effective compression of the feature map and further improving the feature map compression efficiency.
  • the performing enhancement processing on the second feature map to determine the third feature map includes:
  • the encoding end performs enhancement processing on the second feature map to obtain residual information
  • the encoding end determines a third feature map based on the residual information and the second feature map.
  • the enhancement network can be used to enhance the second feature map to obtain residual information, which is used to characterize the compression processing residual corresponding to the second feature map, that is, the residual generated in the process of compressing the first feature map to generate the second feature map through the compression network.
  • residual information which is used to characterize the compression processing residual corresponding to the second feature map, that is, the residual generated in the process of compressing the first feature map to generate the second feature map through the compression network.
  • the residual information and the second feature map are added to obtain the third feature map.
  • the performing enhancement processing on the second feature map to obtain residual information includes:
  • the encoding end uses a second neural network to enhance the second feature map, and upsamples the second feature map to obtain residual information.
  • the second neural network can be used to enhance the second feature map, and the second feature map can be upsampled to obtain residual information.
  • the second neural network is also called an enhancement network.
  • performing upsampling processing on the number of channels of the second feature map includes:
  • the encoding end upsamples the number of channels of the second feature map through a convolutional layer so that the number of channels of the second feature map after the upsampling process is the same as the number of channels of the original feature map.
  • the number of channels of the residual information is the same as the number of channels of the original feature map.
  • the second neural network When the encoding end downsamples the number of channels of the original feature map, the second neural network upsamples the number of channels of the second feature map to obtain residual information.
  • the second neural network When the encoding end downsamples the feature map width and the feature map height of the original feature map, the second neural network upsamples the feature map width and the feature map height of the second feature map to obtain residual information.
  • the second neural network When the number of channels, feature map width and feature map height of the original feature map are downsampled at the encoding end, the second neural network upsamples the number of channels, feature map width and feature map height of the second feature map to obtain residual information.
  • Example 3 corresponds to the above Example 1, and the number of channels of the second feature map in Example 1 is 8.
  • the second feature map is input into the second neural network, and the second feature map is transformed and enhanced through the reflection filling layer, convolution layer 1, residual module and ReLU activation layer in the second neural network, and the number of channels of the second feature map is increased through convolution layer 1.
  • the number of the above residual modules is set to 6.
  • the enhanced second feature map is input into the convolution layer 2 to increase the number of channels of the second feature map.
  • the second feature map is enhanced through convolution layer 3, residual module, ReLU activation layer and reflection filling layer, and then the residual information is output through a convolution layer.
  • the above convolution layer 3 is used to increase the number of channels of the second feature map.
  • Example 3 corresponds to Example 2 above.
  • the second feature map is input into the second neural network, and the second feature map is linearly upsampled by the upsampling module in the second neural network, and the number of channels of the second feature map is increased by 2 times, so that the number of channels of the second feature map is the same as the number of channels of the original feature map.
  • the upsampled second feature map is enhanced by passing it through a subsequent reflection filling layer, a convolutional layer, a residual module, and a ReLU activation layer to obtain residual information.
  • FIG. 7 is a second flow chart of the feature graph processing method provided in the embodiment of the present application.
  • the feature graph processing method provided in this embodiment includes the following steps:
  • the decoding end decodes the target bitstream to obtain a second feature map and a residual feature map.
  • the target bitstream includes a first bitstream and a second bitstream.
  • the first bitstream is decoded to obtain a second feature map
  • the second bitstream is decoded to obtain a residual feature map.
  • S702 The decoding end performs enhancement processing on the second feature map to determine a third feature map.
  • the decoding end may perform enhancement processing on the second feature map to determine a third feature map, wherein the number of channels of the third feature map is greater than the number of channels of the second feature map, and/or the spatial size of the third feature map is greater than the spatial size of the second feature map.
  • the decoding end determines a reconstructed feature map based on the third feature map and the residual feature map.
  • the decoding end may add the third feature map and the residual feature map to obtain a reconstructed feature map.
  • the decoding end decodes the target code stream to obtain a second feature map, enhances the second feature map through an enhancement network, generates residual information, and generates a third feature map based on the residual information and the second feature map; based on the third feature map and the residual feature map obtained by decoding the target code stream, a reconstructed feature map is obtained.
  • the performing enhancement processing on the second feature map to determine the third feature map includes:
  • the decoding end performs enhancement processing on the second feature map to obtain residual information
  • the decoding end determines a third feature map based on the residual information and the second feature map.
  • the second feature map can be enhanced using an enhancement network to obtain residual information, which is used to characterize the compression processing residual corresponding to the second feature map, that is, the residual generated in the process of compressing the first feature map to generate the second feature map through the compression network. After obtaining the residual information, the residual information and the second feature map are added to obtain the third feature map.
  • the performing enhancement processing on the second feature map to obtain residual information includes:
  • the decoding end enhances the second feature map by using a second neural network, and performs upsampling processing on the second feature map to obtain residual information;
  • the upsampling of the second feature map includes at least one of the following:
  • the feature map width and the feature map height of the second feature map are upsampled.
  • the implementation method of the decoding end enhancing the second feature map is consistent with the implementation method of the encoding end enhancing the second feature map, and will not be repeated here.
  • the feature graph processing method provided in the embodiment of the present application can be executed by a feature graph processing device.
  • the feature graph processing device is applied to the encoding end to perform the feature graph processing method as an example to illustrate the feature graph processing device provided in the embodiment of the present application.
  • the embodiment of the present application further provides a feature graph processing device 800, including:
  • the first processing module 801 is used to compress the original feature map to obtain a first feature map
  • a first encoding module 802 configured to encode the first feature map to obtain a first code stream and a second feature map
  • a second processing module 803 is used to perform enhancement processing on the second feature map to determine a third feature map
  • the second encoding module 804 is used to encode the residual feature map to obtain a second bit stream.
  • the first processing module 801 is specifically configured to:
  • the original feature map is enhanced by using a first neural network, and the original feature map is downsampled to obtain a first feature map.
  • the second processing module 803 is specifically configured to:
  • a third feature map is determined.
  • the second processing module 803 is further specifically configured to:
  • the second feature map is enhanced by using a second neural network, and the second feature map is upsampled to obtain residual information.
  • the second processing module 803 is further specifically configured to:
  • the number of channels of the second feature map is upsampled through a convolutional layer so that the number of channels of the second feature map after the upsampling is the same as the number of channels of the original feature map.
  • the feature graph processing method provided in the embodiment of the present application can be executed by a feature graph processing device.
  • the feature graph processing device is applied to a decoding end to perform the feature graph processing method as an example to illustrate the feature graph processing device provided in the embodiment of the present application.
  • the embodiment of the present application further provides a feature graph processing device 900, including:
  • a decoding module 901 is used to decode a target bitstream to obtain a second feature map and a residual feature map;
  • a third processing module 902 is used to perform enhancement processing on the second feature map to determine a third feature map
  • the determination module 903 is used to determine a reconstructed feature map based on the third feature map and the residual feature map.
  • the third processing module 902 is specifically configured to:
  • a third feature map is determined.
  • the third processing module 902 is further specifically configured to:
  • the second feature map is enhanced by using a second neural network, and the second feature map is upsampled to obtain residual information.
  • the original feature map is compressed to obtain a first feature map; the first feature map is encoded to obtain a first code stream and a second feature map; the second feature map is enhanced to determine a third feature map; the residual feature map is encoded to obtain a second code stream.
  • the second code stream is obtained by encoding the residual generated by the original feature map during the compression process, thereby ensuring that in subsequent decoding operations, a reconstructed feature map with a smaller difference from the original feature map can be decoded.
  • the original feature map is compressed to obtain a first feature map with a smaller number of channels and/or spatial size, thereby effectively reducing the channel redundancy and spatial redundancy of the feature map, rather than reducing the pixel-level redundancy of the image as in the related art, thereby achieving effective compression of the feature map, thereby improving the compression efficiency of the feature map.
  • the feature graph processing device applied to the encoding end provided in the embodiment of the present application can implement each process implemented in the method embodiment of Figure 1 and achieve the same technical effect. To avoid repetition, it will not be repeated here.
  • the feature graph processing device applied to the decoding end provided in the embodiment of the present application can implement the various processes implemented in the method embodiment of Figure 7 and achieve the same technical effect. To avoid repetition, it will not be repeated here.
  • the feature graph processing device in the embodiment of the present application can be an electronic device, such as an electronic device with an operating system, or a component in an electronic device, such as an integrated circuit or a chip.
  • the electronic device can be a terminal, or it can be other devices other than a terminal.
  • the terminal can include but is not limited to the types of terminals listed above, and other devices can be servers, network attached storage (NAS), etc., which are not specifically limited in the embodiment of the present application.
  • an embodiment of the present application also provides a communication device 1000, including a processor 1001 and a memory 1002, and the memory 1002 stores programs or instructions that can be executed on the processor 1001.
  • the communication device 1000 is a terminal
  • the program or instructions are executed by the processor 1001 to implement the various steps of the above-mentioned feature map processing method embodiment and can achieve the same technical effect.
  • the embodiment of the present application further provides a terminal, including a processor 1001 and a communication interface, where the processor 1001 is configured to perform the following operations:
  • the residual feature map is determined based on the original feature map and the third feature map.
  • the processor 1001 is configured to perform the following operations:
  • a reconstructed feature map is determined.
  • the terminal embodiment corresponds to the above-mentioned terminal side method embodiment, and each implementation process and implementation mode of the above-mentioned method embodiment can be applied to the terminal embodiment and can achieve the same technical effect.
  • Figure 11 is a schematic diagram of the hardware structure of a terminal implementing the embodiment of the present application.
  • the terminal 1100 includes but is not limited to: a radio frequency unit 1101, a network module 1102, an audio output unit 1103, an input unit 1104, a sensor 1105, a display unit 1106, a user input unit 1107, an interface unit 1108, a memory 1109, and a processor 1110 and other components.
  • the terminal 1100 may also include a power source (such as a battery) for supplying power to each component, and the power source may be logically connected to the processor 1110 through a power management system, so as to implement functions such as managing charging, discharging, and power consumption management through the power management system.
  • a power source such as a battery
  • the terminal structure shown in FIG11 does not constitute a limitation on the terminal, and the terminal may include more or fewer components than shown in the figure, or combine certain components, or arrange components differently, which will not be described in detail here.
  • the input unit 1104 may include a graphics processing unit (GPU) 11041 and a microphone 11042, and the graphics processor 11041 processes the image data of the static picture or video obtained by the image capture device (such as a camera) in the video capture mode or the image capture mode.
  • the display unit 1106 may include a display panel 11061, and the display panel 11061 may be configured in the form of a liquid crystal display, an organic light emitting diode, etc.
  • the user input unit 1107 includes a touch panel 11071 and at least one of other input devices 11072.
  • the touch panel 11071 is also called a touch screen.
  • the touch panel 11071 may include two parts: a touch detection device and a touch controller.
  • Other input devices 11072 may include, but are not limited to, a physical keyboard, function keys (such as a volume control key, a switch key, etc.), a trackball, a mouse, and a joystick, which will not be repeated here.
  • the RF unit 1101 can transmit the data to the processor 1110 for processing; the RF unit 1101 can send uplink data to the network side device.
  • the RF unit 1101 includes but is not limited to an antenna, an amplifier, a transceiver, a coupler, a low noise amplifier, a duplexer, etc.
  • the memory 1109 can be used to store software programs or instructions and various data.
  • the memory 1109 may mainly include a first storage area for storing programs or instructions and a second storage area for storing data, wherein the first storage area may store an operating system, an application program or instruction required for at least one function (such as a sound playback function, an image playback function, etc.), etc.
  • the memory 1109 may include a volatile memory or a non-volatile memory, or the memory 1109 may include both volatile and non-volatile memories.
  • the non-volatile memory may be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or an electrically erasable programmable read-only memory (EEPROM). Or flash memory.
  • ROM read-only memory
  • PROM programmable read-only memory
  • EPROM erasable programmable read-only memory
  • EEPROM electrically erasable programmable read-only memory
  • EEPROM electrically erasable programmable read-only memory
  • Volatile memory can be random access memory (Random Access Memory, RAM), static random access memory (Static RAM, SRAM), dynamic random access memory (Dynamic RAM, DRAM), synchronous dynamic random access memory (Synchronous DRAM, SDRAM), double data rate synchronous dynamic random access memory (Double Data Rate SDRAM, DDRSDRAM), enhanced synchronous dynamic random access memory (Enhanced SDRAM, ESDRAM), synchronous link dynamic random access memory (Synch link DRAM, SLDRAM) and direct memory bus random access memory (Direct Rambus RAM, DRRAM).
  • RAM Random Access Memory
  • SRAM static random access memory
  • DRAM dynamic random access memory
  • DRAM synchronous dynamic random access memory
  • SDRAM double data rate synchronous dynamic random access memory
  • Double Data Rate SDRAM Double Data Rate SDRAM
  • ESDRAM enhanced synchronous dynamic random access memory
  • Synch link DRAM, SLDRAM synchronous link dynamic random access memory
  • Direct Rambus RAM Direct Rambus RAM
  • the memory 1109 in the embodiment of the present application includes but is
  • the processor 1110 may include one or more processing units; optionally, the processor 1110 integrates an application processor and a modem processor, wherein the application processor mainly processes operations related to an operating system, a user interface, and application programs, and the modem processor mainly processes wireless communication signals, such as a baseband processor. It is understandable that the modem processor may not be integrated into the processor 1110.
  • the processor 1110 is configured to perform the following operations:
  • the residual feature map is determined based on the original feature map and the third feature map.
  • processor 1110 is configured to perform the following operations:
  • a reconstructed feature map is determined.
  • An embodiment of the present application also provides a readable storage medium, on which a program or instruction is stored.
  • a program or instruction is stored.
  • the various processes of the above-mentioned feature map processing method embodiment are implemented, and the same technical effect can be achieved. To avoid repetition, it will not be repeated here.
  • the processor is the processor in the terminal described in the above embodiment.
  • the readable storage medium includes a computer readable storage medium, such as a computer read-only memory ROM, a random access memory RAM, a magnetic disk or an optical disk.
  • An embodiment of the present application further provides a chip, which includes a processor and a communication interface, wherein the communication interface is coupled to the processor, and the processor is used to run programs or instructions to implement the various processes of the above-mentioned feature graph method embodiment, and can achieve the same technical effect. To avoid repetition, it will not be repeated here.
  • the chip mentioned in the embodiments of the present application can also be called a system-level chip, a system chip, a chip system or a system-on-chip chip, etc.
  • the embodiments of the present application further provide a computer program/program product, which is stored in a storage medium and is executed by at least one processor to implement the various processes of the above-mentioned feature graph method embodiment and can achieve the same technical effect. To avoid repetition, it will not be repeated here.
  • An embodiment of the present application further provides a system, which includes an encoding end and a decoding end.
  • the encoding end executes each process of the embodiment of the feature map processing method applied to the encoding end
  • the decoding end executes each process of the embodiment of the feature map processing method applied to the decoding end, and can achieve the same technical effect. To avoid repetition, it will not be repeated here.
  • the technical solution of the present application can be embodied in the form of a computer software product, which is stored in a storage medium (such as ROM/RAM, a magnetic disk, or an optical disk), and includes a number of instructions for enabling a terminal (which can be a mobile phone, a computer, a server, an air conditioner, or a network device, etc.) to execute the methods described in each embodiment of the present application.
  • a storage medium such as ROM/RAM, a magnetic disk, or an optical disk
  • a terminal which can be a mobile phone, a computer, a server, an air conditioner, or a network device, etc.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Physics & Mathematics (AREA)
  • Algebra (AREA)
  • General Physics & Mathematics (AREA)
  • Mathematical Analysis (AREA)
  • Mathematical Optimization (AREA)
  • Pure & Applied Mathematics (AREA)
  • Compression Or Coding Systems Of Tv Signals (AREA)
  • Image Analysis (AREA)
  • Image Processing (AREA)

Abstract

本申请公开了一种特征图处理方法、装置及设备，属于图像压缩技术领域。上述特征图处理方法包括：对原始特征图进行压缩处理，获得第一特征图；第一特征图的通道数小于原始特征图的通道数，和/或第一特征图的空间尺寸小于原始特征图的空间尺寸；编码第一特征图，获得第一码流和第二特征图；对第二特征图进行增强处理，确定第三特征图；第三特征图的通道数等于原始特征图的通道数，第三特征图的空间尺寸等于原始特征图的空间尺寸；编码残差特征图，获得第二码流；残差特征图基于原始特征图和第三特征图确定。

Description

特征图处理方法、装置及设备
相关申请的交叉引用
本申请主张在2022年10月14日在中国提交的中国专利申请No.202211261744.X的优先权，其全部内容通过引用包含于此。
技术领域
本申请属于图像压缩技术领域，具体涉及一种特征图处理方法、装置及设备。
背景技术
随着人工智能技术的蓬勃发展，在自动驾驶、智能工业、沉浸式娱乐等领域中，大部分视频流量由终端处理，由终端执行机器视觉任务。在进行机器视觉任务的过程中，需要对图像的特征图进行压缩，达到降低数据量的目的。
然而，相关的压缩方法是利用空间相关性和时间相关性对图像进行压缩，减少图像的时空像素级冗余，这种压缩方法并不能实现对特征图的有效压缩，进而降低了特征图压缩效率。
发明内容
本申请实施例提供一种特征图处理方法、装置及设备，能够解决现有方案并不能实现对特征图的有效压缩，进而降低了特征图压缩效率的问题。
第一方面，提供了一种特征图处理方法，包括：
编码端对原始特征图进行压缩处理，获得第一特征图；所述第一特征图的通道数小于所述原始特征图的通道数，和/或所述第一特征图的空间尺寸小于所述原始特征图的空间尺寸；
所述编码端编码所述第一特征图，获得第一码流和第二特征图；
所述编码端对所述第二特征图进行增强处理，确定第三特征图；所述第三特征图的通道数等于所述原始特征图的通道数，所述第三特征图的空间尺寸等于所述原始特征图的空间尺寸；
所述编码端编码残差特征图，获得第二码流；所述残差特征图基于所述原始特征图和所述第三特征图确定。
第二方面，提供一种特征图处理方法，包括：
解码端解码目标码流，获得第二特征图和残差特征图，所述目标码流包括第一码流和 第二码流，所述第二特征图基于所述第一码流确定，所述残差特征图基于所述第二码流确定；
所述解码端对所述第二特征图进行增强处理，确定第三特征图；所述第三特征图的通道数大于所述第二特征图的通道数，和/或所述第三特征图的空间尺寸大于所述第二特征图的空间尺寸；
所述解码端基于所述第三特征图和所述残差特征图，确定重建特征图。
第三方面，提供了一种特征图处理装置，应用于编码端，所述装置包括：
第一处理模块，用于对原始特征图进行压缩处理，获得第一特征图；所述第一特征图的通道数小于所述原始特征图的通道数，和/或所述第一特征图的空间尺寸小于所述原始特征图的空间尺寸；
第一编码模块，用于编码所述第一特征图，获得第一码流和第二特征图；
第二处理模块，用于对所述第二特征图进行增强处理，确定第三特征图；所述第三特征图的通道数等于所述原始特征图的通道数，所述第三特征图的空间尺寸等于所述原始特征图的空间尺寸；
第二编码模块，用于编码残差特征图，获得第二码流；所述残差特征图基于所述原始特征图和所述第三特征图确定。
第四方面，提供了一种图像处理装置，应用于解码端，所述装置包括：
解码模块，用于解码目标码流，获得第二特征图和残差特征图，所述目标码流包括第一码流和第二码流，所述第二特征图基于所述第一码流确定，所述残差特征图基于所述第二码流确定；
第三处理模块，用于对所述第二特征图进行增强处理，确定第三特征图；所述第三特征图的通道数大于所述第二特征图的通道数，和/或所述第三特征图的空间尺寸大于所述第二特征图的空间尺寸；
确定模块，用于基于所述第三特征图和所述残差特征图，确定重建特征图。
第五方面，提供了一种终端，该终端包括处理器和存储器，所述存储器存储可在所述处理器上运行的程序或指令，所述程序或指令被所述处理器执行时实现如第一方面所述的方法的步骤，或者实现如第二方面所述的方法的步骤。
第六方面，提供了一种可读存储介质，所述可读存储介质上存储程序或指令，所述程序或指令被处理器执行时实现如第一方面所述的方法的步骤，或者实现如第二方面所述的方法的步骤。
第七方面，提供了一种芯片，所述芯片包括处理器和通信接口，所述通信接口和所述处理器耦合，所述处理器用于运行程序或指令，实现如第一方面所述的方法，或者实现如第二方面所述的方法。
第八方面，提供了一种计算机程序/程序产品，所述计算机程序/程序产品被存储在存储介质中，所述计算机程序/程序产品被至少一个处理器执行以实现如第一方面所述的方 法的步骤，或者实现如第二方面所述的方法的步骤。
第九方面，提供了一种系统，所述系统包括编码端和解码端，所述编码端执行如第一方面所述的方法的步骤，所述解码端执行如第二方面所述的方法的步骤。
本申请实施例中，对原始特征图进行压缩处理，获得第一特征图；编码第一特征图，获得第一码流和第二特征图；对第二特征图进行增强处理，确定第三特征图；编码残差特征图，获得第二码流。通过对原始特征图在压缩处理过程中产生的残差进行编码，得到第二码流，进而保证在后续的解码操作中，可以解码获得与原始特征图差异更小的重建特征图。本申请实施例中，通过对原始特征图进行压缩处理，得到通道数和/或空间尺寸更小的第一特征图，有效减少特征图的通道冗余和空间冗余，而并非像相关技术中那样减少图像的像素级冗余，以此实现对特征图的有效压缩，进而提高特征图压缩效率。
附图说明
图1是本申请实施例提供的特征图处理方法的流程示意图之一；
图2是本申请实施例提供的特征图处理方法的框架示意图；
图3是本申请实施例提供的第一神经网络的框架示意图之一；
图4是本申请实施例提供的第一神经网络的框架示意图之二；
图5是本申请实施例提供的第二神经网络的框架示意图之一；
图6是本申请实施例提供的第二神经网络的框架示意图之二；
图7是本申请实施例提供的特征图处理方法的流程示意图之二；
图8是本申请实施例提供的图像处理装置的结构图之一；
图9是本申请实施例提供的图像处理装置的结构图之二；
图10是本申请实施例提供的通信设备的结构图；
图11是本申请实施例提供的终端的硬件结构示意图。
具体实施方式
下面将结合本申请实施例中的附图，对本申请实施例中的技术方案进行清楚描述，显然，所描述的实施例是本申请一部分实施例，而不是全部的实施例。基于本申请中的实施例，本领域普通技术人员所获得的所有其他实施例，都属于本申请保护的范围。
本申请的说明书和权利要求书中的术语“第一”、“第二”等是用于区别类似的对象，而不用于描述特定的顺序或先后次序。应该理解这样使用的术语在适当情况下可以互换，以便本申请的实施例能够以除了在这里图示或描述的那些以外的顺序实施，且“第一”、“第二”所区别的对象通常为一类，并不限定对象的个数，例如第一对象可以是一个，也可以是多个。此外，说明书以及权利要求中“和/或”表示所连接对象的至少其中之一，字符“/”一般表示前后关联对象是一种“或”的关系。
下面结合附图，通过一些实施例及其应用场景对本申请实施例提供的应用于编码端的特征图处理方法进行详细地说明。
请参阅图1，图1是本申请实施例提供的特征图处理方法的流程图之一。本实施例提供的特征图处理方法包括以下步骤：
S101，编码端对原始特征图进行压缩处理，获得第一特征图。
本步骤中，一种可选地实施方式为，编码端直接对原始特征图进行压缩处理，对图像进行特征图提取，得到原始特征图的过程并不在编码端执行。
另一种可选地实施方式为，编码端使用神经网络模型对图像进行特征图提取，得到原始特征图。上述神经网络包括但不限于卷积神经网络(Feature Pyramid Networks，FPN)、Fast区域卷积神经网络(Region Convolutional Neural Networks，R-CNN)、Mask R-CNN或YOLO模型。
以下以使用FPN提取原始特征图为示例进行说明：
示例1：
示例1中的图像由3个颜色通道组成，且分辨率为W×H，其中，W为图像宽度，H为图像高度。使用FPN对第一图像进行特征提取，得到4个原始特征图，分别为P2、P3、P4和P5。其中，P2对应的分辨率为P3对应的分辨率为P4对应的分辨率为P5对应的分辨率为原始特征图对应的通道数均为256。
可选地，可以使用OpenImage V6数据集作为FPN的训练数据集，上述FPN的训练方式为端到端的训练，且上述FPN采用以下损失函数：
其中，L表示FPN对应的损失函数，Lcls表示对数损失，Lreg表示回归损失，Ncls表示对数损失的计算点的个数，Nreg表示回归损失的计算点的个数，λ表示平衡参数，可选地，将平衡参数的数值设置为10。
示例2：
示例2与示例1的区别仅在于，示例2中的FPN采用以下损失函数：
其中，L表示FPN对应的损失函数，Lcls表示对数损失，Lreg表示回归损失，Ncls表示对数损失的计算点的个数，Nreg表示回归损失的计算点的个数，λ表示平衡参数，γ表示加权参数，Lmse表示均方根误差损失。可选地，将平衡参数的数值设置为10，将加权参数设置为1/50。
在示例2中，通过引入均方根误差损失，可以减少编码过程中的损失，并且可以降低编码过程中的码率。
本步骤中，编码端对原始特征图进行压缩处理，获得紧凑特征表示图，即第一特征图。 其中，第一特征图的通道数小于原始特征图的通道数，和/或第一特征图的空间尺寸小于原始特征图的空间尺寸。具体的如何对原始特征图进行压缩处理的实施方式，请参阅后续实施例。
S102，所述编码端编码所述第一特征图，获得第一码流和第二特征图。
本步骤中，在得到第一特征图后，可以通过编解码器对第一特征图进行编码，得到第一码流和重建后的第一特征图(即第二特征图)。
应理解，上述编解码器包括但不限于通用视频编码(Versatile Video Coding，VVC)标准涉及的编解码器、高效视频编码(High Efficiency Video Coding，HEVC)标准涉及的编解码器或者H.264标准涉及的编解码器。
S103，所述编码端对所述第二特征图进行增强处理，确定第三特征图。
上述第三特征图的通道数等于原始特征图的通道数，且第三特征图的空间尺寸等于原始特征图的空间尺寸。具体的如何对第二特征图进行增强处理确定第三特征图的实施方式，请参阅后续实施例。
S104，所述编码端编码残差特征图，获得第二码流。
本步骤中，编码端可以将第三特征图和原始特征图做差，获得残差特征图，并编码残差特征图，获得第二码流。
可选地，在使用编解码器对特征图进行编码之前，可以对特征图进行归一化处理。
具体而言，神经网络编码器对于每次接收到的同一批次(batch)的特征图数据，统计同一batch数据内特征图数据的最大值(normmax)和最小值(normmin)，并通过以下公式对特征图进行归一化处理
valnew＝(valori-normmin)/(normmax-normmin)
其中，valnew表示样本点归一化后的值，valori表示样本点归一化前的值。
应理解，编码端在编码过程中还会对上述最大值(normmax)和最小值(normmin)进行编码，并传输至解码端。通过对特征图进行归一化处理，在后续的对特征图编码压缩的过程中，使得编码数据尽可能分散，减少编码过程中的损失，进而提升编码压缩效果。
一种可选地实施方式为，在使用FPN对图像进行特征图提取，且编解码器为VVC标准涉及的编解码器的情况下，编解码器使用不同的量化参数对残差特征图和第一特征图进行编码，在对第一特征图进行编码时，设置量化参数为22，以降低在低码率下的编码损失。
另一种可选地实施方式为，在使用FPN对图像进行特征图提取，且编解码器为VVC标准涉及的编解码器的情况下，编解码器使用相同的量化参数对残差特征图和第一特征图进行编码。
为便于理解，请参阅图2，图2中的实线表示编码端的特征图处理的数据路径。如图2所示，编码端通过压缩网络对原始特征图进行压缩处理，获得第一特征图；将第一特征图输入至编解码器中进行编码，生成第二特征图；通过增强网络对第二特征图进行增强处 理，生成残差信息；基于残差信息和第二特征图生成第三特征图；将第三特征图和原始特征图做差，获得残差特征图，并编码残差特征图，获得第二码流。
本申请实施例中，对原始特征图进行压缩处理，获得第一特征图；编码第一特征图，获得第一码流和第二特征图；对第二特征图进行增强处理，确定第三特征图；编码残差特征图，获得第二码流。通过对原始特征图在压缩处理过程中产生的残差进行编码，得到第二码流，进而保证在后续的解码操作中，可以解码获得与原始特征图差异更小的重建特征图。本申请实施例中，通过对原始特征图进行压缩处理，得到通道数和/或空间尺寸更小的第一特征图，有效减少特征图的通道冗余和空间冗余，而并非像相关技术中那样减少图像的像素级冗余，以此实现对特征图的有效压缩，进而提高特征图压缩效率。
可选地，所述对原始特征图进行压缩处理，获得第一特征图包括：
所述编码端利用第一神经网络增强所述原始特征图，以及对所述原始特征图进行下采样处理，获得第一特征图。
本实施例中，可以使用第一神经网络增强原始特征图，并对原始特征图进行下采样处理，获得第一特征图。其中，上述第一神经网络又称为压缩网络。
一种可选地实施方式为，第一神经网络对原始特征图的通道数进行下采样处理，降低原始特征图的通道数，得到第一特征图。
另一种可选地实施方式为，第一神经网络对原始特征图的特征图宽度以及特征图高度进行下采样处理，降低原始特征图的空间尺寸，得到第一特征图。
另一种可选地实施方式为，第一神经网络对原始特征图的通道数，和特征图宽度以及特征图高度进行下采样处理，降低原始特征图的通道数和空间尺寸，得到第一特征图。
为便于理解本实施例，以下结合图3和图4通过2个示例进行阐述。
示例1：
在图3示出的应用场景中，将原始特征图输入至第一神经网络中，通过第一神经网络中的反射填充层、卷积层和残差模块和ReLU激活层对原始特征图进行转换并增强，以降低后续编码过程中的损失。可选地，设置上述残差模块的数量为6，卷积层的核尺寸为7。
将增强后的原始特征图输入至由2个卷积层构成的下采样模块，对原始特征图进行下采样处理。可选地，通过图3中的卷积层1将原始特征图的宽度和高度下采样为原来的二分之一；通过图3中的卷积层2将原始特征图的通道数下采样为原来的二分之一。
通过第一神经网络中的ReLU激活层、残差模块、反射填充层和卷积层3对下采样后的原始特征图进行增强，并通过卷积层3将下采样后的原始特征图的通道数降为8，以此得到第一特征图。
示例2：
在图4示出的应用场景中，将原始特征图输入至第一神经网络中，通过第一神经网络中的反射填充层、卷积层和残差模块和ReLU激活层对原始特征图进行转换并增强。
将增强后的原始特征图输入至由2个卷积层构成的下采样模块，对原始特征图进行下 采样处理。可选地，通过图3中的卷积层1将原始特征图的宽度和高度下采样为原来的二分之一。
通过第一神经网络中的ReLU激活层、残差模块、反射填充层和卷积层对下采样后的原始特征图进行增强，以此得到第一特征图。
应理解，第一神经网络对原始特征图进行空间下采样的倍数和通道下采样的倍数在此不做具体限定。
本实施例中，通过对原始特征图进行压缩处理，得到通道数和/或空间尺寸更小的第一特征图，有效减少特征图的通道冗余和空间冗余，以此实现对特征图的有效压缩，进而提高特征图压缩效率。
可选地，所述对所述第二特征图进行增强处理，确定第三特征图包括：
所述编码端对所述第二特征图进行增强处理，获得残差信息；
所述编码端基于所述残差信息和所述第二特征图，确定第三特征图。
本实施例中，可以利用增强网络对第二特征图进行增强处理，获得残差信息，上述残差信息用于表征第二特征图对应的压缩处理残差，即通过压缩网络对第一特征图进行压缩处理生成第二特征图过程中，产生的残差。具体的如何对第二特征图进行增强处理的实施方式，请参阅后续实施例。
在得到残差信息后，对残差信息和第二特征图相加，得到第三特征图。
可选地，所述对所述第二特征图进行增强处理，获得残差信息包括：
所述编码端利用第二神经网络增强所述第二特征图，以及对所述第二特征图进行上采样处理，获得残差信息。
本实施例中，可以使用第二神经网络增强第二特征图，并对第二特征图进行上采样处理，获得残差信息。其中，上述第二神经网络又称为增强网络。
可选地，对所述第二特征图的通道数进行上采样处理包括：
所述编码端通过卷积层对所述第二特征图的通道数进行上采样处理，以使得上采样处理后的第二特征图的通道数与所述原始特征图的通道数相同。
需要说明的是，残差信息的通道数与原始特征图的通道数相同。
在编码端对原始特征图的通道数进行下采样处理的情况下，第二神经网络对第二特征图的通道数进行上采样处理，获得残差信息。
在编码端对原始特征图的特征图宽度以及特征图高度进行下采样的情况下，第二神经网络对第二特征图的特征图宽度以及特征图高度进行上采样处理，获得残差信息。
在编码端对原始特征图的通道数，和特征图宽度以及特征图高度进行下采样处理的情况下，第二神经网络对第二特征图的通道数、特征图宽度以及特征图高度进行上采样处理，获得残差信息。
为便于理解本实施例，以下结合图5和图6通过2个示例进行阐述。
示例3：
需要说明的是，示例3与上述示例1对应，示例1中的第二特征图的通道数为8。
在图5示出的应用场景中，将第二特征图输入至第二神经网络中，通过第二神经网络中的反射填充层、卷积层1和残差模块和ReLU激活层对第二特征图进行转换并增强，并通过卷积层1增加第二特征图的通道数。可选地，设置上述残差模块的数量为6。
将增强后的第二特征图输入至卷积层2中，增加第二特征图的通道数。
通过卷积层3、残差模块、ReLU激活层和反射填充层对第二特征图进行增强，再通过一层卷积层输出残差信息。其中，上述卷积层3用于增加第二特征图的通道数。
示例4：
需要说明的是，示例3与上述示例2对应。
在图6示出的应用场景中，将第二特征图输入至第二神经网络中，通过第二神经网络中的上采样模块对第二特征图进行线性上采样，将第二特征图的通道数增加2倍，使得第二特征图的通道数与原始特征图的通道数相同。
将上采样后的第二特征图通过后续的反射填充层、卷积层和残差模块和ReLU激活层，增强第二特征图，进而得到残差信息。
下面结合附图，通过一些实施例及其应用场景对本申请实施例提供的应用于解码端的特征图处理方法进行详细地说明。
请参阅图7，图7是本申请实施例提供的特征图处理方法的流程图之二。本实施例提供的特征图处理方法包括以下步骤：
S701，解码端解码目标码流，获得第二特征图和残差特征图。
上述目标码流包括第一码流和第二码流。本步骤中，解码第一码流获得第二特征图，解码第二码流获得残差特征图。
S702，所述解码端对所述第二特征图进行增强处理，确定第三特征图。
本步骤中，解码端可以对第二特征图进行增强处理，确定第三特征图。其中，第三特征图的通道数大于第二特征图的通道数，和/或第三特征图的空间尺寸大于第二特征图的空间尺寸。
具体的如何对第二特征图进行增强处理的实施方式，请参阅后续实施例。
S703，所述解码端基于所述第三特征图和所述残差特征图，确定重建特征图。
本步骤中，解码端可以将上述第三特征图残差特征图相加，获得重建特征图。
为便于理解，请参阅图2，图2中的虚线表示编码端的特征图处理的数据路径。
如图2所示，解码端解码目标码流，获得第二特征图，通过增强网络对第二特征图进行增强处理，生成残差信息基于残差信息和第二特征图生成第三特征图；基于第三特征图和解码目标码流得到的残差特征图，得到重建特征图。
可选地，所述对所述第二特征图进行增强处理，确定第三特征图包括：
所述解码端对所述第二特征图进行增强处理，获得残差信息；
所述解码端基于所述残差信息和所述第二特征图，确定第三特征图。
本实施例中，可以利用增强网络对第二特征图进行增强处理，获得残差信息，上述残差信息用于表征第二特征图对应的压缩处理残差，即通过压缩网络对第一特征图进行压缩处理生成第二特征图过程中，产生的残差。在得到残差信息后，对残差信息和第二特征图相加，得到第三特征图。
可选地，所述对所述第二特征图进行增强处理，获得残差信息包括：
所述解码端利用第二神经网络增强所述第二特征图，以及对所述第二特征图进行上采样处理，获得残差信息；
其中，对第二特征图进行上采样处理包括以下至少一项：
对所述第二特征图的通道数进行上采样处理；
对所述第二特征图的特征图宽度以及特征图高度进行上采样处理。
解码端对第二特征图进行增强处理的实施方式，与编码端对第二特征图进行增强处理的实施方式一致，在此不做重复阐述。
本申请实施例提供的特征图处理方法，执行主体可以为特征图处理装置。本申请实施例中以特征图处理装置应用于编码端执行特征图处理方法为例，说明本申请实施例提供的特征图处理装置。
如图8所示，本申请实施例还提供了一种特征图处理装置800，包括：
第一处理模块801，用于对原始特征图进行压缩处理，获得第一特征图；
第一编码模块802，用于编码所述第一特征图，获得第一码流和第二特征图；
第二处理模块803，用于对所述第二特征图进行增强处理，确定第三特征图；
第二编码模块804，用于编码残差特征图，获得第二码流。
可选地，所述第一处理模块801，具体用于：
利用第一神经网络增强所述原始特征图，以及对所述原始特征图进行下采样处理，获得第一特征图。
可选地，所述第二处理模块803，具体用于：
对所述第二特征图进行增强处理，获得残差信息；
基于所述残差信息和所述第二特征图，确定第三特征图。
可选地，所述第二处理模块803，还具体用于：
利用第二神经网络增强所述第二特征图，以及对所述第二特征图进行上采样处理，获得残差信息。
可选地，所述第二处理模块803，还具体用于：
通过卷积层对所述第二特征图的通道数进行上采样处理，以使得上采样处理后的第二特征图的通道数与所述原始特征图的通道数相同。
本申请实施例提供的特征图处理方法，执行主体可以为特征图处理装置。本申请实施例中以特征图处理装置应用于解码端执行特征图处理方法为例，说明本申请实施例提供的特征图处理装置。
如图9所示，本申请实施例还提供了一种特征图处理装置900，包括：
解码模块901，用于解码目标码流，获得第二特征图和残差特征图；
第三处理模块902，用于对所述第二特征图进行增强处理，确定第三特征图；
确定模块903，用于基于所述第三特征图和所述残差特征图，确定重建特征图。
可选地，所述第三处理模块902，具体用于：
对所述第二特征图进行增强处理，获得残差信息；
基于所述残差信息和所述第二特征图，确定第三特征图。
可选地，所述第三处理模块902，还具体用于：
利用第二神经网络增强所述第二特征图，以及对所述第二特征图进行上采样处理，获得残差信息。
本申请实施例中，对原始特征图进行压缩处理，获得第一特征图；编码第一特征图，获得第一码流和第二特征图；对第二特征图进行增强处理，确定第三特征图；编码残差特征图，获得第二码流。通过对原始特征图在压缩处理过程中产生的残差进行编码，得到第二码流，进而保证在后续的解码操作中，可以解码获得与原始特征图差异更小的重建特征图。本申请实施例中，通过对原始特征图进行压缩处理，得到通道数和/或空间尺寸更小的第一特征图，有效减少特征图的通道冗余和空间冗余，而并非像相关技术中那样减少图像的像素级冗余，以此实现对特征图的有效压缩，进而提高特征图压缩效率。
本申请实施例提供的应用于编码端的特征图处理装置能够实现图1的方法实施例实现的各个过程，并达到相同的技术效果，为避免重复，这里不再赘述。
本申请实施例提供的应用于解码端的特征图处理装置能够实现图7的方法实施例实现的各个过程，并达到相同的技术效果，为避免重复，这里不再赘述。
本申请实施例中的特征图处理装置可以是电子设备，例如具有操作系统的电子设备，也可以是电子设备中的部件、例如集成电路或芯片。该电子设备可以是终端，也可以为除终端之外的其他设备。示例性的，终端可以包括但不限于上述所列举的终端的类型，其他设备可以为服务器、网络附属存储器(Network Attached Storage，NAS)等，本申请实施例不作具体限定。
可选地，如图10所示，本申请实施例还提供一种通信设备1000，包括处理器1001和存储器1002，存储器1002上存储有可在所述处理器1001上运行的程序或指令，例如，该通信设备1000为终端时，该程序或指令被处理器1001执行时实现上述特征图处理方法实施例的各个步骤，且能达到相同的技术效果。
本申请实施例还提供一种终端，包括处理器1001和通信接口，处理器1001用于执行以下操作：
对原始特征图进行压缩处理，获得第一特征图；
编码所述第一特征图，获得第一码流和第二特征图；
对所述第二特征图进行增强处理，确定第三特征图；
编码残差特征图，获得第二码流；所述残差特征图基于所述原始特征图和所述第三特征图确定。
或者，处理器1001用于执行以下操作：
解码目标码流，获得第二特征图和残差特征图；
对所述第二特征图进行增强处理，确定第三特征图；
基于所述第三特征图和所述残差特征图，确定重建特征图。
该终端实施例与上述终端侧方法实施例对应，上述方法实施例的各个实施过程和实现方式均可适用于该终端实施例中，且能达到相同的技术效果。具体地，图11为实现本申请实施例的一种终端的硬件结构示意图。
该终端1100包括但不限于：射频单元1101、网络模块1102、音频输出单元1103、输入单元1104、传感器1105、显示单元1106、用户输入单元1107、接口单元1108、存储器1109、以及处理器1110等部件。
本领域技术人员可以理解，终端1100还可以包括给各个部件供电的电源(比如电池)，电源可以通过电源管理系统与处理器1110逻辑相连，从而通过电源管理系统实现管理充电、放电、以及功耗管理等功能。图11中示出的终端结构并不构成对终端的限定，终端可以包括比图示更多或更少的部件，或者组合某些部件，或者不同的部件布置，在此不再赘述。
应理解的是，本申请实施例中，输入单元1104可以包括图形处理器(Graphics Processing Unit，GPU)11041和麦克风11042，图形处理器11041对在视频捕获模式或图像捕获模式中由图像捕获装置(如摄像头)获得的静态图片或视频的图像数据进行处理。显示单元1106可包括显示面板11061，可以采用液晶显示器、有机发光二极管等形式来配置显示面板11061。用户输入单元1107包括触控面板11071以及其他输入设备11072中的至少一种。触控面板11071，也称为触摸屏。触控面板11071可包括触摸检测装置和触摸控制器两个部分。其他输入设备11072可以包括但不限于物理键盘、功能键(比如音量控制按键、开关按键等)、轨迹球、鼠标、操作杆，在此不再赘述。
本申请实施例中，射频单元1101接收来自网络侧设备的下行数据后，可以传输给处理器1110进行处理；射频单元1101可以向网络侧设备发送上行数据。通常，射频单元1101包括但不限于天线、放大器、收发信机、耦合器、低噪声放大器、双工器等。
存储器1109可用于存储软件程序或指令以及各种数据。存储器1109可主要包括存储程序或指令的第一存储区和存储数据的第二存储区，其中，第一存储区可存储操作系统、至少一个功能所需的应用程序或指令(比如声音播放功能、图像播放功能等)等。此外，存储器1109可以包括易失性存储器或非易失性存储器，或者，存储器1109可以包括易失性和非易失性存储器两者。其中，非易失性存储器可以是只读存储器(Read-Only Memory，ROM)、可编程只读存储器(Programmable ROM，PROM)、可擦除可编程只读存储器(Erasable PROM，EPROM)、电可擦除可编程只读存储器(Electrically EPROM，EEPROM) 或闪存。易失性存储器可以是随机存取存储器(Random Access Memory，RAM)，静态随机存取存储器(Static RAM，SRAM)、动态随机存取存储器(Dynamic RAM，DRAM)、同步动态随机存取存储器(Synchronous DRAM，SDRAM)、双倍数据速率同步动态随机存取存储器(Double Data Rate SDRAM，DDRSDRAM)、增强型同步动态随机存取存储器(Enhanced SDRAM，ESDRAM)、同步连接动态随机存取存储器(Synch link DRAM，SLDRAM)和直接内存总线随机存取存储器(Direct Rambus RAM，DRRAM)。本申请实施例中的存储器1109包括但不限于这些和任意其它适合类型的存储器。
处理器1110可包括一个或多个处理单元；可选的，处理器1110集成应用处理器和调制解调处理器，其中，应用处理器主要处理涉及操作系统、用户界面和应用程序等的操作，调制解调处理器主要处理无线通信信号，如基带处理器。可以理解的是，上述调制解调处理器也可以不集成到处理器1110中。
其中，处理器1110用于执行以下操作：
对原始特征图进行压缩处理，获得第一特征图；
编码所述第一特征图，获得第一码流和第二特征图；
对所述第二特征图进行增强处理，确定第三特征图；
编码残差特征图，获得第二码流；所述残差特征图基于所述原始特征图和所述第三特征图确定。
或者，处理器1110用于执行以下操作：
解码目标码流，获得第二特征图和残差特征图；
对所述第二特征图进行增强处理，确定第三特征图；
基于所述第三特征图和所述残差特征图，确定重建特征图。
本申请实施例还提供一种可读存储介质，所述可读存储介质上存储有程序或指令，该程序或指令被处理器执行时实现上述特征图处理方法实施例的各个过程，且能达到相同的技术效果，为避免重复，这里不再赘述。
其中，所述处理器为上述实施例中所述的终端中的处理器。所述可读存储介质，包括计算机可读存储介质，如计算机只读存储器ROM、随机存取存储器RAM、磁碟或者光盘等。
本申请实施例另提供了一种芯片，所述芯片包括处理器和通信接口，所述通信接口和所述处理器耦合，所述处理器用于运行程序或指令，实现上述特征图方法实施例的各个过程，且能达到相同的技术效果，为避免重复，这里不再赘述。
应理解，本申请实施例提到的芯片还可以称为系统级芯片，系统芯片，芯片系统或片上系统芯片等。
本申请实施例另提供了一种计算机程序/程序产品，所述计算机程序/程序产品被存储在存储介质中，所述计算机程序/程序产品被至少一个处理器执行以实现上述特征图方法实施例的各个过程，且能达到相同的技术效果，为避免重复，这里不再赘述。
本申请实施例另提供了一种系统，所述系统包括编码端和解码端，所述编码端执行上述应用于编码端的特征图处理方法实施例的各个过程，所述解码端执行上述应用于解码端的特征图处理方法实施例的各个过程，且能达到相同的技术效果，为避免重复，这里不再赘述。
需要说明的是，在本文中，术语“包括”、“包含”或者其任何其他变体意在涵盖非排他性的包含，从而使得包括一系列要素的过程、方法、物品或者装置不仅包括那些要素，而且还包括没有明确列出的其他要素，或者是还包括为这种过程、方法、物品或者装置所固有的要素。在没有更多限制的情况下，由语句“包括一个……”限定的要素，并不排除在包括该要素的过程、方法、物品或者装置中还存在另外的相同要素。此外，需要指出的是，本申请实施方式中的方法和装置的范围不限按示出或讨论的顺序来执行功能，还可包括根据所涉及的功能按基本同时的方式或按相反的顺序来执行功能，例如，可以按不同于所描述的次序来执行所描述的方法，并且还可以添加、省去、或组合各种步骤。另外，参照某些示例所描述的特征可在其他示例中被组合。
通过以上的实施方式的描述，本领域的技术人员可以清楚地了解到上述实施例方法可借助软件加必需的通用硬件平台的方式来实现，当然也可以通过硬件，但很多情况下前者是更佳的实施方式。基于这样的理解，本申请的技术方案本质上或者说对现有技术做出贡献的部分可以以计算机软件产品的形式体现出来，该计算机软件产品存储在一个存储介质(如ROM/RAM、磁碟、光盘)中，包括若干指令用以使得一台终端(可以是手机，计算机，服务器，空调器，或者网络设备等)执行本申请各个实施例所述的方法。
上面结合附图对本申请的实施例进行了描述，但是本申请并不局限于上述的具体实施方式，上述的具体实施方式仅仅是示意性的，而不是限制性的，本领域的普通技术人员在本申请的启示下，在不脱离本申请宗旨和权利要求所保护的范围情况下，还可做出很多形式，均属于本申请的保护之内。

Claims (21)

  1. 一种特征图处理方法，包括：
    编码端对原始特征图进行压缩处理，获得第一特征图；所述第一特征图的通道数小于所述原始特征图的通道数，和/或所述第一特征图的空间尺寸小于所述原始特征图的空间尺寸；
    所述编码端编码所述第一特征图，获得第一码流和第二特征图；
    所述编码端对所述第二特征图进行增强处理，确定第三特征图；所述第三特征图的通道数等于所述原始特征图的通道数，所述第三特征图的空间尺寸等于所述原始特征图的空间尺寸；
    所述编码端编码残差特征图，获得第二码流；所述残差特征图基于所述原始特征图和所述第三特征图确定。
  2. 根据权利要求1所述的方法，其中，所述对原始特征图进行压缩处理，获得第一特征图包括：
    所述编码端利用第一神经网络增强所述原始特征图，以及对所述原始特征图进行下采样处理，获得第一特征图；
    所述对原始特征图进行下采样处理包括以下至少一项：
    对所述原始特征图的通道数进行下采样处理；
    对所述原始特征图的特征图宽度以及特征图高度进行下采样处理。
  3. 根据权利要求1所述的方法，其中，所述对所述第二特征图进行增强处理，确定第三特征图包括：
    所述编码端对所述第二特征图进行增强处理，获得残差信息；所述残差信息用于表征所述第二特征图对应的压缩处理残差；
    所述编码端基于所述残差信息和所述第二特征图，确定第三特征图。
  4. 根据权利要求3所述的方法，其中，所述对所述第二特征图进行增强处理，获得残差信息包括：
    所述编码端利用第二神经网络增强所述第二特征图，以及对所述第二特征图进行上采样处理，获得残差信息；
    对第二特征图进行上采样处理包括以下至少一项：
    对所述第二特征图的通道数进行上采样处理；
    对所述第二特征图的特征图宽度以及特征图高度进行上采样处理。
  5. 根据权利要求4所述的方法，其中，对所述第二特征图的通道数进行上采样处理包括：
    所述编码端通过卷积层对所述第二特征图的通道数进行上采样处理，以使得上采样处理后的第二特征图的通道数与所述原始特征图的通道数相同。
  6. 一种特征图处理方法，包括：
    解码端解码目标码流，获得第二特征图和残差特征图，所述目标码流包括第一码流和第二码流，所述第二特征图基于所述第一码流确定，所述残差特征图基于所述第二码流确定；
    所述解码端对所述第二特征图进行增强处理，确定第三特征图；所述第三特征图的通道数大于所述第二特征图的通道数，和/或所述第三特征图的空间尺寸大于所述第二特征图的空间尺寸；
    所述解码端基于所述第三特征图和所述残差特征图，确定重建特征图。
  7. 根据权利要求6所述的方法，其中，所述对所述第二特征图进行增强处理，确定第三特征图包括：
    所述解码端对所述第二特征图进行增强处理，获得残差信息，所述残差信息用于表征所述第二特征图对应的压缩处理残差；
    所述解码端基于所述残差信息和所述第二特征图，确定第三特征图。
  8. 根据权利要求7所述的方法，其中，所述对所述第二特征图进行增强处理，获得残差信息包括：
    所述解码端利用第二神经网络增强所述第二特征图，以及对所述第二特征图进行上采样处理，获得残差信息；
    对第二特征图进行上采样处理包括以下至少一项：
    对所述第二特征图的通道数进行上采样处理；
    对所述第二特征图的特征图宽度以及特征图高度进行上采样处理。
  9. 一种特征图处理装置，应用于编码端，所述装置包括：
    第一处理模块，用于对原始特征图进行压缩处理，获得第一特征图；所述第一特征图的通道数小于所述原始特征图的通道数，和/或所述第一特征图的空间尺寸小于所述原始特征图的空间尺寸；
    第一编码模块，用于编码所述第一特征图，获得第一码流和第二特征图；
    第二处理模块，用于对所述第二特征图进行增强处理，确定第三特征图；所述第三特征图的通道数等于所述原始特征图的通道数，所述第三特征图的空间尺寸等于所述原始特征图的空间尺寸；
    第二编码模块，用于编码残差特征图，获得第二码流；所述残差特征图基于所述原始特征图和所述第三特征图确定。
  10. 根据权利要求9所述的装置，其中，所述第一处理模块，具体用于：
    利用第一神经网络增强所述原始特征图，以及对所述原始特征图进行下采样处理，获得第一特征图；
    所述对原始特征图进行下采样处理包括以下至少一项：
    对所述原始特征图的通道数进行下采样处理；
    对所述原始特征图的特征图宽度以及特征图高度进行下采样处理。
  11. 根据权利要求9所述的装置，其中，所述第二处理模块，具体用于：
    对所述第二特征图进行增强处理，获得残差信息；所述残差信息用于表征所述第二特征图对应的压缩处理残差；
    基于所述残差信息和所述第二特征图，确定第三特征图。
  12. 根据权利要求11所述的装置，其中，所述第二处理模块，还具体用于：
    利用第二神经网络增强所述第二特征图，以及对所述第二特征图进行上采样处理，获得残差信息；
    对第二特征图进行上采样处理包括以下至少一项：
    对所述第二特征图的通道数进行上采样处理；
    对所述第二特征图的特征图宽度以及特征图高度进行上采样处理。
  13. 根据权利要求12所述的装置，其中，所述第二处理模块，还具体用于：
    通过卷积层对所述第二特征图的通道数进行上采样处理，以使得上采样处理后的第二特征图的通道数与所述原始特征图的通道数相同。
  14. 一种特征图处理装置，应用于解码端，所述装置包括：
    解码模块，用于解码目标码流，获得第二特征图和残差特征图，所述目标码流包括第一码流和第二码流，所述第二特征图基于所述第一码流确定，所述残差特征图基于所述第二码流确定；
    第三处理模块，用于对所述第二特征图进行增强处理，确定第三特征图；所述第三特征图的通道数大于所述第二特征图的通道数，和/或所述第三特征图的空间尺寸大于所述第二特征图的空间尺寸；
    确定模块，用于基于所述第三特征图和所述残差特征图，确定重建特征图。
  15. 根据权利要求14所述的装置，其中，所述第三处理模块，具体用于：
    对所述第二特征图进行增强处理，获得残差信息，所述残差信息用于表征所述第二特征图对应的压缩处理残差；
    基于所述残差信息和所述第二特征图，确定第三特征图。
  16. 根据权利要求15所述的装置，其中，所述第三处理模块，还具体用于：
    利用第二神经网络增强所述第二特征图，以及对所述第二特征图进行上采样处理，获得残差信息；
    对第二特征图进行上采样处理包括以下至少一项：
    对所述第二特征图的通道数进行上采样处理；
    对所述第二特征图的特征图宽度以及特征图高度进行上采样处理。
  17. 一种终端，包括处理器和存储器，所述存储器存储可在所述处理器上运行的程序或指令，所述程序或指令被所述处理器执行时实现如权利要求1-5中任一项所述的特征图处理方法的步骤，或者实现如权利要求6-8中任一项所述的特征图处理方法的步骤。
  18. 一种可读存储介质，所述可读存储介质上存储程序或指令，所述程序或指令被处理器执行时实现如权利要求1-5中任一项所述的特征图处理方法的步骤，或者实现如权利要求6-8中任一项所述的特征图处理方法的步骤。
  19. 一种芯片，所述芯片包括处理器和通信接口，所述通信接口和所述处理器耦合，所述处理器用于运行程序或指令，实现如权利要求1-5中任一项所述的特征图处理方法的步骤，或者，实现如权利要求6-8中任一项所述的特征图处理方法的步骤。
  20. 一种计算机程序产品，所述计算机程序产品被存储在存储介质中，所述计算机程序产品被至少一个处理器执行以实现如权利要求1-5中任一项所述的特征图处理方法的步骤，或者，所述计算机程序产品被至少一个处理器执行以实现如权利要求6-8中任一项所述的特征图处理方法的步骤。
  21. 一种电子设备，被配置成用于执行如权利要求1-5中任一项所述的通信方法的步骤，或者，被配置成用于执行如权利要求6-8中任一项所述的通信方法的步骤。
PCT/CN2023/123323 2022-10-14 2023-10-08 特征图处理方法、装置及设备 Ceased WO2024078404A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202211261744.XA CN117939167A (zh) 2022-10-14 2022-10-14 特征图处理方法、装置及设备
CN202211261744.X 2022-10-14

Publications (1)

Publication Number Publication Date
WO2024078404A1 true WO2024078404A1 (zh) 2024-04-18

Family

ID=90668819

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2023/123323 Ceased WO2024078404A1 (zh) 2022-10-14 2023-10-08 特征图处理方法、装置及设备

Country Status (2)

Country Link
CN (1) CN117939167A (zh)
WO (1) WO2024078404A1 (zh)

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112218097A (zh) * 2019-07-12 2021-01-12 富士通株式会社 环路滤波装置和图像解码装置
KR20210040873A (ko) * 2020-06-30 2021-04-14 베이징 바이두 넷컴 사이언스 앤 테크놀로지 코., 엘티디. 이미지 충진 방법, 장치, 기기, 저장매체 및 프로그램
US11153566B1 (en) * 2020-05-23 2021-10-19 Tsinghua University Variable bit rate generative compression method based on adversarial learning
CN114463453A (zh) * 2021-12-14 2022-05-10 浙江大华技术股份有限公司 图像重建、编码解码方法、相关装置
CN114820388A (zh) * 2022-06-22 2022-07-29 合肥工业大学 一种基于编解码器结构的图像去雾方法

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112218097A (zh) * 2019-07-12 2021-01-12 富士通株式会社 环路滤波装置和图像解码装置
EP3764651A1 (en) * 2019-07-12 2021-01-13 Fujitsu Limited Loop filter apparatus and image decoding apparatus
US11153566B1 (en) * 2020-05-23 2021-10-19 Tsinghua University Variable bit rate generative compression method based on adversarial learning
KR20210040873A (ko) * 2020-06-30 2021-04-14 베이징 바이두 넷컴 사이언스 앤 테크놀로지 코., 엘티디. 이미지 충진 방법, 장치, 기기, 저장매체 및 프로그램
CN114463453A (zh) * 2021-12-14 2022-05-10 浙江大华技术股份有限公司 图像重建、编码解码方法、相关装置
CN114820388A (zh) * 2022-06-22 2022-07-29 合肥工业大学 一种基于编解码器结构的图像去雾方法

Also Published As

Publication number Publication date
CN117939167A (zh) 2024-04-26

Similar Documents

Publication Publication Date Title
EP3777207B1 (en) Content-specific neural network distribution
Patwa et al. Semantic-preserving image compression
KR20180105294A (ko) 이미지 압축 장치
TWI826160B (zh) 圖像編解碼方法和裝置
CN114554212A (zh) 视频处理装置及方法、计算机存储介质
CN113192147A (zh) 显著性压缩的方法、系统、存储介质、计算机设备及应用
CN110677649A (zh) 基于机器学习的去伪影方法、去伪影模型训练方法及装置
CN118901239A (zh) 图像处理方法、装置、设备、系统、及存储介质
WO2023193629A1 (zh) 区域增强层的编解码方法和装置
CN107018416A (zh) 用于视频和图像压缩的自适应贴片数据大小编码
US20260122242A1 (en) Encoding Method and Apparatus, Decoding Method and Apparatus, Device, Storage Medium, and Computer Program Product
WO2024131692A1 (zh) 图像处理方法、装置及设备
WO2024078404A1 (zh) 特征图处理方法、装置及设备
WO2025160493A1 (en) Systems and methods for layer reconstruction and encoder-driven feature information extraction in video coding for machines
EP4354872A1 (en) Point cloud attribute information encoding and decoding method and apparatus, and related device
WO2024007977A1 (zh) 图像处理方法、装置及设备
CN114339226A (zh) 一种提高画面的流畅度的方法、装置及介质
CN116072132A (zh) 一种音频编码器、解码器、传输系统、方法及介质
CN117395418A (zh) 图像处理方法、装置及设备
Zhu et al. Semantic-aware visual information transmission with key information extraction over wireless networks
CN116233426A (zh) 属性量化、反量化方法、装置及设备
EP4694127A1 (en) Image coding method, image decoding method and apparatus
US20260004465A1 (en) Point cloud encoding processing method, point cloud decoding processing method, and related device
WO2024078403A1 (zh) 图像处理方法、装置及设备
WO2024217361A1 (zh) 图像处理方法及终端

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 23876606

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 23876606

Country of ref document: EP

Kind code of ref document: A1