WO2023197805A1 - 图像处理方法、装置、存储介质及电子设备 - Google Patents

图像处理方法、装置、存储介质及电子设备 Download PDF

Info

Publication number
WO2023197805A1
WO2023197805A1 PCT/CN2023/081240 CN2023081240W WO2023197805A1 WO 2023197805 A1 WO2023197805 A1 WO 2023197805A1 CN 2023081240 W CN2023081240 W CN 2023081240W WO 2023197805 A1 WO2023197805 A1 WO 2023197805A1
Authority
WO
WIPO (PCT)
Prior art keywords
low
frequency information
information
content
image
Prior art date
Application number
PCT/CN2023/081240
Other languages
English (en)
French (fr)
Inventor
郭孟曦
赵世杰
李跃
李军林
张莉
王悦
Original Assignee
北京字节跳动网络技术有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 北京字节跳动网络技术有限公司 filed Critical 北京字节跳动网络技术有限公司
Publication of WO2023197805A1 publication Critical patent/WO2023197805A1/zh

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00—Image enhancement or restoration
    • G06T5/20—Image enhancement or restoration using local operators
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T3/00—Geometric image transformations in the plane of the image
    • G06T3/40—Scaling of whole images or parts thereof, e.g. expanding or contracting
    • G06T3/4007—Scaling of whole images or parts thereof, e.g. expanding or contracting based on interpolation, e.g. bilinear interpolation
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20084—Artificial neural networks [ANN]

Definitions

  • Embodiments of the present disclosure relate to the field of computer data processing technology, such as an image processing method, device, storage medium and electronic equipment.
  • Digital images often need to be resampled during the storage, display and transmission process.
  • the resampled image can adapt to better browsing of images on devices with different resolutions.
  • the image is down-sampled and the down-sampled image is processed. Store and transmit, and then upsample the downsampled image when displayed on the terminal device.
  • the same set of models are used for forward down-sampling processing and reverse up-sampling processing.
  • the down-sampling process will reduce the image resolution, thereby reducing storage and transmission costs.
  • the above-mentioned down-sampling process will also cause the loss of high-frequency information in the image, resulting in the low-resolution image being lost in the image.
  • the details of the reconstructed image are lost during the subsequent upsampling process, thereby reducing the lack of image details in the high-resolution image obtained after the upsampling process of the low-resolution image and reducing the image quality.
  • Embodiments of the present disclosure provide an image processing method, device, storage medium and electronic equipment, so that the obtained low-resolution image can save the content-related information in the original image, thereby improving the image quality while reducing the amount of image data.
  • embodiments of the present disclosure provide an image processing method, which method includes: acquiring an original image, and extracting high-frequency information and low-frequency information in the original image;
  • embodiments of the present disclosure also provide another image processing method, which method includes: acquiring a low-resolution image, and extracting content-related information in low-frequency information and high-frequency information based on the low-resolution image;
  • the high-frequency information is determined based on the content-related information, and the original image corresponding to the low-resolution image is obtained based on the fusion of the high-frequency information and the low-frequency information.
  • embodiments of the present disclosure also provide an image processing device, which includes: an information extraction model configured to acquire an original image and extract high-frequency information and low-frequency information in the original image;
  • a low-resolution image generation module is configured to extract content-related information from the high-frequency information and write the content-related information into the low-frequency information to obtain a low-resolution image corresponding to the original image.
  • embodiments of the present disclosure also provide another image processing device, which device includes: an information extraction model configured to acquire a low-resolution image, and extract low-frequency information and high-frequency information based on the low-resolution image. content-related information;
  • An original image generation module is configured to determine the high-frequency information based on the content-related information, and obtain an original image corresponding to the low-resolution image based on the fusion of the high-frequency information and the low-frequency information.
  • embodiments of the present disclosure also provide an electronic device, the electronic device includes:
  • processors one or more processors
  • a storage device configured to store one or more programs
  • the one or more processors When the one or more programs are executed by the one or more processors, the one or more processors are caused to implement the image processing method described in any one of the embodiments of the present disclosure.
  • embodiments of the disclosure further provide a storage medium containing computer-executable instructions, which when executed by a computer processor are used to perform image processing as described in any embodiment of the disclosure. method.
  • Figure 1 is a schematic flowchart of an image processing method provided by an embodiment of the present disclosure
  • Figure 2 is a schematic flowchart of a spatial rearrangement provided by an embodiment of the present disclosure
  • FIG. 3 is a schematic flowchart of another image processing method provided by an embodiment of the present disclosure.
  • FIG. 4 is a schematic flowchart of another image processing method provided by an embodiment of the present disclosure.
  • Figure 5 is a schematic structural diagram of an image processing device provided by an embodiment of the present disclosure.
  • Figure 6 is a schematic structural diagram of another image processing device provided by an embodiment of the present disclosure.
  • FIG. 7 is a schematic structural diagram of an electronic device provided by an embodiment of the present disclosure.
  • the term “include” and its variations are open-ended, ie, “including but not limited to.”
  • the term “based on” means “based at least in part on.”
  • the term “one embodiment” means “at least one embodiment”; the term “another embodiment” means “at least one additional embodiment”; and the term “some embodiments” means “at least some embodiments”. Relevant definitions of other terms will be given in the description below.
  • the original image before image transmission or image storage, the original image is image downsampled to obtain a low-resolution image corresponding to the original image, thereby reducing the amount of original data. Then, image transmission or image storage is performed on the low-resolution image to reduce the data resources occupied during the transmission or storage process, thereby reducing transmission or storage costs. For example, when displaying the original image, a low-resolution image of the original image is obtained, and the low-resolution image is upsampled to obtain a high-resolution image corresponding to the low-resolution image to present a clearer image. .
  • the methods used by related technologies to downsample the original image in real-time include: mapping the high-frequency information in the original image to the Gaussian distribution, so that the image information on the Gaussian distribution can be extracted in the subsequent upsampling process to obtain a high-resolution image with more original image information.
  • mapping the high-frequency information in the original image to the Gaussian distribution so that the image information on the Gaussian distribution can be extracted in the subsequent upsampling process to obtain a high-resolution image with more original image information.
  • Figure 1 is a technical solution provided by the embodiment of the present disclosure. Schematic flow chart of an image processing method.
  • the embodiments of the present disclosure are suitable for down-sampling and up-sampling images.
  • the method can be executed by the image processing device provided by the embodiments of the present disclosure.
  • the image processing device can be implemented in the form of software and/or hardware, for example, This is achieved through an electronic device, which may be a mobile terminal or a PC.
  • the method in this embodiment includes:
  • S120 Extract content-related information from the high-frequency information, write the content-related information into the low-frequency information, and obtain a low-resolution image corresponding to the original image.
  • the original image to be processed is decomposed into high-frequency information and low-frequency information.
  • the high-frequency information is decomposed and the relevant content of the original image in the high-frequency information is extracted. information, and write the content-related information as steganographic information into the down-sampled low-frequency information to obtain a low-resolution image of the original image, so that the low-resolution image includes more texture information.
  • the reverse upsampling process high-frequency information related to the image content is obtained, and high-resolution images with more texture details are accurately obtained, thereby improving image quality.
  • the original image can be understood as an image that has not been processed.
  • a low-resolution image can be understood as an image that contains the image content of the original image and has a reduced amount of data.
  • the low-frequency information includes image content information
  • the high-frequency information includes Texture information such as edges.
  • the method of extracting high-frequency information and low-frequency information in the original image may include: downsampling the original image to obtain low-frequency information; determining the initialization based on the high-resolution low-frequency image obtained by upsampling the original image and low-frequency information. For high-frequency information, the spatial pixels in the initial high-frequency information are rearranged to the channel dimension to obtain high-frequency information that matches the low-frequency information.
  • the original image can be represented as (H, W, C); where H represents the height data of the original image, W represents the width data of the original image, and C represents the channel data of the original data, for example C can be 3, indicating that the channel data of the original image is 3 channels including RGB channels respectively.
  • the original image is downsampled to obtain the low-frequency information of the original image, and the obtained low-frequency information is directly upsampled to obtain a high-resolution low-frequency image corresponding to the low-frequency information, and based on the image data of the original image and the high-resolution
  • the image data of the low-frequency image determines the initial high-frequency information in the original image.
  • the initial high-frequency information in the original image can be determined based on the image data difference of the above two images.
  • the original image and the high-resolution low-frequency image have the same resolution
  • the corresponding pixels in the original image and the high-resolution low-frequency image are The pixel values are subjected to difference processing respectively to obtain the initial high-frequency information.
  • the spatial pixels in the initial high-frequency information are rearranged to the channel dimension, thereby obtaining high-frequency information that matches the low-frequency information, where the high-frequency information and the low-frequency information have the same resolution.
  • rearranging the spatial pixels in the initial high-frequency information to the channel dimension can be understood as reducing the height data and width data in the initial high-frequency information, and at the same time At this time, the pixel data corresponding to the height data and width data are arranged in other channels, thereby obtaining high-frequency information that matches the resolution of the low-frequency information by increasing the number of channel dimensions.
  • low-frequency information can be expressed as (H/2, W/2, C)
  • high-frequency information can be expressed as H/2, W/2, 4C)
  • the above matching can be understood as matching the image resolutions of low-frequency information and high-frequency information, that is, the height data in the low-frequency information matches the height data in the high-frequency information, and the width data in the low-frequency information matches the Width data in high frequency information matches.
  • the technical solution of the embodiment of the present disclosure has the beneficial effect of obtaining high-frequency information that matches the low-frequency information in that it facilitates the subsequent implicit writing of the content-related information extracted from the high-frequency information into the low-frequency information.
  • steganography can be understood as hiding and storing content-related information in the original image into low-frequency information, forming a low-resolution image containing high-frequency information related to the image content of the original image, making full use of low-resolution image stealth storage
  • the ability to obtain a large amount of information enables subsequent upsampling of the low-resolution image.
  • part of the information related to the image content in the high-frequency information of the original image is reversely extracted to improve the reverse recovery of the image. of image quality.
  • the original image with a resolution of (H, W, C) is down-sampled 2 times, that is, bicubic interpolation is used to down-sample 2 times to obtain the low-frequency information of the original image (H/2, W/2, C);
  • bicubic interpolation to upsample 2 times the low-frequency information (H/2, W/2, C) to obtain a high-resolution low-frequency image (H, W) that is the same as the width data and length data of the original image.
  • the method of extracting high-frequency information and low-frequency information in the original image may also include: performing filtering transformation on the original image to obtain the high-frequency information and low-frequency information in the original image.
  • the filtering transformation can use wavelet transform.
  • Haar transform can also be used to perform high-pass filtering on the input information to obtain separated high-frequency information and low-frequency information.
  • filter transformation is used to perform high-pass filtering on the image information in the original image to obtain the high-frequency information and low-frequency information of the original image.
  • filter transformation is used to separate information from the original image (H, W, C) through high-pass filtering to obtain low-frequency information (H/4, W/4, C) and high-frequency information that are downsampled 4 times. (H/4, W/4, 15C).
  • the multiples used for downsampling the original image include 2 times and 4 times. It should be noted that the above sampling multiples are only introduced as exemplary embodiments. The technical solution of this embodiment can also adopt other sampling multiples, which is not limited in this embodiment.
  • the high-frequency information of the original image is extracted to obtain the content-related information in the high-frequency information, and the content-related information is written into the low-frequency information to obtain the corresponding content of the original image.
  • Low resolution image After obtaining the high-frequency information and low-frequency information of the original image, the high-frequency information of the original image is extracted to obtain the content-related information in the high-frequency information, and the content-related information is written into the low-frequency information to obtain the corresponding content of the original image. Low resolution image.
  • the high-frequency information of the original image includes content-related information and content-irrelevant information.
  • Content-related information can be understood as information related to the image content in the original image; content-independent information can be understood as other information such as image noise in the original image.
  • the method of extracting content-related information in high-frequency information can be extracted using a pre-trained neural network model.
  • high-frequency information can be processed based on a pre-trained information extraction model to obtain content-related information and content-irrelevant information in the high-frequency information.
  • the information extraction model may be a network structure module, and the network structure of the information extraction model may be a structure such as a convolutional neural network, a multi-layer perceptron, etc., which is not limited.
  • the information extraction model may be an attention reversible transformation network model.
  • the method of processing high-frequency information based on a pre-trained information extraction model to obtain content-related information and content-irrelevant information in high-frequency information may include: inputting high-frequency information and low-frequency information into the attention reversible transformation network model , the content-related information and content-irrelevant information in the high-frequency information are obtained, where the low-frequency information provides content-related information for the high-frequency information. auxiliary conditions.
  • the low-frequency information contains the content information of the original image. That is, the low-frequency information is used as an auxiliary condition for extracting content-related information. It is input into the attention reversible transformation network model at the same time as the high-frequency information.
  • the low-frequency information can be included in the attention reversible transformation network model.
  • the content information is used as the learning target of the attention reversible transformation network when extracting information, so that the attention reversible transformation network can accurately extract the image content information of the high-frequency information and the original image when extracting information.
  • the attention reversible transformation network model training method may include: inputting high-frequency information in the original sample image into the attention reversible transformation network model to be trained, and obtaining content-related information and content-independent information output by the attention reversible transformation network model. Information, and the content-related information and content-related information are reversely input into the attention reversible transformation network model to be trained, and the predicted high-frequency information reversely output by the attention reversible transformation network model is obtained.
  • the loss function of the attention reversible transformation network model to be trained is generated based on the predicted high-frequency information and the high-frequency information in the original image, and the model parameters of the attention reversible transformation network model to be trained are adjusted based on the loss function. , iteratively execute the above training process until the trained attention reversible transformation network model is obtained.
  • the high-frequency information (H/2, W/2, 4C) obtained by separating the above information and matching the low-frequency information is further extracted.
  • high-frequency information (H/2, W/2, 4C) is separated into content-related information (H/2, W/2, 4pC) and content-irrelevant information (H/2, W/2, 4pC) according to a separation ratio p. H/2, W/2, 4C-4pC).
  • the separation situation of high-frequency information downsampled 2 times is the content-related information (H/4 , W/4, 15pC) and content-independent information (H/4, W/4, 15C-15pC).
  • the separation ratio p is preset in the attention reversible transformation network model and is not limited to this.
  • the separation ratio p corresponding to downsampling 2 times is greater than the separation ratio p corresponding to downsampling 4 times.
  • different downsampling multiples can correspond to different attention reversible transformation network models.
  • a method of writing content-related information into low-frequency information to obtain a low-resolution image corresponding to the original image may include: performing data fusion on the channel dimension with the content-related information and low-frequency information to obtain a low-resolution image corresponding to the original image.
  • the spatial pixels of the initial high-frequency information in the original image are rearranged to the channel dimension, so that the subsequently obtained high-frequency information is the same as the height data and width data in the low-frequency information.
  • the channel dimension of high-frequency information is larger than that of low-frequency information.
  • the channel dimension of content-related information extracted from high-frequency information is also larger than the channel dimension of low-frequency information.
  • data fusion of content-related information and low-frequency information in the channel dimension may include: determining the first channel data in the low-frequency information and at least one second channel data in the corresponding content-related information; will have a corresponding relationship The first channel data and the second channel data are fused.
  • channel data can be understood as pixel data of multiple channels.
  • the low-frequency information includes three first channels of RGB.
  • the first-channel data can be understood as R channel data, G channel data and B channel data in the low-frequency information.
  • channel data; the second channel data can be understood as the pixel data of multiple channels in content-related information; it is worth noting that since the channel dimension of content-related information is larger than the channel dimension of low-frequency information, correspondingly, the channel dimension of the first channel data is less than the channel dimension of the second channel data.
  • the channel correspondence relationship between the channel data of the first channel and the channel data of the second channel is a one-to-many relationship, that is, the first channel data in the low-frequency information needs to be determined, and corresponding content-related information of at least one second channel data.
  • the corresponding relationship between the first channel data and the second channel data can be set in advance, and the setting method of the corresponding relationship is not limited.
  • the first channel data and the second channel data have a corresponding relationship between the same types of channels.
  • the first channel data includes three RGB channels
  • the second channel data includes nine channels respectively. That is, the number of RGB channels is three. indivual.
  • the R channel in the first channel data corresponds to the three R channels in the second channel data
  • the G channel in the first channel data corresponds to the three G channels in the second channel data
  • the B channel in the first channel data corresponds to the three B channels in the second channel data.
  • multiple channels of the first channel data correspond to multiple channels of the second channel data in sequential polling intervals.
  • the resolution of low-frequency information is (H/2, W/2, C)
  • the channel dimension of the first channel data can be understood as 3 channels
  • the content-related information is (H/2, W/2, 4pC )
  • taking the preset separation ratio p as 75% is an example
  • the channel dimension of the second channel data can be understood as 9 channels.
  • the corresponding relationship may be that the first channel in the first channel data corresponds to the first channel, the fourth channel, and the seventh channel in the second channel data, and the second channel in the first channel data corresponds to the third channel in the second channel data.
  • the third channel in the first channel data corresponds to the third channel, the sixth channel, and the ninth channel in the second channel data.
  • multiple channels of the first channel data correspond to multiple channels of the second channel data in sequence.
  • the resolution of the low-frequency information is (H/2, W/2, C)
  • the content-related information is ( H/2, W/2, 4pC)
  • the corresponding relationship can also be that the first channel in the first channel data corresponds to the first channel to the third channel in the second channel data, and the second channel in the first channel data
  • the third channel in the first channel data corresponds to the seventh channel to the ninth channel in the second channel data.
  • other correspondence relationships are possible, which are not limited in this embodiment.
  • the channel data in the first channel data and the channel data in the second channel data that have the corresponding relationship are data fused to obtain the original image.
  • the corresponding low-resolution image For example, if downsampling is 2 times, the process of obtaining a low-resolution image may include: ((H/2, W/2, C), (H/2, W/2, 4pC))->(H/2 , W/2, C); if downsampling is 4 times, the process of obtaining a low-resolution image may include: ((H/4, W/4, C), (H/2, W/2, 15pC)) ->(H/4, W/4, C).
  • the technical solution of this embodiment also maps the content-independent information to a standard normal distribution.
  • the standard normal distribution can be understood as a normal distribution with a mean of 0 and a variance of 1.
  • the effect of mapping content-independent information to the normal distribution is that in the process of upsampling the low-resolution into the original image, information can be extracted from the normal distribution, and the extracted information can be used as image noise information of the original image. Image processing to restore a high-resolution image that is closer to the original image.
  • the content-irrelevant information may not be processed.
  • the information may be extracted directly based on the standard normal distribution, and the extracted noise data and Perform image processing on low-resolution images to obtain high-resolution images.
  • the technical solution of the embodiment of the present disclosure is to obtain the original image, extract the high-frequency information and low-frequency information in the original image; extract the content-related information in the high-frequency information, write the content-related information into the low-frequency information, and obtain the original image corresponding to Low resolution image.
  • the above technical solution extracts information from the original image to obtain high-frequency information of the original image, decomposes the high-frequency information, extracts content-related information about the original image in the high-frequency information, and uses the content-related information as steganography information Write to the downsampled low-resolution image, thereby achieving a high-resolution image with more texture details during the reverse upsampling process of the low-resolution image, while reducing the amount of image data. Also improves image quality.
  • FIG. 3 is a schematic flowchart of another image processing method provided by an embodiment of the present disclosure. This method can be executed by the image processing device provided by the embodiment of the present disclosure, and the image processing device can be implemented in the form of software and/or hardware, for example, through an electronic device, and the electronic device can be a mobile terminal or a PC, etc. As shown in Figure 3, the method in this embodiment includes:
  • S220 Determine high-frequency information based on content-related information, and obtain the original image corresponding to the low-resolution image based on the fusion of high-frequency information and low-frequency information.
  • the acquired low-resolution image needs to be upsampled to obtain the high-resolution original image.
  • the process of extracting low-frequency information and content-related information from low-resolution images and the process of implicitly writing content-related information into low-frequency information in the above embodiment are reversible processes, and there is a corresponding relationship between the two processing methods.
  • a method for extracting content-related information in low-frequency information and high-frequency information based on a low-resolution image may include: determining a correspondence relationship between a data channel in the low-frequency information and a data channel in the low-resolution image, based on the low-resolution image.
  • the channel data in the image determines the first channel data of the low-frequency information; determines the corresponding relationship between the data channel in the content-related information and the data channel in the low-resolution image, and determines the second channel of the content-related information based on the channel data in the low-resolution image. data.
  • low-resolution images are obtained by data fusion in the channel dimension based on low-frequency information and content-related information. It can be understood as data fusion based on channel data of low-frequency information and channel data of content-related information that have corresponding relationships. owned.
  • First channel data of low-frequency information is determined based on the channel data in the low-resolution image
  • second channel data of the content-related information is determined based on the channel data in the low-resolution image.
  • the order of determining the first channel data and the order of the second channel data is not determined in any order, and can be performed sequentially or simultaneously. This embodiment does not limit this.
  • the channel dimension of the low-resolution image is 3 channels. For example, obtain the correspondence between the data channels in the low-frequency information and the data channels in the low-resolution image. If the correspondence between the data channels is a one-to-one correspondence, the corresponding channel dimensions of the low-frequency information in the low-resolution image are also 3. channel, the resolution of low-frequency information is (H/2, W/2, C); based on the channel data in the low-resolution image, the first channel data in the low-frequency information is determined (H/2, W/2, C) .
  • the method of determining the first channel data may be to copy the channel data in the low-resolution image as the first channel data of the corresponding data channel in the low-frequency information.
  • the corresponding relationship may be that the first channel, the fourth channel and the seventh channel in the content-related information correspond to the first channel in the low-resolution image; the third channel is determined based on the channel data of the first channel in the low-resolution image.
  • Channel data for other channels in the data may be that the first channel, the fourth channel and the seventh channel in the content-related information correspond to the first channel in the low-resolution image; the third channel is determined based on the channel data of the first channel in the low-resolution image.
  • the channel data of the first channel, the fourth channel and the seventh channel in the two-channel data and correspondingly determine the second channel based on the channel correspondence between the remaining channels.
  • the corresponding relationship can also be that the first channel to the third channel in the content-related information correspond to the first channel in the low-resolution image; based on the channel data of the first channel in the low-resolution image, the second channel data is determined respectively.
  • the channel data of the first channel to the third channel and correspondingly determine the channel data of other channels in the second channel data based on the channel correspondence between the remaining channels.
  • the corresponding relationship between the data channel in the content-related information and the data channel in the low-resolution image can also be other corresponding relationships.
  • the third corresponding relationship of the content-related information is determined based on other corresponding relationships and the channel data in the low-resolution image.
  • Two-channel data is not limited in this embodiment.
  • the method of determining the second channel data of the content-related information based on the channel data in the low-resolution image may include: copying the channel data in the low-resolution image as the corresponding data in the content-related information.
  • the second channel data of the data channel may include: copying the channel data in the low-resolution image as the corresponding data in the content-related information.
  • the method of determining the second channel data of the content-related information based on the channel data in the low-resolution image may also include: using a random value of the channel data in the low-resolution image within a preset range as the content-related information.
  • the second channel data of the corresponding data channel in the information are determined based on the channel data of the first channel in the low-resolution image as an example: any of the first channel in the low-resolution image
  • a data can be 230
  • the preset range of the data can be 230 ⁇ 5.
  • the corresponding data in the first channel to the third channel in the second channel data can be randomly sampled in the range of 230 ⁇ 5.
  • the corresponding data in the first channel may be 232
  • the corresponding data in the second channel may be 235
  • the corresponding data in the third channel may be 228.
  • the second channel data of the corresponding data channel in the obtained content-related information can be expressed as (H/2, W/2, 4pC).
  • the method of determining content-irrelevant data may include: performing information resampling in a preset data distribution corresponding to content-irrelevant information to obtain content-irrelevant information.
  • the preset data distribution can be understood as the normal distribution to which the content-independent information of the high-frequency information in the original image is mapped in the process of downsampling the original data to obtain the low-resolution image; of course, it can also be other selected
  • the data distribution is preset, which is not limited in this embodiment.
  • content-independent information is extracted from the above-mapped normal distribution, the extracted information is used as image noise information of the original image, and information is resampled with the content-related image to obtain high-frequency information corresponding to low resolution.
  • information can also be extracted directly based on the standard normal distribution, and content-irrelevant information and content-related information can be resampled to obtain high-frequency information.
  • a method for resampling content-irrelevant information and content-related information to obtain high-frequency information may include: reversely inputting content-related information and content-irrelevant information into an attention reversible transformation network model to obtain attention High-frequency information output by the reversible transformation network model.
  • content-related information is extracted from a low-resolution image. If the upsampling is 2 times, the content-related information is expressed as (H/2, W/2, 4pC); if the upsampling is 4 times, the content-related information is expressed as (H/2, W/2, 4pC); if the upsampling is 4 times, the content-related information is Expressed as (H/2, W/2, 15pC). For example, if the content-irrelevant information is upsampled 2 times, the content-irrelevant information is expressed as (H/2, W/2, 4C-4pC); if the content-irrelevant information is upsampled 4 times, the content-irrelevant information is expressed as (H/4, W /4, 15C-15pC).
  • the process of obtaining high-frequency information can include ((H/2, W/2, 4C-4pC), (H/2, W/2, 4pC))->(H/2 , W/2, 4C);
  • the process of obtaining a low-resolution image can include: ((H/2, W/2, 15pC), (H/2, W/2, 15pC)) ->(H/4, W/4, 15C)).
  • the high-frequency information and the low-frequency information are fused based on Get the original image corresponding to the low-resolution image.
  • the method of obtaining the original image corresponding to the low-resolution image may include: upsampling the low-frequency information to obtain a high-resolution low-frequency image; performing spatial inverse rearrangement of the channel data of the high-frequency information to obtain a high-resolution high-frequency image. , and obtain the original image based on the high-resolution low-frequency image and the high-resolution high-frequency image.
  • the obtained low-frequency information is directly upsampled to obtain a high-resolution low-frequency image corresponding to the low-frequency information; and the data channels in the high-frequency information are spatially inversely rearranged to obtain a high-resolution high-frequency image corresponding to the high-frequency information.
  • image fusion is performed on a high-resolution low-frequency image and a high-resolution high-frequency image to obtain the original image corresponding to the low-resolution image.
  • the original image with a resolution of (H/2, W/2, C) is upsampled 2 times, that is, bicubic interpolation is used to upsample 2 times to obtain a high-resolution low-frequency image (H, W, C); perform spatial inverse rearrangement of high-frequency information (H/2, W/2, 4C) to obtain a high-resolution high-frequency image (H, W, C), and then fuse the high-resolution low-frequency image and the high-resolution high-frequency image to obtain the original image (H, W, C) corresponding to the low-resolution image.
  • the method of obtaining the original image corresponding to the low-resolution image may also include: performing an inverse Haar transform on the high-frequency information and the low-frequency information to obtain the original image.
  • the Haar inverse transform is used to fuse the low-frequency information (H/4, W/4, C) and the high-frequency information (H/4, W/4, 15C) to obtain the original corresponding to the low-resolution image. image(H,W,C).
  • the technical solution of the embodiment of the present disclosure is to obtain a low-resolution image, extract low-frequency information and content-related information in high-frequency information based on the low-resolution image; determine high-frequency information based on the content-related information, and extract content-related information based on the high-frequency information and low-frequency information.
  • the original image corresponding to the low-resolution image is obtained by fusion.
  • the embodiment of the present disclosure also provides an application example to explain the steps of obtaining a low-resolution image based on a high-resolution image, and then reversely obtaining a high-resolution image based on the low-resolution image.
  • this interactive embodiment may be applicable to the process of image transmission.
  • it can be data transmission between the client and the server.
  • the client requests the server to deliver an image/video.
  • the server first transfers the image/video to be transmitted before transmitting the image/video.
  • Multiple video frames i.e., high-resolution images
  • the low-resolution images/videos are transmitted to reduce the amount of transmitted data. Thereby reducing transmission costs.
  • the client After receiving the low-resolution image/video sent by the server, the client obtains the low-resolution image or multiple image frames in the low-resolution video based on the processing method provided in the above embodiment. /High-resolution image/video corresponding to the video for image display.
  • the above image/video transmission can also be image transmission between clients, which is not limited in this embodiment.
  • this application embodiment can also be used in the process of image/video storage.
  • the client when the client stores images locally, it can obtain low-resolution images/videos based on the processing methods provided in the above embodiments based on the image frames in the images to be stored or the videos to be stored, and process the low-resolution images. /video to store.
  • the corresponding high-resolution image/video is obtained based on the processing method provided by the above embodiment for the low-resolution image or the image frame in the low-resolution video to execute Subsequent display or processing.
  • this application embodiment includes the following steps:
  • Step 1 Separate high-frequency and low-frequency information from the high-resolution image (original image) to obtain high-frequency information and low-frequency information respectively.
  • Step 2 Separate the high-frequency information to obtain content-related information and content-irrelevant information respectively.
  • Step 3 Write the content-related information implicitly into the low-frequency information to obtain the low-resolution image corresponding to the high-resolution image.
  • Step 4 Map the content-independent information into Gaussian noise.
  • Step 5 Extract information from low-resolution images to obtain content-related information in low-frequency information and high-frequency information respectively.
  • Step 6 Resample Gaussian noise to obtain content-independent information.
  • Step 7 Fusion of content-irrelevant information and content-related information to obtain high-frequency information.
  • Step 8 Fusion of high-frequency information and low-frequency information to obtain a high-resolution image.
  • steps one to four can be executed by the same device, such as a server, while steps five to step eight can be executed by another device, such as a client; or steps one to four can be executed. 8 are all executed using the same device, such as a server or a client. This embodiment does not limit the execution device of multiple steps.
  • FIG. 5 is a schematic structural diagram of an image processing device provided by an embodiment of the present disclosure. As shown in Figure 5, the device includes: an information extraction model 310 and a low-resolution image generation module 320; wherein,
  • Information extraction model 310 is configured to obtain the original image and extract high-frequency information and low-frequency information in the original image;
  • the low-resolution image generation module 320 is configured to extract content-related information from the high-frequency information, write the content-related information into the low-frequency information, and obtain a low-resolution image corresponding to the original image.
  • the low-resolution image generation module 320 includes:
  • the information extraction submodule is configured to process the high-frequency information based on a pre-trained information extraction model to obtain content-related information and content-independent information in the high-frequency information.
  • the information extraction model is an attention reversible transformation network model
  • the information extraction sub-module includes:
  • An information extraction unit configured to input the high-frequency information and the low-frequency information into the attention reversible transformation network model to obtain content-related information and content-irrelevant information in the high-frequency information, wherein the low-frequency information Information is an auxiliary condition for extracting content-related information from the high-frequency information.
  • the low-resolution image generation module 320 includes:
  • the data fusion submodule is configured to perform data fusion on the channel dimension of the content-related information and the low-frequency information to obtain a low-resolution image corresponding to the original image.
  • the data fusion sub-module includes:
  • a data fusion unit configured to determine the first channel data in the low-frequency information corresponding to at least one second channel data in the content-related information; perform data processing on the first channel data and the second channel data that have a corresponding relationship. Fusion.
  • the information extraction model 310 includes:
  • the first information extraction unit is configured to downsample the original image to obtain low-frequency information; determine the initial high-frequency information based on the original image and the high-resolution low-frequency image obtained by upsampling the low-frequency information, and The spatial pixels in the high-frequency information are rearranged to the channel dimension to obtain high-frequency information that matches the low-frequency information;
  • the second information extraction unit is configured to perform filtering transformation on the original image to obtain high-frequency information and low-frequency information in the original image.
  • FIG. 6 is a schematic structural diagram of another image processing device provided by an embodiment of the present disclosure. As shown in Figure 6, the device includes: an information extraction model 410 and an original image generation module 420; wherein,
  • Information extraction model 410 is configured to obtain a low-resolution image, and extract content-related information in low-frequency information and high-frequency information based on the low-resolution image;
  • the original image generation module 420 is configured to determine the high-frequency information based on the content-related information, and obtain the original image corresponding to the low-resolution image based on the fusion of the high-frequency information and the low-frequency information.
  • the information extraction model 410 includes:
  • the first channel data determination submodule is configured to determine the corresponding relationship between the data channel in the low-frequency information and the data channel in the low-resolution image, and determine the first channel data of the low-frequency information based on the channel data in the low-resolution image.
  • the second channel data determination submodule is configured to determine the corresponding relationship between the data channel in the content-related information and the data channel in the low-resolution image, and determine the content-related information based on the channel data in the low-resolution image. second channel data.
  • the first channel data determination submodule includes:
  • a first channel data determination unit configured to copy the channel data in the low-resolution image as the first channel data of the corresponding data channel in the low-frequency information
  • the second channel data determination sub-module includes:
  • the second channel data determining unit is configured to copy the channel data in the low-resolution image as the second channel data of the corresponding data channel in the content-related information.
  • the device also includes:
  • An information acquisition module configured to perform information resampling in a preset data distribution corresponding to content-irrelevant information before determining the high-frequency information based on the content-related information to obtain content-irrelevant information;
  • the information extraction model 410 includes:
  • a high-frequency information determination submodule is configured to determine the high-frequency information based on the content-related information and the content-independent information obtained by resampling.
  • the high-frequency information determination submodule includes:
  • the high-frequency information determination unit is configured to reversely input the content-related information and the content-irrelevant information to the attention reversible transformation network model to obtain the high-frequency information output by the attention reversible transformation network model.
  • the original image generation module 420 includes:
  • the first original image generation unit is configured to upsample the low-frequency information to obtain a high-resolution low-frequency image; perform spatial inverse rearrangement of the channel data of the high-frequency information to obtain a high-resolution high-frequency image, and based on The high-resolution low-frequency image and the high-resolution high-frequency image are used to obtain an original image;
  • the second original image generating unit is configured to perform Haar inverse transform on the high-frequency information and the low-frequency information to obtain an original image.
  • the device provided by the embodiments of the present disclosure can execute the method provided by any embodiment of the present disclosure, and has corresponding functional modules and beneficial effects for executing the method.
  • Terminal devices in embodiments of the present disclosure may include, but are not limited to, mobile phones, laptops, digital broadcast receivers, PDAs (Personal Digital Assistants), PADs (Tablets), PMPs (Portable Multimedia Players), vehicle-mounted terminals (such as Mobile terminals such as car navigation terminals) and fixed terminals such as digital TVs, desktop computers, etc.
  • the electronic device shown in FIG. 7 is only an example and should not impose any limitations on the functions and scope of use of the embodiments of the present disclosure.
  • the electronic device 400 may include a processing device (eg, central processing unit, graphics processor, etc.) 401 , which may be loaded into a random access device according to a program stored in a read-only memory (ROM) 402 or from a storage device 408 .
  • the program in the memory (RAM) 403 executes various appropriate actions and processes.
  • various programs and data required for the operation of the electronic device 400 are also stored.
  • the processing device 401, ROM 402 and RAM 403 are connected to each other via a bus 404.
  • An input/output (I/O) interface 405 is also connected to bus 404.
  • the following devices may be connected to the I/O interface 405: input devices 406 including, for example, a touch screen, touch pad, keyboard, mouse, camera, microphone, accelerometer, gyroscope, etc.; including, for example, a liquid crystal display (LCD), speakers, vibration An output device 407 such as a computer; a storage device 408 including a magnetic tape, a hard disk, etc.; and a communication device 409.
  • the communication device 409 may allow the electronic device 400 to communicate wirelessly or wiredly with other devices to exchange data.
  • FIG. 7 illustrates electronic device 400 with various means, it should be understood that implementation or availability of all illustrated means is not required. More or fewer means may alternatively be implemented or provided.
  • embodiments of the present disclosure include a computer program product including a computer program carried on a non-transitory computer-readable medium, the computer program containing program code for performing the method illustrated in the flowchart.
  • the computer program may be downloaded and installed from the network via communication device 409, or from storage device 408, or from ROM 402.
  • the processing device 401 When the computer program is executed by the processing device 401, the above-mentioned functions defined in the method of the embodiment of the present disclosure are performed.
  • the electronic device provided by the embodiments of the present disclosure belongs to the same concept as the holographic projection model method provided by the above embodiments.
  • Technical details that are not described in detail in this embodiment can be referred to the above embodiments, and this embodiment has the same features as the above embodiments. beneficial effects.
  • Embodiments of the present disclosure provide a computer storage medium on which a computer program is stored.
  • the program is executed by a processor, the image processing method provided by the above embodiments is implemented.
  • the computer-readable medium mentioned above in the present disclosure may be a computer-readable signal medium or a computer-readable storage medium, or any combination of the above two.
  • the computer-readable storage medium may be, for example, but is not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus or device, or any combination thereof. More specific examples of computer readable storage media may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard drive, random access memory (RAM), read only memory (ROM), removable Programmd read-only memory (EPROM or flash memory), fiber optics, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination of the above.
  • a computer-readable storage medium may be any tangible medium that contains or stores a program for use by or in connection with an instruction execution system, apparatus, or device.
  • a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave carrying computer-readable program code therein. Such propagated data signals may take many forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above.
  • a computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium that can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device .
  • Program code embodied on a computer-readable medium may be transmitted using any suitable medium, including but not limited to: wire, optical cable, RF (radio frequency), etc., or any suitable combination of the above.
  • the client and server can communicate using any currently known or future developed network protocol such as HTTP (Hyper Text Transfer Protocol), and can communicate with digital data in any form or medium.
  • Data communications e.g., communications network
  • communications networks include local area networks (“LAN”), wide area networks (“WAN”), the Internet (e.g., the Internet), and end-to-end networks (e.g., ad hoc end-to-end networks), as well as any currently known or developed in the future network of.
  • the above-mentioned computer-readable medium may be included in the above-mentioned electronic device; it may also exist independently without being assembled into the electronic device.
  • the above-mentioned computer-readable medium carries one or more programs.
  • the electronic device executes the above-mentioned one or more programs.
  • Extract content-related information from the high-frequency information write the content-related information into the low-frequency information, and obtain a low-resolution image corresponding to the original image.
  • the high-frequency information is determined based on the content-related information, and the original image corresponding to the low-resolution image is obtained based on the fusion of the high-frequency information and the low-frequency information.
  • Computer program code for performing the operations of the present disclosure may be written in one or more programming languages, including but not limited to object-oriented programming languages—such as Java, Smalltalk, C++, and Includes conventional procedural programming languages—such as "C” or similar programming languages.
  • the program code may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server.
  • the remote computer can be connected to the user's computer through any kind of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computer (such as an Internet service provider through Internet connection).
  • LAN local area network
  • WAN wide area network
  • Internet service provider such as an Internet service provider through Internet connection
  • each block in the flowchart or block diagram may represent a module, segment, or portion of code that contains one or more logic functions that implement the specified executable instructions.
  • the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown one after another may actually execute substantially in parallel, or they may sometimes execute in the reverse order, depending on the functionality involved.
  • each block of the block diagram and/or flowchart illustration, and combinations of blocks in the block diagram and/or flowchart illustration can be implemented by special purpose hardware-based systems that perform the specified functions or operations. , or can be implemented using a combination of specialized hardware and computer instructions.
  • the units involved in the embodiments of the present disclosure can be implemented in software or hardware. Among them, the name of the unit/module does not constitute a limitation on the unit itself under certain circumstances.
  • FPGAs Field Programmable Gate Arrays
  • ASICs Application Specific Integrated Circuits
  • ASSPs Application Specific Standard Products
  • SOCs Systems on Chips
  • CPLD Complex Programmable Logical device
  • a machine-readable medium may be a tangible medium that may contain or store a program for use by or in connection with an instruction execution system, apparatus, or device.
  • the machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium.
  • Machine-readable media may include, but are not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices or devices, or any suitable combination of the foregoing.
  • machine-readable storage media would include one or more wire-based electrical connections, laptop disks, hard drives, random access memory (RAM), read only memory (ROM), erasable programmable read only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination of the above.
  • RAM random access memory
  • ROM read only memory
  • EPROM or flash memory erasable programmable read only memory
  • CD-ROM portable compact disk read-only memory
  • magnetic storage device or any suitable combination of the above.
  • Example 1 provides an image processing method, which includes:
  • Extract content-related information from the high-frequency information write the content-related information into the low-frequency information, and obtain a low-resolution image corresponding to the original image.
  • Example 2 provides an image processing method, wherein,
  • the extraction of content-related information from the high-frequency information includes:
  • the high-frequency information is processed based on a pre-trained information extraction model to obtain content-related information and content-independent information in the high-frequency information.
  • Example 3 provides an image processing method, wherein,
  • the information extraction model is an attention reversible transformation network model
  • the information extraction model based on pre-training processes the high-frequency information to obtain content-related information and content-independent information in the high-frequency information, including:
  • the high-frequency information and the low-frequency information are input into the attention reversible transformation network model to obtain content-related information and content-irrelevant information in the high-frequency information, where the low-frequency information is the response to the high-frequency information.
  • Frequency information is an auxiliary condition for extracting content-related information.
  • Example 4 provides an image processing method, wherein,
  • Writing the content-related information into the low-frequency information to obtain a low-resolution image corresponding to the original image includes:
  • the content-related information and the low-frequency information are data fused in the channel dimension to obtain a low-resolution image corresponding to the original image.
  • Example 5 provides an image processing method, wherein,
  • the data fusion of the content-related information and the low-frequency information in the channel dimension includes:
  • Example 6 provides an image processing method, wherein,
  • the extraction of high-frequency information and low-frequency information in the original image includes:
  • the original image is downsampled to obtain low-frequency information;
  • the initial high-frequency information is determined based on the high-resolution low-frequency image obtained by upsampling the original image and the low-frequency information, and the spatial pixels in the initial high-frequency information are re-sampled.
  • Example 7 provides an image processing method, including:
  • the high-frequency information is determined based on the content-related information, and the original image corresponding to the low-resolution image is obtained based on the fusion of the high-frequency information and the low-frequency information.
  • Example 8 provides an image processing method, wherein:
  • Extracting content-related information from low-frequency information and high-frequency information based on the low-resolution image includes:
  • Example 9 provides an image processing method, wherein,
  • Determining the first channel data of the low-frequency information based on the channel data in the low-resolution image includes:
  • determining the second channel data of the content-related information based on the channel data in the low-resolution image includes:
  • the channel data in the low-resolution image is copied as the second channel data of the corresponding data channel in the content-related information.
  • Example 10 provides an image processing method. Before determining the high-frequency information based on the content-related information, the method further includes:
  • determining the high-frequency information based on the content-related information includes:
  • the high-frequency information is determined based on the content-related information and the resampled content-independent information.
  • Example 11 provides an image processing method, wherein,
  • Determining the high-frequency information based on the content-related information and the resampled content-independent information includes:
  • the content-related information and the content-irrelevant information are reversely input into the attention reversible transformation network model to obtain the high-frequency information output by the attention reversible transformation network model.
  • Example 12 provides an image processing method, wherein,
  • the original image corresponding to the low-resolution image is obtained based on the fusion of the high-frequency information and the low-frequency information, including:
  • the low-frequency information is upsampled to obtain a high-resolution low-frequency image;
  • the channel data of the high-frequency information is spatially inversely rearranged to obtain a high-resolution high-frequency image, and the high-resolution low-frequency image is obtained based on the high-resolution low-frequency image and the
  • the original image is obtained from the above-mentioned high-resolution and high-frequency image;
  • Example 13 provides an image processing device, which includes:
  • An information extraction model configured to obtain the original image and extract high-frequency information and low-frequency information in the original image
  • a low-resolution image generation module is configured to extract content-related information from the high-frequency information, write the content-related information into the low-frequency information, and obtain a low-resolution image corresponding to the original image.
  • Example 14 provides an image processing device, which includes:
  • An information extraction model configured to obtain a low-resolution image, and extract content-related information in low-frequency information and high-frequency information based on the low-resolution image;
  • An original image generation module is configured to determine the high-frequency information based on the content-related information, and obtain an original image corresponding to the low-resolution image based on the fusion of the high-frequency information and the low-frequency information.

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • General Health & Medical Sciences (AREA)
  • Molecular Biology (AREA)
  • Biophysics (AREA)
  • Computational Linguistics (AREA)
  • Data Mining & Analysis (AREA)
  • Evolutionary Computation (AREA)
  • Artificial Intelligence (AREA)
  • Biomedical Technology (AREA)
  • Computing Systems (AREA)
  • General Engineering & Computer Science (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Mathematical Physics (AREA)
  • Software Systems (AREA)
  • Health & Medical Sciences (AREA)
  • Editing Of Facsimile Originals (AREA)
  • Image Processing (AREA)

Abstract

本公开实施例公开了图像处理方法、装置、存储介质及电子设备。该方法包括获取原始图像，并提取所述原始图像中的高频信息和低频信息(S110)；提取所述高频信息中的内容相关信息，并将所述内容相关信息写入所述低频信息中，以得到所述原始图像对应的低分辨率图像(S120)。

Description

图像处理方法、装置、存储介质及电子设备
本申请要求在2022年4月11日提交中国专利局、申请号为202210377047.4的中国专利申请的优先权，该申请的全部内容通过引用结合在本申请中。
技术领域
本公开实施例涉及计算机数据处理技术领域，例如涉及一种图像处理方法、装置、存储介质及电子设备。
背景技术
数字图像在存储、展示和传输过程中往往需要进行重采样，一方面重采样后的图像可以适应不同分辨率设备更好的浏览图像，另一方面对图像进行下采样处理并将下采样图像进行存储和传输，在终端设备上展示时再对下采样图像进行上采样处理。相关技术中采用同一套模型正向下采样处理，以及逆向上采样处理。在实施的过程中，下采样处理会带来图像分辨率的降低，从而实现存储和传输成本的减少，但是上述下采样处理也会导致图像中高频信息的丢失，从而使得到低分辨率图像在后续进行上采样过程中重建图像的细节丢失，从而降低了低分辨率图像在上采样处理后得到的高分辨率图像的图像细节不足，降低了图像质量。
发明内容
本公开实施例提供了一种图像处理方法、装置、存储介质及电子设备，以实现得到的低分辨率图像保存原图像中的内容相关信息，从而实现在降低图像数据量的同时提高图像质量。
第一方面，本公开实施例提供了一种图像处理方法，该方法包括：获取原始图像，并提取所述原始图像中的高频信息和低频信息；
提取所述高频信息中的内容相关信息，并将所述内容相关信息写入所述低频信息中，以得到所述原始图像对应的低分辨率图像。
第二方面，本公开实施例还提供了另一种图像处理方法，该方法包括：获取低分辨率图像，并基于所述低分辨率图像提取低频信息和高频信息中的内容相关信息；
基于所述内容相关信息确定所述高频信息，并基于所述高频信息和所述低频信息融合得到所述低分辨率图像对应的原始图像。
第三方面，本公开实施例还提供了一种图像处理装置，该装置包括：信息提取模型，设置为获取原始图像，并提取所述原始图像中的高频信息和低频信息；
低分辨率图像生成模块，设置为提取所述高频信息中的内容相关信息，并将所述内容相关信息写入所述低频信息中，以得到所述原始图像对应的低分辨率图像。
第四方面，本公开实施例还提供了另一种图像处理装置，该装置包括：信息提取模型，设置为获取低分辨率图像，并基于所述低分辨率图像提取低频信息和高频信息中的内容相关信息；
原始图像生成模块，设置为基于所述内容相关信息确定所述高频信息，并基于所述高频信息和所述低频信息融合得到所述低分辨率图像对应的原始图像。
第五方面，本公开实施例还提供了一种电子设备，所述电子设备包括：
一个或多个处理器；
存储装置，设置为存储一个或多个程序，
当所述一个或多个程序被所述一个或多个处理器执行，使得所述一个或多个处理器实现如本公开实施例任一所述的图像处理方法。
第六方面，本公开实施例还提供了一种包含计算机可执行指令的存储介质，所述计算机可执行指令在由计算机处理器执行时用于执行如本公开实施例任一所述的图像处理方法。
附图说明
贯穿附图中，相同或相似的附图标记表示相同或相似的元素。应当理解附图是示意性的，原件和元素不一定按照比例绘制。
图1为本公开实施例提供的一种图像处理方法的流程示意图；
图2是本公开实施例提供的一种空间重排的流程示意图；
图3是本公开实施例提供的另一种图像处理方法的流程示意图；
图4是本公开实施例提供的另一种图像处理方法的流程示意图；
图5是本公开实施例提供的一种图像处理装置的结构示意图；
图6是本公开实施例提供的另一种图像处理装置的结构示意图；
图7为本公开实施例提供的一种电子设备的结构示意图。
具体实施方式
应当理解，本公开的方法实施方式中记载的多个步骤可以按照不同的顺序执行，和/或并行执行。此外，方法实施方式可以包括附加的步骤和/或省略执行示出的步骤。本公开的范围在此方面不受限制。
本文使用的术语“包括”及其变形是开放性包括，即“包括但不限于”。术语“基于”是“至少部分地基于”。术语“一个实施例”表示“至少一个实施例”；术语“另一实施例”表示“至少一个另外的实施例”；术语“一些实施例”表示“至少一些实施例”。其他术语的相关定义将在下文描述中给出。
需要注意，本公开中提及的“第一”、“第二”等概念仅用于对不同的装置、模块或单元进行区分，并非用于限定这些装置、模块或单元所执行的功能的顺序或者相互依存关系。
需要注意，本公开中提及的“一个”、“多个”的修饰是示意性而非限制性的，本领域技术人员应当理解，除非在上下文另有明确指出，否则应该理解为“一个或多个”。
一些实施例中，在将图像进行图像传输或者图像存储之前，会对原始图像进行图像下采样处理，得到原始图像对应的低分辨率图像，从而降低原始数据的数据量。进而对该低分辨率图像进行图像传输或者图像存储，以实现减小传输或者存储过程中所占用的数据资源，从而降低传输或者存储成本。例如，在对该原始图像进行展示时，获取该原始图像的低分辨率图像，并对该低分辨率进行上采样处理，得到低分辨率图像对应的高分辨率图像，以呈现更清晰的图像。为了使上采样后的高分辨率图像尽可能地恢复原始图像中的图像细节，相关技术在实时的过程中对原始图像进行下采样处理所采用的方法包括：将原始图像中的高频信息映射到高斯分布上，从而在后续上采样的过程中可以提取高斯分布上的图像信息，以得到具有更多原始图像信息的高分辨率图像。申请人发现上述方法过程中发现有多处局限性；包括： 在得到原始图像的高频信息后，没有对高频信息根据图像内容进一步地进行信息分离，由于所有高频信息无法在无损且可逆的情况下转换到一个容易度量的高斯分布上，导致高频信息中携带的内容相关信息可能会丢失，以及忽视了下采样低分辨率图像能隐形存储大量信息的能力，从而导致图像在上采样时可以采用的图像内容信息过少，恢复出的原始图像细节不足。
针对上述技术问题，为了使上采样后的高分辨率图像中可以恢复更多的原始图像的细节，本公开实施例提供的技术方案，参见图1，图1为本公开实施例所提供的一种图像处理方法流程示意图。本公开实施例适应于在对图像进行下采样以及上采样的情况，该方法可以由本公开实施例提供的图像处理装置来执行，该图像处理装置可以通过软件和/或硬件的形式实现，例如，通过电子设备来实现，该电子设备可以是移动终端或PC端等。如图1，本实施例的方法包括：
S110、获取原始图像，提取所述原始图像中的高频信息和低频信息。
S120、提取所述高频信息中的内容相关信息，将所述内容相关信息写入所述低频信息中，得到所述原始图像对应的低分辨率图像。
本实施例中，对待处理的原始图像进行高频信息和低频信息的分解，并在得到原始图像的高频信息后，对高频信息进行分解，提取到高频信息中关于原始图像的内容相关信息，并将内容相关信息作为隐写信息写入到下采样后的低频信息中，得到原始图像的低分辨率图像，使得该低分辨率图像中包括更多的纹理信息，相应的，在通过逆向上采样处理的过程中得出与图像内容相关的高频信息，准确获得带有更多纹理细节的高分辨率图像，从而实现提高图像质量。
本公开实施例中，原始图像可以理解为未进行处理过的图像。低分辨率图像可以理解为包含原始图像中图像内容，且数据量减小的图像。
例如，为了得到原始图像的低分辨率图像，首先需要对原始图像进行图像信息提取，分别提取原始图像中的高频信息和低频信息，其中，低频信息中包括图像内容信息，高频信息中包括诸如边缘等的纹理信息。
在一些实施例中，提取原始图像中的高频信息和低频信息的方法可以包括：对原始图像进行下采样，得到低频信息；基于原始图像和低频信息上采样得到的高分辨率低频图像确定初始高频信息，对初始高频信息中的空间像素重排至通道维度，得到与低频信息相匹配的高频信息。
在本公开实施例中，原始图像可以表示为(H，W，C)；其中，H表示为原始图像的高度数据，W表示为原始图像的宽度数据，C表示为原始数据的通道数据，例如C可以是3，表示原始图像的通道数据为分别包括RGB通道的3个通道。例如，对原始图像进行下采样处理，得到原始图像的低频信息，并对得到的低频信息直接进行上采样处理得到低频信息对应的高分辨率低频图像，并基于原始图像的图像数据与高分辨率低频图像的图像数据确定原始图像中的初始高频信息。示例性的，可以基于上述两种图像的图像数据差确定原始图像中的初始高频信息，例如原始图像和高分辨率低频图像分辨率相同，将原始图像和高分辨率低频图像中对应像素点的像素值分别进行差值处理，得到初始高频信息。例如，对该初始高频信息中的空间像素进行重新排列，将空间像素排列至通道维度，从而得到与低频信息相匹配的高频信息，其中，高频信息与低频信息的分辨率相同。例如，如图2所示，将该初始高频信息中的空间像素重排至通道维度可以理解为减小初始高频信息中的高度数据和宽度数据，同 时将超出高度数据和宽度数据对应的像素数据排列在其他通道，从而通过增加通道维度数，得到与低频信息的分辨率相匹配的高频信息。示例性的，低频信息可表示为(H/2，W/2，C)，高频信息可表示为H/2，W/2，4C)
需要说明的是，上述相匹配可以理解为低频信息与高频信息的图像分辨率相匹配，即低频信息中的高度数据与高频信息中的高度数据相匹配，以及低频信息中的宽度数据与高频信息中的宽度数据相匹配。本公开实施例的技术方案得到与低频信息匹配的高频信息的有益效果在于便于之后将高频信息中提取出的内容相关信息隐写入至低频信息中。其中，隐写可以理解为将原始图像中的内容相关信息隐藏存储至低频信息中，形成包含原始图像的与图像内容相关的高频信息的低分辨率图像，充分利用了低分辨率图像隐形存储大量信息的能力，从而使后续对该低分辨率图像进行上采样处理，得到高分辨率图像的过程中逆向提取出原始图像高频信息中与图像内容相关的部分信息，以提高逆向恢复处图像的图像质量。
示例性的，对分辨率为(H，W，C)的原始图像进行2倍下采样处理，即使用双三次插值下采样2倍，得到原始图像的低频信息(H/2，W/2，C)；例如，对该低频信息(H/2，W/2，C)采用双三次插值上采样2倍，得到与原始图像的宽度数据和长度数据一样的高分辨率低频图像(H，W，C)，并将上述高分辨率低频图像与原始图像的图像数据相减，得到原始图像的初始高频信息；例如，对该初始高频信息中的空间像素进行重新排列，将空间像素排列至通道维度，从而得到与低频信息相匹配的高频信息(H/2，W/2，4C)。
在一些实施例中，提取原始图像中的高频信息和低频信息的方法还可以包括：对原始图像进行滤波变换，得到原始图像中的高频信息和低频信息。其中，滤波变换可以采用小波变换，例如还可以采用哈尔变换对输入信息进行高通滤波处理，得到分离的高频信息和低频信息。例如，采用滤波变换对原始图像中的图像信息进行高通滤波处理，得到原始图像的高频信息和低频信息。示例性的，使用滤波变换，通过高通滤波对原始图像(H，W，C)进行信息分离，分别得到下采样4倍后的低频信息(H/4，W/4，C)和高频信息(H/4，W/4，15C)。
在上述下采样的过程中，对原始图像进行下采样处理的采用倍数包括2倍以及4倍。需要说明的是，上述采样倍数只是作为示例实施例进行示例性的介绍，本实施例的技术方案还可以采用其他采样倍数，本实施例对此不作限定。
在得到原始图像的高频信息和低频信息之后，对原始图像的高频信息进行信息提取，得到高频信息中的内容相关信息，将内容相关信息写入低频信息中，从而得到原始图像对应的低分辨率图像。
需要解释的是，原始图像的高频信息中包括内容相关信息和内容无关信息。内容相关信息可以理解为与原始图像中的图像内容相关的信息；内容无关信息可以理解为原始图像中的图像噪声等其他信息。例如，提取高频信息中的内容相关信息的方法可以采用预先训练的神经网络模型进行提取。例如，可以基于预先训练的信息提取模型对高频信息进行处理，得到高频信息中的内容相关信息和内容无关信息。例如，信息提取模型可以是网络结构模块，该信息提取模型的网络结构可以是诸如卷积神经网络、多层感知器等的结构，对此不作限定。
在上述实施例的基础上，信息提取模型可以为注意力可逆变换网络模型。相应的，基于预先训练的信息提取模型对高频信息进行处理，得到高频信息中的内容相关信息和内容无关信息的方法可以包括：将高频信息和低频信息输入至注意力可逆变换网络模型中，得到高频信息中的内容相关信息和内容无关信息，其中，低频信息为对高频信息进行内容相关信息提 取的辅助条件。
需要说明的是，低频信息中包含有原始图像的内容信息，即将低频信息作为进行内容相关信息提取的辅助条件，与高频信息同时输入至注意力可逆变换网络模型中，可以将低频信息中包含的内容信息作为注意力可逆变换网络在提取信息时的学习目标，从而实现注意力可逆变换网络在提取信息时将高频信息中与原始图像的图像内容信息准确地进行提取。
在采用注意力可逆变换网络模型进行信息提取之前，需要先对该注意力可逆变换网络模型进行模型训练。例如，注意力可逆变换网络模型训练方法可以包括：将原始样本图像中的高频信息输入至待训练的注意力可逆变换网络模型中，得到注意力可逆变换网络模型输出的内容相关信息和内容无关信息，并将该内容相关信息和内容相关信息逆向输入至该待训练的注意力可逆变换网络模型中，得到注意力可逆变换网络模型反向输出的预测高频信息。基于该预测高频信息和原始图像中的高频信息生成该待训练的注意力可逆变换网络模型的损失函数，并基于该损失函数对该待训练的注意力可逆变换网络模型的模型参数进行调节，迭代执行上述训练过程，直至得到训练完成的注意力可逆变换网络模型。
示例性的，将上述信息分离得到的与低频信息相匹配的高频信息(H/2，W/2，4C)进一步进行信息提取。例如，基于注意力可逆变换网络模型将高频信息(H/2，W/2，4C)按一个分离比例p分离为内容相关信息(H/2，W/2，4pC)和内容无关信息(H/2，W/2，4C-4pC)。需要说明的是，上述是下采样2倍的高频信息的信息分离情况；下采样4倍的高频信息(H/4，W/4，15C)的分离情况为内容相关信息(H/4，W/4，15pC)和内容无关信息(H/4，W/4，15C-15pC)。其中，分离比例p为注意力可逆变换网络模型中预先设置的，对此不作限定。例如，下采样2倍对应的分离比例p大于下采样4倍对应的分离比例p。需要说明的是，不同的下采样倍数可以对应不同的注意力可逆变换网络模型。
在本公开实施中，在分别得到原始图像中的低频信息和原始图像中高频信息的内容相关信息之后，进一步地将内容相关信息写入低频信息中，从而得到原始图像对应的低分辨率图像。例如，将内容相关信息写入低频信息中，得到原始图像对应的低分辨率图像的方法可以包括：将内容相关信息和低频信息在通道维度上进行数据融合，得到原始图像对应的低分辨率图像。
例如，由于为了得到与低频信息相匹配的高频信息，将原始图像中的初始高频信息的空间像素重排至通道维度，从而后续得到的高频信息与低频信息中的高度数据和宽度数据相匹配，高频信息的通道维度大于低频信息的通道维度，相应的，高频信息中提取的内容相关信息的通道维度也大于低频信息的通道维度。
在此基础上，将内容相关信息和低频信息在通道维度上进行数据融合可以包括：确定低频信息中的第一通道数据，对应的内容相关信息中的至少一个第二通道数据；将具有对应关系的第一通道数据和第二通道数据进行数据融合。
本实施例中，通道数据可以理解为多个通道的像素数据，例如，低频信息中包括RGB三个第一通道，第一通道数据可以理解为低频信息中的R通道数据、G通道数据和B通道数据；第二通道数据可以理解为内容相关信息中多个通道的像素数据；值得注意的是，由于内容相关信息的通道维度大于低频信息的通道维度，相应的，第一通道数据的通道维度少于第二通道数据的通道维度，因此，第一通道的通道数据与第二通道的通道数据之间的通道对应关系为一对多的关系，即需要确定低频信息中的第一通道数据，以及对应的内容相关信息中 的至少一个第二通道数据。
本实施例中，可预先设置第一通道数据和第二通道数据的对应关系，且对应关系的设置方式不作限定。例如，第一通道数据与第二通道数据的相同类型通道具有对应关系，示例性的，第一通道数据包括RGB三通道，第二通道数据分别包括九通道，即RGB通道的数量分别为是三个。相应的，将第一通道数据中的R通道与第二通道数据中的三个R通道相对应，将第一通道数据中的G通道与第二通道数据中的三个G通道相对应，将第一通道数据中的B通道与第二通道数据中的三个B通道相对应。例如，第一通道数据的多个通道与第二通道数据中的多个通道依次轮询间隔对应。示例性的，低频信息的分辨率为(H/2，W/2，C)，第一通道数据的通道维度可以理解为3个通道；内容相关信息为(H/2，W/2，4pC)，以预设分离比例p为75％为例，第二通道数据的通道维度可以理解为9个通道。确定第一通道数据中3个通道和第二通道数据中9个通道之间的对应关系。其对应关系可以是第一通道数据中的第一通道对应第二通道数据中的第一通道、第四通道、第七通道，第一通道数据中的第二通道对应第二通道数据中的第二通道、第五通道、第八通道，第一通道数据中的第三通道对应第二通道数据中的第三通道、第六通道、第九通道。例如，第一通道数据的多个通道与第二通道数据中的多个通道依序分组对应，对于低频信息的分辨率为(H/2，W/2，C)，和内容相关信息为(H/2，W/2，4pC)，其对应关系还可以是第一通道数据中的第一通道对应第二通道数据中的第一通道至第三通道，第一通道数据中的第二通道对应第二通道数据中的第四通道、至第六通道，第一通道数据中的第三通道对应第二通道数据中的第七通道至第九通道。当然，还可以是其他的对应关系，本实施例对此不作限定。
例如，在确定第一通道数据和第二通道数据之间的通道对应关系之后，将具有对应关系的第一通道数据中的通道数据和第二通道数据中的通道数据进行数据融合，得到原始图像对应的低分辨率图像。例如，若下采样2倍，则得到低分辨率图像的过程可以包括：((H/2，W/2，C)，(H/2，W/2，4pC))->(H/2，W/2，C)；若下采样4倍，则得到低分辨率图像的过程可以包括：((H/4，W/4，C)，(H/2，W/2，15pC))->(H/4，W/4，C)。
在上述实施例的基础上，在提取到高频信息的内容无关信息之后，本实施例的技术方案还将其内容无关信息映射到一个标准正态分布上。其中，标准正态分布可以理解为均值为0，方差为1的正态分布。将内容无关信息映射到正态分布上的效果在于可以在将低分辨率上采样为原始图像的过程中，对该正态分布进行信息提取，将提取到的信息作为原始图像的图像噪声信息进行图像处理，从而恢复出更加接近于原始图像的高分辨率图像。当然，一些实施例中也可以不对内容无关信息进行处理，相应的在将低分辨率上采样以恢复原始图像的过程中，可以直接基于标准正态分布进行信息提取，将提取到的噪声数据与低分辨率图像进行图像处理，得到高分辨率图像。
本公开实施例的技术方案，通过获取原始图像，提取原始图像中的高频信息和低频信息；提取高频信息中的内容相关信息，将内容相关信息写入低频信息中，得到原始图像对应的低分辨率图像。上述技术方案通过对原始图像进行信息提取，得到原始图像的高频信息，并对高频信息进行分解，提取到高频信息中关于原始图像的内容相关信息，并将内容相关信息作为隐写信息写入到下采样后的低分辨率图像中，从而实现在对该低分辨率图像进行逆向上采样处理的过程中，获得带有更多纹理细节的高分辨率图像，在降低图像数据量的同时提高图像质量。
图3为本公开实施例所提供的另一种图像处理方法流程示意图。该方法可以由本公开实施例提供的图像处理装置来执行，该图像处理装置可以通过软件和/或硬件的形式实现，例如，通过电子设备来实现，该电子设备可以是移动终端或PC端等。如图3，本实施例的方法包括：
S210、获取低分辨率图像，基于低分辨率图像提取低频信息和高频信息中的内容相关信息。
S220、基于内容相关信息确定高频信息，并基于高频信息和低频信息融合得到低分辨率图像对应的原始图像。
在本公开实施例中，为了得到低分辨率对应的原始图像，需要对获取到的低分辨率图像进行上采样处理，得到高分辨率的原始图像。
例如，为了得到低分辨率图像对应的原始图像，首先需要对低分辨率图像进行信息提取，分别提取低分辨率图像中的低频信息和高频信息中的内容相关信息。其中，内容相关信息包括在原始图像下采样处理得到低分率图像的过程中，隐写入低分辨率图像的部分高频信息。需要说明的是，从低分辨率图像中提取低频信息和内容相关信息的过程，与上述实施例中，将内容相关信息隐写入低频信息的过程为可逆过程，二者处理方式存在对应关系。
在一些实施例中，基于低分辨率图像提取低频信息和高频信息中的内容相关信息的方法可以包括：确定低频信息中数据通道与低分辨率图像中数据通道的对应关系，基于低分辨率图像中的通道数据确定低频信息的第一通道数据；确定内容相关信息中数据通道与低分辨率图像中数据通道的对应关系，基于低分辨率图像中的通道数据确定内容相关信息的第二通道数据。
例如，低分辨率图像是基于低频信息和内容相关信息在通道维度上进行数据融合所得到的，可以理解为是基于具有对应关系的低频信息的通道数据和内容相关信息的通道数据进行数据融合所得到的。相应的，在提取低分辨率图像中的低频信息和内容相关信息的过程中，需要确定低分辨率图像中低频信息中数据通道与低分辨率图像中数据通道的对应关系，以及内容相关信息中数据通道与低分辨率图像中数据通道的对应关系。基于低分辨率图像中的通道数据确定低频信息的第一通道数据，以及基于低分辨率图像中的通道数据确定内容相关信息的第二通道数据。
本公开实施例的技术方案中值得注意的是，确定第一通道数据的顺序和第而通道数据的顺序不分先后，可以顺序执行，也可以同时执行，本实施例对此不做限定。
示例性的，若低分辨率图像的分辨率为(H/2，W/2，C)，则在C＝3的情况下，可以理解为低分辨率图像的通道维度为3个通道。例如，获取低频信息中数据通道与低分辨率图像中数据通道的对应关系，若其数据通道的对应关系为一一对应的关系，相应的低分辨率图像中低频信息的通道维度也为3个通道，低频信息的分辨率为(H/2，W/2，C)；基于低分辨率图像中的通道数据对应确定低频信息中的第一通道数据(H/2，W/2，C)。
例如，确定第一通道数据的方法可以是将低分辨率图像中通道数据进行数据复制，作为低频信息中对应数据通道的第一通道数据。
例如，获取内容相关信息中数据通道与低分辨率图像中数据通道的对应关系。例如，其对应关系可以是内容相关信息中的第一通道、第四通道和第七通道与低分辨率图像中的第一通道对应；基于低分辨率图像中第一通道的通道数据分别确定第二通道数据中的第一通道、第四通道和第七通道的通道数据；并相应的基于其余通道之间的通道对应关系确定第二通道 数据中其他通道的通道数据。例如，其对应关系还可以是内容相关信息中的第一通道至第三通道对应低分辨率图像中的第一通道；基于低分辨率图像中第一通道的通道数据分别确定第二通道数据中的第一通道至第三通道的通道数据，并相应的基于其余通道之间的通道对应关系确定第二通道数据中其他通道的通道数据。当然，内容相关信息中数据通道与低分辨率图像中数据通道的对应关系还可以是其他的对应关系，相应的，基于其他的对应关系以及低分辨率图像中的通道数据确定内容相关信息的第二通道数据，本实施例对此不作限定。
在上述实施例的基础上，基于低分辨率图像中的通道数据确定内容相关信息的第二通道数据的方法可以包括：将低分辨率图像中的通道数据进行数据复制，作为内容相关信息中对应数据通道的第二通道数据。
在一些实施例中，基于低分辨率图像中的通道数据确定内容相关信息的第二通道数据的方法还可以包括：将低分辨率图像中的通道数据在预设范围内的随机数值作为内容相关信息中对应数据通道的第二通道数据。示例性的，以基于低分辨率图像中第一通道的通道数据分别确定第二通道数据中的第一通道至第三通道的通道数据为例进行介绍：低分辨率图像中第一通道中任一数据例如可以为230，该数据的预设范围可以是230±5，相应的，第二通道数据中的第一通道至第三通道中对应的数据可以是在230±5的范围随机采样得到，例如，第二通道数据内第一通道中对应数据可以是232，第二通道中对应的数据为235、第三通道中对应的数据为228。例如，得到内容相关信息中对应数据通道的第二通道数据可以表示为(H/2，W/2，4pC)。
本实施例中，为了基于内容相关信息确定低分辨率图像对应的高频信息，以及为了提高得到高分辨率数据的还原的真实性，需要获取内容无关数据，并基于内容相关数据和内容无关数据确定该高频信息。例如，确定内容无关数据的方法可以包括：在内容无关信息对应的预设数据分布中，进行信息重采样，得到内容无关信息。其中，预设数据分布可以理解为在对原始数据进行下采样得到低分辨率图像的过程中，将原始图像中高频信息的内容无关信息所映射到的正态分布；当然还可以是选用的其他预设数据分布，本实施例对此不作限定。
例如，对上述映射过的正态分布进行内容无关信息提取，将提取到的信息作为原始图像的图像噪声信息，并与内容相关图像进行信息重采样，得到低分辨率对应的高频信息。当然，一些实施例中也可以直接基于标准正态分布进行信息提取，将内容无关信息与内容相关信息进行信息重采样，得到高频信息。
在一些实施例中，将内容无关信息与内容相关信息进行信息重采样，得到高频信息的方法可以包括：将内容相关信息和内容无关信息反向输入至注意力可逆变换网络模型，得到注意力可逆变换网络模型输出的高频信息。
示例性的，从低分辨率图像中提取出内容相关信息，若上采样2倍，则内容相关信息表示为(H/2，W/2，4pC)；若上采样4倍，则内容相关信息表示为(H/2，W/2，15pC)。例如，获取内容无关信息若上采样2倍，则内容无关信息表示为(H/2，W/2，4C-4pC)；若上采样4倍，则内容无关信息表示为(H/4，W/4，15C-15pC)。例如，若下采样2倍，则得到高频信息的过程可以包括((H/2，W/2，4C-4pC)，(H/2，W/2，4pC))->(H/2，W/2，4C)；若下采样4倍，则得到低分辨率图像的过程可以包括：((H/2，W/2，15pC)，(H/2，W/2，15pC))->(H/4，W/4，15C))。
例如，在得到低分辨率图像的低频信息与高频信息之后，基于高频信息和低频信息融合 得到低分辨率图像对应的原始图像。例如，得到低分辨率图像对应的原始图像的方法可以包括：对低频信息进行上采样，得到高分辨率低频图像；对高频信息的通道数据进行空间逆重排，得到高分辨率高频图像，并基于高分辨率低频图像和高分辨率高频图像得到原始图像。
例如，对得到的低频信息直接进行上采样处理得到低频信息对应的高分辨率低频图像；以及对高频信息中的数据通道进行空间逆重排，得到高频信息对应的高分辨率高频图像，例如，将高分辨率低频图像和高分辨率高频图像进行图像融合，得到低分辨率图像对应的原始图像。
示例性的，对分辨率为(H/2，W/2，C)的原始图像进行2倍上采样处理，即使用双三次插值上采样2倍，得到高分辨率低频图像(H，W，C)；将高频信息(H/2，W/2，4C)进行空间逆重排，得到与高分辨率低频图像的宽度数据和长度数据一样的高分辨率高频图像(H，W，C)，进而将高分辨率低频图像和高分辨率高频图像进行图像融合，得到低分辨率图像对应的原始图像(H，W，C)。
在一些实施例中，得到低分辨率图像对应的原始图像的方法还可以包括：对高频信息和低频信息进行哈尔逆变换，得到原始图像。
示例性的，使用哈尔逆变换对低频信息(H/4，W/4，C)和高频信息(H/4，W/4，15C)进行信息融合，得到低分辨率图像对应的原始图像(H，W，C)。
本公开实施例的技术方案，通过获取低分辨率图像，基于低分辨率图像提取低频信息和高频信息中的内容相关信息；基于内容相关信息确定高频信息，并基于高频信息和低频信息融合得到低分辨率图像对应的原始图像。通过上述技术方案，获得了带有更多纹理细节的高分辨率图像，从而提高图像质量。
在上述实施例的基础上，本公开实施例还提供了一个应用实施例，用于解释基于高分辨率图像得到低分辨率图像，再基于低分辨率图像反向得到高分辨率图像的步骤。
在对下述应用实施例进行介绍之前先对该实施例的应用场景进行适应性的介绍。例如，该交互实施例可以适用于图像传输的过程中。示例性的，可以是客户端与服务器之间的数据传输，例如，客户端向服务器请求下发图像/视频，相应的，服务器在进行图像/视频传输之前，先将待传输图像或待传输视频中的多个视频帧(即高分辨率图像)基于上述实施例的方式进行处理，得到对应的低分辨率图像/视频，并对低分辨率图像/视频进行传输，以减小传输数据量，从而减小传输成本。客户端在收到服务器下发的低分辨率图像/视频之后，对该低分辨率图像或低分辨率视频中的多个图像帧分别基于上述实施例提供的处理方式，得到该低分辨率图像/视频对应的高分辨率图像/视频进行图像展示。当然，上述图像/视频传输还可以是客户端和客户端之间的图像传输，本实施例对此不作限定。例如，该应用实施例还可以是用于图像/视频存储的过程中。示例性的，客户端将图像进行本地存储的过程中，可以对待存储图像或待存储视频中的图像帧基于上述实施例提供的处理方式，得到低分辨率图像/视频，并对低分辨率图像/视频进行存储。当需要对存储的图像/视频进行处理或者展示时，对该低分辨率图像或低分辨率视频中的图像帧基于上述实施例提供的处理方式，得对应的高分辨率图像/视频，以执行后续的展示或者处理。
上述应用场景只是作为本应用实施例的示例应用场景，本实施例还可以应用于其他应用场景，在此本实施例不再一一赘述。
如图4所示，该应用实施例包括如下步骤：
步骤一：将高分辨率图像(原始图像)进行高低频信息分离，分别得到高频信息和低频信息。
步骤二：对高频信息进行信息分离，分别得到内容相关信息和内容无关信息。
步骤三：将内容相关信息隐写入低频信息中，得到高分辨率图像对应的低分辨率图像。
步骤四：将内容无关信息映射为高斯噪声。
步骤五：对低分辨率图像进行信息提取，分别得到低频信息和高频信息中的内容相关信息。
步骤六：对高斯噪声进行重采样，得到内容无关信息。
步骤七：将内容无关信息和内容相关信息进行信息融合，得到高频信息。
步骤八：将高频信息和低频信息进行高低频信息融合，得到高分辨率图像。
值得注意的是，上述多个步骤中，步骤一至步骤四可以采用同一装置执行，例如采用服务器执行，同时步骤五至步骤八采用另一装置执行，例如采用客户端执行；还可以是步骤一至步骤八均采用同一装置执行，例如服务器或者客户端执行，本实施例对多个步骤的执行装置不作限制。
图5是本公开实施例所提供的一种图像处理装置的结构示意图。如图5所示，所述装置包括：信息提取模型310和低分辨率图像生成模块320；其中，
信息提取模型310，设置为获取原始图像，提取所述原始图像中的高频信息和低频信息；
低分辨率图像生成模块320，设置为提取所述高频信息中的内容相关信息，将所述内容相关信息写入所述低频信息中，得到所述原始图像对应的低分辨率图像。
本公开实施例的技术方案，
在上述实施例的基础上，低分辨率图像生成模块320，包括：
信息提取子模块，设置为基于预先训练的信息提取模型对所述高频信息进行处理，得到所述高频信息中的内容相关信息和内容无关信息。
在上述实施例的基础上，所述信息提取模型为注意力可逆变换网络模型；
相应的，信息提取子模块，包括：
信息提取单元，设置为将所述高频信息和所述低频信息输入至所述注意力可逆变换网络模型中，得到所述高频信息中的内容相关信息和内容无关信息，其中，所述低频信息为对所述高频信息进行内容相关信息提取的辅助条件。
在上述实施例的基础上，低分辨率图像生成模块320，包括：
数据融合子模块，设置为将所述内容相关信息和所述低频信息在通道维度上进行数据融合，得到原始图像对应的低分辨率图像。
在上述实施例的基础上，数据融合子模块，包括：
数据融合单元，设置为确定低频信息中的第一通道数据，对应的所述内容相关信息中的至少一个第二通道数据；将具有对应关系的第一通道数据和所述第二通道数据进行数据融合。
在上述实施例的基础上，信息提取模型310，包括：
第一信息提取单元，设置为对所述原始图像进行下采样，得到低频信息；基于所述原始图像和所述低频信息上采样得到的高分辨率低频图像确定初始高频信息，对所述初始高频信息中的空间像素重排至通道维度，得到与所述低频信息相匹配的高频信息；
或者，
第二信息提取单元，设置为对所述原始图像进行滤波变换，得到所述原始图像中的高频信息和低频信息。
图6是本公开实施例所提供的另一种图像处理装置的结构示意图。如图6所示，所述装置包括：信息提取模型410和原始图像生成模块420；其中，
信息提取模型410，设置为获取低分辨率图像，基于所述低分辨率图像提取低频信息和高频信息中的内容相关信息；
原始图像生成模块420，设置为基于所述内容相关信息确定所述高频信息，并基于所述高频信息和所述低频信息融合得到所述低分辨率图像对应的原始图像。本公开实施例的技术方案，
在上述实施例的基础上，信息提取模型410，包括：
第一通道数据确定子模块，设置为确定所述低频信息中数据通道与所述低分辨率图像中数据通道的对应关系，基于所述低分辨率图像中通道数据确定所述低频信息的第一通道数据；
第二通道数据确定子模块，设置为确定所述内容相关信息中数据通道与所述低分辨率图像中数据通道的对应关系，基于所述低分辨率图像中的通道数据确定所述内容相关信息的第二通道数据。
在上述实施例的基础上，第一通道数据确定子模块，包括：
第一通道数据确定单元，设置为将所述低分辨率图像中的通道数据进行数据复制，作为所述低频信息中对应数据通道的第一通道数据；
以及，第二通道数据确定子模块，包括：
第二通道数据确定单元，设置为将所述低分辨率图像中的通道数据进行数据复制，作为所述内容相关信息中对应数据通道的第二通道数据。
在上述实施例的基础上，该装置还包括：
信息获取模块，设置为在基于所述内容相关信息确定所述高频信息之前，在内容无关信息对应的预设数据分布中，进行信息重采样，得到内容无关信息；
相应的，信息提取模型410，包括：
高频信息确定子模块，设置为基于所述内容相关信息和重采样得到的内容无关信息确定所述高频信息。
在上述实施例的基础上，高频信息确定子模块，包括：
高频信息确定单元，设置为将所述内容相关信息和所述内容无关信息反向输入至注意力可逆变换网络模型，得到所述注意力可逆变换网络模型输出的高频信息。
在上述实施例的基础上，原始图像生成模块420，包括：
第一原始图像生成单元，设置为对所述低频信息进行上采样，得到高分辨率低频图像；对所述高频信息的通道数据进行空间逆重排，得到高分辨率高频图像，并基于所述高分辨率低频图像和所述高分辨率高频图像得到原始图像；
或者，
第二原始图像生成单元，设置为对所述高频信息和所述低频信息进行哈尔逆变换，得到原始图像。
本公开实施例所提供的装置可执行本公开任意实施例所提供的方法，具备执行方法相应的功能模块和有益效果。
值得注意的是，上述装置所包括的多个单元和模块只是按照功能逻辑进行划分的，但并不局限于上述的划分，只要能够实现相应的功能即可；另外，多个功能单元的具体名称也只是为了便于相互区分，并不用于限制本公开实施例的保护范围。
下面参考图7，其示出了适于用来实现本公开实施例的电子设备(例如图7中的终端设备或服务器)400的结构示意图。本公开实施例中的终端设备可以包括但不限于诸如移动电话、笔记本电脑、数字广播接收器、PDA(个人数字助理)、PAD(平板电脑)、PMP(便携式多媒体播放器)、车载终端(例如车载导航终端)等等的移动终端以及诸如数字TV、台式计算机等等的固定终端。图7示出的电子设备仅仅是一个示例，不应对本公开实施例的功能和使用范围带来任何限制。
如图7所示，电子设备400可以包括处理装置(例如中央处理器、图形处理器等)401，其可以根据存储在只读存储器(ROM)402中的程序或者从存储装置408加载到随机访问存储器(RAM)403中的程序而执行多种适当的动作和处理。在RAM403中，还存储有电子设备400操作所需的多种程序和数据。处理装置401、ROM 402以及RAM 403通过总线404彼此相连。输入/输出(I/O)接口405也连接至总线404。
通常，以下装置可以连接至I/O接口405：包括例如触摸屏、触摸板、键盘、鼠标、摄像头、麦克风、加速度计、陀螺仪等的输入装置406；包括例如液晶显示器(LCD)、扬声器、振动器等的输出装置407；包括例如磁带、硬盘等的存储装置408；以及通信装置409。通信装置409可以允许电子设备400与其他设备进行无线或有线通信以交换数据。虽然图7示出了具有多种装置的电子设备400，但是应理解的是，并不要求实施或具备所有示出的装置。可以替代地实施或具备更多或更少的装置。
根据本公开的实施例，上文参考流程图描述的过程可以被实现为计算机软件程序。例如，本公开的实施例包括一种计算机程序产品，其包括承载在非暂态计算机可读介质上的计算机程序，该计算机程序包含用于执行流程图所示的方法的程序代码。在这样的实施例中，该计算机程序可以通过通信装置409从网络上被下载和安装，或者从存储装置408被安装，或者从ROM402被安装。在该计算机程序被处理装置401执行时，执行本公开实施例的方法中限定的上述功能。
本公开实施例提供的电子设备与上述实施例提供的全息投影模型方法属于同一构思，未在本实施例中详尽描述的技术细节可参见上述实施例，并且本实施例与上述实施例具有相同的有益效果。
本公开实施例提供了一种计算机存储介质，其上存储有计算机程序，该程序被处理器执行时实现上述实施例所提供的图像处理方法。
需要说明的是，本公开上述的计算机可读介质可以是计算机可读信号介质或者计算机可读存储介质或者是上述两者的任意组合。计算机可读存储介质例如可以是——但不限于——电、磁、光、电磁、红外线、或半导体的系统、装置或器件，或者任意以上的组合。计算机可读存储介质的更具体的例子可以包括但不限于：具有一个或多个导线的电连接、便携式计算机磁盘、硬盘、随机访问存储器(RAM)、只读存储器(ROM)、可擦式可编程只读存储器(EPROM或闪存)、光纤、便携式紧凑磁盘只读存储器(CD-ROM)、光存储器件、磁存储器件、或者上述的任意合适的组合。在本公开中，计算机可读存储介质可以是任何包含或存储程序的有形介质，该程序可以被指令执行系统、装置或者器件使用或者与其结合使用。而 在本公开中，计算机可读信号介质可以包括在基带中或者作为载波一部分传播的数据信号，其中承载了计算机可读的程序代码。这种传播的数据信号可以采用多种形式，包括但不限于电磁信号、光信号或上述的任意合适的组合。计算机可读信号介质还可以是计算机可读存储介质以外的任何计算机可读介质，该计算机可读信号介质可以发送、传播或者传输用于由指令执行系统、装置或者器件使用或者与其结合使用的程序。计算机可读介质上包含的程序代码可以用任何适当的介质传输，包括但不限于：电线、光缆、RF(射频)等等，或者上述的任意合适的组合。
在一些实施方式中，客户端、服务器可以利用诸如HTTP(Hyper Text Transfer Protocol，超文本传输协议)之类的任何当前已知或未来研发的网络协议进行通信，并且可以与任意形式或介质的数字数据通信(例如，通信网络)互连。通信网络的示例包括局域网(“LAN”)，广域网(“WAN”)，网际网(例如，互联网)以及端对端网络(例如，ad hoc端对端网络)，以及任何当前已知或未来研发的网络。
上述计算机可读介质可以是上述电子设备中所包含的；也可以是单独存在，而未装配入该电子设备中。
上述计算机可读介质承载有一个或者多个程序，当上述一个或者多个程序被该电子设备执行时，使得该电子设备：
获取原始图像，提取所述原始图像中的高频信息和低频信息；
提取所述高频信息中的内容相关信息，将所述内容相关信息写入所述低频信息中，得到所述原始图像对应的低分辨率图像。
或者，
获取低分辨率图像，基于所述低分辨率图像提取低频信息和高频信息中的内容相关信息；
基于所述内容相关信息确定所述高频信息，并基于所述高频信息和所述低频信息融合得到所述低分辨率图像对应的原始图像。
可以以一种或多种程序设计语言或其组合来编写用于执行本公开的操作的计算机程序代码，上述程序设计语言包括但不限于面向对象的程序设计语言—诸如Java、Smalltalk、C++，还包括常规的过程式程序设计语言—诸如“C”语言或类似的程序设计语言。程序代码可以完全地在用户计算机上执行、部分地在用户计算机上执行、作为一个独立的软件包执行、部分在用户计算机上部分在远程计算机上执行、或者完全在远程计算机或服务器上执行。在涉及远程计算机的情形中，远程计算机可以通过任意种类的网络——包括局域网(LAN)或广域网(WAN)—连接到用户计算机，或者，可以连接到外部计算机(例如利用因特网服务提供商来通过因特网连接)。
附图中的流程图和框图，图示了按照本公开多种实施例的系统、方法和计算机程序产品的可能实现的体系架构、功能和操作。在这点上，流程图或框图中的每个方框可以代表一个模块、程序段、或代码的一部分，该模块、程序段、或代码的一部分包含一个或多个用于实现规定的逻辑功能的可执行指令。也应当注意，在有些作为替换的实现中，方框中所标注的功能也可以以不同于附图中所标注的顺序发生。例如，两个接连地表示的方框实际上可以基本并行地执行，它们有时也可以按相反的顺序执行，这依所涉及的功能而定。也要注意的是，框图和/或流程图中的每个方框、以及框图和/或流程图中的方框的组合，可以用执行规定的功能或操作的专用的基于硬件的系统来实现，或者可以用专用硬件与计算机指令的组合来实现。
描述于本公开实施例中所涉及到的单元可以通过软件的方式实现，也可以通过硬件的方式来实现。其中，单元/模块的名称在某种情况下并不构成对该单元本身的限定。
本文中以上描述的功能可以至少部分地由一个或多个硬件逻辑部件来执行。例如，非限制性地，可以使用的示范类型的硬件逻辑部件包括：现场可编程门阵列(FPGA)、专用集成电路(ASIC)、专用标准产品(ASSP)、片上系统(SOC)、复杂可编程逻辑设备(CPLD)等等。
在本公开的上下文中，机器可读介质可以是有形的介质，其可以包含或存储以供指令执行系统、装置或设备使用或与指令执行系统、装置或设备结合地使用的程序。机器可读介质可以是机器可读信号介质或机器可读储存介质。机器可读介质可以包括但不限于电子的、磁性的、光学的、电磁的、红外的、或半导体系统、装置或设备，或者上述内容的任何合适组合。机器可读存储介质的更具体示例会包括基于一个或多个线的电气连接、便携式计算机盘、硬盘、随机存取存储器(RAM)、只读存储器(ROM)、可擦除可编程只读存储器(EPROM或快闪存储器)、光纤、便捷式紧凑盘只读存储器(CD-ROM)、光学储存设备、磁储存设备、或上述内容的任何合适组合。
根据本公开的一个或多个实施例，【示例一】提供了一种图像处理方法，该方法包括：
获取原始图像，提取所述原始图像中的高频信息和低频信息；
提取所述高频信息中的内容相关信息，将所述内容相关信息写入所述低频信息中，得到所述原始图像对应的低分辨率图像。
根据本公开的一个或多个实施例，【示例二】提供了一种图像处理方法，其中，
所述提取所述高频信息中的内容相关信息，包括：
基于预先训练的信息提取模型对所述高频信息进行处理，得到所述高频信息中的内容相关信息和内容无关信息。
根据本公开的一个或多个实施例，【示例三】提供了一种图像处理方法，其中，
所述信息提取模型为注意力可逆变换网络模型；
所述基于预先训练的信息提取模型对所述高频信息进行处理，得到所述高频信息中的内容相关信息和内容无关信息，包括：
将所述高频信息和所述低频信息输入至所述注意力可逆变换网络模型中，得到所述高频信息中的内容相关信息和内容无关信息，其中，所述低频信息为对所述高频信息进行内容相关信息提取的辅助条件。
根据本公开的一个或多个实施例，【示例四】提供了一种图像处理方法，其中，
所述将所述内容相关信息写入所述低频信息中，得到所述原始图像对应的低分辨率图像，包括：
将所述内容相关信息和所述低频信息在通道维度上进行数据融合，得到原始图像对应的低分辨率图像。
根据本公开的一个或多个实施例，【示例五】提供了一种图像处理方法，其中，
所述将所述内容相关信息和所述低频信息在通道维度上进行数据融合，包括：
确定低频信息中的第一通道数据，对应的所述内容相关信息中的至少一个第二通道数据；将具有对应关系的第一通道数据和所述第二通道数据进行数据融合。
根据本公开的一个或多个实施例，【示例六】提供了一种图像处理方法，其中，
所述提取所述原始图像中的高频信息和低频信息，包括：
对所述原始图像进行下采样，得到低频信息；基于所述原始图像和所述低频信息上采样得到的高分辨率低频图像确定初始高频信息，对所述初始高频信息中的空间像素重排至通道维度，得到与所述低频信息相匹配的高频信息；
或者，
对所述原始图像进行滤波变换，得到所述原始图像中的高频信息和低频信息。
根据本公开的一个或多个实施例，【示例七】提供了一种图像处理方法，包括：
获取低分辨率图像，基于所述低分辨率图像提取低频信息和高频信息中的内容相关信息；
基于所述内容相关信息确定所述高频信息，并基于所述高频信息和所述低频信息融合得到所述低分辨率图像对应的原始图像。
根据本公开的一个或多个实施例，【示例八】提供了一种图像处理方法，其中，：
所述基于所述低分辨率图像提取低频信息和高频信息中的内容相关信息，包括：
确定所述低频信息中数据通道与所述低分辨率图像中数据通道的对应关系，基于所述低分辨率图像中的通道数据确定所述低频信息的第一通道数据；
确定所述内容相关信息中数据通道与所述低分辨率图像中数据通道的对应关系，基于所述低分辨率图像中的通道数据确定所述内容相关信息的第二通道数据。
根据本公开的一个或多个实施例，【示例九】提供了一种图像处理方法，其中，
所述基于所述低分辨率图像中的通道数据确定所述低频信息的第一通道数据，包括：
将所述低分辨率图像中的通道数据进行数据复制，作为所述低频信息中对应数据通道的第一通道数据；
以及，基于所述低分辨率图像中的通道数据确定所述内容相关信息的第二通道数据，包括：
将所述低分辨率图像中的通道数据进行数据复制，作为所述内容相关信息中对应数据通道的第二通道数据。
根据本公开的一个或多个实施例，【示例十】提供了一种图像处理方法，在基于所述内容相关信息确定所述高频信息之前，所述方法还包括：
在内容无关信息对应的预设数据分布中，进行信息重采样，得到内容无关信息；
相应的，基于所述内容相关信息确定所述高频信息，包括：
基于所述内容相关信息和重采样得到的内容无关信息确定所述高频信息。
根据本公开的一个或多个实施例，【示例十一】提供了一种图像处理方法，其中，
所述基于所述内容相关信息和重采样得到的内容无关信息确定所述高频信息，包括：
将所述内容相关信息和所述内容无关信息反向输入至注意力可逆变换网络模型，得到所述注意力可逆变换网络模型输出的高频信息。
根据本公开的一个或多个实施例，【示例十二】提供了一种图像处理方法，其中，
所述基于所述高频信息和所述低频信息融合得到所述低分辨率图像对应的原始图像，包括：
对所述低频信息进行上采样，得到高分辨率低频图像；对所述高频信息的通道数据进行空间逆重排，得到高分辨率高频图像，并基于所述高分辨率低频图像和所述高分辨率高频图像得到原始图像；
或者，
对所述高频信息和所述低频信息进行哈尔逆变换，得到原始图像。
根据本公开的一个或多个实施例，【示例十三】提供了一种图像处理装置，该装置包括：
信息提取模型，设置为获取原始图像，提取所述原始图像中的高频信息和低频信息；
低分辨率图像生成模块，设置为提取所述高频信息中的内容相关信息，将所述内容相关信息写入所述低频信息中，得到所述原始图像对应的低分辨率图像。
根据本公开的一个或多个实施例，【示例十四】提供了一种图像处理装置，该装置包括：
信息提取模型，设置为获取低分辨率图像，基于所述低分辨率图像提取低频信息和高频信息中的内容相关信息；
原始图像生成模块，设置为基于所述内容相关信息确定所述高频信息，并基于所述高频信息和所述低频信息融合得到所述低分辨率图像对应的原始图像。
此外，虽然采用特定次序描绘了多种操作，但是这不应当理解为要求这些操作以所示出的特定次序或以顺序次序执行来执行。在一定环境下，多任务和并行处理可能是有利的。同样地，虽然在上面论述中包含了若干具体实现细节，但是这些不应当被解释为对本公开的范围的限制。在单独的实施例的上下文中描述的某些特征还可以组合地实现在单个实施例中。相反地，在单个实施例的上下文中描述的多种特征也可以单独地或以任何合适的子组合的方式实现在多个实施例中。

Claims (16)

  1. 一种图像处理方法，包括：
    获取原始图像，并提取所述原始图像中的高频信息和低频信息；
    提取所述高频信息中的内容相关信息，并将所述内容相关信息写入所述低频信息中，以得到所述原始图像对应的低分辨率图像。
  2. 根据权利要求1所述的方法，其中，所述提取所述高频信息中的内容相关信息，包括：
    基于预先训练的信息提取模型对所述高频信息进行处理，得到所述高频信息中的内容相关信息和内容无关信息。
  3. 根据权利要求2所述的方法，其中，所述信息提取模型为注意力可逆变换网络模型；
    所述基于预先训练的信息提取模型对所述高频信息进行处理，得到所述高频信息中的内容相关信息和内容无关信息，包括：
    将所述高频信息和所述低频信息输入至所述注意力可逆变换网络模型中，得到所述高频信息中的内容相关信息和内容无关信息，其中，所述低频信息为对所述高频信息进行内容相关信息提取的辅助条件。
  4. 根据权利要求1所述的方法，其中，所述将所述内容相关信息写入所述低频信息中，以得到所述原始图像对应的低分辨率图像，包括：
    将所述内容相关信息和所述低频信息在通道维度上进行数据融合，以得到所述原始图像对应的低分辨率图像。
  5. 根据权利要求4所述的方法，其中，所述将所述内容相关信息和所述低频信息在通道维度上进行数据融合，包括：
    确定所述低频信息中的第一通道数据，以及对应的所述内容相关信息中的至少一个第二通道数据；将具有对应关系的第一通道数据和所述第二通道数据进行数据融合。
  6. 根据权利要求1所述的方法，其中，所述提取所述原始图像中的高频信息和低频信息，包括：
    对所述原始图像进行下采样，得到低频信息；基于所述原始图像和所述低频信息上采样得到的高分辨率低频图像确定初始高频信息，并对所述初始高频信息中的空间像素重排至通道维度，以得到与所述低频信息相匹配的高频信息；
    或者，
    对所述原始图像进行滤波变换，得到所述原始图像中的高频信息和低频信息。
  7. 一种图像处理方法，包括：
    获取低分辨率图像，并基于所述低分辨率图像提取低频信息和高频信息中的内容相关信息；
    基于所述内容相关信息确定所述高频信息，并基于所述高频信息和所述低频信息融合得到所述低分辨率图像对应的原始图像。
  8. 根据权利要求7所述的方法，其中，所述基于所述低分辨率图像提取低频信息和高频信息中的内容相关信息，包括：
    确定所述低频信息中数据通道与所述低分辨率图像中数据通道的对应关系，并基于所述低分辨率图像中的通道数据确定所述低频信息的第一通道数据；
    确定所述内容相关信息中数据通道与所述低分辨率图像中数据通道的对应关系，并基于所述低分辨率图像中的通道数据确定所述内容相关信息的第二通道数据。
  9. 根据权利要求8所述的方法，其中，所述基于所述低分辨率图像中的通道数据确定所述低频信息的第一通道数据，包括：
    将所述低分辨率图像中的通道数据进行数据复制，作为所述低频信息中对应数据通道的第一通道数据；
    以及，基于所述低分辨率图像中的通道数据确定所述内容相关信息的第二通道数据，包括：
    将所述低分辨率图像中的通道数据进行数据复制，作为所述内容相关信息中对应数据通道的第二通道数据。
  10. 根据权利要求7所述的方法，在所述基于所述内容相关信息确定所述高频信息之前，还包括：
    在内容无关信息对应的预设数据分布中，进行信息重采样，以得到内容无关信息；
    所述基于所述内容相关信息确定所述高频信息，包括：
    基于所述内容相关信息和重采样得到的内容无关信息确定所述高频信息。
  11. 根据权利要求10所述的方法，其中，所述基于所述内容相关信息和重采样得到的内容无关信息确定所述高频信息，包括：
    将所述内容相关信息和所述内容无关信息反向输入至注意力可逆变换网络模型，得到所述注意力可逆变换网络模型输出的高频信息。
  12. 根据权利要求7所述的方法，其中，所述基于所述高频信息和所述低频信息融合得到所述低分辨率图像对应的原始图像，包括：
    对所述低频信息进行上采样，以得到高分辨率低频图像；对所述高频信息的通道数据进行空间逆重排，以得到高分辨率高频图像，并基于所述高分辨率低频图像和所述高分辨率高频图像得到原始图像；
    或者，
    对所述高频信息和所述低频信息进行哈尔逆变换，以得到原始图像。
  13. 一种图像处理装置，包括：
    信息提取模型，设置为获取原始图像，并提取所述原始图像中的高频信息和低频信息；
    低分辨率图像生成模块，设置为提取所述高频信息中的内容相关信息，并将所述内容相关信息写入所述低频信息中，以得到所述原始图像对应的低分辨率图像。
  14. 一种图像处理装置，包括：
    信息提取模型，设置为获取低分辨率图像，并基于所述低分辨率图像提取低频信息和高频信息中的内容相关信息；
    原始图像生成模块，设置为基于所述内容相关信息确定所述高频信息，并基于所述高频信息和所述低频信息融合得到所述低分辨率图像对应的原始图像。
  15. 一种电子设备，包括：
    一个或多个处理器；
    存储装置，设置为存储一个或多个程序，
    当所述一个或多个程序被所述一个或多个处理器执行，使得所述一个或多个处理器实现如权利要求1-12中任一所述的图像处理方法。
  16. 一种包含计算机可执行指令的存储介质，所述计算机可执行指令在由计算机处理器 执行时用于执行如权利要求1-12中任一所述的图像处理方法。
PCT/CN2023/081240 2022-04-11 2023-03-14 图像处理方法、装置、存储介质及电子设备 WO2023197805A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202210377047.4A CN114742738A (zh) 2022-04-11 2022-04-11 图像处理方法、装置、存储介质及电子设备
CN202210377047.4 2022-04-11

Publications (1)

Publication Number Publication Date
WO2023197805A1 true WO2023197805A1 (zh) 2023-10-19

Family

ID=82281689

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2023/081240 WO2023197805A1 (zh) 2022-04-11 2023-03-14 图像处理方法、装置、存储介质及电子设备

Country Status (2)

Country Link
CN (1) CN114742738A (zh)
WO (1) WO2023197805A1 (zh)

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114742738A (zh) * 2022-04-11 2022-07-12 北京字节跳动网络技术有限公司 图像处理方法、装置、存储介质及电子设备

Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20110211765A1 (en) * 2010-03-01 2011-09-01 Sony Corporation Image processing apparatus, image processnig method, and program
CN111754406A (zh) * 2020-06-22 2020-10-09 北京大学深圳研究生院 图像分辨率处理方法、装置、设备及可读存储介质
CN112468830A (zh) * 2019-09-09 2021-03-09 阿里巴巴集团控股有限公司 视频图像处理方法、装置及电子设备
CN112801872A (zh) * 2021-02-02 2021-05-14 上海眼控科技股份有限公司 一种图像处理方法、装置、设备及存储介质
CN113496465A (zh) * 2020-03-20 2021-10-12 微软技术许可有限责任公司 图像缩放
CN113870104A (zh) * 2020-06-30 2021-12-31 微软技术许可有限责任公司 超分辨率图像重建
CN114742738A (zh) * 2022-04-11 2022-07-12 北京字节跳动网络技术有限公司 图像处理方法、装置、存储介质及电子设备

Patent Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20110211765A1 (en) * 2010-03-01 2011-09-01 Sony Corporation Image processing apparatus, image processnig method, and program
CN112468830A (zh) * 2019-09-09 2021-03-09 阿里巴巴集团控股有限公司 视频图像处理方法、装置及电子设备
CN113496465A (zh) * 2020-03-20 2021-10-12 微软技术许可有限责任公司 图像缩放
CN111754406A (zh) * 2020-06-22 2020-10-09 北京大学深圳研究生院 图像分辨率处理方法、装置、设备及可读存储介质
CN113870104A (zh) * 2020-06-30 2021-12-31 微软技术许可有限责任公司 超分辨率图像重建
CN112801872A (zh) * 2021-02-02 2021-05-14 上海眼控科技股份有限公司 一种图像处理方法、装置、设备及存储介质
CN114742738A (zh) * 2022-04-11 2022-07-12 北京字节跳动网络技术有限公司 图像处理方法、装置、存储介质及电子设备

Also Published As

Publication number Publication date
CN114742738A (zh) 2022-07-12

Similar Documents

Publication Publication Date Title
WO2022105638A1 (zh) 图像退化处理方法、装置、存储介质及电子设备
WO2022227886A1 (zh) 超分修复网络模型生成方法、图像超分修复方法及装置
CN113870104A (zh) 超分辨率图像重建
JP2023547917A (ja) 画像分割方法、装置、機器および記憶媒体
WO2023197805A1 (zh) 图像处理方法、装置、存储介质及电子设备
CN113689372B (zh) 图像处理方法、设备、存储介质及程序产品
EP4200793A1 (en) Neural network-based devices and method to remove video coding artifacts in video stream
WO2020062494A1 (zh) 图像处理方法和装置
CN114519667A (zh) 一种图像超分辨率重建方法及系统
CN114494022B (zh) 模型训练方法、超分辨率重建方法、装置、设备及介质
CN110310293B (zh) 人体图像分割方法及设备
CN112418249A (zh) 掩膜图像生成方法、装置、电子设备和计算机可读介质
CN111935425A (zh) 视频降噪方法、装置、电子设备及计算机可读介质
WO2024032331A9 (zh) 图像处理方法及装置、电子设备、存储介质
JP2023538825A (ja) ピクチャのビデオへの変換の方法、装置、機器および記憶媒体
WO2024082933A1 (zh) 视频处理方法、装置、电子设备及存储介质
WO2023179360A1 (zh) 视频处理方法、装置、电子设备及存储介质
WO2023179385A1 (zh) 一种视频超分方法、装置、设备及存储介质
WO2022252883A1 (zh) 图像修复模型的训练方法及图像修复方法、装置及设备
CN111815535B (zh) 图像处理方法、装置、电子设备和计算机可读介质
CN111798385B (zh) 图像处理方法及装置、计算机可读介质和电子设备
CN111738958B (zh) 图片修复方法、装置、电子设备及计算机可读介质
CN112215774B (zh) 模型训练和图像去雾方法、装置、设备和计算机可读介质
WO2023020283A1 (zh) 图像处理方法、装置、设备、介质及程序产品
WO2023093838A1 (zh) 超分图像处理方法、装置、设备及介质

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 23787453

Country of ref document: EP

Kind code of ref document: A1