WO2024032331A1 - 图像处理方法及装置、电子设备、存储介质 - Google Patents

图像处理方法及装置、电子设备、存储介质 Download PDF

Info

Publication number
WO2024032331A1
WO2024032331A1 PCT/CN2023/107968 CN2023107968W WO2024032331A1 WO 2024032331 A1 WO2024032331 A1 WO 2024032331A1 CN 2023107968 W CN2023107968 W CN 2023107968W WO 2024032331 A1 WO2024032331 A1 WO 2024032331A1
Authority
WO
WIPO (PCT)
Prior art keywords
image
processing
characteristic parameters
processing unit
output
Prior art date
Application number
PCT/CN2023/107968
Other languages
English (en)
French (fr)
Other versions
WO2024032331A9 (zh
Inventor
钱刘熠辉
唐宏
武娟
刘晓军
徐晓青
Original Assignee
中国电信股份有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 中国电信股份有限公司 filed Critical 中国电信股份有限公司
Publication of WO2024032331A1 publication Critical patent/WO2024032331A1/zh
Publication of WO2024032331A9 publication Critical patent/WO2024032331A9/zh

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T3/00—Geometric image transformations in the plane of the image
    • G06T3/40—Scaling of whole images or parts thereof, e.g. expanding or contracting
    • G06T3/4053—Scaling of whole images or parts thereof, e.g. expanding or contracting based on super-resolution, i.e. the output image resolution being higher than the sensor resolution
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G06N3/084—Backpropagation, e.g. using gradient descent
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T3/00—Geometric image transformations in the plane of the image
    • G06T3/40—Scaling of whole images or parts thereof, e.g. expanding or contracting
    • G06T3/4046—Scaling of whole images or parts thereof, e.g. expanding or contracting using neural networks

Definitions

  • the present application relates to the field of computer technology, specifically, to an image processing method and device, electronic equipment, storage media, and program products.
  • Image super-resolution is an important image processing technology in computer vision. Its goal is to reconstruct high-resolution images based on low-resolution images.
  • common image super-resolution methods are prone to losing details during the reconstruction process, and the resulting high-resolution images are not very effective, which affects the accuracy of image super-resolution construction.
  • embodiments of the present application provide an image processing method and device, electronic equipment, storage media, and program products.
  • an image processing method includes:
  • the image processing model includes multiple processing units
  • the image to be processed is input to the image processing model, and the image to be processed is processed through the multiple processing units in sequence to obtain the characteristic parameters output by each processing unit; wherein, after passing through the multiple processing units When the m+1th processing unit in the processing unit processes the image to be processed, the m+1th processing unit processes the characteristic parameters output by the mth processing unit to obtain the mth processing unit.
  • Characteristic parameters output by m+1 processing units where m is an integer, and the number of characteristic parameters output by the m+1-th processing unit is greater than the number of characteristic parameters output by the m-th processing unit;
  • a target image with a corresponding resolution higher than the resolution of the image to be processed is generated.
  • an image processing device includes:
  • An acquisition module configured to acquire an image to be processed and an image processing model; the image processing model includes multiple processing units;
  • a processing module configured to input the image to be processed into the image processing model, and to process the image to be processed through the plurality of processing units in sequence to obtain the characteristic parameters output by each processing unit; wherein, in In the process of processing the image to be processed by the m+1th processing unit among the plurality of processing units, the characteristic parameters output by the mth processing unit are processed by the m+1th processing unit. , obtain the characteristic parameters output by the m+1th processing unit, where m is an integer, and the number of characteristic parameters output by the m+1th processing unit is greater than the number of characteristic parameters output by the mth processing unit;
  • a generation module configured to generate a target image with a corresponding resolution higher than the resolution of the image to be processed based on the characteristic parameters output by the last processing unit among the plurality of processing units.
  • an electronic device including:
  • processors one or more processors
  • a storage device is used to store one or more programs, and when the one or more programs are executed by the one or more processors, the electronic device implements the image processing method as described above.
  • a computer-readable storage medium on which computer-readable instructions are stored.
  • the computer-readable instructions are executed by a processor of an electronic device, the electronic device is caused to perform the above steps.
  • a computer program product which includes a computer program.
  • the computer instructions are executed by a processor, the image processing method as described above is implemented.
  • Figure 1 is a flow chart of an image processing method according to an exemplary embodiment of the present application
  • Figure 2 is a schematic structural diagram of an image processing model shown in an exemplary embodiment of the present application.
  • Figure 3 is another structural schematic diagram of an image processing model shown in an exemplary embodiment of the present application.
  • Figure 4 is a processing flow chart of a processing subunit shown in an exemplary embodiment of the present application.
  • Figure 5 is a schematic structural diagram of an image processing model according to another exemplary embodiment of the present application.
  • Figure 6 is a processing flow chart of a pre-processing unit according to an exemplary embodiment of the present application.
  • Figure 7 is a schematic structural diagram of an image processing model according to another exemplary embodiment of the present application.
  • Figure 8 is a schematic structural diagram of an image processing model according to another exemplary embodiment of the present application.
  • Figure 9 is an optimization flow chart of an image processing model shown in another exemplary embodiment of the present application.
  • Figure 10 is a flow chart of step S910 in the embodiment shown in Figure 9 in an exemplary embodiment
  • Figure 11 is a schematic diagram of acquiring a standard image according to an exemplary embodiment of the present application.
  • Figure 12 is a flow chart of step S930 in the embodiment shown in Figure 9 in an exemplary embodiment
  • Figure 13 is a flow chart of an image processing method according to an exemplary embodiment of the present application.
  • Figure 14 is a schematic structural diagram of an image processing model shown in another exemplary embodiment of the present application.
  • Figure 15 is a processing block diagram of an image processing model shown in another exemplary embodiment of the present application.
  • Figure 16 is a schematic structural diagram of an image processing device according to an exemplary embodiment of the present application.
  • FIG. 17 shows a schematic structural diagram of a computer system suitable for implementing an electronic device according to an embodiment of the present application.
  • embodiments of the present application propose an image processing method and device, electronic equipment, storage media, and program products, which can improve the accuracy of image super-resolution construction.
  • Figure 1 is a flow chart of an image processing method according to an exemplary embodiment of the present application.
  • the image processing method may include steps S110 to S130, which are described in detail as follows:
  • Step S110 Obtain the image to be processed and the image processing model; the image processing model includes multiple processing units.
  • the image processing model is a machine learning model used to improve the resolution of images. Its specific type can be flexibly set according to actual needs, for example, including but not limited to convolutional neural networks, recurrent neural networks, etc., among which , Convolutional neural networks include but are not limited to residual networks.
  • the image processing model includes multiple processing units for processing input data.
  • the multiple processing units are connected in sequence. That is to say, as shown in Figure 2, the image processing model 200 includes multiple processing units 210.
  • the previous processing unit The output terminal is connected to the input terminal of the subsequent processing unit.
  • the image to be processed is an image that needs to be improved in resolution, which can be a video frame in a video, or Can be a separate image.
  • the format of the image to be processed includes but is not limited to bmp (bitmap image), jpeg (Joint Photographic Experts Group, Joint Photographic Experts Group), png (Portable Network Graphics, portable network graphics), tiff (Tag Image File Format, tag image file format) etc.
  • the image to be processed and the image processing model can be obtained.
  • Step S120 input the image to be processed into the image processing model, and sequentially process the image to be processed through multiple processing units to obtain the characteristic parameters output by each processing unit; wherein, after passing the m+1th of the multiple processing units In the process of processing the image to be processed by the processing unit, the characteristic parameters output by the m-th processing unit are processed by the m+1-th processing unit to obtain the characteristic parameters output by the m+1-th processing unit, m is an integer, The number of feature parameters output by the m+1th processing unit is greater than the number of feature parameters output by the mth processing unit.
  • Feature parameters are parameters that characterize the characteristics of the image to be processed, and their forms include but are not limited to feature maps, feature vectors, etc.
  • n is an integer, and its value range can be [1, M]. That is to say, m is an integer greater than or equal to 1 and less than or equal to M. Among them, M is the number of processing units, and M is an integer greater than 1.
  • the image to be processed is input to the image processing model, and multiple processing units included in the image processing model will process the image to be processed in sequence.
  • the characteristic parameters output by the m-th processing unit will be input to the m+1-th processing unit, so that the characteristic parameters output by the m-th processing unit can be processed by the m+1-th processing unit.
  • the characteristic parameters output by the first processing unit are obtained; the characteristics output by the first processing unit are The parameters are input to the second processing unit, so that the characteristic parameters output by the first processing unit are processed by the second processing unit to obtain the characteristic parameters output by the second processing unit; the characteristic parameters output by the second processing unit are Input to the third processing unit, and process the characteristic parameters output by the second processing unit through the third processing unit, and so on, until the characteristic parameters output by the last processing unit are obtained.
  • the number of characteristic parameters output by the m+1th processing unit is greater than that of the mth processing unit.
  • the number of feature parameters output by each unit that is, the number of feature parameters output by each processing unit is greater than the number of feature units output by its previous processing unit, that is, based on the data transmission direction, the feature parameters output by multiple processing units.
  • the number increases successively, so that the image processing model is biased towards extracting the global structural features of the image in the early stage of processing, and is biased towards extracting the detailed texture features of the image in the later stage of processing.
  • the image processing model can achieve a progressive formula from coarse to fine.
  • the generation of high-resolution images improves the structural integrity and texture refinement of high-resolution images, and improves the accuracy of image super-resolution construction.
  • the specific number of characteristic parameters output by each processing unit can be flexibly set according to actual needs.
  • the number of feature parameters output by each processing unit can be learned by the image processing model itself, or can be set by the model development engineer. That is, the number of feature parameters output by each processing unit can be the image processing model. hyperparameters.
  • Step S130 Based on the characteristic parameters output by the last processing unit among the plurality of processing units, generate a target image with a corresponding resolution higher than the resolution of the image to be processed.
  • the image processing module can generate a target image based on the characteristic parameters output by the last processing unit, where the resolution of the target image is higher than the resolution of the image to be processed.
  • the image processing model may also include an upsampling unit, which processes the feature parameters output by the last processing unit to obtain a feature map, and upsamples the feature map to obtain the target image.
  • the specific way in which the upsampling unit performs upsampling can be flexibly set according to actual needs.
  • upsampling can be performed through at least one of nearest neighbor interpolation, bilinear interpolation, bicubic interpolation, and the like.
  • the number of upsampling units and the upsampling ratio can be flexibly set according to actual needs.
  • the image processing model 200 can include two upsampling units 220 , each upsampling unit 220 The upsampling ratio is 2, resulting in 4-fold upsampling.
  • the image processing method provided in this embodiment can be applied to different scenarios, such as image repair, image fusion, image editing and other scenarios.
  • it can be applied to a virtual reality (Virtual Reality, VR) scenario.
  • VR Virtual Reality
  • the user can wear a VR device (for example, VR glasses or a helmet, etc.), and the VR device can deliver immersive visual and auditory information.
  • Allowing users to obtain an immersive experience in which the VR device can determine the user's current head position and field of view direction through sensors integrated in the body or external to the room, and use the VR device to The corresponding visible area in the panoramic image or panoramic video is displayed on the device screen, but the data volume of the panoramic image frame is large, usually reaching tens of millions of pixels (for example, 4K resolution, 8K resolution), etc., resulting in the panorama being
  • the process of transmitting images or panoramic videos from data providers to VR devices takes up a lot of memory and network bandwidth. Therefore, in order to reduce memory pressure and network bandwidth consumption, data providers can transmit low-resolution images to VR devices.
  • the low-resolution image is used as the image to be processed, and the low-resolution image is restored to the high-resolution image through the image processing method provided in this embodiment, so that not only the high-resolution image can be obtained, but also the bandwidth consumption of the network can be reduced.
  • the image to be processed and the image processing model are obtained.
  • the image processing model includes multiple processing units; the image to be processed is input to the image processing model, and the image to be processed is processed through the multiple processing units in sequence to obtain each processing unit.
  • Characteristic parameters output by the unit among them, in the process of processing the image to be processed through the m+1th processing unit among multiple processing units, the characteristic parameters output by the m+1th processing unit to the mth processing unit Perform processing to obtain the characteristic parameters output by the m+1th processing unit, m is an integer, and the number of characteristic parameters output by the m+1th processing unit is greater than the number of characteristic parameters output by the mth processing unit; based on multiple processing
  • the feature parameters output by the last processing unit in the unit generate a target image with a corresponding resolution higher than the resolution of the image to be processed.
  • the image processing model in the process of processing the image to be processed by the image processing model, the number of feature parameters As the data transmission direction increases in sequence, the image processing model is biased towards extracting the global structural features of the image in the early stage of processing, and is biased towards extracting the detailed texture features of the image in the later stage of processing. In other words, the image processing model can achieve from coarse to fine. Progressive generation of high-resolution images, thereby improving the structural integrity and texture refinement of high-resolution images, and improving the accuracy of image super-resolution construction.
  • FIG. 4 is a flow chart of an image processing method proposed based on the embodiment shown in FIG. 1 .
  • the image processing method may also include steps S410, S411-Step S414, detailed introduction is as follows:
  • Step S410 sequentially process the image to be processed through multiple processing sub-units included in the m-th processing unit, obtain the characteristic parameters output by each processing sub-unit included in the m-th processing unit, and convert the The characteristic parameters output by the last processing subunit are as Characteristic parameters output by the mth processing unit.
  • each processing unit contains multiple processing sub-units, and the number of characteristic parameters output by multiple sub-units belonging to the same processing unit is the same.
  • the specific number of processing sub-units included in each processing unit can be flexible according to actual needs. Settings, the number of processing sub-units contained in different processing units can be the same or different.
  • the type of processing sub-unit can be flexibly set according to actual needs. In one example, if the image processing model is a convolutional neural network, the processing sub-unit can be a convolution block.
  • the image to be processed can be processed sequentially through multiple processing sub-units included in the m-th processing unit, thereby obtaining each parameter included in the m-th processing unit.
  • the characteristic parameters output by the m-th processing sub-unit, and the characteristic parameters output by the last processing sub-unit in the m-th processing unit are used as the characteristic parameters output by the m-th processing unit, and then the characteristic parameters output by the m-th processing unit are Input to the m+1th processing unit. That is to say, as shown in FIG.
  • each processing unit 210 includes multiple processing sub-units 211 , and the output end of the previous processing sub-unit 211 is connected to the input end of the next processing sub-unit 211 .
  • the characteristic parameters output by the first processing sub-unit of the processing unit are first obtained, and the characteristic parameters output by the first processing sub-unit of the processing unit are input to the second processing unit of the processing unit.
  • a processing subunit is used to process the characteristic parameters output by the first processing subunit of the processing unit through the second processing subunit of the processing unit to obtain the characteristic parameters output by the second processing subunit of the processing unit. , and so on, until the characteristic parameters output by the last processing sub-unit of the processing unit are obtained, the characteristic parameters output by the last processing sub-unit of the processing unit are input to the first of the next processing unit of the processing unit Process subunits.
  • Step S411 In the process of processing the image to be processed by the j+1th processing sub-unit in the m-th processing unit, the characteristic parameters output by the j-th processing sub-unit in the m-th processing unit are input to the The first processing layer in j+1 processing subunits obtains the characteristic parameters output by the first processing layer; where j is an integer.
  • j is an integer, and its value range is determined based on the number of processing sub-units included in the processing unit.
  • the j-th processing sub-unit is any processing sub-unit in the processing unit to which it belongs.
  • Each processing subunit contains multiple processing layers and belongs to the multi-layer processing of the same processing subunit.
  • the number of feature parameters output by the layer is the same.
  • the specific number of processing layers included in each processing subunit can be flexibly set according to actual needs.
  • the number of processing layers included in different processing subunits can be the same or different.
  • the type of processing layer included in the processing subunit can be flexibly set according to actual needs.
  • the j+1th processing subunit in the mth processing unit also contains multiple processing layers.
  • first The characteristic parameters output by the j-th processing sub-unit in the m-th processing unit are input to the first layer of processing layer in the j+1-th processing sub-unit, so as to pass through the first layer in the j+1-th processing sub-unit.
  • the processing layer processes the characteristic parameters output by the j-th processing sub-unit in the m-th processing unit to obtain the characteristic parameters output by the first-layer processing layer in the j+1-th processing sub-unit.
  • Step S412 Input the characteristic parameters output by the first layer of processing layer to the next layer of processing layer of the first layer of processing layer until the characteristic parameters output by the penultimate layer of processing layer in the j+1th processing sub-unit are obtained.
  • the characteristic parameters output by the first processing layer in the j+1th processing subunit are input to the second processing layer in the j+1th processing subunit for processing, and the j+1th processing subunit is obtained.
  • the characteristic parameters output by the second processing layer in the j+1th processing subunit are then input to the third processing layer in the j+1th processing subunit. , until the characteristic parameters output by the penultimate processing layer in the j+1th processing subunit are obtained.
  • Step S413 Process the characteristic parameters input to the first processing layer and the characteristic parameters output from the penultimate processing layer through the last processing layer in the j+1th processing subunit to obtain the output of the last processing layer. characteristic parameters.
  • the characteristic parameters input to the first processing layer in the j+1th processing subunit and the characteristic parameters output from the penultimate processing layer in the j+1th processing subunit are input to the j+1th processing
  • the last processing layer in the sub-unit obtains the characteristic parameters output by the last processing layer in the j+1th processing sub-unit.
  • the multi-layer processing layers 2110 are connected in sequence, and the input of the first layer of processing layer 2110 The terminal is connected to the input terminal of the last processing layer 2110.
  • the feature parameters output by the upper processing layer are input to the next processing layer, and the input feature parameters of the first processing layer (i.e., input to the first processing layer special Characteristic parameters) are input to the last processing layer.
  • the processing layers included in the processing subunit can be: the first convolutional layer and the first normalization layer.
  • a nonlinear rectification layer a second convolution layer, a second normalization layer, and an overlay layer
  • the first convolution layer and the second convolution layer can fuse and modify the input feature parameters through convolution operations
  • the first The normalization layer and the second normalization layer can normalize the input feature parameters to a certain range and improve the convergence stability of the model
  • the nonlinear rectification layer can avoid the problem of neurons not being activated when the model is deep, speeding up The overall convergence rate of the model
  • the input end of the overlay layer is connected to the input end of the first convolutional layer and the output end of the second normalization layer, so that the characteristic parameters of the first convolutional layer and the second normalization layer can be input
  • the output feature parameters are superimposed.
  • Step S414 use the feature parameters output by the last layer as the feature parameters output by the j+1th processing subunit.
  • each processing unit contains multiple processing sub-units with the same number of output characteristic parameters, and each processing sub-unit contains multiple processing layers
  • the last layer of processing The layer processes the characteristic parameters input to the first layer processing layer and the characteristic parameters output from the penultimate layer, so that the beginning and end of each processing sub-unit can be connected to ensure that the correction of the characteristic parameters of the image in each processing sub-unit is limited. to avoid image distortion caused by over-correction.
  • FIG. 6 is a flow chart of an image processing method proposed based on the embodiment shown in FIG. 1 .
  • the image processing method may also include steps S610 to S620. The details are as follows:
  • Step S610 sequentially process the image to be processed through multiple preprocessing units to obtain the characteristic parameters output by each preprocessing unit; wherein, the image to be processed is processed through the n+1th preprocessing unit among the multiple preprocessing units.
  • the characteristic parameters output by the nth preprocessing unit are processed by the n+1th preprocessing unit to obtain the characteristic parameters output by the n+1th preprocessing unit, n is an integer, and the n+1th The number of characteristic parameters output by the first preprocessing unit is smaller than the number of characteristic parameters output by the nth preprocessing unit.
  • the image processing model also includes multiple preprocessing units.
  • the specific number of preprocessing units can be determined based on It can be set flexibly according to actual needs, and its number can be equal to or different from the number of processing units.
  • the specific type of preprocessing unit can be flexibly set according to actual needs. For example, it can be a neural network used for convolution operations.
  • n is an integer
  • the n-th preprocessing unit is any preprocessing unit among multiple preprocessing units included in the image processing model.
  • the image processing model contains multiple preprocessing units that process the image to be processed in sequence. Among them, the characteristic parameters output by the nth preprocessing unit will be input to the n+1th preprocessing unit, so that the characteristic parameters output by the nth preprocessing unit are processed by the n+1th preprocessing unit to obtain the nth preprocessing unit. The characteristic parameters output by n+1 preprocessing units are then obtained, and then the characteristic parameters output by each preprocessing unit are obtained.
  • the characteristic parameters output by the first preprocessing unit are obtained; the first preprocessing unit
  • the characteristic parameters output by the unit are input to the second preprocessing unit, so that the characteristic parameters output by the first preprocessing unit are processed by the second preprocessing unit to obtain the characteristic parameters output by the second preprocessing unit;
  • the characteristic parameters output by the two preprocessing units are input to the third preprocessing unit, and the characteristic parameters output by the second preprocessing unit are processed through the third preprocessing unit, and so on until the last preprocessing unit is obtained. Characteristic parameters output by the processing unit.
  • the number of feature parameters output by each preprocessing unit is smaller than the number of feature units output by the previous preprocessing unit. That is to say, based on the data transmission direction, the number of feature parameters output by multiple preprocessing units decreases in sequence.
  • Step S610 Input the characteristic parameters output by the last pre-processing unit among the plurality of pre-processing units to the first processing unit among the plurality of processing units to obtain the characteristic parameters output by the first processing unit.
  • multiple preprocessing units 230 are connected in sequence, the output end of the previous preprocessing unit 230 is connected to the input end of the next preprocessing unit 230 , and the output of the last preprocessing unit 230 The terminal is connected to the input terminal of the first processing module 210.
  • the characteristic parameters output by the mth processing unit Under the condition that the number of feature parameters output by the M-m+1th pre-processing unit matches (for example, is equal), in step S120, the m-th processing unit outputs the
  • the process of processing the characteristic parameters to obtain the characteristic parameters output by the m+1th processing unit may include: inputting the characteristic parameters output by the mth processing unit and the characteristic parameters output by the M-m+1th preprocessing unit into the m+1 processing unit, obtain the characteristic parameters output by the m+1th processing unit.
  • step S130 based on the characteristic parameters output by the last processing unit among the plurality of processing units, the process of generating a target image with a corresponding resolution higher than the resolution of the image to be processed may include: based on the last processing unit The output characteristic parameters and the characteristic parameters output by the first preprocessing unit among the plurality of preprocessing units generate a target image with a corresponding resolution higher than the resolution of the image to be processed.
  • multiple pre-processing units are set up in the image processing model, and the multiple pre-processing units are connected in sequence.
  • the output end of the last pre-processing unit is connected to the input end of the first processing unit, and, according to the data transmission direction, The number of feature parameters output by multiple preprocessing units is successively reduced, thereby improving the accuracy of the image processing model.
  • FIG. 9 is a flow chart of an image processing method proposed based on the embodiment shown in FIG. 1 .
  • the image processing method may also include steps S910 to S940, which are described in detail as follows:
  • Step S910 Obtain a sample image, and extract multiple standard images from the sample image.
  • the sample image refers to an image serving as a sample, which may be a high-resolution image.
  • multiple images can be extracted from the sample images, and the extracted images can be used as standard images.
  • the specific method of extracting multiple standard images from the sample image can be flexibly set according to actual needs.
  • Step S920 Reduce the resolution of each standard image to obtain the input image corresponding to each standard image.
  • the resolution of each standard image can be reduced to obtain the input image corresponding to each standard image. That is, the resolution of the input image is smaller than the resolution of the corresponding standard image.
  • the specific method of reducing the resolution of the standard image can be flexibly set according to actual needs.
  • the standard image can be down-sampled to obtain an input image corresponding to the standard image.
  • the specific method of downsampling can be flexibly set according to actual needs.
  • downsampling can be performed through bilinear downsampling.
  • multiple downsampling methods can also be set.
  • one of the downsampling methods can be randomly selected from multiple downsampling methods to downsample the standard image.
  • the downsampling rate can be flexibly set according to actual needs, for example, it can be set to 4 times, 2 times, etc.
  • Step S930 Process each input image through the image processing model to obtain an output image corresponding to each input image.
  • the input image can be input to the image processing model, so that the image processing model processes the input image to obtain an output image corresponding to the input image.
  • Step S940 Calculate the loss value of the image processing model based on the difference between the output image corresponding to each input image and the standard image, and adjust the parameters of the image processing model based on the calculated loss value.
  • the difference between the output image and the corresponding standard image can be calculated, the loss value of the image processing model can be calculated based on the difference, and the parameters of the image processing model can be adjusted according to the loss value of the image processing model. , to optimize the image processing model.
  • the specific method for calculating the difference between the output image and the corresponding standard image can be flexibly set according to actual needs.
  • the difference between the output image and the corresponding standard image can be calculated in the following way:
  • L MSE is the difference between the output image and the corresponding standard image
  • w is the width of the image
  • h is the height of the image
  • I HR (x, y) is the pixel in the standard image with the abscissa x and the ordinate y
  • the pixel value of the point, I SR (x, y) is the pixel value of the pixel point corresponding to the x coordinate in the output image and the y coordinate in the output image.
  • the loss value of the image processing model can also be calculated based on the difference between the output image and the corresponding standard image; optionally, the image processing model can be The model is trained in batches, that is, the number of input pictures input to the image processing model is multiple each time. Correspondingly, multiple output pictures are obtained for each training. After calculating the difference between these multiple output pictures and the corresponding standard pictures, The average of the differences between multiple output images and the corresponding standard image can be used as the loss value of the image processing model.
  • steps S910 to S940 can be applied to the training phase of the image processing model, or can also be applied to the optimization phase after the training of the image processing model is completed.
  • a sample image is obtained, multiple standard images are extracted from the sample image, the resolution of each standard image is reduced, an input image corresponding to each standard image is obtained, and each input image is processed through an image processing model.
  • Process obtain the output image corresponding to each input image, calculate the loss value of the image processing model based on the difference between the output image corresponding to each input image and the standard image, and adjust the parameters of the image processing model based on the calculated loss value , in this way, through one sample image, multiple standard images and corresponding input images for training can be obtained, thereby increasing the number of training samples and reducing the difficulty of obtaining training samples, and the size of the standard image is smaller than the sample image
  • the size can improve the processing efficiency of the image processing model, reduce the amount of computing resources occupied during model training, and improve the model convergence speed.
  • FIG. 10 is a flowchart of step S910 shown in FIG. 9 in an exemplary embodiment.
  • the process of extracting multiple standard images from the sample image may include steps S911 to S912. The details are as follows:
  • Step S911 Determine multiple regions whose sizes match the first size from the sample image.
  • the first size is the size of a preset area, including width and height.
  • the width and height can be equal or different.
  • the specific size of the first size can be flexibly set according to actual needs.
  • multiple regions can be determined from the sample image, and the size of each region matches the first size. There may or may not be overlap between adjacent areas.
  • the specific method of determining multiple regions whose sizes match the first size from the sample image can be flexibly set according to actual needs.
  • the sample image may be divided into multiple non-overlapping regions based on the first size.
  • the sample image can be moved based on the width direction step size and the height direction step size to determine regions with multiple sizes matching the first size; wherein the width direction step size is The step length of moving in the width direction, and the step length of the height direction is the step length of moving in the height direction; that is to say, the sizes of the multiple areas determined are all the first size, and the two adjacent areas in the width direction
  • the interval between regions is the width direction step, and the interval between two adjacent regions in the height direction is the height direction step.
  • the step length in the width direction and the step length in the height direction may be equal or unequal, and their specific lengths can be flexibly set according to actual needs.
  • sample image 1210 multiple regions 1211 with sizes d large ⁇ d large can be determined in sequence, and the distance between the center points of adjacent regions 1211 is d space , and a region with size d small can be randomly determined in each region 1211
  • the image block of ⁇ d small is used as the standard image 1212, in which the number of determined areas N large is as follows:
  • a multiple size matching the first size can be set on the left edge of the sample image. sampling windows, and the distance between the positions of the multiple sampling windows in the height direction is the height direction step. After sampling in multiple sampling windows, move multiple sampling windows to the left based on the width direction step until the sampling window reaches the sample image. The right edge of , thus obtaining multiple areas.
  • Step S912 Randomly extract an image whose size matches the second size from each area, and use the extracted image as a standard image, where the first size is larger than the second size.
  • the second size is the size of the standard image, and the second size includes the width and height of the standard image, where the width and height of the standard image may be equal or unequal.
  • an image whose size matches the second size can be randomly extracted from each region, and the extracted image can be used as a standard image.
  • the extracted image can be flipped horizontally to obtain a standard image.
  • multiple regions whose sizes match the first size are determined from the sample image, an image whose size matches the second size is randomly extracted from each region, and the extracted image is used as a standard image. , where the first size is larger than the second size.
  • FIG. 12 is a flow chart of step S930 in the embodiment shown in FIG. 9 in an exemplary embodiment.
  • the process of processing each input image through the image processing model to obtain the output image corresponding to each input image may include steps S931 to S933. The details are as follows:
  • Step S931 Obtain an input image set; the input image set contains input images corresponding to each standard image.
  • the input image set contains input images corresponding to each standard image. Since there are multiple standard images, correspondingly, the input image set contains multiple input images.
  • Step S932 Obtain multiple input images from the input image set, input the multiple input images to the image processing model, and obtain output images corresponding to each of the multiple input images.
  • multiple input images can be obtained from the input image set, and the multiple input images can be input to the image processing model, so that the image processing model can process the input images in batches to obtain the input The corresponding output image of each image.
  • the specific number of input images obtained from the input image set each time can be flexibly set according to actual needs. For example, it can be set to 10 images, 20 images, etc., or all input images can be obtained from the input image set. , and input the obtained input images to the image processing model together.
  • Step S933 Re-acquire multiple input images from the input image set, and input the re-acquired multiple input images into the image processing model until an output image corresponding to each input image in the input image set is obtained.
  • the input image set can be obtained first, and the input image set contains the input image corresponding to each standard image; multiple input images are obtained from the input image set, and the multiple input images are input to the image processing model to obtain multiple input images.
  • the output images corresponding to each of the input images are re-obtained from the input image set, and the re-obtained multiple input images are input to the image processing model until the output corresponding to each input image in the input image set is obtained. images, allowing batch training of image processing models.
  • image processing methods include:
  • Step S1301 obtain a sample image.
  • the sample image may be a panoramic image or other types of images.
  • Step S1302 Perform partition sampling on the sample image to obtain a standard image.
  • the image block and the symmetrical image of the image block are used as standard images.
  • N p is the number of standard images.
  • Step S1303 downsample the standard image to obtain the input image.
  • downsampling can be performed to obtain the input image of each standard image.
  • Step S1304 Train the initial image processing model based on the input image and the standard image to obtain the image processing model.
  • the input image can be processed through the initial image processing model to obtain the output image.
  • the loss value of the initial image processing model is determined, and the parameters of the initial image processing model are adjusted based on the loss value. , to optimize the initial image processing model.
  • the initial image processing model may be a progressive residual network, as shown in Figure 14.
  • the initial image processing model includes M convolution units 1410, M processing units 1420, 2 upsampling units 1430 and the output layer 1440, where, according to the data transmission direction, the number of feature maps output by the previous processing unit 1420 is less than the number of feature maps output by the next processing unit 1420, mth
  • the number of feature maps output by the processing unit 1420 is the same as the number of feature maps output by the M-m+1th convolution unit 1410, and the output end of the m-th processing unit 1420 is the same as the number of feature maps output by the M-m+1th convolution unit 1410.
  • each processing unit 1420 includes multiple convolution blocks 1421 with the same number of output feature maps, and the number of feature maps output by each convolution block 1421 is equal to the number of feature maps output by the processing unit to which it belongs.
  • Each convolution block 1421 contains 6 sequentially connected processing layers. According to the data transmission direction, the 6 processing layers are the first convolution layer, the first normalization layer, the nonlinear rectification layer, and the second convolution layer.
  • the convolution layer, the second normalization layer and the pixel-level overlay layer wherein the input end of the first convolution layer is connected to the input end of the pixel-level overlay layer (ie, the output end of the second normalization layer), so that the pixel-level overlay layer
  • the feature map corresponding to the input end of the first convolution layer and the feature map output by the second normalization layer can be superimposed, thereby limiting the correction amplitude of the feature map for each convolution block and avoiding deviations caused by excessive correction.
  • Each upsampling unit 1430 can upsample the input feature map by 2 times, and the output layer 1440 can convert the input feature map into an image in a specified form for output, for example, it can be converted into RBG form.
  • rectangular frames with the same filling pattern output the same number of feature maps.
  • Step S1305 Obtain the image to be processed.
  • Step S1306 Process the image to be processed through the image processing model to obtain the target image.
  • the sample image after obtaining the sample image, perform partition sampling on the sample image to obtain the image block, horizontally flip the image block to obtain the symmetrical image of the image block, and use the image block and the symmetrical image of the image block as The standard image is then downsampled to obtain the input image.
  • the input image and the standard image are used as training data.
  • the image processing model is trained based on the training data.
  • the low-resolution image to be processed is input to Image processing model to obtain high-resolution target images.
  • the high-resolution target image refers to an image with a higher resolution than the low-resolution input image.
  • the "high resolution” and “low resolution” here are not used to limit the resolution of the image. The specific range is only used to limit the relative relationship between image resolutions.
  • the training process includes 1.1-1.3.
  • the details are as follows:
  • Obtain training data Use the size of 416 pixels ⁇ 416 pixels as a template, crop the image from the upper left corner to the lower right corner of a single high-resolution sample image, and the distance between the center points of two adjacent cropping areas is 208 pixels .
  • there are a total of 15 ⁇ 7 105 areas.
  • an area of 208 pixels ⁇ 208 pixels is randomly selected as an image block, and each image block is flipped horizontally to obtain a symmetrical image.
  • the image block and symmetrical image are used as standard images to obtain 210 standard images. .
  • bilinear downsampling is performed to obtain a 52 ⁇ 52 pixel input image, each 52 ⁇ 52 pixel low-resolution input image and 208 ⁇ 208 pixel high-resolution standard
  • the mapping relationship between images is the goal that the image processing model needs to learn.
  • the image processing model uses a progressive residual network. According to the data transmission direction, it includes a convolution unit with a number of feature maps of 32, a convolution unit with a number of feature maps of 48, Convolution unit with 64 feature maps, 6 convolution blocks with 32 feature maps, 6 convolution blocks with 48 feature maps, 6 convolution blocks with 64 feature maps, 2 upsampling layer and an output layer, where the output end of the convolution unit with a feature map number of 32 is connected to the output end of the last convolution block with a feature map number of 32, and the output end of the convolution unit with a feature map number of 48 is connected to The output end of the last convolution block with a feature map number of 48 is connected, and the output end of the convolution unit with a feature map number of 64 is connected to the output end of the last convolution block with a feature map number of 64.
  • Each upsampling layer The image can be enlarged 2 times.
  • the input images are input to the image processing model in batches of 16 for model training, so that the model can output high-resolution output images with 4 times upsampling at the tail end; due to the input of each round of training 16 input images, therefore, each round of training will get 16 output images.
  • the model undergoes backpropagation to optimize the parameters of each layer in the model.
  • the image processing model can be set to be trained for a total of 1500 cycles, of which the first 1000 cycles are trained with a learning rate of 0.0001, and the last 500 cycles are trained with a learning rate of 0.00001.
  • image evaluation indicators include but are not limited to PSNR (Peak Signal-to-Noise Ratio, peak signal-to-noise ratio), SSIM (Structural SIMilarity, structural similarity), VIFP (Visual Information Fidelity in Pixel domain, visual information in the pixel domain) fidelity), etc.
  • larger-scale training data is obtained based on block sampling of a single sample image, which effectively improves the feature diversity of the training data and reduces the risk of over-fitting in model training; the number of feature maps output from multiple outputs gradually increases.
  • the progressive deep generation network model composed of large convolution blocks can make the shallow convolution blocks tend to store the global structural features of the image, and the deep convolution blocks learn how to generate the global structure features in the shallow convolution blocks.
  • FIG. 16 is a block diagram of an image processing device according to an exemplary embodiment of the present application. As shown in Figure 16, the device includes:
  • the acquisition module 1601 is configured to acquire the image to be processed and the image processing model; the image processing model includes multiple processing units;
  • the processing module 1602 is configured to input the image to be processed into the image processing model, and sequentially process the image to be processed through multiple processing units to obtain the characteristic parameters output by each processing unit; wherein, after passing through the first of the multiple processing units During the processing of the image to be processed by the m+1 processing unit, the feature parameters output by the m+1th processing unit are processed. Process the numbers to obtain the characteristic parameters output by the m+1th processing unit, m is an integer, and the number of characteristic parameters output by the m+1th processing unit is greater than the number of characteristic parameters output by the mth processing unit;
  • the generation module 1603 is configured to generate a target image with a corresponding resolution higher than the resolution of the image to be processed based on the characteristic parameters output by the last processing unit among the plurality of processing units.
  • the image processing device may also include corresponding modules to implement other steps in the image processing method provided by the above embodiments.
  • the image processing device provided by the above embodiments and the image processing method provided by the above embodiments belong to With the same concept, the specific manner in which each module and unit performs operations has been described in detail in the method embodiments and will not be described again here.
  • Embodiments of the present application also provide an electronic device, including: one or more processors; a storage device configured to store one or more programs. When one or more programs are executed by one or more processors, The electronic device is allowed to implement the image processing method provided in the above embodiments.
  • FIG. 17 shows a schematic structural diagram of a computer system suitable for implementing an electronic device according to an embodiment of the present application.
  • the computer system 1700 includes a central processing unit (Central Processing Unit, CPU) 1701, which can be loaded into a random computer according to a program stored in a read-only memory (Read-Only Memory, ROM) 1702 or from a storage part 1708. Access the program in the memory (Random Access Memory, RAM) 1703 to perform various appropriate actions and processing, such as performing the method in the above embodiment. In RAM 1703, various programs and data required for system operation are also stored.
  • CPU 1701, ROM 1702 and RAM 1703 are connected to each other through bus 1704.
  • An input/output (I/O) interface 1705 is also connected to bus 1704.
  • the following components are connected to the I/O interface 1705: an input part 1706 including a keyboard, a mouse, etc.; an output part 1707 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., a speaker, etc. ; a storage section 1708 including a hard disk, etc.; and a communication section 1709 including a network interface card such as a LAN (Local Area Network) card, a modem, etc.
  • the communication section 1709 performs communication processing via a network such as the Internet.
  • Driver 1710 is also connected to I/O interface 1705 as needed.
  • Detachable Removable media 1711 such as magnetic disks, optical disks, magneto-optical disks, semiconductor memories, etc., are installed on the drive 1710 as needed, so that computer programs read therefrom are installed into the storage portion 1708 as needed.
  • the process described above with reference to the flowchart may be implemented as a computer software program.
  • embodiments of the present application include a computer program product including a computer program carried on a computer-readable medium, the computer program including a computer program for performing the method shown in the flowchart.
  • the computer program may be downloaded and installed from the network via communications portion 1709, and/or installed from removable media 1711.
  • CPU central processing unit
  • the computer-readable medium shown in the embodiments of the present application may be a computer-readable signal medium or a computer-readable storage medium, or any combination of the above two.
  • the computer-readable storage medium may be, for example, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination thereof.
  • Computer readable storage media may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard drive, random access memory (RAM), read only memory (ROM), removable Programmable Read-Only Memory (Erasable Programmable Read Only Memory, EPROM), flash memory, optical fiber, portable compact disk read-only memory (Compact Disc Read-Only Memory, CD-ROM), optical storage device, magnetic storage device, or any of the above suitable The combination.
  • a computer-readable storage medium may be any tangible medium that contains or stores a program for use by or in connection with an instruction execution system, apparatus, or device.
  • a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, in which a computer-readable computer program is carried. Such propagated data signals may take many forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above.
  • a computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium that can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device .
  • Computer programs embodied on computer-readable media may be transmitted using any suitable medium, including but not limited to: wireless, wired, etc., or any suitable combination of the above.
  • each block in the flow chart or block diagram may represent a module, program segment, or part of the code.
  • the above-mentioned module, program segment, or part of the code includes one or more executable components for implementing the specified logical function. instruction.
  • the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown one after another may actually execute substantially in parallel, or they may sometimes execute in the reverse order, depending on the functionality involved.
  • each block in the block diagram or flowchart illustration, and combinations of blocks in the block diagram or flowchart illustration can be implemented by special purpose hardware-based systems that perform the specified functions or operations, or may be implemented by special purpose hardware-based systems that perform the specified functions or operations. Achieved by a combination of specialized hardware and computer instructions.
  • the units involved in the embodiments of this application can be implemented in software or hardware, and the described units can also be provided in a processor. Among them, the names of these units do not constitute a limitation on the unit itself under certain circumstances.
  • Another aspect of the application also provides a computer-readable storage medium on which computer-readable instructions are stored.
  • the computer-readable instructions When the computer-readable instructions are executed by the processor of the electronic device, the electronic device implements the method as described above.
  • the computer-readable storage medium may be included in the electronic device described in the above embodiments, or may exist separately without being assembled into the electronic device.
  • Another aspect of the present application also provides a computer program product or computer program, which includes computer instructions.
  • the computer instructions When the computer instructions are executed by a processor, the methods provided in the above embodiments are implemented.
  • the computer instructions can be stored in a computer-readable storage medium; the processor of the electronic device can read the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the electronic device executes the above-mentioned embodiments. provided method.
  • the present disclosure is applicable to the field of positioning and navigation technology to solve the problem of inaccurate location-based positioning in related technologies, so as to achieve the effect that the corrected initial positioning point can more accurately describe the position of the device to be positioned.

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Artificial Intelligence (AREA)
  • Evolutionary Computation (AREA)
  • Biomedical Technology (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Health & Medical Sciences (AREA)
  • Biophysics (AREA)
  • Computational Linguistics (AREA)
  • Data Mining & Analysis (AREA)
  • General Health & Medical Sciences (AREA)
  • Molecular Biology (AREA)
  • Computing Systems (AREA)
  • General Engineering & Computer Science (AREA)
  • Mathematical Physics (AREA)
  • Software Systems (AREA)
  • Image Processing (AREA)

Abstract

本申请的实施例揭示了一种图像处理方法及装置、电子设备、存储介质，该方法包括：获取待处理图像和包含多个处理单元的图像处理模型，将待处理图像输入至图像处理模型，依次通过多个处理单元对待处理图像进行处理，得到每个处理单元输出的特征参数；在通过第m+1个处理单元进行处理的过程中，通过第m+1个处理单元对第m个处理单元输出的特征参数进行处理，得到第m+1个处理单元输出的特征参数，第m+1个处理单元输出的特征参数的数量大于第m个处理单元输出的特征参数的数量；基于最后一个处理单元输出的特征参数，生成分辨率高于待处理图像分辨率的目标图像。本申请实施例的技术方案能够提升图像超分辨率构建的准确性。

Description

图像处理方法及装置、电子设备、存储介质
相关申请的交叉引用
本公开要求于2022年08月12日提交的申请号为202210971621.9、名称为“图像处理方法及装置、电子设备、存储介质”的中国专利申请的优先权，该中国专利申请的全部内容通过引用全部并入本文。
技术领域
本申请涉及计算机技术领域，具体而言，涉及一种图像处理方法及装置、电子设备、存储介质、程序产品。
背景技术
随着图像处理技术的快速发展，对图像的分辨率要求越来越高。图像超分辨率是计算机视觉中的一种重要的图像处理技术，其目标是基于低分辨率图像重建出高分辨率图像。但是常见的图像超分辨率方法在重建过程中容易丢失细节，所得的高分辨率图像的效果欠佳，影响图像超分辨率构建的准确性。
发明内容
为解决上述技术问题，本申请的实施例提供了一种图像处理方法及装置、电子设备、存储介质、程序产品。
根据本申请实施例的一个方面，提供了一种图像处理方法，所述方法包括：
获取待处理图像以及图像处理模型；所述图像处理模型包含多个处理单元；
将所述待处理图像输入至所述图像处理模型，并依次通过所述多个处理单元对所述待处理图像进行处理，得到每个处理单元输出的特征参数；其中，在通过所述多个处理单元中的第m+1个处理单元对所述待处理图像进行处理的过程中，通过所述第m+1个处理单元对第m个处理单元输出的特征参数进行处理，得到所述第m+1个处理单元输出的特征参数，所述m为整数，第m+1个处理单元输出的特征参数的数量大于第m个处理单元输出的特征参数的数量；
基于所述多个处理单元中的最后一个处理单元输出的特征参数，生成所对应的分辨率高于所述待处理图像的分辨率的目标图像。
根据本申请实施例的一个方面，提供了一种图像处理装置，所述装置包括：
获取模块，配置为获取待处理图像以及图像处理模型；所述图像处理模型包含多个处理单元；
处理模块，配置为将所述待处理图像输入至所述图像处理模型，并依次通过所述多个处理单元对所述待处理图像进行处理，得到每个处理单元输出的特征参数；其中，在通过所述多个处理单元中的第m+1个处理单元对所述待处理图像进行处理的过程中，通过所述第m+1个处理单元对第m个处理单元输出的特征参数进行处理，得到所述第m+1个处理单元输出的特征参数，所述m为整数，第m+1个处理单元输出的特征参数的数量大于第m个处理单元输出的特征参数的数量；
生成模块，配置为基于所述多个处理单元中的最后一个处理单元输出的特征参数，生成所对应的分辨率高于所述待处理图像的分辨率的目标图像。
根据本申请实施例的一个方面，提供了一种电子设备，包括：
一个或多个处理器；
存储装置，用于存储一个或多个程序，当所述一个或多个程序被所述一个或多个处理器执行时，使得所述电子设备实现如前所述的图像处理方法。
根据本申请实施例的一个方面，提供了一种计算机可读存储介质，其上存储有计算机可读指令，当所述计算机可读指令被电子设备的处理器执行时，使电子设备执行如前所述的图像处理方法。
根据本申请实施例的一个方面，提供了一种计算机程序产品，包括计算机程序，所述计算机指令被处理器执行时实现如前所述的图像处理方法。
应当理解的是，以上的一般描述和后文的细节描述仅是示例性和解释性的，并不能限制本申请。
附图说明
此处的附图被并入说明书中并构成本说明书的一部分，示出了符合本 申请的实施例，并与说明书一起用于解释本申请的原理。显而易见地，下面描述中的附图仅仅是本申请的一些实施例，对于本领域普通技术者来讲，在不付出创造性劳动的前提下，还可以根据这些附图获得其他的附图。在附图中：
图1是本申请的一示例性实施例示出的图像处理方法的流程图；
图2是本申请的一示例性实施例示出的图像处理模型的结构示意图；
图3是本申请的一示例性实施例示出的图像处理模型的另一种结构示意图；
图4是本申请的一示例性实施例示出的处理子单元的处理流程图；
图5是本申请的另一示例性实施例示出的图像处理模型的结构示意图；
图6是本申请的一示例性实施例示出的预处理单元的处理流程图；
图7是本申请的另一示例性实施例示出的图像处理模型的结构示意图；
图8是本申请的另一示例性实施例示出的图像处理模型的结构示意图；
图9是本申请的另一示例性实施例示出的图像处理模型的优化流程图；
图10是图9所示实施例中的步骤S910在一示例性实施例中的流程图；
图11是本申请的一示例性实施例示出的获取标准图像的示意图；
图12是图9所示实施例中的步骤S930在一示例性实施例中的流程图；
图13是本申请的一示例性实施例示出的图像处理方法的流程图；
图14是本申请的另一示例性实施例示出的图像处理模型的结构示意图；
图15是本申请的另一示例性实施例示出的图像处理模型的处理框图；
图16是本申请的一示例性实施例示出的图像处理装置的结构示意图；
图17示出了适于用来实现本申请实施例的电子设备的计算机系统的结构示意图。
具体实施方式
这里将详细地对示例性实施例执行说明，其示例表示在附图中。下面的描述涉及附图时，除非另有表示，不同附图中的相同数字表示相同或相似的要素。以下示例性实施例中所描述的实施方式并不代表与本申请相一致的所有实施方式。相反，它们仅是与如所附权利要求书中所详述的、本申请的一些方面相一致的装置和方法的例子。
附图中所示的方框图仅仅是功能实体，不一定必须与物理上独立的实体相对应。即，可以采用软件形式来实现这些功能实体，或在一个或多个硬件模块或集成电路中实现这些功能实体，或在不同网络和/或处理器装置和/或微控制器装置中实现这些功能实体。
附图中所示的流程图仅是示例性说明，不是必须包括所有的内容和操作/步骤，也不是必须按所描述的顺序执行。例如，有的操作/步骤还可以分解，而有的操作/步骤可以合并或部分合并，因此实际执行的顺序有可能根据实际情况改变。
还需要说明的是：在本申请中提及的“多个”是指两个或者两个以上。“和/或”描述关联对象的关联关系，表示可以存在三种关系，例如，A和/或B可以表示：单独存在A，同时存在A和B，单独存在B这三种情况。字符“/”一般表示前后关联对象是一种“或”的关系。
相关技术中，图像超分辨率方法在重建过程中容易丢失细节，所得的高分辨率图像的效果欠佳，影响图像超分辨率构建的准确性。基于此，本申请的实施例提出了一种图像处理方法及装置、电子设备、存储介质、程序产品，从而可以提升图像超分辨率构建的准确性。
参见图1，图1是本申请的一示例性实施例示出的一种图像处理方法的流程图。如图1所示，在一示例性实施例中，该图像处理方法可以包括步骤S110至步骤S130，详细介绍如下：
步骤S110，获取待处理图像以及图像处理模型；图像处理模型包含多个处理单元。
需要说明的是，图像处理模型为一种机器学习模型，用于提升图像的分辨率，其具体类型可以根据实际需要灵活设置，例如，包括但不限于卷积神经网络、循环神经网络等，其中，卷积神经网路包括但不限于残差网络。
图像处理模型包含多个处理单元，用于对输入的数据进行处理，多个处理单元依次连接，也就是说，参见图2所示，图像处理模型200包括多个处理单元210，前一个处理单元的输出端与后一个处理单元的输入端连接。
待处理图像为需要提升分辨率的图像，其可以是视频中的视频帧，也 可以是单独的图像。待处理图像的格式包括但不限于bmp(位图图像)，jpeg(Joint Photographic Experts Group，联合图像专家组)，png(Portable Network Graphics，便携式网络图形)，tiff(Tag Image File Format，标签图像文件格式)等。
在需要提升图像的分辨率时，可以获取待处理图像以及图像处理模型。
步骤S120，将待处理图像输入至图像处理模型，并依次通过多个处理单元对待处理图像进行处理，得到每个处理单元输出的特征参数；其中，在通过多个处理单元中的第m+1个处理单元对待处理图像进行处理的过程中，通过第m+1个处理单元对第m个处理单元输出的特征参数进行处理，得到第m+1个处理单元输出的特征参数，m为整数，第m+1个处理单元输出的特征参数的数量大于第m个处理单元输出的特征参数的数量。
特征参数为表征待处理图像的特征的参数，其形式包括但不限于特征图、特征向量等。
m为整数，其取值范围可以是[1，M]，也就是说，m为大于等于1、且小于等于M的整数。其中，M为处理单元的数量，M为大于1的整数。
为了提升待处理图像的分辨率，本实施例中，将待处理图像输入至图像处理模型，图像处理模型包含的多个处理单元会依次对待处理图像进行处理。
其中，多个处理单元依次连接，因此，第m个处理单元输出的特征参数会输入至第m+1个处理单元，以通过第m+1个处理单元对第m个处理单元输出的特征参数进行处理，得到第m+1个处理单元输出的特征参数，进而得到每个处理单元输出的特征参数。也就是说，多个处理单元依次对待处理图像进行处理的过程中，第一个处理单元对待处理图像进行处理后，得到第一个处理单元输出的特征参数；将第一个处理单元输出的特征参数输入至第二个处理单元，以通过第二个处理单元对第一个处理单元输出的特征参数进行处理，得到第二个处理单元输出的特征参数；将第二个处理单元输出的特征参数输入至第三个处理单元，并通过第三个处理单元对第二个处理单元输出的特征参数进行处理，并依此类推，直至得到最后一个处理单元输出的特征参数。
并且，第m+1个处理单元输出的特征参数的数量大于第m个处理单 元输出的特征参数的数量，即，每个处理单元输出的特征参数的数量大于其上一个处理单元输出的特征单元的数量，也就是说，基于数据传输方向，多个处理单元输出的特征参数的数量依次递增，使得在处理前期图像处理模型偏向于提取图像的全局结构特征，在处理后期偏向于提取图像的细节纹理特征，也就是说，图像处理模型可以实现由粗略到精细的递进式高分辨率图像的生成，提升高分辨率图像的结构完整性和纹理精致度，提升图像超分辨率构建的准确性。
其中，每个处理单元输出的特征参数的具体数量可以根据实际需要灵活设置。可选的，每个处理单元输出的特征参数的数量可以由图像处理模型自行学习得到，或者，也可以由模型开发工程师设置，即，每个处理单元输出的特征参数的数量可以是图像处理模型的超参数。
步骤S130，基于多个处理单元中的最后一个处理单元输出的特征参数，生成所对应的分辨率高于待处理图像的分辨率的目标图像。
在得到最后一个处理单元输出的特征参数后，图像处理模块可以基于最后一个处理单元输出的特征参数生成目标图像，其中，目标图像的分辨率高于待处理图像的分辨率。
可选的，图像处理模型还可以包括上采样单元，通过上采样单元对最后一个处理单元输出的特征参数进行处理得到特征图，并对特征图进行上采样，得到目标图片。其中，上采样单元进行上采样的具体方式可以根据实际需要灵活设置，例如，可以通过最近邻插值，双线性插值，双三次插值等中的至少一种来进行上采样。上采样单元的个数以及上采样的倍率可以根据实际需要灵活设置，例如，在一个示例中，参见图3所示，图像处理模型200可以包括2个上采样单元220，每个上采样单元220的上采样倍率为2，从而进行4倍率上采样。
需要说明的是，本实施例提供的图像处理方法可以应用于不同场景，例如，图像修复、图像融合、图像编辑等场景。在一可选的示例中，可以应用于虚拟现实(Virtual Reality，VR)场景，例如，用户可以佩戴VR设备(例如，VR眼镜或头盔等)，VR设备可以通过传递沉浸式的视觉和听觉信息使用户获得身临其境的体验，其中，VR设备可以通过集成在机身或者外置于房间的传感器判断用户目前的头部位置和视野方向，并在VR 设备屏幕上展示全景图像或全景视频中对应的可视区域，但是全景图像帧的数据量较大，通常能够达到上千万像素级(例如，4K分辨率、8K分辨率)等，导致将全景图像或全景视频由数据提供方传输至VR设备的过程中占用大量的内存和网络带宽，因此，为了降低内存压力以及网络带宽消耗，数据提供方可以将低分辨率图像传输至VR设备，VR设备将低分辨率图像作为待处理图像，并通过本实施例提供的图像处理方法将低分辨率图像还原为高分辨率图像，从而不仅可以获得高分辨率图像，还可以降低网络的带宽消耗。
本实施例中，获取待处理图像以及图像处理模型，图像处理模型包含多个处理单元；将待处理图像输入至图像处理模型，并依次通过多个处理单元对待处理图像进行处理，得到每个处理单元输出的特征参数；其中，在通过多个处理单元中的第m+1个处理单元对待处理图像进行处理的过程中，通过第m+1个处理单元对第m个处理单元输出的特征参数进行处理，得到第m+1个处理单元输出的特征参数，m为整数，第m+1个处理单元输出的特征参数的数量大于第m个处理单元输出的特征参数的数量；基于多个处理单元中的最后一个处理单元输出的特征参数，生成所对应的分辨率高于待处理图像的分辨率的目标图像，也就是说，在图像处理模型处理待处理图像的过程中，特征参数的数量随着数据传输方向依次增加，使得在处理前期图像处理模型偏向于提取图像的全局结构特征，在处理后期偏向于提取图像的细节纹理特征，也就是说，图像处理模型可以实现由粗略到精细的递进式高分辨率图像的生成，从而提升高分辨率图像的结构完整性和纹理精致度，提升图像超分辨率构建的准确性。
在一示例性实施例中，参见图4所示，图4为图1所示实施例的基础上，提出的图像处理方法的流程图。如图4所示，在每个处理单元包含输出的特征参数的数量相同的多个处理子单元，每个处理子单元包含多层处理层的条件下，图像处理方法还可以包括步骤S410、步骤S411-步骤S414，详细介绍如下：
步骤S410，依次通过第m个处理单元包含的多个处理子单元对待处理图像进行处理，得到第m个处理单元包含的每个处理子单元输出的特征参数，并将第m个处理单元中的最后一个处理子单元输出的特征参数作为 第m个处理单元输出的特征参数。
需要说明的是，每个处理单元包含多个处理子单元，且属于同一处理单元的多个子单元输出的特征参数的数量相同，每个处理单元包含的处理子单元的具体数量可以根据实际需要灵活设置，不同处理单元包含的处理子单元的数量可以相同，也可以不同。处理子单元的类型可以根据实际需要灵活设置，在一个示例中，若图像处理模型为卷积神经网络，处理子单元可以是卷积块。
为了获取到第m个处理单元输出的特征参数，本实施例中，可以通过第m个处理单元所包含的多个处理子单元依次对待处理图像进行处理，从而得到第m个处理单元包含的每个处理子单元输出的特征参数，并将第m个处理单元中的最后一个处理子单元输出的特征参数作为第m个处理单元输出的特征参数，然后，将第m个处理单元输出的特征参数输入至第m+1个处理单元。也就是说，参见图5所示，每个处理单元210包含多个处理子单元211，上一个处理子单元211的输出端与下一个处理子单元211的输入端连接。在每个处理单元的处理过程中，先获取该处理单元的第一个处理子单元输出的特征参数，将该处理单元的第一个处理子单元输出的特征参数输入至该处理单元的第二个处理子单元，以通过该处理单元的第二个处理子单元对该处理单元的第一个处理子单元输出的特征参数进行处理，得到该处理单元的第二个处理子单元输出的特征参数，并依此类推，直至得到该处理单元的最后一个处理子单元输出的特征参数，将该处理单元的最后一个处理子单元输出的特征参数输入至该处理单元的下一个处理单元的第一个处理子单元。
步骤S411，在通过第m个处理单元中的第j+1个处理子单元对待处理图像进行处理的过程中，将第m个处理单元中的第j个处理子单元输出的特征参数输入至第j+1个处理子单元中的第一层处理层，得到第一层处理层输出的特征参数；其中，j为整数。
需要说明的是，j为整数，其取值范围基于处理单元包含的处理子单元的数量确定，第j个处理子单元为其所属的处理单元中的任一处理子单元。
每个处理子单元包含多层处理层，且属于同一处理子单元的多层处理 层输出的特征参数的数量相同，每个处理子单元包含的处理层的具体数量可以根据实际需要灵活设置，不同处理子单元包含的处理层的数量可以相同，也可以不同。处理子单元包含的处理层的类型可以根据实际需要灵活设置。
第m个处理单元中的第j+1个处理子单元也包含多层处理层，在通过第m个处理单元中的第j+1个处理子单元对待处理图像进行处理的过程中，先将第m个处理单元中的第j个处理子单元输出的特征参数输入至第j+1个处理子单元中的第一层处理层，以通过第j+1个处理子单元中的第一层处理层对第m个处理单元中的第j个处理子单元输出的特征参数进行处理，得到第j+1个处理子单元中的第一层处理层输出的特征参数。
步骤S412，将第一层处理层输出的特征参数输入至第一层处理层的下一层处理层，直至得到第j+1个处理子单元中的倒数第二层处理层输出的特征参数。
将第j+1个处理子单元中的第一层处理层输出的特征参数输入至第j+1个处理子单元中的第二层处理层中进行处理，得到第j+1个处理子单元中的第二层处理层输出的特征参数，再将第j+1个处理子单元中的第二层处理层输出的特征参数输入至第j+1个处理子单元中的第三层处理层，直至得到第j+1个处理子单元中的倒数第二层处理层输出的特征参数。
步骤S413，通过第j+1个处理子单元中的最后一层处理层对输入第一层处理层的特征参数和倒数第二层处理层输出的特征参数进行处理，得到最后一层处理层输出的特征参数。
将输入至第j+1个处理子单元中的第一层处理层的特征参数与第j+1个处理子单元中的倒数第二层处理层输出的特征参数输入至第j+1个处理子单元中的最后一层处理层，得到第j+1个处理子单元中的最后一层处理层输出的特征参数。
也就是说，参见图5所示，每个处理单元210的每个处理子单元211所包含的多层处理层2110中，多层处理层2110依次连接，并且，第一层处理层2110的输入端与最后一层处理层2110的输入端连接。在每个处理子单元的处理过程中，上一层处理层输出的特征参数输入至下一层处理层中，并且，第一层处理层的输入特征参数(即，输入至第一层处理层的特 征参数)输入至最后一层处理层。
在一个示例中，若图像处理模型为卷积神经网络，处理子单元为卷积块，则按照数据传输方向，处理子单元包含的处理层可以依次为：第一卷积层、第一标准化层、非线性整流层、第二卷积层、第二标准化层、叠加层，其中，第一卷积层、第二卷积层能够通过卷积运算对输入特征参数进行融合和内容修正；第一标准化层和第二标准化层能够将输入特征参数归一化至一定的范围内，提高模型的收敛稳定性；非线性整流层能够避免在模型较深的情况下神经元不被激活的问题，加快模型的整体收敛速率；叠加层的输入端与第一层卷积层的输入端以及第二层标准化层的输出端连接，从而可以将输入第一层卷积层的特征参数以及第二标准化层输出的特征参数进行叠加。
步骤S414，将最后一层输出的特征参数作为第j+1个处理子单元输出的特征参数。
本实施例中，在每个处理单元包含输出的特征参数的数量相同的多个处理子单元，每个处理子单元包含多层处理层的条件下，每个处理子单元中，最后一层处理层对输入第一层处理层的特征参数和倒数第二层输出的特征参数进行处理，从而能够将每个处理子单元的首尾连接，保障图像的特征参数在每一个处理子单元的修正是有限的，避免过度修正导致图像失真的情况。
在一示例性实施例中，参见图6所示，图6为图1所示实施例的基础上，提出的图像处理方法的流程图。如图6所示，在图像处理模型还包括多个预处理单元的条件下，图像处理方法还可以包括步骤S610-步骤S620，详细介绍如下：
步骤S610，依次通过多个预处理单元对待处理图像进行处理，得到每个预处理单元输出的特征参数；其中，在通过多个预处理单元中的第n+1个预处理单元对待处理图像进行处理的过程中，通过第n+1个预处理单元对第n个预处理单元输出的特征参数进行处理，得到第n+1个预处理单元输出的特征参数，n为整数，第n+1个预处理单元输出的特征参数的数量小于第n个预处理单元输出的特征参数的数量。
图像处理模型还包括多个预处理单元，预处理单元的具体数量可以根 据实际需要灵活设置，其数量可以与处理单元的数量相等，也可以不等。预处理单元的具体类型可以根据实际需要灵活设置，例如，可以是用于卷积运算的神经网络。
n为整数，第n个预处理单元为图像处理模型包括的多个预处理单元中的任意一个预处理单元。
图像处理模型包含的多个预处理单元会依次对待处理图像进行处理。其中，第n个预处理单元输出的特征参数会输入至第n+1个预处理单元，以通过第n+1个预处理单元对第n个预处理单元输出的特征参数进行处理，得到第n+1个预处理单元输出的特征参数，进而得到每个预处理单元输出的特征参数。也就是说，多个预处理单元依次对待处理图像进行处理的过程中，第一个预处理单元对待处理图像进行处理后，得到第一个预处理单元输出的特征参数；将第一个预处理单元输出的特征参数输入至第二个预处理单元，以通过第二个预处理单元对第一个预处理单元输出的特征参数进行处理，得到第二个预处理单元输出的特征参数；将第二个预处理单元输出的特征参数输入至第三个预处理单元，并通过第三个预处理单元对第二个预处理单元输出的特征参数进行处理，并依此类推，直至得到最后一个预处理单元输出的特征参数。
并且，每个预处理单元输出的特征参数的数量小于其上一个预处理单元输出的特征单元的数量，也就是说，基于数据传输方向，多个预处理单元输出的特征参数的数量依次递减。
步骤S610，将多个预处理单元中的最后一个预处理单元输出的特征参数输入至多个处理单元中的第一个处理单元，得到第一个处理单元输出的特征参数。
将最后一个预处理单元输出的特征参数输入至多个处理单元中的第一个处理单元，以通过第一个处理单元对最后一个预处理单元输出的特征参数进行处理，得到第一个处理单元输出的特征参数。
也就是说，参见图7所示，多个预处理单元230依次连接，上一个预处理单元230的输出端与下一个预处理单元230的输入端连接，并且，最后一个预处理单元230的输出端与第一个处理模块210的输入端连接。
可选的，在一示例性实施方式中，在第m个处理单元输出的特征参数 的数量与第M-m+1个预处理单元输出的特征参数的数量相匹配(例如，相等)的条件下，步骤S120中，通过第m+1个处理单元对第m个处理单元输出的特征参数进行处理，得到第m+1个处理单元输出的特征参数的过程可以包括：将第m个处理单元输出的特征参数以及第M-m+1个预处理单元输出的特征参数输入至第m+1个处理单元，得到第m+1个处理单元输出的特征参数。
对应的，步骤S130中，基于多个处理单元中的最后一个处理单元输出的特征参数，生成所对应的分辨率高于待处理图像的分辨率的目标图像的过程可以包括：基于最后一个处理单元输出的特征参数以及多个预处理单元中的第一个预处理单元输出的特征参数，生成所对应的分辨率高于待处理图像的分辨率的目标图像。
也就是说，参见图8所示，在第m个处理单元输出的特征参数的数量与第M-m+1个预处理单元输出的特征参数的数量相匹配(例如，相等)的条件下，第m个处理单元输出端与第M-m+1个预处理单元的输出端连接，这样，可以保障待处理图像在不同单元中的特征参数能够传递至模型尾端，缓解了在深度神经网络中增加深度导致的梯度消失的问题，提升图像处理模型的准确度。
本实施例中，图像处理模型中设置多个预处理单元，多个预处理单元依次连接，最后一个预处理单元的输出端与第一个处理单元的输入端连接，并且，按照数据传输方向，多个预处理单元输出的特征参数的数量依次减少，从而提升图像处理模型的精度。
在一示例性实施例中，参见图9所示，图9为图1所示实施例的基础上，提出的图像处理方法的流程图。如图9所示，图像处理方法还可以包括步骤S910-步骤S940，详细介绍如下：
步骤S910，获取样本图像，并从样本图像中提取出多张标准图像。
其中，样本图像是指作为样本的图像，其可以是高分辨率的图像。
为了提升图像处理模型的训练样本数量，本实施例中，可以从样本图像中提取出多张图像，将提取出的图像作为标准图像。
其中，从样本图像中提取出多张标准图像的具体方式可以根据实际需要灵活设置。
步骤S920，降低每张标准图像的分辨率，得到每张标准图像对应的输入图像。
为了获取训练样本，在得到多张标准图像后，可以降低每张标准图像的分辨率，得到每张标准图像对应的输入图像。也就是说，输入图像的分辨率小于对应标准图像的分辨率。
其中，降低标准图像的分辨率的具体方式可以根据实际需要灵活设置，例如，可以对标准图像进行下采样，得到标准图像对应的输入图像。下采样的具体方式可以根据实际需要灵活设置，例如，可以通过双线性下采样的方式进行下采样。为了提升图像处理模型的泛化能力，还可以设置多种下采样方式，针对每张标准图像，可以随机从多种下采样方式中选择其中一种下采样方式对标准图像进行下采样。下采样的倍率可以根据实际需要灵活设置，例如，可以设置为4倍、2倍等。
步骤S930，通过图像处理模型对每张输入图像进行处理，得到每张输入图像对应的输出图像。
在得到低分辨率的输入图像后，可以将输入图像输入至图像处理模型，以使图像处理模型对输入图像进行处理，得到输入图像对应的输出图像。
步骤S940，基于每张输入图像对应的输出图像和标准图像之间的差异，计算图像处理模型的损失值，并基于计算出的损失值调整图像处理模型的参数。
为了计算图像处理模型的损失，本实施例中，可以计算输出图像与对应标准图像之间的差异，并根据差异计算图像处理模型的损失值，根据图像处理模型的损失值调整图像处理模型的参数，以优化图像处理模型。
其中，计算输出图像与对应标准图像之间的差异的具体方式可以根据实际需要灵活设置。可选的，可以通过以下方式计算输出图像与对应标准图像之间的差异：
其中，LMSE为输出图像与对应标准图像之间的差异，w为图像的宽度，h为图像的高度，IHR(x，y)为标准图像中横坐标为x，纵坐标为y的像素点的像素值，ISR(x，y)为对应输出图像中横坐标为x，纵坐标为y的像素点的像素值。
可选的，在计算出每张输出图像与对应标准图像之间的差异后，还可以基于输出图像与对应标准图像之间的差异计算图像处理模型的损失值；可选的，可以对图像处理模型进行批量训练，即，每次输入图像处理模型的输入图片的数量为多张，对应的，每次训练得到多张输出图片，在计算出这多张输出图片与对应标准图片的差异后，可以将多张输出图片与对应标准图像之间的差异的平均值作为图像处理模型的损失值。
需要说明的是，步骤S910-步骤S940的过程可以应用于图像处理模型的训练阶段，或者，也可以应用于图像处理模型训练完成后的优化阶段。
本实施例中，获取样本图像，并从样本图像中提取出多张标准图像，降低每张标准图像的分辨率，得到每张标准图像对应的输入图像，通过图像处理模型对每张输入图像进行处理，得到每张输入图像对应的输出图像，基于每张输入图像对应的输出图像和标准图像之间的差异，计算图像处理模型的损失值，并基于计算出的损失值调整图像处理模型的参数，这样，通过一张样本图像，可以获取到多张用于进行训练的标准图像和对应的输入图像，从而提升训练样本的数量，降低训练样本的获取难度，并且，标准图像的尺寸小于样本图像的尺寸，可以提升图像处理模型的处理效率，降低模型训练过程中所占用的计算资源量，提升模型收敛速度。
在一示例性实施例中，参见图10所示，图10为图9所示的步骤S910在一示例性实施例中的流程图。如图10所示，从样本图像中提取出多张标准图像的过程可以包括步骤S911-步骤S912，详细介绍如下：
步骤S911，从样本图像中确定多个尺寸与第一尺寸相匹配的区域。
第一尺寸为预先设置的区域的尺寸，包括宽度和高度，宽度和高度可以相等，也可以不同，第一尺寸的具体大小可以根据实际需要灵活设置。
本实施例中，可以从样本图像中确定出多个区域，并且，每个区域的尺寸与第一尺寸相匹配。其中，相邻区域之间可以存在重叠，也可以不存在重叠。
其中，从样本图像中确定多个尺寸与第一尺寸相匹配的区域的具体方式可以根据实际需要灵活设置。
在一可选的实施方式中，可以基于第一尺寸将样本图像划分为多个互相不重叠的区域。
在另一可选的实施方式中，可以基于宽度方向步长和高度方向步长在样本图像上进行移动，以确定出多个尺寸与第一尺寸相匹配的区域；其中，宽度方向步长为在宽度方向上移动的步长，高度方向步长为在高度方向上移动的步长；也就是说，确定出的多个区域的尺寸均为第一尺寸，且，宽度方向相邻的两个区域之间的间隔为宽度方向步长，高度方向相邻的两个区域之间的间隔为高度方向步长。其中，宽度方向步长和高度方向步长可以相等，也可以不等，其具体长度可以根据实际需要灵活设置。其中，假设样本图像的宽度为w′，高度为h′，宽度方向步长和高度方向步长均为dspace，区域的宽度和高度均为dlarge，则参见图11所示，样本图像1210中，可以依次确定出多个尺寸为dlarge×dlarge的区域1211，并且，相邻区域1211的中心点之间的间距为dspace，每个区域1211中可以随机确定出一个尺寸为dsmall×dsmall的图像块作为标准图像1212，其中，确定出的区域的个数Nlarge如下：
为向下取整函数。
需要说明的是，基于宽度方向步长和高度方向步长在样本图像上进行移动的具体方式可以根据实际需要灵活设置，例如，可以在样本图像的左边缘设置尺寸与第一尺寸相匹配的多个采样窗口，且多个采样窗口的位置在高度方向上的间距为高度方向步长，在多个采样窗口采样后，基于宽度方向步长向左移动多个采样窗口，直至采样窗口到达样本图像的右边缘，从而得到多个区域。或者，可以在样本图像的左上角设置一个采样窗口，在采样窗口采样后，可以按照从左到右、从上到下的方式在样本图像上进行移动，其中，从左到右移动的步长为宽度方向步长，从上到下移动的步长为高度方向步长。
步骤S912，从每个区域中随机提取一张尺寸与第二尺寸相匹配的图像，并将提取出的图像作为标准图像，其中，第一尺寸大于第二尺寸。
其中，第二尺寸为标准图像的尺寸，第二尺寸包含标准图像的宽度和高度，其中，标准图像的宽度和高度可以相等，也可以不等。
在从样本图像中确定出多个区域后，可以从每个区域中随机提取一张尺寸与第二尺寸相匹配的图像，并将提取出的图像作为标准图像。
可选的，还可以对提取出的图像进行翻转，并将翻转后得到的图像也作为标准图像。例如，可以对提取出的图像进行水平翻转，得到标准图像。
本实施例中，从样本图像中确定多个尺寸与第一尺寸相匹配的区域，从每个区域中随机提取一张尺寸与第二尺寸相匹配的图像，并将提取出的图像作为标准图像，其中，第一尺寸大于第二尺寸，这样，在不同区域中进行随机采样，以得到标准图像，可以保障用于训练的标准图像具有良好的特征多样性，从而能够有效地避免图像处理模型在训练过程中产生过拟合的问题，提高图像处理模型的泛化能力。
在一示例性实施例中，参见图12所示，图12为图9所示实施例中的步骤S930在一示例性实施例中的流程图。如图12所示，通过图像处理模型对每张输入图像进行处理，得到每张输入图像对应的输出图像的过程可以包括步骤S931-步骤S933，详细介绍如下：
步骤S931，获取输入图像集合；输入图像集合包含每张标准图像对应的输入图像。
其中，输入图像集合包含每张标准图像对应的输入图像，由于标准图像的数量为多张，对应的，输入图像集合中包含多张输入图像。
步骤S932，从输入图像集合中获取多张输入图像，并将多张输入图像输入至图像处理模型，得到多张输入图像各自对应的输出图像。
为了提升图像处理模型的优化速度，本实施例中，可以从输入图像集合中获取多张输入图像，并将多张输入图像输入至图像处理模型，使得图像处理模型可以批量处理输入图像，得到输入图像各自对应的输出图像。
需要说明的是，每次从输入图像集合中获取的输入图像的具体数量可以根据实际需要灵活设置，例如，可以设置为10张、20张等，或者，可以从输入图像集合中获取全部输入图像，并将获取到的输入图像一起输入至图像处理模型。
步骤S933，从输入图像集合中重新获取多张输入图像，并将重新获取的多张输入图像输入至图像处理模型，直至得到输入图像集合中的每张输入图像对应的输出图像。
在得到获取到的多张输入图像各自对应的输出图像后，若输入图像集合中还存在未输入至图像处理模型的输入图像，则从输入图像集合中重新 获取多张输入图像，并将重新获取的多张输入图像输入至图像处理模型，直至得到输入图像集合包含的每张输入图像对应的输出图像。
本实施例中，可以先获取输入图像集合，输入图像集合包含每张标准图像对应的输入图像；从输入图像集合中获取多张输入图像，并将多张输入图像输入至图像处理模型，得到多张输入图像各自对应的输出图像，从输入图像集合中重新获取多张输入图像，并将重新获取的多张输入图像输入至图像处理模型，直至得到输入图像集合中的每张输入图像对应的输出图像，从而可以对图像处理模型进行批量训练。
以下对本申请实施例的一个具体应用场景进行详细说明。请参见图13所示，图像处理方法包括：
步骤S1301，获取样本图像。
其中，样本图像可以是全景图像，也可以是其它类型的图像。
步骤S1302，对样本图像进行分区采样，得到标准图像。
可选的，可以从样本图像的左上角开始，以尺寸dlarge×dlarge为模板，dspace为步长逐步向样本图像的右下角平移，得到多个区域。从每个区域中以尺寸dsmall×dsmall为模板随机采样一个图像块，并对图像块进行翻转，得到图像块的对称图像，将图像块以及图像块的对称图像作为标准图像。
其中，得到的标准图像的数量为：
其中，Np为标准图像的数量。
步骤S1303，对标准图像进行下采样，得到输入图像。
针对每张标准图像，可以进行下采样，得到每张标准图像的输入图像。
步骤S1304，基于输入图像和标准图像对初始图像处理模型进行训练，得到图像处理模型。
可选的，可以通过初始图像处理模型对输入图像进行处理，得到输出图像，基于输出图像与标准图像之间的差异，确定初始图像处理模型的损失值，基于损失值调整初始图像处理模型的参数，以优化初始图像处理模型。
其中，初始图像处理模型可以是递进式残差网络，参见图14所示，按照数据传输方向，初始图像处理模型包含依次连接的M个卷积单元1410、 M个处理单元1420、2个上采样单元1430以及输出层1440，其中，按照数据传输方向，上一个处理单元1420输出的特征图的数量小于下一个处理单元1420输出的特征图的数量，第m个处理单元1420输出的特征图的数量与第M-m+1个卷积单元1410输出的特征图的数量相同，并且，第m个处理单元1420的输出端与第M-m+1个卷积单元1410输出端连接；每个处理单元1420包含多个输出的特征图的数量相同的多个卷积块1421，每个卷积块1421输出的特征图的数量等于其所属的处理单元输出的特征图的数量，每个卷积块1421包含6层依次连接的处理层，按照数据传输方向，6层处理层依次为第一卷积层、第一标准化层、非线性整流层、第二卷积层、第二标准化层和像素级叠加层，其中，第一卷积层的输入端与像素级叠加层的输入端(即，第二标准化层的输出端)连接，这样，像素级叠加层可以对第一卷积层的输入端对应的特征图与第二标准化层输出的特征图进行叠加，从而限制每个卷积块对特征图的修正幅度，避免修正过大导致出现偏差的情况。每个上采样单元1430可以对输入的特征图进行2倍上采样，输出层1440可以将输入的特征图转换为指定形式的图像进行输出，例如，可以转换为RBG形式。其中，图14中，填充图案相同的矩形框所输出的特征图数量相同。
步骤S1305，获取待处理图像。
步骤S1306，通过图像处理模型对待处理图像进行处理，得到目标图像。
也就是说，参见图15所示，在获取到样本图像后，对样本图像进行分区采样得到图像块，对图像块进行水平翻转得到图像块的对称图像，将图像块和图像块的对称图像作为标准图像，再对标准图像进行下采样，得到输入图像，将输入图像和标准图像作为训练数据，基于训练数据对图像处理模型进行训练，在训练完成后，将低分辨率的待处理图像输入至图像处理模型，得到高分辨率的目标图像。需要说明的是，高分辨率的目标图像是指比低分辨率的输入图像的分辨率高的图像，此处的“高分辨率”、“低分辨率”并不用于限定图像的分辨率的具体范围，仅用于限定图像分辨率的相对关系。
为了更好的理解，此处以一个示例进行说明，训练过程包括1.1-1.3， 详细介绍如下：
1.1获取训练数据：以尺寸416像素×416像素为模板，从单幅高分辨率样本图像的左上角开始往右下角裁剪图像，且相邻两个裁剪区域的中心点之间的距离为208像素。此时，宽度方向上的区域数量为(3328-416)/208+1＝15个，高度方向上的区域数量为(1664-416)/208+1＝7个，因此总共有15×7＝105个区域。对于每一个区域，从中随机选择一个208像素×208像素的区域作为图像块，并且对于每一个图像块进行水平翻转得到一幅对称图像，将图像块以及对称图像作为标准图像，得到210个标准图像。对于每一个208×208像素的标准图像，进行双线性下采样得到52×52像素的输入图像，每一个52×52像素的低分辨率的输入图像和208×208像素的高分辨率的标准图像之间的映射关系便是图像处理模型需要学习的目标。
1.2构造图像处理模型并基于训练数据进行训练：图像处理模型采用递进式残差网络，按照数据传输方向，其包含特征图数量为32的卷积单元、特征图数量为48的卷积单元、特征图数量为64的卷积单元、6个特征图数量为32的卷积块、6个特征图数量为48的卷积块、6个特征图数量为64的卷积块、2个上采样层以及一个输出层，其中，特征图数量为32的卷积单元的输出端与最后一个特征图数量为32的卷积块的输出端连接，特征图数量为48的卷积单元的输出端与最后一个特征图数量为48的卷积块的输出端连接，特征图数量为64的卷积单元的输出端与最后一个特征图数量为64的卷积块的输出端连接，每一个上采样层可以将图像放大2倍。在训练过程中，以每批次16张方式将输入图像分批输入至图像处理模进行模型训练，使模型能够在尾端输出4倍上采样的高分辨率的输出图像；由于每轮训练输入16张输入图像，因此，每轮训练会得到16张输出图像，计算每轮训练过程中输出图像与对应标准图像之间的差异，并基于差异确定图像处理模型的损失，利用损失值对图像处理模型进行反向传播以优化模型中各个层的参数。其中，可以设置图像处理模型总共训练1500个周期，其中前1000个周期以学习率0.0001进行训练，后500个周期的学习率为0.00001。
1.3对图像处理模型进行测试：在完成1500个周期的训练后，可以从全景图像中随机采样10幅尺寸为3328像素×1664像素的原始高分辨率图 像，对原始高分辨率图像进行4倍双线性下采样，得到尺寸为832像素×416像素的低分辨率测试图像，将低分辨率(832像素×416像素)的测试图像输入至图像处理模型，并利用图像处理模型对测试图像的信息进行补充和融合，最终得到水平长度和垂直高度均放大4倍的高分辨率(3328像素×1664像素)测试图像。将模型生成的高分辨率的测试图像和原始高分辨率图像进行对比，并基于图像评价指标对二者的相似性进行度量，以确定图像处理模型的性能。其中，图像评价指标包括但不限于PSNR(Peak Signal-to-Noise Ratio，峰值信噪比)，SSIM(Structural SIMilarity，结构相似性)，VIFP(Visual Information Fidelity in Pixel domain，像素域中的视觉信息保真度)等。
本实施例中，基于单幅样本图像分块采样得到较大规模的训练数据，有效提高了训练数据的特征多样性，降低模型训练过拟合的风险；由多个输出的特征图数量逐渐增大的卷积块组成的递进式深度生成网络模型，能够使浅层的卷积块倾向于存储图像的全局结构特征，深层的卷积块学习如何在浅层卷积块生成的全局结构特征基础上生成细节纹理特征，从而提升最终生成的高分辨图像的结构稳定性和纹理精细度，减少生成图像过程中不符合逻辑的噪声区域的数量；由多级跨步连接组成的残差网络结构中，每一个卷积块的首尾通过像素级叠加保证该级卷积块对图像的修正不会偏差过大，整个模型首尾特征图数量相同的单元之间的跨步连接保证图像的原始特征信息能够传递至生成的高分辨率图像，能够有效地减少模型的参数总量，提高模型的训练速度和预测阶段的计算效率，并且，避免生成的高分辨率图像出现失真的情况。
参见图16，图16是本申请的一示例性实施例示出的图像处理装置的框图。如图16所示，该装置包括：
获取模块1601，配置为获取待处理图像以及图像处理模型；图像处理模型包含多个处理单元；
处理模块1602，配置为将待处理图像输入至图像处理模型，并依次通过多个处理单元对待处理图像进行处理，得到每个处理单元输出的特征参数；其中，在通过多个处理单元中的第m+1个处理单元对待处理图像进行处理的过程中，通过第m+1个处理单元对第m个处理单元输出的特征参 数进行处理，得到第m+1个处理单元输出的特征参数，m为整数，第m+1个处理单元输出的特征参数的数量大于第m个处理单元输出的特征参数的数量；
生成模块1603，配置为基于多个处理单元中的最后一个处理单元输出的特征参数，生成所对应的分辨率高于待处理图像的分辨率的目标图像。
需要说明的是，图像处理装置还可以包含对应的模块以实现前述实施例所提供的图像处理方法中的其它步骤，上述实施例所提供的图像处理装置与上述实施例所提供的图像处理方法属于同一构思，其中各个模块和单元执行操作的具体方式已经在方法实施例中进行了详细描述，此处不再赘述。
本申请的实施例还提供了一种电子设备，包括：一个或多个处理器；存储装置，用于存储一个或多个程序，当一个或多个程序被一个或多个处理器执行时，使得电子设备实现上述各个实施例中提供的图像处理方法。
图17示出了适于用来实现本申请实施例的电子设备的计算机系统的结构示意图。
需要说明的是，图17示出的电子设备的计算机系统1700仅是一个示例，不应对本申请实施例的功能和使用范围带来任何限制。
如图17所示，计算机系统1700包括中央处理单元(Central Processing Unit，CPU)1701，其可以根据存储在只读存储器(Read-Only Memory，ROM)1702中的程序或者从储存部分1708加载到随机访问存储器(Random Access Memory，RAM)1703中的程序而执行各种适当的动作和处理，例如执行上述实施例中的方法。在RAM 1703中，还存储有系统操作所需的各种程序和数据。CPU 1701、ROM 1702以及RAM 1703通过总线1704彼此相连。输入/输出(Input/Output，I/O)接口1705也连接至总线1704。
以下部件连接至I/O接口1705：包括键盘、鼠标等的输入部分1706；包括诸如阴极射线管(Cathode Ray Tube，CRT)、液晶显示器(Liquid Crystal Display，LCD)等以及扬声器等的输出部分1707；包括硬盘等的储存部分1708；以及包括诸如LAN(Local Area Network，局域网)卡、调制解调器等的网络接口卡的通信部分1709。通信部分1709经由诸如因特网的网络执行通信处理。驱动器1710也根据需要连接至I/O接口1705。可拆 卸介质1711，诸如磁盘、光盘、磁光盘、半导体存储器等等，根据需要安装在驱动器1710上，以便于从其上读出的计算机程序根据需要被安装入存储部分1708。
特别地，根据本申请的实施例，上文参考流程图描述的过程可以被实现为计算机软件程序。例如，本申请的实施例包括一种计算机程序产品，其包括承载在计算机可读介质上的计算机程序，该计算机程序包含用于执行流程图所示的方法的计算机程序。在这样的实施例中，该计算机程序可以通过通信部分1709从网络上被下载和安装，和/或从可拆卸介质1711被安装。在该计算机程序被中央处理单元(CPU)1701执行时，执行本申请的系统中限定的各种功能。
需要说明的是，本申请实施例所示的计算机可读介质可以是计算机可读信号介质或者计算机可读存储介质或者是上述两者的任意组合。计算机可读存储介质例如可以是电、磁、光、电磁、红外线、或半导体的系统、装置或器件，或者任意以上的组合。计算机可读存储介质的更具体的例子可以包括但不限于：具有一个或多个导线的电连接、便携式计算机磁盘、硬盘、随机访问存储器(RAM)、只读存储器(ROM)、可擦式可编程只读存储器(Erasable Programmable Read Only Memory，EPROM)、闪存、光纤、便携式紧凑磁盘只读存储器(Compact Disc Read-Only Memory，CD-ROM)、光存储器件、磁存储器件、或者上述的任意合适的组合。在本申请中，计算机可读存储介质可以是任何包含或存储程序的有形介质，该程序可以被指令执行系统、装置或者器件使用或者与其结合使用。而在本申请中，计算机可读的信号介质可以包括在基带中或者作为载波一部分传播的数据信号，其中承载了计算机可读的计算机程序。这种传播的数据信号可以采用多种形式，包括但不限于电磁信号、光信号或上述的任意合适的组合。计算机可读的信号介质还可以是计算机可读存储介质以外的任何计算机可读介质，该计算机可读介质可以发送、传播或者传输用于由指令执行系统、装置或者器件使用或者与其结合使用的程序。计算机可读介质上包含的计算机程序可以用任何适当的介质传输，包括但不限于：无线、有线等等，或者上述的任意合适的组合。
附图中的流程图和框图，图示了按照本申请各种实施例的系统、方法 和计算机程序产品的可能实现的体系架构、功能和操作。其中，流程图或框图中的每个方框可以代表一个模块、程序段、或代码的一部分，上述模块、程序段、或代码的一部分包含一个或多个用于实现规定的逻辑功能的可执行指令。也应当注意，在有些作为替换的实现中，方框中所标注的功能也可以以不同于附图中所标注的顺序发生。例如，两个接连地表示的方框实际上可以基本并行地执行，它们有时也可以按相反的顺序执行，这依所涉及的功能而定。也要注意的是，框图或流程图中的每个方框、以及框图或流程图中的方框的组合，可以用执行规定的功能或操作的专用的基于硬件的系统来实现，或者可以用专用硬件与计算机指令的组合来实现。
描述于本申请实施例中所涉及到的单元可以通过软件的方式实现，也可以通过硬件的方式来实现，所描述的单元也可以设置在处理器中。其中，这些单元的名称在某种情况下并不构成对该单元本身的限定。
本申请的另一方面还提供了一种计算机可读存储介质，其上存储有计算机可读指令，该计算机可读指令被电子设备的处理器执行时，使电子设备实现如前所述的方法。该计算机可读存储介质可以是上述实施例中描述的电子设备中所包含的，也可以是单独存在，而未装配入该电子设备中。
本申请的另一方面还提供了一种计算机程序产品或计算机程序，该计算机程序产品或计算机程序包括计算机指令，计算机指令被处理器执行时实现上述各个实施例中提供的方法。其中，该计算机指令可以存储在计算机可读存储介质中；电子设备的处理器可以从计算机可读存储介质读取该计算机指令，处理器执行该计算机指令，使得该电子设备执行上述各个实施例中提供的方法。
工业实用性
本公开适用于定位导航技术领域，用以解决相关技术中基于位置的定位不准确的问题，达到校正后的初始定位点能更准确地描述待定位设备的位置的效果。
上述内容，仅为本申请的较佳示例性实施例，并非用于限制本申请的实施方案，本领域普通技术人员根据本申请的主要构思和精神，可以十分方便地进行相应的变通或修改，故本申请的保护范围应以权利要求书所要求的保护范围为准。

Claims (11)

  1. 一种图像处理方法，所述方法包括：
    获取待处理图像以及图像处理模型；所述图像处理模型包含多个处理单元；
    将所述待处理图像输入至所述图像处理模型，并依次通过所述多个处理单元对所述待处理图像进行处理，得到每个处理单元输出的特征参数；其中，在通过所述多个处理单元中的第m+1个处理单元对所述待处理图像进行处理的过程中，通过所述第m+1个处理单元对第m个处理单元输出的特征参数进行处理，得到所述第m+1个处理单元输出的特征参数，所述m为整数，第m+1个处理单元输出的特征参数的数量大于第m个处理单元输出的特征参数的数量；
    基于所述多个处理单元中的最后一个处理单元输出的特征参数，生成所对应的分辨率高于所述待处理图像的分辨率的目标图像。
  2. 如权利要求1所述的方法，其中，每个处理单元包含输出的特征参数的数量相同的多个处理子单元，每个处理子单元包含多层处理层；所述方法还包括：
    依次通过所述第m个处理单元包含的多个处理子单元对所述待处理图像进行处理，得到所述第m个处理单元包含的每个处理子单元输出的特征参数，并将所述第m个处理单元中的最后一个处理子单元输出的特征参数作为所述第m个处理单元输出的特征参数；
    其中，在通过所述第m个处理单元中的第j+1个处理子单元对所述待处理图像进行处理的过程中，将所述第m个处理单元中的第j个处理子单元输出的特征参数输入至所述第j+1个处理子单元中的第一层处理层，得到所述第一层处理层输出的特征参数；其中，所述j为整数；
    将所述第一层处理层输出的特征参数输入至所述第一层处理层的下一层处理层，直至得到所述第j+1个处理子单元中的倒数第二层处理层输出的特征参数；
    通过所述第j+1个处理子单元中的最后一层处理层对所述倒数第二层处理层输出的特征参数以及输入所述第一层处理层的特征参数进行处理，得到所述最后一层处理层输出的特征参数；
    将所述最后一层输出的特征参数作为所述第j+1个处理子单元输出的特征参数。
  3. 如权利要求1所述的方法，其中，所述图像处理模型还包括多个预处理单元；所述方法还包括：
    依次通过所述多个预处理单元对所述待处理图像进行处理，得到每个预处理单元输出的特征参数；其中，在通过所述多个预处理单元中的第n+1个预处理单元对所述待处理图像进行处理的过程中，通过所述第n+1个预处理单元对第n个预处理单元输出的特征参数进行处理，得到所述第n+1个预处理单元输出的特征参数，所述n为整数，第n+1个预处理单元输出的特征参数的数量小于第n个预处理单元输出的特征参数的数量；
    将所述多个预处理单元中的最后一个预处理单元输出的特征参数输入至所述多个处理单元中的第一个处理单元，得到所述第一个处理单元输出的特征参数。
  4. 如权利要求3所述的方法，其中，所述处理单元的数量与所述预处理单元的数量均为M，所述通过所述第m+1个处理单元对第m个处理单元输出的特征参数进行处理，得到所述第m+1个处理单元输出的特征参数，包括：
    将所述第m个处理单元输出的特征参数以及第M-m+1个预处理单元输出的特征参数输入至所述第m+1个处理单元，得到所述第m+1个处理单元输出的特征参数；其中，所述第m个处理单元输出的特征参数的数量与所述第M-m+1个预处理单元输出的特征参数的数量相匹配；
    所述基于所述多个处理单元中的最后一个处理单元输出的特征参数，生成所对应的分辨率高于所述待处理图像的分辨率的目标图像，包括：
    基于所述最后一个处理单元输出的特征参数以及所述多个预处理单元中的第一个预处理单元输出的特征参数，生成所对应的分辨率高于所述待处理图像的分辨率的目标图像。
  5. 如权利要求1-4中任一项所述的方法，其中，所述方法还包括：
    获取样本图像，并从所述样本图像中提取出多张标准图像；
    降低每张标准图像的分辨率，得到所述每张标准图像对应的输入图像；
    通过所述图像处理模型对所述每张输入图像进行处理，得到所述每张 输入图像对应的输出图像；
    基于所述每张输入图像对应的输出图像和标准图像之间的差异，计算所述图像处理模型的损失值，并基于计算出的损失值调整所述图像处理模型的参数。
  6. 如权利要求5所述的方法，其中，所述从所述样本图像中提取出多张标准图像，包括：
    从所述样本图像中确定多个尺寸与第一尺寸相匹配的区域；
    从每个区域中随机提取一张尺寸与第二尺寸相匹配的图像，并将提取出的图像作为目标图像，其中，所述第一尺寸大于所述第二尺寸。
  7. 如权利要求5所述的方法，其中，所述通过所述图像处理模型对所述每张输入图像进行处理，得到所述每张输入图像对应的输出图像，包括：
    获取输入图像集合；所述输入图像集合包含所述每张标准图像对应的输入图像；
    从所述输入图像集合中获取多张输入图像，并将所述多张输入图像输入至所述图像处理模型，得到所述多张输入图像各自对应的输出图像；
    从所述输入图像集合中重新获取多张输入图像，并将重新获取的多张输入图像输入至所述图像处理模型，直至得到所述输入图像集合中的每张输入图像对应的输出图像。
  8. 一种图像处理装置，其中，所述装置包括：
    获取模块，配置为获取待处理图像以及图像处理模型；所述图像处理模型包含多个处理单元；
    处理模块，配置为将所述待处理图像输入至所述图像处理模型，并依次通过所述多个处理单元对所述待处理图像进行处理，得到每个处理单元输出的特征参数；其中，在通过所述多个处理单元中的第m+1个处理单元对所述待处理图像进行处理的过程中，通过所述第m+1个处理单元对第m个处理单元输出的特征参数进行处理，得到所述第m+1个处理单元输出的特征参数，所述m为整数，第m+1个处理单元输出的特征参数的数量大于第m个处理单元输出的特征参数的数量；
    生成模块，配置为基于所述多个处理单元中的最后一个处理单元输出 的特征参数，生成所对应的分辨率高于所述待处理图像的分辨率的目标图像。
  9. 一种电子设备，其中，包括：
    一个或多个处理器；
    存储装置，用于存储一个或多个计算机程序，当所述一个或多个计算机程序被所述一个或多个处理器执行时，使得所述电子设备实现权利要求1-7中的任一项所述的图像处理方法。
  10. 一种计算机可读存储介质，其中，其上存储有计算机程序，当所述计算机程序被电子设备的处理器执行时，使所述电子设备实现权利要求1-7中的任一项所述的图像处理方法。
  11. 一种计算机程序产品，包括计算机程序，所述计算机程序被处理器执行时实现权利要求1-7中的任一项所述的图像处理方法。
PCT/CN2023/107968 2022-08-12 2023-07-18 图像处理方法及装置、电子设备、存储介质 WO2024032331A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202210971621.9A CN115311145B (zh) 2022-08-12 2022-08-12 图像处理方法及装置、电子设备、存储介质
CN202210971621.9 2022-08-12

Publications (2)

Publication Number Publication Date
WO2024032331A1 true WO2024032331A1 (zh) 2024-02-15
WO2024032331A9 WO2024032331A9 (zh) 2024-03-28

Family

ID=83862745

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2023/107968 WO2024032331A1 (zh) 2022-08-12 2023-07-18 图像处理方法及装置、电子设备、存储介质

Country Status (2)

Country Link
CN (1) CN115311145B (zh)
WO (1) WO2024032331A1 (zh)

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115311145B (zh) * 2022-08-12 2024-06-11 中国电信股份有限公司 图像处理方法及装置、电子设备、存储介质

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2003281524A (ja) * 2002-03-25 2003-10-03 Fuji Photo Film Co Ltd 画像処理方法および装置並びにプログラム
CN107481192A (zh) * 2017-08-11 2017-12-15 北京市商汤科技开发有限公司 图像处理方法、装置、存储介质、计算机程序和电子设备
CN109191382A (zh) * 2018-10-18 2019-01-11 京东方科技集团股份有限公司 图像处理方法、装置、电子设备及计算机可读存储介质
CN109903219A (zh) * 2019-02-28 2019-06-18 深圳市商汤科技有限公司 图像处理方法及装置、电子设备、计算机可读存储介质
CN113096132A (zh) * 2020-01-08 2021-07-09 东华医为科技有限公司 图像处理的方法、装置、存储介质和电子设备
CN115311145A (zh) * 2022-08-12 2022-11-08 中国电信股份有限公司 图像处理方法及装置、电子设备、存储介质

Family Cites Families (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111192200A (zh) * 2020-01-02 2020-05-22 南京邮电大学 基于融合注意力机制残差网络的图像超分辨率重建方法
CN111476737B (zh) * 2020-04-15 2022-02-11 腾讯科技（深圳）有限公司 一种图像处理方法、智能设备及计算机可读存储介质
US11288771B2 (en) * 2020-04-29 2022-03-29 Adobe Inc. Texture hallucination for large-scale image super-resolution
KR102488037B1 (ko) * 2020-09-22 2023-01-13 인천대학교 산학협력단 인공 지능에 기반한 실시간 초해상도 구현 방법 및 장치
CN112270645B (zh) * 2020-11-03 2022-05-03 中南民族大学 多阶特征循环增强的渐进高倍人脸超分辨率系统及其方法
CN112991203B (zh) * 2021-03-08 2024-05-07 Oppo广东移动通信有限公司 图像处理方法、装置、电子设备及存储介质
CN113191950B (zh) * 2021-05-07 2022-08-12 西南交通大学 一种超分辨率人脸图像重建方法
CN113421187B (zh) * 2021-06-10 2023-01-03 山东师范大学 一种超分辨率重建方法、系统、存储介质、设备
CN113962861A (zh) * 2021-10-29 2022-01-21 中国联合网络通信集团有限公司 图像重建方法、装置、电子设备和计算机可读介质
CN114266771B (zh) * 2022-03-02 2022-06-03 深圳市智源空间创新科技有限公司 基于改进扩展特征金字塔模型的管道缺陷检测方法及装置

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2003281524A (ja) * 2002-03-25 2003-10-03 Fuji Photo Film Co Ltd 画像処理方法および装置並びにプログラム
CN107481192A (zh) * 2017-08-11 2017-12-15 北京市商汤科技开发有限公司 图像处理方法、装置、存储介质、计算机程序和电子设备
CN109191382A (zh) * 2018-10-18 2019-01-11 京东方科技集团股份有限公司 图像处理方法、装置、电子设备及计算机可读存储介质
CN109903219A (zh) * 2019-02-28 2019-06-18 深圳市商汤科技有限公司 图像处理方法及装置、电子设备、计算机可读存储介质
CN113096132A (zh) * 2020-01-08 2021-07-09 东华医为科技有限公司 图像处理的方法、装置、存储介质和电子设备
CN115311145A (zh) * 2022-08-12 2022-11-08 中国电信股份有限公司 图像处理方法及装置、电子设备、存储介质

Also Published As

Publication number Publication date
CN115311145A (zh) 2022-11-08
WO2024032331A9 (zh) 2024-03-28
CN115311145B (zh) 2024-06-11

Similar Documents

Publication Publication Date Title
Dong et al. Multi-scale boosted dehazing network with dense feature fusion
TWI728465B (zh) 圖像處理方法和裝置、電子設備及儲存介質
CN110782490A (zh) 一种具有时空一致性的视频深度图估计方法及装置
US20100124383A1 (en) Systems and methods for resolution-invariant image representation
WO2023284401A1 (zh) 图像美颜处理方法、装置、存储介质与电子设备
EP4207051A1 (en) Image super-resolution method and electronic device
CN111626308B (zh) 一种基于轻量卷积神经网络的实时光流估计方法
WO2024032331A1 (zh) 图像处理方法及装置、电子设备、存储介质
CN111242999B (zh) 基于上采样及精确重匹配的视差估计优化方法
CN108876716B (zh) 超分辨率重建方法及装置
Guan et al. Srdgan: learning the noise prior for super resolution with dual generative adversarial networks
WO2020259123A1 (zh) 一种调整图像画质方法、装置及可读存储介质
CN111861888A (zh) 图像处理方法、装置、电子设备及存储介质
CN113902611A (zh) 图像美颜处理方法、装置、存储介质与电子设备
Liu et al. Arbitrary-scale super-resolution via deep learning: A comprehensive survey
CN117036581A (zh) 基于二维神经渲染的体渲染方法、系统、设备及介质
WO2023197805A1 (zh) 图像处理方法、装置、存储介质及电子设备
CN111696034A (zh) 图像处理方法、装置及电子设备
WO2023123873A1 (zh) 一种基于注意力机制稠密光流计算方法
Wang Interpolation and sharpening for image upsampling
CN115375539A (zh) 图像分辨率增强、多帧图像超分辨率系统和方法
WO2021008322A1 (zh) 图像处理方法、装置及设备
CN114565532A (zh) 视频美颜处理方法、装置、存储介质与电子设备
CN112365400A (zh) 一种快速光场角度超分辨重建方法
Feng et al. Applying holo360 video and image super-resolution generative adversarial networks to virtual reality immersion

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 23851554

Country of ref document: EP

Kind code of ref document: A1