WO2019101140A1 - 高分辨率图片生成方法、计算机设备及存储介质 - Google Patents

高分辨率图片生成方法、计算机设备及存储介质 Download PDF

Info

Publication number
WO2019101140A1
WO2019101140A1 PCT/CN2018/116987 CN2018116987W WO2019101140A1 WO 2019101140 A1 WO2019101140 A1 WO 2019101140A1 CN 2018116987 W CN2018116987 W CN 2018116987W WO 2019101140 A1 WO2019101140 A1 WO 2019101140A1
Authority
WO
WIPO (PCT)
Prior art keywords
picture
neural network
network model
deep neural
resolution
Prior art date
Application number
PCT/CN2018/116987
Other languages
English (en)
French (fr)
Inventor
戴宇荣
高立钊
付强
陈芳民
姚达
田恒锋
谢渝彬
周刘纪
王涛
吴永坚
黄俊洪
Original Assignee
腾讯科技（深圳）有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 腾讯科技（深圳）有限公司 filed Critical 腾讯科技（深圳）有限公司
Publication of WO2019101140A1 publication Critical patent/WO2019101140A1/zh

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T3/00—Geometric image transformations in the plane of the image
    • G06T3/40—Scaling of whole images or parts thereof, e.g. expanding or contracting
    • G06T3/4053—Scaling of whole images or parts thereof, e.g. expanding or contracting based on super-resolution, i.e. the output image resolution being higher than the sensor resolution
    • G06T3/4076—Scaling of whole images or parts thereof, e.g. expanding or contracting based on super-resolution, i.e. the output image resolution being higher than the sensor resolution using the original low-resolution images to iteratively correct the high-resolution images
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T3/00—Geometric image transformations in the plane of the image
    • G06T3/40—Scaling of whole images or parts thereof, e.g. expanding or contracting
    • G06T3/4046—Scaling of whole images or parts thereof, e.g. expanding or contracting using neural networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T3/00—Geometric image transformations in the plane of the image
    • G06T3/40—Scaling of whole images or parts thereof, e.g. expanding or contracting
    • G06T3/4053—Scaling of whole images or parts thereof, e.g. expanding or contracting based on super-resolution, i.e. the output image resolution being higher than the sensor resolution
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00—Image enhancement or restoration
    • G06T5/90—Dynamic range modification of images or parts thereof
    • G06T5/92—Dynamic range modification of images or parts thereof based on global image properties
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/048—Activation functions

Definitions

  • the present application relates to the field of picture processing, and in particular, to a high-resolution picture generation method, a computer device, and a storage medium.
  • multimedia information such as improving the stimulation of multimedia information to users, so high-resolution multimedia information (picture information or video information, etc.) has become the mainstream multimedia file.
  • the interactive terminal When the two parties need to perform high-resolution multimedia information interaction, the interactive terminal often needs a large number of storage media to store high-resolution multimedia, and the interactive terminal also needs high-speed broadband for high-resolution multimedia transmission operation, which greatly improves the interaction.
  • the cost of information exchange between the two terminals, and the requirements of the above storage medium and bandwidth also cause the information exchange efficiency of both parties to the interaction to be reduced.
  • SR image super-resolution
  • a high resolution picture generating method a computer device, and a storage medium are provided.
  • the embodiment of the present application provides a high-resolution image generating method, including:
  • the computer device acquires at least one deep neural network model, wherein the deep neural network model is generated according to a corresponding high resolution picture, a picture conversion algorithm, and a deep neural network framework;
  • the computer device acquires a low resolution picture, which is generated according to a corresponding high resolution picture and a picture conversion algorithm
  • the deep neural network model comprises a plurality of non-linearly transformed convolution layers that alternately use different parameter matrices as convolution template parameters.
  • a computer device comprising a memory and a processor, the memory storing computer readable instructions, the computer readable instructions being executed by the processor such that the processor performs the following steps:
  • the deep neural network model is generated according to a corresponding high resolution picture, a picture conversion algorithm, and a deep neural network framework;
  • the deep neural network model comprises a plurality of non-linearly transformed convolution layers that alternately use different parameter matrices as convolution template parameters.
  • One or more non-volatile storage media storing computer readable instructions, when executed by one or more processors, cause one or more processors to perform the following steps:
  • the deep neural network model is generated according to a corresponding high resolution picture, a picture conversion algorithm, and a deep neural network framework;
  • the deep neural network model comprises a plurality of non-linearly transformed convolution layers that alternately use different parameter matrices as convolution template parameters.
  • FIG. 1A is an internal structural diagram of a computer device of a high-resolution picture generating method of the present application
  • 1B is another internal structural diagram of a computer device of the high-resolution picture generating method of the present application.
  • FIG. 1 is a flow chart of an embodiment of a high resolution picture generating method of the present application
  • step S104 of an embodiment of a high-resolution image generating method of the present application
  • FIG. 3 is a flowchart of creating a deep neural network model in an embodiment of a high-resolution image generating method of the present application
  • FIG. 4 is a flow chart showing the creation of a deep neural network model in an embodiment of the high-resolution image generating method of the present application
  • FIG. 5 is a schematic structural diagram of an embodiment of a high-resolution image generating apparatus of the present application.
  • FIG. 6 is a schematic structural diagram of a picture conversion module according to an embodiment of a high-resolution picture generating apparatus of the present application.
  • FIG. 7 is a schematic structural diagram of a corresponding model generating device in an embodiment of a high-resolution image generating apparatus of the present application.
  • FIG. 8 is a schematic structural diagram of a sub-picture conversion module of a corresponding model generation device according to an embodiment of the high-resolution picture generating apparatus of the present application;
  • FIG. 9 is a schematic structural diagram of a model generation module of a corresponding model generation device in an embodiment of the high-resolution image generation device of the present application.
  • FIG. 10 is a schematic structural diagram of a low-resolution sub-picture feature point extracting unit of a model generating module of a corresponding model generating device in an embodiment of the high-resolution image generating apparatus of the present application;
  • 11A is a schematic diagram showing the working principle of a specific embodiment of a high-resolution picture generating method and a high-resolution picture generating apparatus according to the present application;
  • 11B is a timing chart showing the operation of a high-resolution picture generating method and a high-resolution picture generating apparatus according to an embodiment of the present application;
  • FIG. 12 is a schematic structural diagram of a deep neural network model of a specific embodiment of a high-resolution image generating method and a high-resolution image generating apparatus according to the present application;
  • FIG. 13 is a schematic diagram showing the working environment structure of an electronic device in which the high-resolution image generating apparatus of the present application is located.
  • the high-resolution picture generating method and the picture generating apparatus of the present application may be disposed in any electronic device for performing a high-resolution picture converting operation on the received low-resolution picture.
  • the electronic device includes, but is not limited to, a wearable device, a headset, a healthcare platform, a personal computer, a server computer, a handheld or laptop device, a mobile device (such as a mobile phone, a personal digital assistant (PDA), a media player) And so on), multiprocessor systems, consumer electronics, small computers, mainframe computers, distributed computing environments including any of the above systems or devices, and the like.
  • the electronic device is preferably a mobile terminal or a fixed terminal for information interaction.
  • the high-resolution image generating method and the image generating apparatus of the present application improve the conversion accuracy of the compressed low-resolution image to the high-resolution image by creating a deep neural network model with a nonlinear conversion convolution layer, due to the part
  • the interactive terminal can only store and transmit low-resolution pictures, thereby effectively reducing the interaction cost of the multimedia picture information of the interactive terminals, improving the interaction efficiency of the multimedia picture information of the interactive terminals, and solving the existing high-resolution pictures.
  • FIG. 1A shows an internal block diagram of a computer device in one embodiment.
  • the computer device may specifically be a terminal comprising a processor, a memory, a network interface, an input device and a display screen connected by a system bus.
  • the memory comprises a non-volatile storage medium and an internal memory.
  • the non-volatile storage medium of the computer device stores an operating system, and may also store a computer program that, when executed by the processor, causes the processor to implement a high-resolution image generating method.
  • the internal memory may also store a computer program that, when executed by the processor, causes the processor to perform a high resolution picture generation method.
  • the display screen of the computer device may be a liquid crystal display or an electronic ink display screen
  • the input device of the computer device may be a touch layer covered on the display screen, or a button, a trackball or a touchpad provided on the computer device casing, and It can be an external keyboard, trackpad or mouse.
  • FIG. 1B shows an internal block diagram of a computer device in one embodiment.
  • the computer device may in particular be a server comprising a processor, a memory and a network interface connected by a system bus.
  • the memory includes a nonvolatile storage medium and an internal memory.
  • the non-volatile storage medium of the computer device can store an operating system, a database, and computer readable instructions.
  • the computer readable instructions when executed, may cause the processor to perform a high resolution picture generation method for storing data, such as a stored depth neural network model.
  • the server's processor is used to provide computing and control capabilities that support the operation of the entire server.
  • the network interface of the server is used for communicating with an external terminal through a network connection, for example, the converted high resolution picture can be sent to the terminal or the like.
  • the structure shown in FIG. 1A or FIG. 1B is only a block diagram of a partial structure related to the solution of the present application, and does not constitute a limitation of a terminal or a server to which the solution of the present application is applied.
  • the specific server may include a comparison diagram. More or fewer components are shown, or some components are combined, or have different component arrangements. It will be understood by those skilled in the art that the structure shown in FIG. 1A or FIG.
  • 1B is only a block diagram of a partial structure related to the solution of the present application, and does not constitute a limitation on a server to which the solution of the present application is applied, and a specific server. More or fewer components than those shown in the figures may be included, or some components may be combined, or have different component arrangements.
  • FIG. 1 is a flowchart of an embodiment of a high resolution picture generating method according to the present application.
  • the high-resolution image generating method of the present embodiment can be implemented by using the above-mentioned terminal or server.
  • the high-resolution image generating method of this embodiment includes:
  • Step S101 Acquire at least one deep neural network model, wherein the deep neural network model is generated by the model generating device according to the corresponding high resolution image, the image conversion algorithm, and the deep neural network framework;
  • Step S102 acquiring a low-resolution picture, where the low-resolution picture is generated by the picture generating device according to the corresponding high-resolution picture and the picture conversion algorithm;
  • Step S103 determining a corresponding depth neural network model according to the low resolution picture
  • Step S104 converting the low resolution picture into a high resolution picture through the deep neural network model.
  • step S101 the high resolution picture generating device (electronic device) acquires at least one deep neural network model from the model generating device.
  • the model generation device herein may be a background server or user terminal for generating a deep neural network model.
  • the deep neural network model here is a machine learning model for quickly converting a corresponding low resolution picture into a high resolution picture.
  • the machine learning model generates a general algorithm for converting low-resolution images into high-resolution images by learning conversion data of a large number of low-resolution images and high-resolution images.
  • the deep neural network model can be based on corresponding high resolution pictures, image conversion algorithms, and deep neural network frameworks.
  • the high-resolution image can set the image type that the deep neural network can adapt, such as a close-up picture or a distant picture of a character;
  • the picture conversion algorithm refers to a conversion algorithm for converting a high-resolution picture into a low-resolution picture, such as a picture compression algorithm.
  • the deep neural network framework refers to the preset structure of the deep neural network model, such as the input convolutional layer, the output convolutional layer, etc., the parameter structure of the deep neural network framework and the corresponding deep neural network model Corresponding deep neural network model.
  • the high-resolution image generating device can simultaneously acquire multiple different depth neural network models for high-resolution images of different parameters. Perform the build operation.
  • the above-described deep neural network model can be pre-generated by the model generation device, thereby effectively improving the conversion efficiency of the high-resolution image generating device to convert the high-resolution image in real time.
  • the high-resolution image generating device may be disposed in the model generating device or may be disposed on another mobile or fixed device, and the subsequent high-resolution image generating device may acquire the deep neural network model from the model generating device.
  • step S102 the high resolution picture generating means acquires a low resolution picture from the picture generating device.
  • the picture generation device herein may be a background server or a user terminal for generating a low resolution picture.
  • the low resolution picture here is generated by the picture generating device according to the corresponding high resolution picture and picture conversion algorithm.
  • the picture generating device processes the high resolution picture that needs to be transmitted according to a picture conversion algorithm to generate a corresponding low resolution picture.
  • the high resolution picture here is the same as the high resolution picture of the deep neural network model generated in step S101, and the picture conversion algorithm here is also the same as the picture conversion algorithm for generating the depth neural network model in step S101.
  • the high-resolution image generating device can perform high-resolution image conversion on the low-resolution image acquired in this step according to the depth neural network acquired in step S101.
  • the low-resolution picture is generated by a picture generating device, and the high-resolution picture generating device may be disposed in the picture generating device to reduce the picture information storage amount of the picture generating device by the low-resolution picture.
  • the high resolution picture generating device may also be disposed on other mobile or fixed devices to reduce the amount of picture information transmission by the picture generating device to the device where the high resolution picture generating device is located by the low resolution picture.
  • step S103 the high resolution picture generating means determines the depth neural network model corresponding to the low resolution picture from the plurality of depth neural network models acquired in step S101 based on the low resolution picture acquired in step S102.
  • the deep neural network model may classify the pictures applicable to each deep neural network model using a high resolution picture or a picture conversion algorithm.
  • the corresponding low resolution picture may also be classified by using the corresponding high resolution picture or picture conversion algorithm. Therefore, if a high resolution picture corresponding to a low resolution picture and a high resolution picture corresponding to a deep neural network model are of the same type, the low resolution picture may be considered to correspond to the depth neural network model.
  • step S104 the high resolution picture generating means converts the low resolution picture acquired in step S102 into a corresponding high resolution picture by the depth neural network model acquired in step S103.
  • FIG. 2 is a flowchart of step S104 of an embodiment of the high-resolution image generating method of the present application.
  • the step S104 includes:
  • Step S201 the high-resolution image generating device performs a bicubic interpolation and enlargement operation on the low-resolution image to obtain a low-resolution image after the zoom-in operation; thus, the low-resolution image has the same number of image feature points as the high-resolution image. .
  • Step S202 The high-resolution image generating device acquires a picture feature point of the low-resolution picture based on the pixel brightness value of the picture pixel in the low-resolution picture after the enlargement operation.
  • the pixel brightness value of the picture pixel of the low-resolution sub-picture is only used as the picture feature point of the low-resolution sub-picture to simplify Deep neural network model.
  • Step S203 the high-resolution image generating device converts the picture feature points of the low-resolution picture into the picture feature points of the high-resolution picture through the deep neural network model.
  • the picture feature points of the high resolution picture here are also the pixel brightness values of the picture pixels in the high resolution picture.
  • Step S204 the high-resolution image generating device synthesizes the picture pixels of the high-resolution picture according to the picture feature points of the high-resolution picture acquired in step S203 and the blue-red density offset of the low-resolution picture, thereby acquiring the low resolution.
  • the high resolution image corresponding to the rate image.
  • Such a high-resolution image generating device can realize the conversion and restoration of high-resolution images only by searching the depth neural network model and transforming the resolution image using the deep neural network model, thereby greatly improving the high-resolution image generating device.
  • the conversion efficiency of high-resolution pictures and the information interaction efficiency of high-resolution image generating devices can realize the conversion and restoration of high-resolution images only by searching the depth neural network model and transforming the resolution image using the deep neural network model, thereby greatly improving the high-resolution image generating device.
  • the high-resolution image generation method of the present embodiment improves the accuracy of restoring the compressed low-resolution image to a high-resolution image by creating a deep neural network model with a nonlinear conversion convolution layer, thereby reducing the interaction.
  • the interaction cost of the multimedia picture information of the two terminals improves the interaction efficiency of the multimedia picture information of the interactive terminals.
  • FIG. 3 is a flowchart of creating a deep neural network model in an embodiment of a high resolution picture generating method of the present application.
  • the model generation device generates a depth neural network model according to a high resolution picture, a picture conversion algorithm, and a deep neural network framework. Specifically include:
  • Step S301 dividing the high resolution picture into a plurality of high resolution sub-pictures
  • Step S302 performing image conversion on the high-resolution sub-picture using a picture conversion algorithm to obtain a low-resolution sub-picture corresponding to the high-resolution sub-picture;
  • Step S303 the low-resolution sub-picture is used as an input sample of the deep neural network framework, and the high-resolution sub-picture is used as an output contrast sample of the deep neural network framework to generate a corresponding deep neural network model.
  • step S301 the model generation device performs a segmentation operation on the high-resolution image used for machine learning, such as dividing the average into 4 equal parts or 9 equal parts, etc., to acquire a plurality of high-resolution sub-pictures.
  • step S302 the model generation device performs image conversion on the high-resolution sub-picture using a preset picture conversion algorithm to obtain a low-resolution sub-picture corresponding to the high-resolution sub-picture. Specifically:
  • the model generating device performs a picture reduction operation on the high-resolution sub-picture acquired in step S301 according to the set scaling ratio, such as reducing each high-resolution sub-picture to a quarter of the original size.
  • the model generation device performs a picture compression operation on the high-resolution sub-picture after the picture reduction operation by using a set compression algorithm, such as reducing the quality of the picture from 100% to 85%.
  • step S303 the model generation device uses the low-resolution sub-picture acquired in step S302 as an input sample of the depth neural network framework, and the high-resolution sub-picture acquired in step S301 is used as an output contrast sample of the depth neural network framework to generate a corresponding depth.
  • Neural network model For details, please refer to FIG. 4 , which is a flowchart of creating a deep neural network model in an embodiment of the high-resolution image generating method of the present application.
  • the step S303 includes:
  • step S401 the model generation device extracts picture feature points of the low-resolution sub-picture.
  • the model generating device may first perform a bicubic interpolation and amplification operation on the low-resolution sub-picture to obtain a low-resolution sub-picture after the amplification operation. This ensures that the number of picture feature points of the low-resolution sub-picture is consistent with the number of picture feature points of the high-resolution sub-picture.
  • the model generation device then converts the low-resolution sub-picture after the enlargement operation from the RGB color space to the YCbCr color space, and acquires the pixel brightness value of the picture pixel of the low-resolution sub-picture in the YCbCr color space as the low-resolution sub-picture. Picture feature points.
  • the low-resolution sub-picture in the YCbCr color space defines a picture by the pixel luminance value (Y), the blue density offset value (Cb), and the red density offset value (Cr) of the picture pixel. Since the human eye is more sensitive to the pixel brightness value and has greater tolerance to color, the pixel brightness value of the picture pixel of the low-resolution sub-picture is only used as the picture feature point of the low-resolution sub-picture. The amount of calculation of the deep neural network model is reduced without affecting the result of the picture conversion.
  • the same model generating device can obtain the pixel brightness value of the picture pixel in the high resolution sub-picture in the YCbCr color space as the picture feature point of the high resolution sub-picture.
  • Step S402 the model generating device creates an input convolution layer of the deep neural network model according to the picture feature points of the low-resolution sub-picture acquired in step S401, the number of convolution kernels of the input convolution layer, and the feature map size of the input convolution layer. And the corresponding input activation layer.
  • the picture feature point of the low-resolution sub-picture is the input of the input convolution layer
  • the number of convolution kernels of the input convolution layer is used to represent the feature extraction mode for feature extraction from the picture feature points of the low-resolution sub-picture.
  • the number of feature maps entered into the convolutional layer is used to adjust the complexity of the deep neural network model. The larger the number of convolution kernels of the input convolutional layer, the larger the feature map size of the input convolutional layer, the higher the complexity of the deep neural network model, and the more accurate the image conversion result.
  • the number of convolution kernels input to the convolution layer is set larger, such as 14-18.
  • the feature extraction of the picture feature points may be performed using the red pixel brightness value, the blue pixel brightness value, and the green pixel brightness value ( That is, the number of convolution kernels input to the convolutional layer is 3).
  • the convolution template parameter of the input convolution layer is set to 5*5
  • the feature map size of the input convolution layer is 28*28, that is, the parameter matrix of 32*32 is sequentially traversed by using the parameter matrix of 5*5.
  • a feature map of 28*28 size is available.
  • the resulting input convolutional layer has an output of 28*28*3.
  • the input activation layer is used to nonlinearly process the output of the input convolutional layer to ensure that the output of the input convolutional layer is differentiable, thereby ensuring the normal operation of the parameter training of the subsequent deep neural network model.
  • Step S403 the output data of the input activation layer of the depth neural network model obtained by the model generation device according to step S402, the number of convolution kernels of the nonlinear conversion convolution layer, the feature map size of the nonlinear conversion convolution layer, and the nonlinear conversion volume
  • the layered convolution template parameters create multiple nonlinear transformation convolution layers of the deep neural network model and corresponding nonlinear transformation activation layers. If five nonlinear conversion convolution layers and corresponding nonlinear conversion activation layers are provided, a plurality of nonlinear conversion convolution layers and corresponding non-linear conversion activation layers are sequentially connected, such as the output and non-linearity of the nonlinear conversion convolutional layer A1.
  • the input of the linear conversion active layer B1 is connected, the output of the nonlinear conversion active layer B1 is connected to the input of the nonlinear conversion convolutional layer A2, and the output of the nonlinear conversion convolutional layer A2 is connected to the input of the nonlinear conversion activation layer B2... .
  • the output data of the input active layer of the deep neural network model here is the input of the first nonlinearly transformed convolutional layer.
  • the number of convolution kernels of the nonlinearly converted convolutional layer is used to represent the number of feature extraction modes for feature extraction from the output data of the input active layer, the feature map size of the nonlinearly converted convolutional layer, and the nonlinear conversion convolutional layer.
  • the convolution template parameters are used to adjust the complexity of the deep neural network model.
  • the number of convolution kernels of the nonlinear conversion convolutional layer is set smaller, such as set to 4-6. That is, the number of convolution kernels of the input convolutional layer is greater than the number of convolution kernels of the non-linear conversion convolutional layer.
  • the model generation device alternately uses the first parameter matrix and the second parameter matrix to set convolution template parameters of all the nonlinear conversion convolution layers.
  • the first parameter matrix is 1*1 and the German-day parameter matrix is 3*3.
  • the convolution template parameter of the nonlinear conversion convolutional layer A1 is 1*1
  • the convolution template parameter of the nonlinear conversion convolutional layer A2 is 3*3
  • the convolution template parameter of the nonlinear conversion convolutional layer A3 is 1*1. 1.
  • This can effectively improve the nonlinear characteristics of the deep neural network model and reduce the variation of the feature map size of the nonlinearly converted convolutional layer, thereby further reducing the computational complexity of the deep neural network model and ensuring the subsequent deep neural network model.
  • the convergence of the deep neural network model during parameter training is not prone to over-fitting.
  • non-linear transformation activation layer is needed to nonlinearly process the output of the corresponding nonlinear conversion convolutional layer to ensure that the output of the input convolutional layer is differentiable, thereby ensuring the parameter training of the subsequent deep neural network model. normal operation.
  • Step S404 the model generating device creates the depth according to the output data of the last nonlinear conversion active layer of the deep neural network model acquired in step S403, the number of convolution kernels of the pre-output convolution layer, and the feature map size of the pre-output convolution layer.
  • the output data of the last nonlinear conversion active layer of the deep neural network model is the input of the input convolutional layer
  • the number of convolution kernels of the pre-output convolutional layer is used to represent the output data of the active layer from the last nonlinear conversion.
  • the number of feature extraction modes on which feature extraction is performed, and the feature map size of the input convolution layer are used to adjust the complexity of the deep neural network model.
  • the number of convolution kernels of the pre-output convolutional layer should be equal to the number of convolution kernels of the input convolutional layer, such as 14-18.
  • the pre-output active layer is required to nonlinearly process the output of the corresponding pre-output convolutional layer to ensure that the output of the pre-output convolutional layer is differentiable, thereby ensuring normal parameter training of the subsequent deep neural network model. run.
  • Step S405 the model generating device creates an output volume of the deep neural network model according to the data of the pre-output active layer of the depth neural network model acquired in step S404, the number of convolution kernels of the output convolution layer, and the feature map size of the output convolution layer. Laminated.
  • the output data of the pre-output active layer of the deep neural network model is the input of the output convolutional layer
  • the number of convolution kernels of the output convolutional layer is used to represent the feature extraction for feature extraction from the output data of the pre-output active layer.
  • the number of modes, the feature map size of the pre-output active layer is used to adjust the complexity of the deep neural network model.
  • the feature map size of the output convolution layer should be the same as the feature map size of the input convolution layer, so that the same number of picture feature points can be output, and the output comparison sample (high resolution sub-picture Picture feature points) for comparison operations.
  • Step S406 the model generation device generates a corresponding depth neural network model based on the data of the output convolution layer of the depth neural network model acquired in step S405 and the picture feature points of the high resolution sub-picture.
  • the image feature points of the high-resolution sub-picture are used to compare the output data of the output convolution layer of the deep neural network model, and the parameters of the deep neural network model are corrected according to the comparison result, so that the output convolution of the final deep neural network model is performed.
  • the difference between the output data of the layer and the picture feature points of the high-resolution sub-picture is smaller than the set value, so that the corresponding deep neural network model can be determined.
  • the model generation device can perform parameters on the deep neural network model (such as a convolution template parameter w) by using a PReLU algorithm (Parametric Rectified Linear Unit) in advance. And the bias parameter b) initialization operation, which makes the deep neural network model converge faster.
  • parameters on the deep neural network model such as a convolution template parameter w
  • PReLU algorithm Parametric Rectified Linear Unit
  • the model generation device can also perform the iterative operation on the deep neural network model by using the Adam (Adaptive Moment Estimation) algorithm, so that the parameters in the deep neural network model can be quickly and accurately obtained.
  • Adam Adaptive Moment Estimation
  • the deep neural network model of the present embodiment effectively reduces the computational complexity of the deep neural network model by setting the nonlinear convolutional convolution layer with a small number of convolution kernels.
  • the number of convolution kernels of the pre-output convolutional layer is equal to the number of convolution kernels of the input convolutional layer
  • the feature map size of the output convolutional layer is the same as the feature map size of the input convolutional layer, further improving the depth neural network model. Operational accuracy.
  • a computer device is also provided, the internal structure of which may be as shown in FIG. 1A or FIG. 1B, the computer device includes a high-resolution image generating device, and the high-resolution image generating device includes each Modules, each of which may be implemented in whole or in part by software, hardware or a combination thereof.
  • FIG. 5 is a schematic structural diagram of an embodiment of a high-resolution image generating device of the present application.
  • the high-resolution image generating apparatus of the present embodiment can be implemented by using the high-resolution image generating method described above.
  • the high-resolution image generating apparatus 50 of the present embodiment includes a deep neural network model acquiring module 51 and a low-resolution image acquiring module 52.
  • the deep neural network model obtaining module 51 is configured to acquire at least one deep neural network model, wherein the deep neural network model is generated by other terminals according to a corresponding high resolution picture, a picture conversion algorithm, and a deep neural network framework; the low resolution picture obtaining module 52 For obtaining a low resolution picture, wherein the low resolution picture is generated by other terminals according to the corresponding high resolution picture and the picture conversion algorithm; the depth neural network model determining module 53 is configured to determine the corresponding depth neural network according to the low resolution picture.
  • the model; the picture conversion module 54 is configured to convert the low resolution picture into a high resolution picture through the deep neural network model.
  • FIG. 6 is a schematic structural diagram of a picture conversion module according to an embodiment of a high resolution picture generating apparatus of the present application.
  • the picture conversion module 54 includes a low resolution picture enlargement operation unit 61, a low resolution picture feature point acquisition unit 62, a picture feature point conversion unit 63, and a high resolution picture acquisition unit 64.
  • the low-resolution picture enlargement operation unit 61 is configured to perform a bicubic interpolation and enlargement operation on the low-resolution picture to obtain a low-resolution picture after the enlargement operation; and the low-resolution picture feature point acquisition unit 62 is configured to be based on the low after the amplification operation
  • the pixel brightness value of the picture pixel in the resolution picture is obtained, and the picture feature point of the low resolution picture is obtained;
  • the picture feature point conversion unit 63 is configured to convert the picture feature point of the low resolution picture into a high resolution picture by using the deep neural network model.
  • the picture feature point; the high-resolution picture obtaining unit 64 is configured to acquire a high-resolution picture according to the picture feature point of the high-resolution picture and the blue-red density offset of the low-resolution picture.
  • the deep neural network model acquiring module 51 first acquires at least one deep neural network model from the model generating device.
  • the model generation device herein may be a background server or a user terminal for generating a deep neural network model.
  • the deep neural network model here is a machine learning model for quickly converting a corresponding low resolution picture into a high resolution picture.
  • the machine learning model generates a general algorithm for converting low-resolution images into high-resolution images by learning conversion data of a large number of low-resolution images and high-resolution images.
  • the deep neural network model can be based on corresponding high resolution pictures, image conversion algorithms, and deep neural network frameworks.
  • the high-resolution image can set the image type that the deep neural network can adapt, such as a close-up picture or a distant picture of a character;
  • the picture conversion algorithm refers to a conversion algorithm for converting a high-resolution picture into a low-resolution picture, such as a picture compression algorithm.
  • the deep neural network framework refers to the preset structure of the deep neural network model, such as the input convolutional layer, the output convolutional layer, etc., the parameter structure of the deep neural network framework and the corresponding deep neural network model Corresponding deep neural network model.
  • the high-resolution image generating device can simultaneously acquire multiple different depth neural network models for high-resolution images of different parameters. Perform the build operation.
  • the above-described deep neural network model can be pre-generated by the model generation device, thereby effectively improving the conversion efficiency of the high-resolution image generating device to convert the high-resolution image in real time.
  • the high-resolution image generating device may be disposed in the model generating device or may be disposed on another mobile or fixed device, and the subsequent high-resolution image generating device may acquire the deep neural network model from the model generating device.
  • the low resolution picture acquisition module 52 then retrieves the low resolution picture from the picture generation device.
  • the picture generation device herein may be a background server or a user terminal for generating a low resolution picture.
  • the low resolution picture here is generated by the picture generating device according to the corresponding high resolution picture and picture conversion algorithm.
  • the picture generating device processes the high resolution picture that needs to be transmitted according to a picture conversion algorithm to generate a corresponding low resolution picture.
  • the high-resolution image here is the same as the high-resolution image that generates the deep neural network model.
  • the image conversion algorithm here is also the same as the image conversion algorithm that generates the deep neural network model.
  • Such a high-resolution image generating device can perform high-resolution image conversion on low-resolution images according to the deep neural network acquired by the deep neural network model acquisition module.
  • the low-resolution picture is generated by a picture generating device, and the high-resolution picture generating device may be disposed in the picture generating device to reduce the picture information storage amount of the picture generating device by the low-resolution picture.
  • the high resolution picture generating device may also be disposed on other mobile or fixed devices to reduce the amount of picture information transmission by the picture generating device to the device where the high resolution picture generating device is located by the low resolution picture.
  • the depth neural network model determining module 53 determines the deep neural network corresponding to the low resolution picture from the plurality of deep neural network models acquired by the deep neural network model acquisition module according to the low resolution picture acquired by the low resolution picture acquisition module 52. model.
  • the deep neural network model may use a high-resolution picture or picture conversion algorithm to classify the pictures applicable to each deep neural network model when generating multiple deep neural network models.
  • the corresponding low resolution picture may also be classified by using the corresponding high resolution picture or picture conversion algorithm. Therefore, if a high resolution picture corresponding to a low resolution picture and a high resolution picture corresponding to a deep neural network model are of the same type, the low resolution picture may be considered to correspond to the depth neural network model.
  • the final picture conversion module 54 converts the low resolution picture acquired by the low resolution picture acquisition module 52 into the corresponding high resolution picture through the depth neural network model acquired by the depth neural network model determination module 53.
  • the specific process includes:
  • the low-resolution picture enlargement operation unit 61 of the picture conversion module 54 performs a bicubic interpolation and enlargement operation on the low-resolution picture to obtain a low-resolution picture after the enlargement operation; thus, the low-resolution picture has the same number as the high-resolution picture. Picture feature points.
  • the low-resolution picture feature point acquisition unit 62 of the picture conversion module 54 acquires the picture feature points of the low-resolution picture based on the pixel brightness values of the picture pixels in the low-resolution picture after the enlargement operation.
  • the pixel brightness value of the picture pixel of the low-resolution sub-picture is only used as the picture feature point of the low-resolution sub-picture to simplify Deep neural network model.
  • the picture feature point conversion unit 63 of the picture conversion module 54 converts picture feature points of the low resolution picture into picture feature points of the high resolution picture through the depth neural network model.
  • the picture feature points of the high resolution picture here are also the pixel brightness values of the picture pixels in the high resolution picture.
  • the high-resolution image acquisition unit 64 of the picture conversion module 54 synthesizes a picture of the high-resolution picture according to the picture feature point of the high-resolution picture acquired by the picture feature point conversion unit 63 and the blue-red density offset of the low-resolution picture. a pixel to obtain a high resolution picture corresponding to the low resolution picture.
  • the high-resolution image generating device 50 can realize the conversion and restoration of the high-resolution image only by searching the depth neural network model and converting the resolution image using the deep neural network model, thereby greatly improving the high-resolution image generating device.
  • the high-resolution image conversion efficiency and the information interaction efficiency of the high-resolution image generating device can realize the conversion and restoration of the high-resolution image only by searching the depth neural network model and converting the resolution image using the deep neural network model, thereby greatly improving the high-resolution image generating device.
  • the high-resolution picture generating apparatus of the present embodiment improves the accuracy of restoring the compressed low-resolution picture to a high-resolution picture by creating a deep neural network model having a nonlinear conversion convolution layer, thereby reducing the interaction.
  • the interaction cost of the multimedia picture information of the two terminals improves the interaction efficiency of the multimedia picture information of the interactive terminals.
  • FIG. 7 is a schematic structural diagram of a corresponding model generating device in an embodiment of the high-resolution image generating apparatus of the present application.
  • the model generation device 70 includes a picture segmentation module 71, a sub-picture conversion module 72, and a model generation module 73.
  • the picture segmentation module 71 is configured to divide the high-resolution picture into a plurality of high-resolution sub-pictures; the sub-picture conversion module 72 is configured to perform picture conversion on the high-resolution sub-picture using the picture conversion algorithm to obtain a high-resolution sub-picture corresponding The low-resolution sub-picture is used; the model generation module 73 is configured to use the low-resolution sub-picture as the input sample of the deep neural network framework, and the high-resolution sub-picture as the output contrast sample of the deep neural network framework to generate the corresponding deep neural network model. .
  • FIG. 8 is a schematic structural diagram of a sub-picture conversion module of a corresponding model generation device according to an embodiment of the high-resolution image generating apparatus of the present application.
  • the sub-picture conversion module 72 includes a picture reduction operation unit 81 and a picture compression unit 82.
  • the picture reduction operation unit 81 is configured to perform a picture reduction operation on the high-resolution sub-picture according to the set scaling ratio; the picture compression unit 82 is configured to perform a picture compression operation on the high-resolution sub-picture after the picture reduction operation using the set compression algorithm. To obtain a low-resolution sub-picture corresponding to the high-resolution sub-picture.
  • FIG. 9 is a schematic structural diagram of a model generating module of a corresponding model generating device according to an embodiment of the high-resolution image generating apparatus of the present application.
  • the model generation module 73 includes a high-resolution sub-picture feature point extraction unit 91, a feature map size setting unit 92, a convolution kernel number setting unit 93, a convolution template parameter setting unit 94, a model parameter initialization unit 95, and a model parameter operation unit.
  • 96 A low-resolution sub-picture feature point extracting unit 97, an input convolutional layer and an active layer creating unit 98, a nonlinear conversion convolutional layer and an active layer creating unit 99, a pre-output convolutional layer and an active layer creating unit 9A, and an output The convolution layer creation unit 9B and the model generation unit 9C.
  • the high-resolution sub-picture feature point extracting unit 91 is configured to acquire a picture feature point of the high-resolution sub-picture based on the pixel brightness value of the picture pixel in the high-resolution sub-picture.
  • the feature size setting unit 92 is for setting the feature map size of the input convolution layer and the feature map size of the output convolution layer.
  • the convolution kernel number setting unit 93 is for setting the number of convolution kernels of the input convolutional layer, the number of convolution kernels of the pre-output convolutional layer, and the number of convolution kernels of the non-linear conversion convolutional layer.
  • the convolution template parameter setting unit 94 is for alternately using the parameter matrices of 1*1 and 3*3 to set the convolution template parameters of all the non-linearly converted convolutional layers.
  • the model parameter initializing unit 95 is configured to perform a parameter initializing operation on the deep neural network model using the PReLU algorithm.
  • the model parameter operation unit 96 is configured to perform an iterative operation on the deep neural network model using the Adam algorithm to acquire parameters in the deep neural network model.
  • the low-resolution sub-picture feature point extracting unit 97 is configured to extract picture feature points of the low-resolution sub-picture.
  • the input convolutional layer and activation layer creation unit 98 is configured to create an input volume of the deep neural network model according to the picture feature points of the low-resolution sub-picture, the number of convolution kernels of the input convolutional layer, and the feature map size of the input convolutional layer.
  • the nonlinear conversion convolutional layer and activation layer creation unit 99 is configured to input the data of the active layer according to the depth neural network model, the number of convolution kernels of the nonlinear conversion convolution layer, the feature map size of the nonlinear conversion convolution layer, and the non- Linear convolution convolutional layer convolution template parameters, creating multiple nonlinear transformation convolutional layers of the deep neural network model and corresponding nonlinear transformation activation layers.
  • the pre-output convolutional layer and activation layer creating unit 9A is configured to use data of a plurality of nonlinear conversion activation layers of the deep neural network model, a number of convolution kernels of the pre-output convolutional layer, and a feature map size of the pre-output convolution layer, Create a pre-output convolutional layer of the deep neural network model and a pre-output activation layer.
  • the output convolutional layer creation unit 9B is configured to create an output convolution of the deep neural network model according to the data of the pre-output active layer of the deep neural network model, the number of convolution kernels of the output convolutional layer, and the feature map size of the output convolutional layer.
  • the model generation unit 9C is configured to generate a corresponding depth neural network model based on the data of the output convolution layer of the depth neural network model and the picture feature points of the high resolution sub-picture.
  • FIG. 10 is a schematic structural diagram of a low-resolution sub-picture feature point extracting unit of a model generating module of a corresponding model generating device according to an embodiment of the high-resolution image generating apparatus of the present application.
  • the low-resolution sub-picture feature point extracting unit 97 includes a sub-picture interpolation enlarging operation sub-unit 101 and a low-resolution sub-picture feature point extracting sub-unit 102.
  • the sub-picture interpolation and amplification operation sub-unit 101 is configured to perform a bicubic interpolation and amplification operation on the low-resolution sub-picture to obtain a low-resolution sub-picture after the enlargement operation; and the low-resolution sub-picture feature point extraction sub-unit 102 is configured to be based on the amplification The pixel brightness value of the picture pixel in the low resolution sub-picture after the operation, and the picture feature point of the low-resolution sub-picture is obtained.
  • the process of generating the corresponding deep neural network model by the model generation device 70 of this embodiment includes:
  • the picture segmentation module 71 performs a segmentation operation on the high-resolution image used for machine learning, such as dividing the average into 4 equal parts or 9 equal parts to obtain a plurality of high-resolution sub-pictures.
  • the sub-picture conversion module 72 then performs picture conversion on the high-resolution sub-picture using a preset picture conversion algorithm to obtain a low-resolution sub-picture corresponding to the high-resolution sub-picture. Specifically:
  • the picture reduction operation unit 81 of the sub-picture conversion module 72 performs a picture reduction operation on the high-resolution sub-picture acquired by the picture segmentation module according to the set scaling ratio, such as reducing each high-resolution sub-picture to a quarter of the original size.
  • the picture compression unit 82 of the sub-picture conversion module 72 performs a picture compression operation on the high-resolution sub-picture after the picture reduction operation using a set compression algorithm, such as reducing the quality of the picture from 100% to 85%. In this way, a low-resolution sub-picture corresponding to the high-resolution sub-picture can be obtained.
  • the model generation module 73 uses the low-resolution sub-picture acquired by the sub-picture conversion module 72 as an input sample of the depth neural network framework, and the high-resolution sub-picture obtained by the picture segmentation module 71 serves as an output contrast sample of the deep neural network framework to generate a corresponding Deep neural network model.
  • the process of creating a specific deep neural network model includes:
  • the low-resolution sub-picture feature point extraction unit 97 of the model generation module 73 extracts picture feature points of the low-resolution sub-picture. Specifically, the sub-picture interpolation and amplification operation sub-unit 101 of the low-resolution sub-picture feature point extraction unit 97 performs a bicubic interpolation and amplification operation on the low-resolution sub-picture to obtain a low-resolution sub-picture after the enlargement operation. This ensures that the number of picture feature points of the low-resolution sub-picture is consistent with the number of picture feature points of the high-resolution sub-picture.
  • the low-resolution sub-picture feature point extraction sub-unit 102 of the low-resolution sub-picture feature point extracting unit 97 converts the low-resolution sub-picture after the enlargement operation from the RGB color space to the YCbCr color space, and acquires the low resolution in the YCbCr color space. Rate the pixel brightness value of the picture pixel of the sub-picture as the picture feature point of the low-resolution sub-picture.
  • the low-resolution sub-picture in the YCbCr color space defines a picture by the pixel luminance value (Y), the blue density offset value (Cb), and the red density offset value (Cr) of the picture pixel. Since the human eye is more sensitive to the pixel brightness value and has greater tolerance to color, the pixel brightness value of the picture pixel of the low-resolution sub-picture is only used as the picture feature point of the low-resolution sub-picture. The amount of calculation of the deep neural network model is reduced without affecting the result of the picture conversion.
  • the same high-resolution sub-picture feature point extracting unit 91 can acquire the pixel brightness value of the picture pixel in the high-resolution sub-picture in the YCbCr color space as the picture feature point of the high-resolution sub-picture.
  • the input convolution layer and the activation layer creation unit 98 of the model generation module 73 obtains the picture feature points of the low-resolution sub-picture acquired by the low-resolution sub-picture feature point extraction unit 97, the number of convolution kernels of the input convolution layer, and Enter the feature map size of the convolutional layer, create the input convolutional layer of the deep neural network model, and the corresponding input activation layer.
  • the picture feature point of the low-resolution sub-picture is the input of the input convolution layer
  • the number of convolution kernels of the input convolution layer is used to represent the feature extraction mode for feature extraction from the picture feature points of the low-resolution sub-picture.
  • the number of feature maps entered into the convolutional layer is used to adjust the complexity of the deep neural network model. The larger the number of convolution kernels of the input convolutional layer, the larger the feature map size of the input convolutional layer, the higher the complexity of the deep neural network model, and the more accurate the image conversion result.
  • the convolution kernel number setting unit 93 can set more feature extraction modes, so the number of convolution kernels input to the convolution layer is set larger, such as 14-18.
  • the feature extraction of the picture feature points may be performed using the red pixel brightness value, the blue pixel brightness value, and the green pixel brightness value ( That is, the number of convolution kernels input to the convolutional layer is 3).
  • the convolution template parameter of the input convolution layer is set to 5*5
  • the feature map size of the input convolution layer is 28*28, that is, the parameter matrix of 32*32 is sequentially traversed by using the parameter matrix of 5*5.
  • a feature map of 28*28 size is available.
  • the resulting input convolutional layer has an output of 28*28*3.
  • the input activation layer is used to nonlinearly process the output of the input convolutional layer to ensure that the output of the input convolutional layer is differentiable, thereby ensuring the normal operation of the parameter training of the subsequent deep neural network model.
  • the nonlinear conversion convolutional layer of the model generation module 73 and the activation layer creation unit 99 output data of the input activation layer of the deep neural network model acquired by the input convolution layer and the activation layer creation unit 98, and the nonlinear conversion convolution layer
  • the number of convolution kernels, the feature map size of the nonlinearly transformed convolutional layer, and the convolution template parameters of the nonlinearly transformed convolutional layer create multiple nonlinear transformation convolutional layers of the deep neural network model and corresponding nonlinear transformation activation Floor.
  • nonlinear conversion convolution layers and corresponding nonlinear conversion activation layers are provided, a plurality of nonlinear conversion convolution layers and corresponding non-linear conversion activation layers are sequentially connected, such as the output and non-linearity of the nonlinear conversion convolutional layer A1.
  • the input of the linear conversion active layer B1 is connected, the output of the nonlinear conversion active layer B1 is connected to the input of the nonlinear conversion convolutional layer A2, and the output of the nonlinear conversion convolutional layer A2 is connected to the input of the nonlinear conversion activation layer B2... .
  • the output data of the input active layer of the deep neural network model here is the input of the first nonlinearly transformed convolutional layer.
  • the number of convolution kernels of the nonlinearly converted convolutional layer is used to represent the number of feature extraction modes for feature extraction from the output data of the input active layer, the feature map size of the nonlinearly converted convolutional layer, and the nonlinear conversion convolutional layer.
  • the convolution template parameters are used to adjust the complexity of the deep neural network model.
  • the convolution kernel number setting unit 93 can set fewer feature extraction modes, so the number of convolution kernels of the non-linear conversion convolution layer is set smaller, as set to 4- 6 and so on. That is, the number of convolution kernels of the input convolutional layer is greater than the number of convolution kernels of the non-linear conversion convolutional layer.
  • the convolution template parameter setting unit 94 sets the first parameter matrix and the second parameter matrix, and sets convolution template parameters of all the nonlinear conversion convolution layers.
  • the first parameter matrix is 1*1
  • the German-day parameter matrix is 3*3.
  • the convolution template parameter of the nonlinear conversion convolutional layer A1 is 1*1
  • the convolution template parameter of the nonlinear conversion convolutional layer A2 is 3*3
  • the convolution template parameter of the nonlinear conversion convolutional layer A3 is 1*1. 1.
  • the convergence of the deep neural network model during parameter training is not prone to over-fitting.
  • non-linear transformation activation layer is needed to nonlinearly process the output of the corresponding nonlinear conversion convolutional layer to ensure that the output of the input convolutional layer is differentiable, thereby ensuring the parameter training of the subsequent deep neural network model. normal operation.
  • the pre-output convolutional layer of the model generation module 73 and the activation layer creation unit 9A output data of the last nonlinear conversion activation layer of the deep neural network model acquired by the nonlinear conversion convolutional layer and the activation layer creation unit 99, The number of convolution kernels of the output convolutional layer and the feature map size of the pre-output convolutional layer, the pre-output convolutional layer of the deep neural network model and the pre-output activation layer;
  • the output data of the last nonlinear conversion active layer of the deep neural network model is the input of the input convolutional layer
  • the number of convolution kernels of the pre-output convolutional layer is used to represent the output data of the active layer from the last nonlinear conversion.
  • the number of feature extraction modes on which feature extraction is performed, and the feature map size of the input convolution layer are used to adjust the complexity of the deep neural network model.
  • the number of convolution kernels of the pre-output convolution layer set by the convolution kernel number setting unit 93 should be equal to the number of convolution kernels of the input convolution layer, such as 14-18, etc. .
  • the same feature extraction mode can be used to generate the high-resolution sub-picture.
  • the pre-output active layer is required to nonlinearly process the output of the corresponding pre-output convolutional layer to ensure that the output of the pre-output convolutional layer is differentiable, thereby ensuring normal parameter training of the subsequent deep neural network model. run.
  • the output convolutional layer creation unit 9B of the model generation module 73 calculates the data of the pre-output activation layer of the deep neural network model acquired by the pre-output convolution layer and the activation layer creation unit 9A, the number of convolution kernels of the output convolution layer, and Output the feature map size of the convolutional layer to create the output convolutional layer of the deep neural network model.
  • the output data of the pre-output active layer of the deep neural network model is the input of the output convolutional layer
  • the number of convolution kernels of the output convolutional layer is used to represent the feature extraction for feature extraction from the output data of the pre-output active layer.
  • the number of modes, the feature map size of the pre-output active layer is used to adjust the complexity of the deep neural network model.
  • the feature map size of the output convolution layer set by the feature picture size setting unit 92 should be the same as the feature picture size of the input convolution layer, so that the same number of picture feature points can be output, compared with the output.
  • the sample (picture feature points of the high-resolution sub-picture) is compared.
  • the model generation unit 9C of the model generation module 73 generates a corresponding depth neural network model based on the data of the output convolution layer of the depth neural network model acquired by the output convolution layer creation unit 9B and the picture feature points of the high resolution sub-picture. .
  • the model generation unit 9C compares the output data of the output convolution layer of the deep neural network model using the picture feature points of the high-resolution sub-picture, and corrects the parameters of the deep neural network model according to the comparison result, so that the final deep neural network model
  • the difference between the output data of the output convolutional layer and the picture feature points of the high-resolution sub-picture is less than the set value, so that the corresponding deep neural network model can be determined.
  • the model parameter initializing unit 95 may perform parameters (such as a convolution template) on the deep neural network model by using a PReLU algorithm (Parametric Rectified Linear Unit) in advance.
  • the parameter w and the bias parameter b) are initialized so that the convergence of the deep neural network model is faster.
  • the model parameter operation unit 96 can also perform an iterative operation on the deep neural network model by using the Adam (Adaptive Moment Estimation) algorithm, so that the parameters in the deep neural network model can be quickly and accurately obtained.
  • Adam Adaptive Moment Estimation
  • the deep neural network model of the present embodiment effectively reduces the computational complexity of the deep neural network model by setting the nonlinear convolutional convolution layer with a small number of convolution kernels.
  • the number of convolution kernels of the pre-output convolutional layer is equal to the number of convolution kernels of the input convolutional layer
  • the feature map size of the output convolutional layer is the same as the feature map size of the input convolutional layer, further improving the depth neural network model. Operational accuracy.
  • FIG. 11A is a schematic diagram showing the working principle of a high-resolution image generating method and a high-resolution image generating device according to the present application
  • FIG. 11B is a high-resolution image generating method of the present application.
  • the model generation device is a background server C, and the background server C is connected to the background server D, the user terminal c, and the user terminal d, respectively.
  • the background server C is used to generate a deep neural network model
  • the background server D is used to provide high-resolution pictures for training
  • the user terminal c and the user terminal d are two user terminals for information interaction.
  • the high resolution image generation step in this embodiment includes:
  • step S1101 the background server C receives a plurality of high resolution pictures for training from the background server D.
  • Step S1102 The background server C performs segmentation and image conversion on the received high resolution picture for training by using a preset image conversion algorithm, and generates a plurality of high resolution sub-pictures and corresponding plurality of low-resolution sub-pictures.
  • the background server C uses the low-resolution sub-picture as an input sample of the deep neural network framework, and the high-resolution sub-picture serves as an output contrast sample of the deep neural network framework to generate a corresponding deep neural network model.
  • the pixel luminance value of the picture pixel in the low-resolution sub-picture is used as the picture feature point of the low-resolution sub-picture
  • the pixel brightness value of the picture pixel in the high-resolution sub-picture is used as the picture feature point of the high-resolution sub-picture.
  • the deep neural network model is shown in Figure 12, which includes an input convolutional layer (A0) and corresponding input activation layer (B0), and five nonlinear conversion convolution layers (A1, A2, A3, A4, A5). And a corresponding non-linear conversion active layer (B1, B2, B3, B4, B5), a pre-output convolution layer (A6) and a pre-output active layer (B6) and an output convolution layer (A7).
  • the number of convolution kernels of the input convolutional layer and the pre-output convolutional layer is 16, the number of convolution kernels of the non-linear conversion convolutional layer is 5, and the number of convolution kernels of the output convolutional layer is 1.
  • the feature map size of the input convolutional layer is the same as the feature map size of the output convolutional layer.
  • the convolution template parameters of the five non-linear conversion convolutional layers and the pre-output convolutional layer are 1*1, 3*3, 1*1, 3*3, 1*1, 3*3, respectively.
  • the feature map size of the input convolutional layer is the same as the feature map size of the output convolutional layer.
  • step S1104 the background server C uses the PReLU algorithm to perform parameter initialization operations on the deep neural network model; then the Adam algorithm is used to perform an iterative operation on the deep neural network model to obtain parameters in the deep neural network model (such as convolution template parameters w, etc.) .
  • step S1105 the background server C sends the deep neural network model obtained by the training to the user terminal c and the user terminal d.
  • Step S1106 When the user terminal c needs to send a certain high resolution picture to the user terminal d, the user terminal c may convert the high resolution picture into a corresponding low resolution picture by using a corresponding picture conversion algorithm, and then the low resolution is performed. The rate picture is sent to the user terminal d.
  • step S1107 when the user terminal d receives the low resolution picture, the corresponding deep neural network model is found locally.
  • Step S1108 The user terminal d converts the low resolution picture into a corresponding high resolution picture by using the deep neural network model.
  • the user terminal c does not need to use a large amount of storage space to store high-resolution pictures, and does not need to use a high bandwidth to transmit high-resolution pictures; at the same time, the user terminal d does not need to complicate the low-resolution pictures.
  • the image conversion operation only needs to follow the preset depth neural network model to achieve accurate conversion of low resolution images to high resolution images.
  • the whole deep neural network model is simple in structure, has high convergence and is not easy to be overcomposed. Therefore, the whole deep neural network model has less computational complexity, shorter training time and higher accuracy.
  • the high-resolution image generating method, generating device and storage medium of the present application improve the accuracy of converting a low-resolution image into a high-resolution image by creating a deep neural network model with a nonlinear conversion convolution layer, thereby reducing the accuracy
  • the interaction cost of multimedia picture information on both sides of the interactive terminal improves the interaction efficiency of the multimedia picture information of the interactive terminals; effectively solves the existing high-resolution picture generation method and the high-level image generation device has high information interaction cost And the technical problem of low information interaction efficiency.
  • ком ⁇ онент can be, but is not limited to being, a process running on a processor, a processor, an object, an executable application, a thread of execution, a program, and/or a computer.
  • a component can be, but is not limited to being, a process running on a processor, a processor, an object, an executable application, a thread of execution, a program, and/or a computer.
  • an application running on a controller and the controller can be a component.
  • One or more components can reside within a process and/or thread of execution, and a component can be located on a computer and/or distributed between two or more computers.
  • Example electronic device 1312 includes, but is not limited to, a wearable device, a headset, a healthcare platform, a personal computer, a server computer, a handheld or laptop device, a mobile device (such as a mobile phone, a personal digital assistant (PDA), media playback) And so on), multiprocessor systems, consumer electronics, small computers, mainframe computers, distributed computing environments including any of the above systems or devices, and the like.
  • Computer readable instructions may be distributed via computer readable media (discussed below).
  • Computer readable instructions may be implemented as program modules, such as functions, objects, application programming interfaces (APIs), data structures, etc. that perform particular tasks or implement particular abstract data types.
  • program modules such as functions, objects, application programming interfaces (APIs), data structures, etc. that perform particular tasks or implement particular abstract data types.
  • APIs application programming interfaces
  • data structures such as lists, etc. that perform particular tasks or implement particular abstract data types.
  • the functionality of the computer readable instructions can be combined or distributed at will in various environments.
  • FIG. 13 illustrates an example of an electronic device 1312 that includes one or more of the high resolution picture generation devices of the present application.
  • electronic device 1312 includes at least one processing unit 1316 and memory 1318.
  • memory 1318 can be volatile (such as RAM), non-volatile (such as ROM, flash memory, etc.) or some combination of the two. This configuration is illustrated in Figure 13 by dashed line 1314.
  • electronic device 1312 may include additional features and/or functionality.
  • device 1312 may also include additional storage devices (eg, removable and/or non-removable) including, but not limited to, magnetic storage devices, optical storage devices, and the like.
  • additional storage is illustrated by storage device 1320 in FIG.
  • computer readable instructions for implementing one or more embodiments provided herein may be in storage device 1320.
  • Storage device 1320 can also store other computer readable instructions for implementing an operating system, applications, and the like.
  • Computer readable instructions may be loaded into memory 1318 for execution by, for example, processing unit 1316.
  • Computer storage media includes volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storage of information such as computer readable instructions or other data.
  • Memory 1318 and storage device 1320 are examples of computer storage media.
  • Computer storage media includes, but is not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital versatile disk (DVD) or other optical storage device, magnetic tape cassette, magnetic tape, magnetic disk storage device or other magnetic storage device, Or any other medium that can be used to store desired information and that can be accessed by electronic device 1312. Any such computer storage media may be part of the electronic device 1312.
  • the electronic device 1312 may also include a communication connection 1326 that allows the electronic device 1312 to communicate with other devices.
  • Communication connection 1326 may include, but is not limited to, a modem, a network interface card (NIC), an integrated network interface, a radio frequency transmitter/receiver, an infrared port, a USB connection, or other interface for connecting electronic device 1312 to other electronic devices.
  • Communication connection 1326 can include a wired connection or a wireless connection.
  • Communication connection 1326 can transmit and/or receive communication media.
  • Computer readable medium can include a communication medium.
  • Communication media typically embodies computer readable instructions or other data in "modulated data signals" such as carrier waves or other transport mechanisms, and includes any information delivery media.
  • modulated data signal can include a signal that one or more of the signal characteristics are set or changed in such a manner as to encode the information into the signal.
  • the electronic device 1312 can include an input device 1324, such as a keyboard, mouse, pen, voice input device, touch input device, infrared camera, video input device, and/or any other input device.
  • Output device 1322 such as one or more displays, speakers, printers, and/or any other output device, may also be included in device 1312.
  • Input device 1324 and output device 1322 can be coupled to electronic device 1312 via a wired connection, a wireless connection, or any combination thereof.
  • an input device or output device from another electronic device can be used as the input device 1324 or output device 1322 of the electronic device 1312.
  • the components of electronic device 1312 can be connected by various interconnects, such as a bus.
  • interconnects may include Peripheral Component Interconnect (PCI) (such as Fast PCI), Universal Serial Bus (USB), Firewire (IEEE 1394), optical bus architecture, and the like.
  • PCI Peripheral Component Interconnect
  • USB Universal Serial Bus
  • Firewire IEEE 1394
  • optical bus architecture and the like.
  • the components of electronic device 1312 can be interconnected by a network.
  • memory 1318 can be comprised of multiple physical memory units that are interconnected by a network located in different physical locations.
  • storage devices for storing computer readable instructions may be distributed across a network.
  • electronic device 1330 accessible via network 1328 can store computer readable instructions for implementing one or more embodiments provided herein.
  • the electronic device 1312 can access the electronic device 1330 and download a portion or all of the computer readable instructions for execution.
  • electronic device 1312 can download a plurality of computer readable instructions as needed, or some of the instructions can be executed at electronic device 1312 and some of the instructions can be executed at electronic device 1330.
  • the one or more operations may constitute computer readable instructions stored on one or more computer readable media that, when executed by an electronic device, cause the computing device to perform the operations.
  • the order in which some or all of the operations are described should not be construed as implying that the operations must be sequential. Those skilled in the art will appreciate alternative rankings that have the benefit of this specification. Moreover, it should be understood that not all operations must be present in every embodiment provided herein.
  • Each functional unit in the embodiment of the present application may be integrated into one processing module, or each unit may exist physically separately, or two or more units may be integrated into one module.
  • the above integrated modules can be implemented in the form of hardware or in the form of software functional modules.
  • the integrated modules, if implemented in the form of software functional modules and sold or used as stand-alone products, may also be stored in a computer readable storage medium.
  • the above mentioned storage medium may be a read only memory, a magnetic disk or an optical disk or the like.
  • the various steps in the various embodiments of the present application are not necessarily performed in the order indicated by the steps. Except as explicitly stated herein, the execution of these steps is not strictly limited, and the steps may be performed in other orders. Moreover, at least some of the steps in the embodiments may include multiple sub-steps or multiple stages, which are not necessarily performed at the same time, but may be executed at different times, and the execution of these sub-steps or stages The order is also not necessarily sequential, but may be performed alternately or alternately with other steps or at least a portion of the sub-steps or stages of the other steps.
  • Non-volatile memory can include read only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory.
  • Volatile memory can include random access memory (RAM) or external cache memory.
  • RAM is available in a variety of formats, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronization chain.
  • SRAM static RAM
  • DRAM dynamic RAM
  • SDRAM synchronous DRAM
  • DDRSDRAM double data rate SDRAM
  • ESDRAM enhanced SDRAM
  • Synchlink DRAM SLDRAM
  • Memory Bus Radbus
  • RDRAM Direct RAM
  • DRAM Direct Memory Bus Dynamic RAM
  • RDRAM Memory Bus Dynamic RAM

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Evolutionary Computation (AREA)
  • Artificial Intelligence (AREA)
  • General Health & Medical Sciences (AREA)
  • General Engineering & Computer Science (AREA)
  • Data Mining & Analysis (AREA)
  • Biophysics (AREA)
  • Biomedical Technology (AREA)
  • Molecular Biology (AREA)
  • Computing Systems (AREA)
  • Computational Linguistics (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Mathematical Physics (AREA)
  • Software Systems (AREA)
  • Health & Medical Sciences (AREA)
  • Image Processing (AREA)
  • Editing Of Facsimile Originals (AREA)

Abstract

本申请提供一种高分辨率图片生成方法，其包括：获取至少一个深度神经网络模型；获取低分辨率图片；根据低分辨率图片，确定对应的深度神经网络模型；通过深度神经网络模型，将低分辨率图片转换为高分辨率图片；其中深度神经网络模型包括交替使用不同参数矩阵作为卷积模板参数的多个非线性转换卷积层。

Description

高分辨率图片生成方法、计算机设备及存储介质
本申请要求于2017年11月24日提交中国专利局，申请号为2017111911011，申请名称为“高分辨率图片生成方法、生成装置及存储介质”的中国专利申请的优先权，其全部内容通过引用结合在本申请中。
技术领域
本申请涉及图片处理领域，特别是涉及一种高分辨率图片生成方法、计算机设备及存储介质。
背景技术
随着科技的发展，人们对多媒体信息的要求越来越高，如提升多媒体信息对用户感观的刺激，因此高分辨率多媒体信息(图片信息或视频信息等)成为了主流的多媒体文件。
当交互双方需要进行高分辨率多媒体信息交互时，交互终端往往需要大量的存储介质对高分辨率多媒体进行存储操作，同时交互终端也往往需要高速宽带进行高分辨多媒体传输操作，这样会大大提高交互终端双方的信息交互成本，同时上述存储介质以及带宽的要求也造成了交互终端双方的信息交互效率的降低。
因此急需一种可以将压缩后的低分辨率(Low Resolution,LR)图像转换为原来的高分辨率(High Resolution,HR)图像的图像超分辨率技术(Super Resolution,SR)。
发明内容
根据本申请提供的各种实施例，提供一种高分辨率图片生成方法、计算机设备及存储介质。
本申请实施例提供一种高分辨率图片生成方法，其包括：
计算机设备获取至少一个深度神经网络模型，其中所述深度神经网络模型 是根据对应的高分辨率图片、图片转换算法以及深度神经网络框架生成的；
计算机设备获取低分辨率图片，所述低分辨率图片是根据对应的高分辨率图片以及图片转换算法生成的；
计算机设备根据所述低分辨率图片，确定对应的深度神经网络模型；
计算机设备通过所述深度神经网络模型，将所述低分辨率图片转换为高分辨率图片；
其中所述深度神经网络模型包括交替使用不同参数矩阵作为卷积模板参数的多个非线性转换卷积层。
一种计算机设备，包括存储器和处理器，所述存储器中存储有计算机可读指令，所述计算机可读指令被所述处理器执行时，使得所述处理器执行如下步骤：
获取至少一个深度神经网络模型，其中所述深度神经网络模型是根据对应的高分辨率图片、图片转换算法以及深度神经网络框架生成的；
获取低分辨率图片，所述低分辨率图片是根据对应的高分辨率图片以及图片转换算法生成的；
根据所述低分辨率图片，确定对应的深度神经网络模型；
通过所述深度神经网络模型，将所述低分辨率图片转换为高分辨率图片；
其中所述深度神经网络模型包括交替使用不同参数矩阵作为卷积模板参数的多个非线性转换卷积层。
一个或多个存储有计算机可读指令的非易失性存储介质，所述计算机可读指令被一个或多个处理器执行时，使得一个或多个处理器执行如下步骤：
获取至少一个深度神经网络模型，其中所述深度神经网络模型是根据对应的高分辨率图片、图片转换算法以及深度神经网络框架生成的；
获取低分辨率图片，所述低分辨率图片是根据对应的高分辨率图片以及图片转换算法生成的；
根据所述低分辨率图片，确定对应的深度神经网络模型；
通过所述深度神经网络模型，将所述低分辨率图片转换为高分辨率图片；
其中所述深度神经网络模型包括交替使用不同参数矩阵作为卷积模板参数的多个非线性转换卷积层。
附图说明
为了更清楚地说明本申请实施例中的技术方案，下面将对实施例描述中所需要使用的附图作简单地介绍，显而易见地，下面描述中的附图仅仅是本申请的一些实施例，对于本领域普通技术人员来讲，在不付出创造性劳动的前提下，还可以根据这些附图获得其他的附图。
图1A为本申请的高分辨率图片生成方法的计算机设备的内部结构图；
图1B为本申请的高分辨率图片生成方法的计算机设备的另一内部结构图；
图1为本申请的高分辨率图片生成方法的一实施例的流程图；
图2为本申请的高分辨率图片生成方法的一实施例的步骤S104的流程图；
图3为本申请的高分辨率图片生成方法的一实施例中的深度神经网络模型的创建流程图；
图4为本申请的高分辨率图片生成方法的一实施例中的深度神经网络模型的创建细化流程图；
图5为本申请的高分辨率图片生成装置的一实施例的结构示意图；
图6为本申请的高分辨率图片生成装置的一实施例的图片转换模块的结构示意图；
图7为本申请的高分辨率图片生成装置的一实施例中对应的模型生成设备的结构示意图；
图8为本申请的高分辨率图片生成装置的一实施例中对应的模型生成设备的子图片转换模块的结构示意图；
图9为本申请的高分辨率图片生成装置的一实施例中对应的模型生成设备的模型生成模块的结构示意图；
图10为本申请的高分辨率图片生成装置的一实施例中对应的模型生成设备的模型生成模块的低分辨率子图片特征点提取单元的结构示意图；
图11A为本申请的高分辨率图片生成方法及高分辨率图片生成装置的具体 实施例的工作原理示意图；
图11B为本申请的高分辨率图片生成方法及高分辨率图片生成装置的具体实施例的工作时序图；
图12为本申请的高分辨率图片生成方法及高分辨率图片生成装置的具体实施例的深度神经网络模型的结构示意图；
图13为本申请的高分辨率图片生成装置所在的电子设备的工作环境结构示意图。
具体实施方式
为了使本申请的目的、技术方案及优点更加清楚明白，以下结合附图及实施例，对本申请进行进一步详细说明。应当理解，此处所描述的具体实施例仅仅用以解释本申请，并不用于限定本申请。
在以下的说明中，本申请的具体实施例将参考由一部或多部计算机所执行之作业的步骤及符号来说明，除非另有述明。因此，其将可了解到这些步骤及操作，其中有数次提到为由计算机执行，包括了由代表了以一结构化型式中的数据之电子信号的计算机处理单元所操纵。此操纵转换该数据或将其维持在该计算机之内存系统中的位置处，其可重新配置或另外以本领域技术人员所熟知的方式来改变该计算机之运作。该数据所维持的数据结构为该内存之实体位置，其具有由该数据格式所定义的特定特性。但是，本申请原理以上述文字来说明，其并不代表为一种限制，本领域技术人员将可了解到以下所述的多种步骤及操作亦可实施在硬件当中。
本申请的高分辨率图片生成方法及图片生成装置可设置在任何的电子设备中，用于对接收到的低分辨率图片进行高分辨率图片转换操作。该电子设备包括但不限于可穿戴设备、头戴设备、医疗健康平台、个人计算机、服务器计算机、手持式或膝上型设备、移动设备(比如移动电话、个人数字助理(PDA)、媒体播放器等等)、多处理器系统、消费型电子设备、小型计算机、大型计算机、包括上述任意系统或设备的分布式计算环境，等等。该电子设备优选为用于进行信息交互的移动终端或固定终端。
本申请的高分辨率图片生成方法及图片生成装置通过创建具有非线性转换 卷积层的深度神经网络模型，提高了压缩后的低分辨率图片还原为高分辨率图片的转换准确度，由于部分交互终端可只存储以及传输低分辨率图片，因此可有效的降低交互终端双方的多媒体图片信息的交互成本，提高了交互终端双方的多媒体图片信息的交互效率；解决了现有的高分辨率图片生成方法以及高分辨率图片生成装置的信息交互成本较高且信息交互效率较低的技术问题。
在一个实施例中，如图1A所示，图1A示出了一个实施例中计算机设备的内部结构图。该计算机设备具体可以是终端，该终端包括通过系统总线连接的处理器、存储器、网络接口、输入装置和显示屏。其中，存储器包括非易失性存储介质和内存储器。该计算机设备的非易失性存储介质存储有操作系统，还可存储有计算机程序，该计算机程序被处理器执行时，可使得处理器实现高分辨率图片生成方法。该内存储器中也可储存有计算机程序，该计算机程序被处理器执行时，可使得处理器执行高分辨率图片生成方法。计算机设备的显示屏可以是液晶显示屏或者电子墨水显示屏，计算机设备的输入装置可以是显示屏上覆盖的触摸层，也可以是计算机设备外壳上设置的按键、轨迹球或触控板，还可以是外接的键盘、触控板或鼠标等。
在一个实施例中，如图1B所示，图1B示出了一个实施例中计算机设备的内部结构图。该计算机设备具体可以是服务器，该服务器包括通过系统总线连接的处理器、存储器和网络接口。存储器包括非易失性存储介质和内存储器。该计算机设备的非易失性存储介质可存储操作系统、数据库和计算机可读指令。该计算机可读指令被执行时，可使得处理器执行一种高分辨率图片生成方法，数据库用于存储数据，如存储深度神经网络模型。该服务器的处理器用于提供计算和控制能力，支撑整个服务器的运行。该服务器的网络接口用于与外部的终端通过网络连接通信，比如可将转换后的高分辨率图片发送至终端等。图1A或图1B中示出的结构，仅仅是与本申请方案相关的部分结构的框图，并不构成对本申请方案所应用于其上的终端或服务器的限定，具体的服务器可以包括比图中所示更多或更少的部件，或者组合某些部件，或者具有不同的部件布置。本领域技术人员可以理解，图1A或图1B中示出的结构，仅仅是与本申请方案相关的部分结构的框图，并不构成对本申请方案所应用于其上的服务器的限定，具体的服务器可以包括比图中所示更多或更少的部件，或者组合某些部件，或 者具有不同的部件布置。
请参照图1，图1为本申请的高分辨率图片生成方法的一实施例的流程图。本实施例的高分辨率图片生成方法可使用上述的终端或者服务器进行实施，本实施例的高分辨率图片生成方法包括：
步骤S101，获取至少一个深度神经网络模型，其中深度神经网络模型由模型生成设备根据对应的高分辨率图片、图片转换算法以及深度神经网络框架生成；
步骤S102，获取低分辨率图片，其中低分辨率图片由图片生成设备根据对应的高分辨率图片以及图片转换算法生成；
步骤S103，根据低分辨率图片，确定对应的深度神经网络模型；
步骤S104，通过深度神经网络模型，将低分辨率图片转换为高分辨率图片。
下面详细说明本实施例的高分辨率图片生成方法的高分辨率图片的生成过程。
在步骤S101中，高分辨率图片生成装置(电子设备)从模型生成设备上获取至少一个深度神经网络模型。
这里的模型生成设备可为用于生成深度神经网络模型的后台服务器或用户终端。
这里的深度神经网络模型是用来快速将对应的低分辨率图片转换为高分辨率图片的机器学习模型。该机器学习模型通过对大量低分辨率图片和高分辨率图片的转换数据进行学习，生成低分辨率图片转换为高分辨率图片的通用算法。
该深度神经网络模型可根据对应的高分辨率图片、图片转换算法以及深度神经网络框架。其中高分辨率图片可设定该深度神经网络可适应的图片类型，如人物近景图片或远景图片等；图片转换算法是指高分辨率图片转换为低分辨率图片的转换算法，如图片压缩算法以及图片分割算法等；深度神经网络框架是指该深度神经网络模型的预设结构，如输入卷积层、输出卷积层等结构，该深度神经网络框架和对应的深度神经网络模型的参数构成对应的深度神经网络模型。
由于深度神经网络模型与高分辨率图片、图片转换算法以及深度神经网络 框架均相关，因此高分辨率图片生成装置可同时获取多个不同的深度神经网络模型，以便对不同参数的高分辨率图片进行生成操作。
上述深度神经网络模型可由模型生成设备预先生成，从而可有效的提高高分辨率图片生成装置实时转换高分辨率图片的转换效率。高分辨率图片生成装置可设置在该模型生成设备中，也可设置在其他的移动或固定设备上，后续高分辨率图片生成装置可从模型生成设备上获取深度神经网络模型。
在步骤S102中，高分辨率图片生成装置从图片生成设备上获取低分辨率图片。
这里的图片生成设备可为用于生成低分辨率图片的后台服务器或用户终端。
这里的低分辨率图片由图片生成设备根据对应的高分辨率图片以及图片转换算法生成。为了降低图片生成设备的图片信息储存量以及图片信息传输量，图片生成设备会将需要传输的高分辨率图片按图片转换算法进行处理，以生成相应的低分辨率图片。
这里的高分辨率图片与步骤S101中生成深度神经网络模型的高分辨图片相同，这里的图片转换算法也与步骤S101中生成深度神经网络模型的图片转换算法相同。这样高分辨率图片生成装置可根据步骤S101获取的深度神经网络对本步骤获取的低分辨率图片进行图片高分辨率转换。
上述低分辨率图片由图片生成设备生成，该高分辨率图片生成装置可设置在该图片生成设备中，以通过该低分辨率图片降低图片生成设备的图片信息储存量。该高分辨率图片生成装置也可设置在其他的移动或固定设备上，以通过该低分辨率图片降低图片生成设备向高分辨率图片生成装置所在设备的图片信息传输量。
在步骤S103中，高分辨率图片生成装置根据步骤S102获取的低分辨率图片，从步骤S101中获取的多个深度神经网络模型中确定与低分辨率图片对应的深度神经网络模型。
具体的，深度神经网络模型在生成多个深度神经网络模型时，可使用高分 辨率图片或图片转换算法对每个深度神经网络模型适用的图片进行分类。同时图片生成设备在生成低分辨率图片时，可也使用对应的高分辨率图片或图片转换算法对相应的低分辨率图片进行分类。因此如某个低分辨率图片对应的高分辨率图片和某个深度神经网络模型对应的高分辨率图片的类型相同，则可认为该低分辨率图片与该深度神经网络模型相对应。
在步骤S104中，高分辨率图片生成装置通过步骤S103获取的深度神经网络模型，将步骤S102获取的低分辨率图片转换为对应的高分辨率图片。具体请参照图2，图2为本申请的高分辨率图片生成方法的一实施例的步骤S104的流程图。该步骤S104包括：
步骤S201，高分辨率图片生成装置对低分辨率图片进行双立方插值放大操作，以得到放大操作后的低分辨率图片；这样使得低分辨率图片具有与高分辨率图片相同数量的图片特征点。
步骤S202，高分辨率图片生成装置基于放大操作后的低分辨率图片中图片像素的像素亮度值，获取低分辨率图片的图片特征点。
由于人眼对像素亮度值敏感度较大，对色彩的容错性较大，因此这里仅以低分辨率子图片的图片像素的像素亮度值，作为低分辨率子图片的图片特征点，以简化深度神经网络模型。
步骤S203，高分辨率图片生成装置通过深度神经网络模型，将低分辨率图片的图片特征点转换为高分辨率图片的图片特征点。这里的高分辨率图片的图片特征点也是高分辨率图片中图片像素的像素亮度值。
步骤S204，高分辨率图片生成装置根据步骤S203获取的高分辨率图片的图片特征点以及低分辨率图片的蓝色红色浓度偏移量，合成高分辨率图片的图片像素，从而获取该低分辨率图片对应的高分辨率图片。
这样高分辨率图片生成装置可仅仅通过查找深度神经网络模型以及使用深度神经网络模型对分辨率图片进行转换两个步骤实现了高分辨率图片的转换还原，大大提高了高分辨率图片生成装置的高分辨率图片的转换效率以及高分辨率图片生成装置的信息交互效率。
这样即完成了实施例的高分辨率图片生成方法的高分辨率图片的生成过程。
本实施例的高分辨率图片生成方法通过创建具有非线性转换卷积层的深度神经网络模型，提高了将压缩后的低分辨率图片转换还原为高分辨率图片的准确性，从而降低了交互终端双方的多媒体图片信息的交互成本，提高了交互终端双方的多媒体图片信息的交互效率。
图3为本申请的高分辨率图片生成方法的一实施例中的深度神经网络模型的创建流程图。本实施例的高分辨率图片生成方法中模型生成设备根据高分辨率图片、图片转换算法以及深度神经网络框架生成深度神经网络模型。具体包括：
步骤S301，将高分辨率图片分割为多个高分辨率子图片；
步骤S302，使用图片转换算法对高分辨率子图片进行图片转换，以得到高分辨率子图片对应的低分辨率子图片；
步骤S303，以低分辨率子图片作为深度神经网络框架的输入样本，高分辨率子图片作为深度神经网络框架的输出对比样本，生成对应的深度神经网络模型。
下面详细说明上述深度神经网络模型生成的各步骤的具体流程。
在步骤S301中，模型生成设备对用来进行机器学习的高分辨率图片进行分割操作，如平均分割为4等分或9等分等，以获取多个高分辨率子图片。
在步骤S302中，模型生成设备使用预设的图片转换算法对高分辨率子图片进行图片转换，以得到高分辨率子图片对应的低分辨率子图片。具体可为：
模型生成设备按设定缩放比例，对步骤S301获取的高分辨率子图片进行图片缩小操作，如将每张高分辨率子图片缩小为原来尺寸的四分之一等。
随后模型生成设备使用设定压缩算法对图片缩小操作后的高分辨率子图片进行图片压缩操作，如将图片的品质由100％降低到85％等。
这样即可得到高分辨率子图片对应的低分辨率子图片。
在步骤S303中，模型生成设备以步骤S302获取的低分辨率子图片作为深度神经网络框架的输入样本，步骤S301获取的高分辨率子图片作为深度神经网络框架的输出对比样本，生成对应的深度神经网络模型。具体请参照图4，图4为本申请的高分辨率图片生成方法的一实施例中的深度神经网络模型的创建细化流程图。该步骤S303包括：
步骤S401，模型生成设备提取低分辨率子图片的图片特征点。具体的，模型生成设备可首先对低分辨率子图片进行双立方插值放大操作，以得到放大操作后的低分辨率子图片。这样可保证低分辨率子图片的图片特征点的数量与高分辨率子图片的图片特征点的数量一致。
随后模型生成设备将放大操作后的低分辨率子图片从RGB彩色空间转换到YCbCr彩色空间，并获取YCbCr彩色空间中低分辨率子图片的图片像素的像素亮度值，作为低分辨率子图片的图片特征点。
其中YCbCr彩色空间中的低分辨率子图片通过图片像素的像素亮度值(Y)、蓝色浓度偏移值(Cb)以及红色浓度偏移值(Cr)来定义图片。由于人眼对像素亮度值敏感度较大，对色彩的容错性较大，因此这里仅以低分辨率子图片的图片像素的像素亮度值，作为低分辨率子图片的图片特征点，以在不影响图片转换结果的情况下减小深度神经网络模型的计算量。
同样模型生成设备可获取YCbCr彩色空间中高分辨率子图片中图片像素的像素亮度值，作为高分辨率子图片的图片特征点。
步骤S402，模型生成设备根据步骤S401获取的低分辨率子图片的图片特征点、输入卷积层的卷积核数量以及输入卷积层的特征图尺寸，创建深度神经网络模型的输入卷积层以及对应的输入激活层。
这里的低分辨率子图片的图片特征点为该输入卷积层的输入，输入卷积层的卷积核数量用于表示从低分辨率子图片的图片特征点上进行特征提取的特征提取模式的数量，输入卷积层的特征图尺寸用于调整深度神经网络模型的复杂度。其中输入卷积层的卷积核数量越大，输入卷积层的特征图尺寸越大，深度神经网络模型的复杂度越高，图片转换结果也就越准确。
这里为了提高图片转换结果的准确性，可设置较多的特征提取模式，因此输入卷积层的卷积核数量设置的较大，如设置为14-18等。
如输入的低分辨率子图片的图片特征点为32*32的特征点矩阵，则可使用红色的像素亮度值、蓝色的像素亮度值以及绿色的像素亮度值进行图片特征点的特征提取(即输入卷积层的卷积核数量为3)。如设定该输入卷积层的卷积模板参数为5*5，则该输入卷积层的特征图尺寸为28*28，即使用5*5的参数矩阵依次遍历32*32的特征点矩阵可得到28*28尺寸的特征图。因此得到的输入卷积层的输出为28*28*3。
此外这里还需要使用输入激活层对输入卷积层的输出进行非线性处理，以保证输入卷积层的输出是可微的，进而保证后续深度神经网络模型的参数训练的正常运行。
步骤S403，模型生成设备根据步骤S402获取的深度神经网络模型的输入激活层的输出数据、非线性转换卷积层的卷积核数量、非线性转换卷积层的特征图尺寸以及非线性转换卷积层的卷积模板参数，创建深度神经网络模型的多个非线性转换卷积层以及对应的非线性转换激活层。如设置5个非线性转换卷积层以及对应的非线性转换激活层，多个非线性转换卷积层以及对应的非线性转换激活层依次连接，如非线性转换卷积层A1的输出与非线性转换激活层B1的输入连接，非线性转换激活层B1的输出与非线性转换卷积层A2的输入连接，非线性转换卷积层A2的输出与非线性转换激活层B2的输入连接……。
这里的深度神经网络模型的输入激活层的输出数据为第一个非线性转换卷积层的输入。非线性转换卷积层的卷积核数量用于表示从输入激活层的输出数据上进行特征提取的特征提取模式的数量，非线性转换卷积层的特征图尺寸和非线性转换卷积层的卷积模板参数用于调整深度神经网络模型的复杂度。
这里为了减小深度神经网络模型的运算量，可设置较少的特征提取模式，因此非线性转换卷积层的卷积核数量设置的较小，如设置为4-6等。即输入卷积层的卷积核数量大于非线性转换卷积层的卷积核数量。
具体的，模型生成设备交替使用第一参数矩阵和第二参数矩阵，设置所有 非线性转换卷积层的卷积模板参数。这里优选第一参数矩阵为1*1，德日参数矩阵为3*3。如非线性转换卷积层A1的卷积模板参数为1*1，非线性转换卷积层A2的卷积模板参数为3*3，非线性转换卷积层A3的卷积模板参数为1*1……。这样可以有效的提高深度神经网络模型的非线性特性，减小非线性转换卷积层的特征图尺寸的变化量，从而进一步减小深度神经网络模型的运算量，保证了后续深度神经网络模型的参数训练时的深度神经网络模型的收敛性，也不容易发生过拟合的现象。
此外这里还需要使用非线性转换激活层对相应的非线性转换卷积层的输出进行非线性处理，以保证输入卷积层的输出是可微的，进而保证后续深度神经网络模型的参数训练的正常运行。
步骤S404，模型生成设备根据步骤S403获取的深度神经网络模型的最后一个非线性转换激活层的输出数据、预输出卷积层的卷积核数量以及预输出卷积层的特征图尺寸，创建深度神经网络模型的预输出卷积层以及预输出激活层；
这里的深度神经网络模型的最后一个非线性转换激活层的输出数据为该输入卷积层的输入，预输出卷积层的卷积核数量用于表示从最后一个非线性转换激活层的输出数据上进行特征提取的特征提取模式的数量，输入卷积层的特征图尺寸用于调整深度神经网络模型的复杂度。
这里为了提高图片转换结果的准确性，预输出卷积层的卷积核数量应该与输入卷积层的卷积核数量相等，如均设置为14-18等。这样经过非线性转换激活层的非线性转换后，可使用同样的特征提取模式来生成高分辨率子图片。
此外这里还需要使用预输出激活层对相应的预输出卷积层的输出进行非线性处理，以保证预输出卷积层的输出是可微的，进而保证后续深度神经网络模型的参数训练的正常运行。
步骤S405，模型生成设备根据步骤S404获取的深度神经网络模型的预输出激活层的数据、输出卷积层的卷积核数量以及输出卷积层的特征图尺寸，创建深度神经网络模型的输出卷积层。
这里的深度神经网络模型的预输出激活层的输出数据为该输出卷积层的输 入，输出卷积层的卷积核数量用于表示从预输出激活层的输出数据上进行特征提取的特征提取模式的数量，预输出激活层的特征图尺寸用于调整深度神经网络模型的复杂度。
为了保证图片转换结果的准确性，输出卷积层的特征图尺寸应与输入卷积层的特征图尺寸相同，这样可以输出相同数量的图片特征点，与输出对比样本(高分辨率子图片的图片特征点)进行对比操作。
步骤S406，模型生成设备基于步骤S405获取的深度神经网络模型的输出卷积层的数据以及高分辨率子图片的图片特征点，生成对应的深度神经网络模型。
使用高分辨率子图片的图片特征点对深度神经网络模型的输出卷积层的输出数据进行比较，并根据比较结果对深度神经网络模型的参数进行修正，使得最后深度神经网络模型的输出卷积层的输出数据与高分辨率子图片的图片特征点的差异小于设定值，这样即可确定对应的深度神经网络模型。
具体的，这里为了加快深度神经网络模型的机器训练速度，这里模型生成设备可以预先使用PReLU算法(Parametric Rectified Linear Unit，参数化修正线性单元)对深度神经网络模型进行参数(如卷积模板参数w以及偏置参数b)初始化操作，这样可使得深度神经网络模型的收敛速度更快。
同时模型生成设备还可使用Adam(Adaptive Moment Estimation，适应性矩估计)算法对深度神经网络模型进行迭代运算，从而可快速准确的获取该深度神经网络模型中的参数。
这样即完成了本实施例的深度神经网络模型的生成过程。
本实施例的深度神经网络模型通过多个卷积核数量较小的非线性转换卷积层的设置，有效的减少了深度神经网络模型的运算量。同时预输出卷积层的卷积核数量与输入卷积层的卷积核数量相等，输出卷积层的特征图尺寸与输入卷积层的特征图尺寸相同，进一步提高了深度神经网络模型的运算准确性。
在一个实施例中，还提供了一种计算机设备，该计算机设备的内部结构可如图1A或图1B所示，该计算机设备包括高分辨率图片生成装置，高分辨率图片 生成装置中包括各个模块，每个模块可全部或部分通过软件、硬件或其组合来实现。
本申请还提供一种高分辨率图片生成装置，请参照图5，图5为本申请的高分辨率图片生成装置的一实施例的结构示意图。本实施例的高分辨率图片生成装置可使用上述的高分辨率图片生成方法进行实施，本实施例的高分辨率图片生成装置50包括深度神经网络模型获取模块51、低分辨率图片获取模块52、深度神经网络模型确定模块53以及图片转换模块54。
深度神经网络模型获取模块51用于获取至少一个深度神经网络模型，其中深度神经网络模型由其他终端根据对应的高分辨率图片、图片转换算法以及深度神经网络框架生成；低分辨率图片获取模块52用于获取低分辨率图片，其中低分辨率图片由其他终端根据对应的高分辨率图片以及图片转换算法生成；深度神经网络模型确定模块53用于根据低分辨率图片，确定对应的深度神经网络模型；图片转换模块54用于通过深度神经网络模型，将低分辨率图片转换为高分辨率图片。
请参照图6，图6为本申请的高分辨率图片生成装置的一实施例的图片转换模块的结构示意图。该图片转换模块54包括低分辨率图片放大操作单元61、低分辨率图片特征点获取单元62、图片特征点转换单元63以及高分辨率图片获取单元64。
低分辨率图片放大操作单元61用于对低分辨率图片进行双立方插值放大操作，以得到放大操作后的低分辨率图片；低分辨率图片特征点获取单元62用于基于放大操作后的低分辨率图片中图片像素的像素亮度值，获取低分辨率图片的图片特征点；图片特征点转换单元63用于通过深度神经网络模型，将低分辨率图片的图片特征点转换为高分辨率图片的图片特征点；高分辨率图片获取单元64用于根据高分辨率图片的图片特征点以及低分辨率图片的蓝色红色浓度偏移量，获取高分辨率图片。
本实施例的高分辨率图片生成装置50使用时，首先深度神经网络模型获取模块51从模型生成设备上获取至少一个深度神经网络模型。
这里的模型生成设备可为用于生成深度神经网络模型的后台服务器或用户 终端。
这里的深度神经网络模型是用来快速将对应的低分辨率图片转换为高分辨率图片的机器学习模型。该机器学习模型通过对大量低分辨率图片和高分辨率图片的转换数据进行学习，生成低分辨率图片转换为高分辨率图片的通用算法。
该深度神经网络模型可根据对应的高分辨率图片、图片转换算法以及深度神经网络框架。其中高分辨率图片可设定该深度神经网络可适应的图片类型，如人物近景图片或远景图片等；图片转换算法是指高分辨率图片转换为低分辨率图片的转换算法，如图片压缩算法以及图片分割算法等；深度神经网络框架是指该深度神经网络模型的预设结构，如输入卷积层、输出卷积层等结构，该深度神经网络框架和对应的深度神经网络模型的参数构成对应的深度神经网络模型。
由于深度神经网络模型与高分辨率图片、图片转换算法以及深度神经网络框架均相关，因此高分辨率图片生成装置可同时获取多个不同的深度神经网络模型，以便对不同参数的高分辨率图片进行生成操作。
上述深度神经网络模型可由模型生成设备预先生成，从而可有效的提高高分辨率图片生成装置实时转换高分辨率图片的转换效率。高分辨率图片生成装置可设置在该模型生成设备中，也可设置在其他的移动或固定设备上，后续高分辨率图片生成装置可从模型生成设备上获取深度神经网络模型。
随后低分辨率图片获取模块52从图片生成设备上获取低分辨率图片。
这里的图片生成设备可为用于生成低分辨率图片的后台服务器或用户终端。
这里的低分辨率图片由图片生成设备根据对应的高分辨率图片以及图片转换算法生成。为了降低图片生成设备的图片信息储存量以及图片信息传输量，图片生成设备会将需要传输的高分辨率图片按图片转换算法进行处理，以生成相应的低分辨率图片。
这里的高分辨率图片与生成深度神经网络模型的高分辨图片相同，这里的图片转换算法也与生成深度神经网络模型的图片转换算法相同。这样高分辨率 图片生成装置可根据深度神经网络模型获取模块获取的深度神经网络对低分辨率图片进行图片高分辨率转换。
上述低分辨率图片由图片生成设备生成，该高分辨率图片生成装置可设置在该图片生成设备中，以通过该低分辨率图片降低图片生成设备的图片信息储存量。该高分辨率图片生成装置也可设置在其他的移动或固定设备上，以通过该低分辨率图片降低图片生成设备向高分辨率图片生成装置所在设备的图片信息传输量。
然后深度神经网络模型确定模块53根据低分辨率图片获取模块52获取的低分辨率图片，从深度神经网络模型获取模块获取的多个深度神经网络模型中确定与低分辨率图片对应的深度神经网络模型。
具体的，深度神经网络模型在生成多个深度神经网络模型时，可使用高分辨率图片或图片转换算法对每个深度神经网络模型适用的图片进行分类。同时图片生成设备在生成低分辨率图片时，可也使用对应的高分辨率图片或图片转换算法对相应的低分辨率图片进行分类。因此如某个低分辨率图片对应的高分辨率图片和某个深度神经网络模型对应的高分辨率图片的类型相同，则可认为该低分辨率图片与该深度神经网络模型相对应。
最后图片转换模块54通过深度神经网络模型确定模块53获取的深度神经网络模型，将低分辨率图片获取模块52获取的低分辨率图片转换为对应的高分辨率图片。具体流程包括：
图片转换模块54的低分辨率图片放大操作单元61对低分辨率图片进行双立方插值放大操作，以得到放大操作后的低分辨率图片；这样使得低分辨率图片具有与高分辨率图片相同数量的图片特征点。
图片转换模块54的低分辨率图片特征点获取单元62基于放大操作后的低分辨率图片中图片像素的像素亮度值，获取低分辨率图片的图片特征点。
由于人眼对像素亮度值敏感度较大，对色彩的容错性较大，因此这里仅以低分辨率子图片的图片像素的像素亮度值，作为低分辨率子图片的图片特征点，以简化深度神经网络模型。
图片转换模块54的图片特征点转换单元63通过深度神经网络模型，将低分辨率图片的图片特征点转换为高分辨率图片的图片特征点。这里的高分辨率图片的图片特征点也是高分辨率图片中图片像素的像素亮度值。
图片转换模块54的高分辨率图片获取单元64根据图片特征点转换单元63获取的高分辨率图片的图片特征点以及低分辨率图片的蓝色红色浓度偏移量，合成高分辨率图片的图片像素，从而获取该低分辨率图片对应的高分辨率图片。
这样高分辨率图片生成装置50可仅仅通过查找深度神经网络模型以及使用深度神经网络模型对分辨率图片进行转换两个步骤实现了高分辨率图片的转换还原，大大提高了高分辨率图片生成装置的高分辨率图片的转换效率以及高分辨率图片生成装置的信息交互效率。
这样即完成了实施例的高分辨率图片生成装置50的高分辨率图片的生成过程。
本实施例的高分辨率图片生成装置通过创建具有非线性转换卷积层的深度神经网络模型，提高了将压缩后的低分辨率图片转换还原为高分辨率图片的准确性，从而降低了交互终端双方的多媒体图片信息的交互成本，提高了交互终端双方的多媒体图片信息的交互效率。
请参照图7，图7为本申请的高分辨率图片生成装置的一实施例中对应的模型生成设备的结构示意图。该模型生成设备70包括图片分割模块71、子图片转换模块72以及模型生成模块73。
图片分割模块71用于将高分辨率图片分割为多个高分辨率子图片；子图片转换模块72用于使用图片转换算法对高分辨率子图片进行图片转换，以得到高分辨率子图片对应的低分辨率子图片；模型生成模块73用于以低分辨率子图片作为深度神经网络框架的输入样本，高分辨率子图片作为深度神经网络框架的输出对比样本，生成对应的深度神经网络模型。
请参照图8，图8为本申请的高分辨率图片生成装置的一实施例中对应的模型生成设备的子图片转换模块的结构示意图。该子图片转换模块72包括图片 缩小操作单元81以及图片压缩单元82。
图片缩小操作单元81用于按设定缩放比例，对高分辨率子图片进行图片缩小操作；图片压缩单元82用于使用设定压缩算法对图片缩小操作后的高分辨率子图片进行图片压缩操作，以得到高分辨率子图片对应的低分辨率子图片。
请参照图9，图9为本申请的高分辨率图片生成装置的一实施例中对应的模型生成设备的模型生成模块的结构示意图。
该模型生成模块73包括高分辨率子图片特征点提取单元91、特征图尺寸设置单元92、卷积核数量设置单元93、卷积模板参数设置单元94、模型参数初始化单元95、模型参数运算单元96、低分辨率子图片特征点提取单元97、输入卷积层及激活层创建单元98、非线性转换卷积层及激活层创建单元99、预输出卷积层及激活层创建单元9A、输出卷积层创建单元9B以及模型生成单元9C。
高分辨率子图片特征点提取单元91用于基于高分辨率子图片中图片像素的像素亮度值，获取高分辨率子图片的图片特征点。特征图尺寸设置单元92用于设置输入卷积层的特征图尺寸与输出卷积层的特征图尺寸。卷积核数量设置单元93用于设置输入卷积层的卷积核数量、预输出卷积层的卷积核数量以及非线性转换卷积层的卷积核数量。卷积模板参数设置单元94用于交替使用1*1和3*3的参数矩阵，设置所有非线性转换卷积层的卷积模板参数。模型参数初始化单元95用于采用PReLU算法对深度神经网络模型进行参数初始化操作。模型参数运算单元96用于使用Adam算法对深度神经网络模型进行迭代运算，以获取深度神经网络模型中的参数。低分辨率子图片特征点提取单元97用于提取低分辨率子图片的图片特征点。输入卷积层及激活层创建单元98用于根据低分辨率子图片的图片特征点、输入卷积层的卷积核数量以及输入卷积层的特征图尺寸，创建深度神经网络模型的输入卷积层以及对应的输入激活层。非线性转换卷积层及激活层创建单元99用于根据深度神经网络模型的输入激活层的数据、非线性转换卷积层的卷积核数量、非线性转换卷积层的特征图尺寸以及非线性转换卷积层的卷积模板参数，创建深度神经网络模型的多个非线性转换 卷积层以及对应的非线性转换激活层。预输出卷积层及激活层创建单元9A用于根据深度神经网络模型的多个非线性转换激活层的数据、预输出卷积层的卷积核数量以及预输出卷积层的特征图尺寸，创建深度神经网络模型的预输出卷积层以及预输出激活层。输出卷积层创建单元9B用于根据深度神经网络模型的预输出激活层的数据、输出卷积层的卷积核数量以及输出卷积层的特征图尺寸，创建深度神经网络模型的输出卷积层。模型生成单元9C用于基于深度神经网络模型的输出卷积层的数据以及高分辨率子图片的图片特征点，生成对应的深度神经网络模型。
请参照图10，图10为本申请的高分辨率图片生成装置的一实施例中对应的模型生成设备的模型生成模块的低分辨率子图片特征点提取单元的结构示意图。该低分辨率子图片特征点提取单元97包括子图片插值放大操作子单元101以及低分辨率子图片特征点提取子单元102。
子图片插值放大操作子单元101用于对低分辨率子图片进行双立方插值放大操作，以得到放大操作后的低分辨率子图片；低分辨率子图片特征点提取子单元102用于基于放大操作后的低分辨率子图片中图片像素的像素亮度值，获取低分辨率子图片的图片特征点。
本实施例的模型生成设备70生成对应的深度神经网络模型的流程包括：
首先图片分割模块71对用来进行机器学习的高分辨率图片进行分割操作，如平均分割为4等分或9等分等，以获取多个高分辨率子图片。
随后子图片转换模块72使用预设的图片转换算法对高分辨率子图片进行图片转换，以得到高分辨率子图片对应的低分辨率子图片。具体可为：
子图片转换模块72的图片缩小操作单元81按设定缩放比例，对图片分割模块获取的高分辨率子图片进行图片缩小操作，如将每张高分辨率子图片缩小为原来尺寸的四分之一等。
子图片转换模块72的图片压缩单元82使用设定压缩算法对图片缩小操作后的高分辨率子图片进行图片压缩操作，如将图片的品质由100％降低到85％等。这样即可得到高分辨率子图片对应的低分辨率子图片。
随后模型生成模块73以子图片转换模块72获取的低分辨率子图片作为深度神经网络框架的输入样本，图片分割模块71获取的高分辨率子图片作为深度神经网络框架的输出对比样本，生成对应的深度神经网络模型。具体深度神经网络模型的创建流程包括：
一、模型生成模块73的低分辨率子图片特征点提取单元97提取低分辨率子图片的图片特征点。具体的，低分辨率子图片特征点提取单元97的子图片插值放大操作子单元101对低分辨率子图片进行双立方插值放大操作，以得到放大操作后的低分辨率子图片。这样可保证低分辨率子图片的图片特征点的数量与高分辨率子图片的图片特征点的数量一致。
低分辨率子图片特征点提取单元97的低分辨率子图片特征点提取子单元102将放大操作后的低分辨率子图片从RGB彩色空间转换到YCbCr彩色空间，并获取YCbCr彩色空间中低分辨率子图片的图片像素的像素亮度值，作为低分辨率子图片的图片特征点。
其中YCbCr彩色空间中的低分辨率子图片通过图片像素的像素亮度值(Y)、蓝色浓度偏移值(Cb)以及红色浓度偏移值(Cr)来定义图片。由于人眼对像素亮度值敏感度较大，对色彩的容错性较大，因此这里仅以低分辨率子图片的图片像素的像素亮度值，作为低分辨率子图片的图片特征点，以在不影响图片转换结果的情况下减小深度神经网络模型的计算量。
同样高分辨率子图片特征点提取单元91可获取YCbCr彩色空间中高分辨率子图片中图片像素的像素亮度值，作为高分辨率子图片的图片特征点。
二、模型生成模块73的输入卷积层及激活层创建单元98根据低分辨率子图片特征点提取单元97获取的低分辨率子图片的图片特征点、输入卷积层的卷积核数量以及输入卷积层的特征图尺寸，创建深度神经网络模型的输入卷积层以及对应的输入激活层。
这里的低分辨率子图片的图片特征点为该输入卷积层的输入，输入卷积层的卷积核数量用于表示从低分辨率子图片的图片特征点上进行特征提取的特征提取模式的数量，输入卷积层的特征图尺寸用于调整深度神经网络模型的复杂 度。其中输入卷积层的卷积核数量越大，输入卷积层的特征图尺寸越大，深度神经网络模型的复杂度越高，图片转换结果也就越准确。
这里为了提高图片转换结果的准确性，卷积核数量设置单元93可设置较多的特征提取模式，因此输入卷积层的卷积核数量设置的较大，如设置为14-18等。
如输入的低分辨率子图片的图片特征点为32*32的特征点矩阵，则可使用红色的像素亮度值、蓝色的像素亮度值以及绿色的像素亮度值进行图片特征点的特征提取(即输入卷积层的卷积核数量为3)。如设定该输入卷积层的卷积模板参数为5*5，则该输入卷积层的特征图尺寸为28*28，即使用5*5的参数矩阵依次遍历32*32的特征点矩阵可得到28*28尺寸的特征图。因此得到的输入卷积层的输出为28*28*3。
此外这里还需要使用输入激活层对输入卷积层的输出进行非线性处理，以保证输入卷积层的输出是可微的，进而保证后续深度神经网络模型的参数训练的正常运行。
三、模型生成模块73的非线性转换卷积层及激活层创建单元99根据输入卷积层及激活层创建单元98获取的深度神经网络模型的输入激活层的输出数据、非线性转换卷积层的卷积核数量、非线性转换卷积层的特征图尺寸以及非线性转换卷积层的卷积模板参数，创建深度神经网络模型的多个非线性转换卷积层以及对应的非线性转换激活层。如设置5个非线性转换卷积层以及对应的非线性转换激活层，多个非线性转换卷积层以及对应的非线性转换激活层依次连接，如非线性转换卷积层A1的输出与非线性转换激活层B1的输入连接，非线性转换激活层B1的输出与非线性转换卷积层A2的输入连接，非线性转换卷积层A2的输出与非线性转换激活层B2的输入连接……。
这里的深度神经网络模型的输入激活层的输出数据为第一个非线性转换卷积层的输入。非线性转换卷积层的卷积核数量用于表示从输入激活层的输出数据上进行特征提取的特征提取模式的数量，非线性转换卷积层的特征图尺寸和非线性转换卷积层的卷积模板参数用于调整深度神经网络模型的复杂度。
这里为了减小深度神经网络模型的运算量，卷积核数量设置单元93可设置较少的特征提取模式，因此非线性转换卷积层的卷积核数量设置的较小，如设置为4-6等。即输入卷积层的卷积核数量大于非线性转换卷积层的卷积核数量。
具体的，卷积模板参数设置单元94第一参数矩阵和第二参数矩阵，设置所有非线性转换卷积层的卷积模板参数。这里优选第一参数矩阵为1*1，德日参数矩阵为3*3。如非线性转换卷积层A1的卷积模板参数为1*1，非线性转换卷积层A2的卷积模板参数为3*3，非线性转换卷积层A3的卷积模板参数为1*1……。这样可以有效的提高深度神经网络模型的非线性特性，减小非线性转换卷积层的特征图尺寸的变化量，从而进一步减小深度神经网络模型的运算量，保证了后续深度神经网络模型的参数训练时的深度神经网络模型的收敛性，也不容易发生过拟合的现象。
此外这里还需要使用非线性转换激活层对相应的非线性转换卷积层的输出进行非线性处理，以保证输入卷积层的输出是可微的，进而保证后续深度神经网络模型的参数训练的正常运行。
四、模型生成模块73的预输出卷积层及激活层创建单元9A根据非线性转换卷积层及激活层创建单元99获取的深度神经网络模型的最后一个非线性转换激活层的输出数据、预输出卷积层的卷积核数量以及预输出卷积层的特征图尺寸，创建深度神经网络模型的预输出卷积层以及预输出激活层；
这里的深度神经网络模型的最后一个非线性转换激活层的输出数据为该输入卷积层的输入，预输出卷积层的卷积核数量用于表示从最后一个非线性转换激活层的输出数据上进行特征提取的特征提取模式的数量，输入卷积层的特征图尺寸用于调整深度神经网络模型的复杂度。
这里为了提高图片转换结果的准确性，卷积核数量设置单元93设置的预输出卷积层的卷积核数量应该与输入卷积层的卷积核数量相等，如均设置为14-18等。这样经过非线性转换激活层的非线性转换后，可使用同样的特征提取模式来生成高分辨率子图片。
此外这里还需要使用预输出激活层对相应的预输出卷积层的输出进行非线 性处理，以保证预输出卷积层的输出是可微的，进而保证后续深度神经网络模型的参数训练的正常运行。
五、模型生成模块73的输出卷积层创建单元9B根据预输出卷积层及激活层创建单元9A获取的深度神经网络模型的预输出激活层的数据、输出卷积层的卷积核数量以及输出卷积层的特征图尺寸，创建深度神经网络模型的输出卷积层。
这里的深度神经网络模型的预输出激活层的输出数据为该输出卷积层的输入，输出卷积层的卷积核数量用于表示从预输出激活层的输出数据上进行特征提取的特征提取模式的数量，预输出激活层的特征图尺寸用于调整深度神经网络模型的复杂度。
为了保证图片转换结果的准确性，特征图尺寸设置单元92设置的输出卷积层的特征图尺寸应与输入卷积层的特征图尺寸相同，这样可以输出相同数量的图片特征点，与输出对比样本(高分辨率子图片的图片特征点)进行对比操作。
六、模型生成模块73的模型生成单元9C基于输出卷积层创建单元9B获取的深度神经网络模型的输出卷积层的数据以及高分辨率子图片的图片特征点，生成对应的深度神经网络模型。
模型生成单元9C使用高分辨率子图片的图片特征点对深度神经网络模型的输出卷积层的输出数据进行比较，并根据比较结果对深度神经网络模型的参数进行修正，使得最后深度神经网络模型的输出卷积层的输出数据与高分辨率子图片的图片特征点的差异小于设定值，这样即可确定对应的深度神经网络模型。
具体的，这里为了加快深度神经网络模型的机器训练速度，这里模型参数初始化单元95可以预先使用PReLU算法(Parametric Rectified Linear Unit，参数化修正线性单元)对深度神经网络模型进行参数(如卷积模板参数w以及偏置参数b)初始化操作，这样可使得深度神经网络模型的收敛速度更快。
同时模型参数运算单元96还可使用Adam(Adaptive Moment Estimation，适应性矩估计)算法对深度神经网络模型进行迭代运算，从而可快速准确的获取该 深度神经网络模型中的参数。
这样即完成了本实施例的模型生成设备70的深度神经网络模型的生成过程。
本实施例的深度神经网络模型通过多个卷积核数量较小的非线性转换卷积层的设置，有效的减少了深度神经网络模型的运算量。同时预输出卷积层的卷积核数量与输入卷积层的卷积核数量相等，输出卷积层的特征图尺寸与输入卷积层的特征图尺寸相同，进一步提高了深度神经网络模型的运算准确性。
下面通过一具体实施例说明本申请的高分辨率图片生成方法及高分辨率图片生成装置的工作原理。请参照图11A和图11B，图11A为本申请的高分辨率图片生成方法及高分辨率图片生成装置的具体实施例的工作原理示意图；图11B为本申请的高分辨率图片生成方法及高分辨率图片生成装置的具体实施例的工作时序图。
其中模型生成设备为后台服务器C，后台服务器C分别与后台服务器D、用户终端c以及用户终端d连接。其中后台服务器C用于生成深度神经网络模型，后台服务器D用于提供训练用高分辨率图片，用户终端c和用户终端d为进行信息交互的两个用户终端。本具体实施例中的高分辨率图片生成步骤包括：
步骤S1101，后台服务器C从后台服务器D处接收多个训练用高分辨率图片。
步骤S1102，后台服务器C采用预设的图片转换算法对接收的训练用高分辨率图片进行分割以及图片转换，生成多个高分辨率子图片以及对应的多个低分辨率子图片。
步骤S1103，后台服务器C以低分辨率子图片作为深度神经网络框架的输入样本，高分辨率子图片作为深度神经网络框架的输出对比样本，生成对应的深度神经网络模型。
其中以低分辨率子图片中图片像素的像素亮度值作为低分辨率子图片的图片特征点，高分辨率子图片中图片像素的像素亮度值作为高分辨率子图片的图片特征点。
该深度神经网络模型如图12所示，其包括一个输入卷积层(A0)和对应的输入激活层(B0)、五个非线性转换卷积层(A1、A2、A3、A4、A5)以及对应的非线性转换激活层(B1、B2、B3、B4、B5)、一个预输出卷积层(A6)以及预输出激活层(B6)以及一个输出卷积层(A7)。
其中输入卷积层和预输出卷积层的卷积核数量均为16，非线性转换卷积层的卷积核数量均为5，输出卷积层的卷积核数量为1。输入卷积层的特征图尺寸和输出卷积层的特征图尺寸相同。五个非线性转换卷积层以及预输出卷积层的卷积模板参数依次为1*1、3*3、1*1、3*3、1*1、3*3。输入卷积层的特征图尺寸与输出卷积层的特征图尺寸相同。
步骤S1104，后台服务器C采用PReLU算法对深度神经网络模型进行参数初始化操作；随后使用Adam算法对深度神经网络模型进行迭代运算，以获取深度神经网络模型中的参数(如卷积模板参数w等)。
步骤S1105，后台服务器C将训练获得的深度神经网络模型发送至用户终端c以及用户终端d。
步骤S1106，当用户终端c需要向用户终端d发送某个高分辨率图片时，用户终端c可采用对应的图片转换算法将高分辨率图片转换为对应的低分辨率图片，随后将该低分辨率图片发送至用户终端d。
步骤S1107，用户终端d接收到该低分辨率图片时，在本地找到对应的深度神经网络模型。
步骤S1108，用户终端d通过该深度神经网络模型，将该低分辨率图片转换为对应的高分辨率图片。
这样即完成了本具体实施例的高分辨率图片的转换过程。
用户终端c不需要使用大量的存储空间对高分辨率图片进行存储操作，也不需要耗费较高的带宽对高分辨率图片进行传输操作；同时用户终端d也不需要对低分辨率图片进行复杂的图片转换操作，只需要按照预先设置的深度神经网络模型即可实现低分辨率图片到高分辨率图片的准确转换。
同时整个深度神经网络模型结构简单，具有较高的收敛性也不易发生过拟 合，因此整个深度神经网络模型的运算量较小，训练时间较短，准确性较高。
本申请的高分辨率图片生成方法、生成装置及存储介质通过创建具有非线性转换卷积层的深度神经网络模型，提高了将低分辨率图片转换为高分辨率图片的准确性，从而降低了交互终端双方的多媒体图片信息的交互成本，提高了交互终端双方的多媒体图片信息的交互效率；有效的解决了现有的高分辨率图片生成方法以及高分辨率图片生成装置的信息交互成本较高且信息交互效率较低的技术问题。
如本申请所使用的术语“组件”、“模块”、“系统”、“接口”、“进程”等等一般地旨在指计算机相关实体：硬件、硬件和软件的组合、软件或执行中的软件。例如，组件可以是但不限于是运行在处理器上的进程、处理器、对象、可执行应用、执行的线程、程序和/或计算机。通过图示，运行在控制器上的应用和该控制器二者都可以是组件。一个或多个组件可以有在于执行的进程和/或线程内，并且组件可以位于一个计算机上和/或分布在两个或更多计算机之间。
图13和随后的讨论提供了对实现本申请所述的高分辨率图片生成装置所在的电子设备的工作环境的简短、概括的描述。图13的工作环境仅仅是适当的工作环境的一个实例并且不旨在建议关于工作环境的用途或功能的范围的任何限制。实例电子设备1312包括但不限于可穿戴设备、头戴设备、医疗健康平台、个人计算机、服务器计算机、手持式或膝上型设备、移动设备(比如移动电话、个人数字助理(PDA)、媒体播放器等等)、多处理器系统、消费型电子设备、小型计算机、大型计算机、包括上述任意系统或设备的分布式计算环境，等等。
尽管没有要求，但是在“计算机可读指令”被一个或多个电子设备执行的通用背景下描述实施例。计算机可读指令可以经由计算机可读介质来分布(下文讨论)。计算机可读指令可以实现为程序模块，比如执行特定任务或实现特定抽象数据类型的功能、对象、应用编程接口(API)、数据结构等等。典型地，该计算机可读指令的功能可以在各种环境中随意组合或分布。
图13图示了包括本申请的高分辨率图片生成装置中的一个或多个实施例的电子设备1312的实例。在一种配置中，电子设备1312包括至少一个处理单元1316和存储器1318。根据电子设备的确切配置和类型，存储器1318可以是 易失性的(比如RAM)、非易失性的(比如ROM、闪存等)或二者的某种组合。该配置在图13中由虚线1314图示。
在其他实施例中，电子设备1312可以包括附加特征和/或功能。例如，设备1312还可以包括附加的存储装置(例如可移除和/或不可移除的)，其包括但不限于磁存储装置、光存储装置等等。这种附加存储装置在图13中由存储装置1320图示。在一个实施例中，用于实现本文所提供的一个或多个实施例的计算机可读指令可以在存储装置1320中。存储装置1320还可以存储用于实现操作系统、应用程序等的其他计算机可读指令。计算机可读指令可以载入存储器1318中由例如处理单元1316执行。
本文所使用的术语“计算机可读介质”包括计算机存储介质。计算机存储介质包括以用于存储诸如计算机可读指令或其他数据之类的信息的任何方法或技术实现的易失性和非易失性、可移除和不可移除介质。存储器1318和存储装置1320是计算机存储介质的实例。计算机存储介质包括但不限于RAM、ROM、EEPROM、闪存或其他存储器技术、CD-ROM、数字通用盘(DVD)或其他光存储装置、盒式磁带、磁带、磁盘存储装置或其他磁存储设备、或可以用于存储期望信息并可以被电子设备1312访问的任何其他介质。任意这样的计算机存储介质可以是电子设备1312的一部分。
电子设备1312还可以包括允许电子设备1312与其他设备通信的通信连接1326。通信连接1326可以包括但不限于调制解调器、网络接口卡(NIC)、集成网络接口、射频发射器/接收器、红外端口、USB连接或用于将电子设备1312连接到其他电子设备的其他接口。通信连接1326可以包括有线连接或无线连接。通信连接1326可以发射和/或接收通信媒体。
术语“计算机可读介质”可以包括通信介质。通信介质典型地包含计算机可读指令或诸如载波或其他传输机构之类的“己调制数据信号”中的其他数据，并且包括任何信息递送介质。术语“己调制数据信号”可以包括这样的信号：该信号特性中的一个或多个按照将信息编码到信号中的方式来设置或改变。
电子设备1312可以包括输入设备1324，比如键盘、鼠标、笔、语音输入设备、触摸输入设备、红外相机、视频输入设备和/或任何其他输入设备。设备1312中也可以包括输出设备1322，比如一个或多个显示器、扬声器、打印 机和/或任意其他输出设备。输入设备1324和输出设备1322可以经由有线连接、无线连接或其任意组合连接到电子设备1312。在一个实施例中，来自另一个电子设备的输入设备或输出设备可以被用作电子设备1312的输入设备1324或输出设备1322。
电子设备1312的组件可以通过各种互连(比如总线)连接。这样的互连可以包括外围组件互连(PCI)(比如快速PCI)、通用串行总线(USB)、火线(IEEE1394)、光学总线结构等等。在另一个实施例中，电子设备1312的组件可以通过网络互连。例如，存储器1318可以由位于不同物理位置中的、通过网络互连的多个物理存储器单元构成。
本领域技术人员将认识到，用于存储计算机可读指令的存储设备可以跨越网络分布。例如，可经由网络1328访问的电子设备1330可以存储用于实现本申请所提供的一个或多个实施例的计算机可读指令。电子设备1312可以访问电子设备1330并且下载计算机可读指令的一部分或所有以供执行。可替代地，电子设备1312可以按需要下载多条计算机可读指令，或者一些指令可以在电子设备1312处执行并且一些指令可以在电子设备1330处执行。
本文提供了实施例的各种操作。在一个实施例中，所述的一个或多个操作可以构成一个或多个计算机可读介质上存储的计算机可读指令，其在被电子设备执行时将使得计算设备执行所述操作。描述一些或所有操作的顺序不应当被解释为暗示这些操作必需是顺序相关的。本领域技术人员将理解具有本说明书的益处的可替代的排序。而且，应当理解，不是所有操作必需在本文所提供的每个实施例中存在。
而且，尽管已经相对于一个或多个实现方式示出并描述了本公开，但是本领域技术人员基于对本说明书和附图的阅读和理解将会想到等价变型和修改。本公开包括所有这样的修改和变型，并且仅由所附权利要求的范围限制。特别地关于由上述组件(例如元件、资源等)执行的各种功能，用于描述这样的组件的术语旨在对应于执行所述组件的指定功能(例如其在功能上是等价的)的任意组件(除非另外指示)，即使在结构上与执行本文所示的本公开的示范性实现方式中的功能的公开结构不等同。此外，尽管本公开的特定特征已经相对于若干实现方式中的仅一个被公开，但是这种特征可以与如可以对给定或特定应 用而言是期望和有利的其他实现方式的一个或多个其他特征组合。而且，就术语“包括”、“具有”、“含有”或其变形被用在具体实施方式或权利要求中而言，这样的术语旨在以与术语“包含”相似的方式包括。
本申请实施例中的各功能单元可以集成在一个处理模块中，也可以是各个单元单独物理存在，也可以两个或两个以上单元集成在一个模块中。上述集成的模块既可以采用硬件的形式实现，也可以采用软件功能模块的形式实现。所述集成的模块如果以软件功能模块的形式实现并作为独立的产品销售或使用时，也可以存储在一个计算机可读取存储介质中。上述提到的存储介质可以是只读存储器，磁盘或光盘等。上述的各装置或系统，可以执行相应方法实施例中的方法。
应该理解的是，虽然本申请各实施例中的各个步骤并不是必然按照步骤标号指示的顺序依次执行。除非本文中有明确的说明，这些步骤的执行并没有严格的顺序限制，这些步骤可以以其它的顺序执行。而且，各实施例中至少一部分步骤可以包括多个子步骤或者多个阶段，这些子步骤或者阶段并不必然是在同一时刻执行完成，而是可以在不同的时刻执行，这些子步骤或者阶段的执行顺序也不必然是依次进行，而是可以与其它步骤或者其它步骤的子步骤或者阶段的至少一部分轮流或者交替地执行。
本领域普通技术人员可以理解实现上述实施例方法中的全部或部分流程，是可以通过计算机程序来指令相关的硬件来完成，所述的程序可存储于一非易失性计算机可读取存储介质中，该程序在执行时，可包括如上述各方法的实施例的流程。其中，本申请所提供的各实施例中所使用的对存储器、存储、数据库或其它介质的任何引用，均可包括非易失性和/或易失性存储器。非易失性存储器可包括只读存储器(ROM)、可编程ROM(PROM)、电可编程ROM(EPROM)、电可擦除可编程ROM(EEPROM)或闪存。易失性存储器可包括随机存取存储器(RAM)或者外部高速缓冲存储器。作为说明而非局限，RAM以多种形式可得，诸如静态RAM(SRAM)、动态RAM(DRAM)、同步DRAM(SDRAM)、双数据率SDRAM(DDRSDRAM)、增强型SDRAM(ESDRAM)、同步链路(Synchlink)DRAM(SLDRAM)、存储器总线(Rambus)直接RAM (RDRAM)、直接存储器总线动态RAM(DRDRAM)、以及存储器总线动态RAM(RDRAM)等。

Claims (36)

  1. 一种高分辨率图片生成方法，其特征在于，包括：
    计算机设备获取至少一个深度神经网络模型，其中所述深度神经网络模型是根据对应的高分辨率图片、图片转换算法以及深度神经网络框架生成的；
    所述计算机设备获取低分辨率图片，其中所述低分辨率图片是根据对应的高分辨率图片以及图片转换算法生成的；
    所述计算机设备根据所述低分辨率图片，确定对应的深度神经网络模型；
    所述计算机设备通过所述深度神经网络模型，将所述低分辨率图片转换为高分辨率图片；
    其中所述深度神经网络模型包括交替使用不同参数矩阵作为卷积模板参数的多个非线性转换卷积层。
  2. 根据权利要求1所述的方法，其特征在于，所述根据高分辨率图片、图片转换算法以及深度神经网络框架生成所述深度神经网络模型的步骤包括：
    所述计算机设备将所述高分辨率图片分割为多个高分辨率子图片；
    所述计算机设备使用所述图片转换算法对所述高分辨率子图片进行图片转换，得到所述高分辨率子图片对应的低分辨率子图片；所述计算机设备将所述低分辨率子图片作为所述深度神经网络框架的输入样本，所述高分辨率子图片作为所述深度神经网络框架的输出对比样本，所述计算机设备生成对应的深度神经网络模型。
  3. 根据权利要求2所述的方法，其特征在于，所述计算机设备使用所述图片转换算法对所述高分辨率子图片进行图片转换，得到所述高分辨率子图片对应的低分辨率子图片的步骤包括：
    所述计算机设备按设定缩放比例，对所述高分辨率子图片进行图片缩小操作；所述计算机设备使用设定压缩算法对所述图片缩小操作后的高分辨率子图片进行图片压缩操作，得到所述高分辨率子图片对应的低分辨率子图片。
  4. 根据权利要求2所述的方法，其特征在于，所述计算机设备生成对应的深度神经网络模型的步骤包括：
    所述计算机设备提取所述低分辨率子图片的图片特征点；
    所述计算机设备根据所述低分辨率子图片的图片特征点、输入卷积层的卷积核数量以及输入卷积层的特征图尺寸，创建所述深度神经网络模型的输入卷积层以及对应的输入激活层；
    所述计算机设备根据所述深度神经网络模型的输入激活层的数据、非线性转换卷积层的卷积核数量、非线性转换卷积层的特征图尺寸以及非线性转换卷积层的卷积模板参数，创建所述深度神经网络模型的多个非线性转换卷积层以及对应的非线性转换激活层；
    所述计算机设备根据所述深度神经网络模型的多个非线性转换激活层的数据、预输出卷积层的卷积核数量以及预输出卷积层的特征图尺寸，创建所述深度神经网络模型的预输出卷积层以及预输出激活层；
    所述计算机设备根据所述深度神经网络模型的预输出激活层的数据、输出卷积层的卷积核数量以及输出卷积层的特征图尺寸，创建所述深度神经网络模型的输出卷积层；所述计算机设备基于所述深度神经网络模型的输出卷积层的数据以及所述高分辨率子图片的图片特征点，生成对应的深度神经网络模型。
  5. 根据权利要求4所述的方法，其特征在于，所述计算机设备提取所述低分辨率子图片的图片特征点的步骤包括：
    所述计算机设备对所述低分辨率子图片进行双立方插值放大操作，得到放大操作后的低分辨率子图片；所述计算机设备基于所述放大操作后的低分辨率子图片中图片像素的像素亮度值，获取所述低分辨率子图片的图片特征点。
  6. 根据权利要求4所述的方法，其特征在于，所述方法还包括：
    所述计算机设备基于所述高分辨率子图片中图片像素的像素亮度值，获取所述高分辨率子图片的图片特征点。
  7. 根据权利要求4所述的方法，其特征在于，所述方法还包括：
    所述计算机设备设置输入卷积层的特征图尺寸与输出卷积层的特征图尺寸；
    其中所述输入卷积层的特征图尺寸与输出卷积层的特征图尺寸相同。
  8. 根据权利要求4所述的方法，其特征在于，所述方法还包括：
    所述计算机设备设置所述输入卷积层的卷积核数量和所述预输出卷积层的卷积核数量；
    其中所述输入卷积层的卷积核数量与所述预输出卷积层的卷积核数量相等。
  9. 根据权利要求8所述的方法，其特征在于，所述方法还包括：
    所述计算机设备设置所述非线性转换卷积层的卷积核数量；
    其中所述输入卷积层的卷积核数量大于所述非线性转换卷积层的卷积核数量。
  10. 根据权利要求4所述的方法，其特征在于，所述计算机设备根据所述深度神经网络模型的输入激活层的数据、非线性转换卷积层的卷积核数量、非线性转换卷积层的特征图尺寸以及非线性转换卷积层的卷积模板参数，创建所述深度神经网络模型的多个非线性转换卷积层以及对应的非线性转换激活层的步骤包括：
    所述计算机设备交替使用第一参数矩阵和第二参数矩阵，设置所有非线性转换卷积层的卷积模板参数。
  11. 根据权利要求4所述的方法，其特征在于，所述方法包括：
    所述计算机设备采用PReLU算法对所述深度神经网络模型进行参数初始化操作；
    所述计算机设备使用Adam算法对所述深度神经网络模型进行迭代运算，获取所述深度神经网络模型中的参数。
  12. 根据权利要求1所述的方法，其特征在于，所述计算机设备通过所述深度神经网络模型，将所述低分辨率图片转换为高分辨率图片的步骤包括：
    所述计算机设备对所述低分辨率图片进行双立方插值放大操作，得到放大操作后的低分辨率图片；
    所述计算机设备基于所述放大操作后的低分辨率图片中图片像素的像素亮度值，获取所述低分辨率图片的图片特征点；
    所述计算机设备通过所述深度神经网络模型，将所述低分辨率图片的图片特征点转换为高分辨率图片的图片特征点；
    所述计算机设备根据所述高分辨率图片的图片特征点以及所述低分辨率图片的蓝色红色浓度偏移量，获取所述高分辨率图片。
  13. 一种计算机设备，包括存储器和处理器，所述存储器中存储有计算机可读指令，所述计算机可读指令被所述处理器执行时，使得所述处理器执行如下步骤：
    获取至少一个深度神经网络模型，其中所述深度神经网络模型是根据对应的高分辨率图片、图片转换算法以及深度神经网络框架生成的；
    获取低分辨率图片，其中所述低分辨率图片是根据对应的高分辨率图片以及图片转换算法生成的；
    根据所述低分辨率图片，确定对应的深度神经网络模型；通过所述深度神经网络模型，将所述低分辨率图片转换为高分辨率图片；
    其中所述深度神经网络模型包括交替使用不同参数矩阵作为卷积模板参数的多个非线性转换卷积层。
  14. 根据权利要求1所述的计算机设备，其特征在于，所述根据高分辨率图片、图片转换算法以及深度神经网络框架生成所述深度神经网络模型的步骤包括：
    将所述高分辨率图片分割为多个高分辨率子图片；
    使用所述图片转换算法对所述高分辨率子图片进行图片转换，以得到所述高分辨率子图片对应的低分辨率子图片；将所述低分辨率子图片作为所述深度神经网络框架的输入样本，所述高分辨率子图片作为所述深度神经网络框架的输出对比样本，生成对应的深度神经网络模型。
  15. 根据权利要求14所述的计算机设备，其特征在于，所述使用所述图片转换算法对所述高分辨率子图片进行图片转换，得到所述高分辨率子图片对应的低分辨率子图片的步骤包括：
    按设定缩放比例，对所述高分辨率子图片进行图片缩小操作；使用设定压缩算法对所述图片缩小操作后的高分辨率子图片进行图片压缩操作，以得到所述高分辨率子图片对应的低分辨率子图片。
  16. 根据权利要求14所述的计算机设备，其特征在于，所述计算机可读指令还使得所述处理器执行如下步骤：
    提取所述低分辨率子图片的图片特征点；
    根据所述低分辨率子图片的图片特征点、输入卷积层的卷积核数量以及输入卷积层的特征图尺寸，创建所述深度神经网络模型的输入卷积层以及对应的输入激活层；
    根据所述深度神经网络模型的输入激活层的数据、非线性转换卷积层的卷积核数量、非线性转换卷积层的特征图尺寸以及非线性转换卷积层的卷积模板参数，创建所述深度神经网络模型的多个非线性转换卷积层以及对应的非线性转换激活层；
    根据所述深度神经网络模型的多个非线性转换激活层的数据、预输出卷积层的卷积核数量以及预输出卷积层的特征图尺寸，创建所述深度神经网络模型的预输出卷积层以及预输出激活层；
    根据所述深度神经网络模型的预输出激活层的数据、输出卷积层的卷积核数量以及输出卷积层的特征图尺寸，创建所述深度神经网络模型的输出卷积层；
    基于所述深度神经网络模型的输出卷积层的数据以及所述高分辨率子图片的图片特征点，生成对应的深度神经网络模型。
  17. 根据权利要求16所述的计算机设备，其特征在于，所述提取所述低分辨率子图片的图片特征点的步骤包括：
    对所述低分辨率子图片进行双立方插值放大操作，以得到放大操作后的低分辨率子图片；基于所述放大操作后的低分辨率子图片中图片像素的像素亮度值，获取所述低分辨率子图片的图片特征点。
  18. 根据权利要求16所述的计算机设备，其特征在于，所述计算机可读指令还使得所述处理器执行如下步骤：
    基于所述高分辨率子图片中图片像素的像素亮度值，获取所述高分辨率子图片的图片特征点。
  19. 根据权利要求16所述的计算机设备，其特征在于，所述计算机可读指令还使得所述处理器执行如下步骤：
    设置输入卷积层的特征图尺寸与输出卷积层的特征图尺寸；
    其中所述输入卷积层的特征图尺寸与输出卷积层的特征图尺寸相同。
  20. 根据权利要求16所述的计算机设备，其特征在于，所述计算机可读指令还使得所述处理器执行如下步骤：
    设置所述输入卷积层的卷积核数量和所述预输出卷积层的卷积核数量；
    其中所述输入卷积层的卷积核数量与所述预输出卷积层的卷积核数量相等。
  21. 根据权利要求20所述的计算机设备，其特征在于，所述计算机可读指令还使得所述处理器执行如下步骤：
    设置所述非线性转换卷积层的卷积核数量；
    其中所述输入卷积层的卷积核数量大于所述非线性转换卷积层的卷积核数量。
  22. 根据权利要求16所述的计算机设备，其特征在于，所述根据所述深度神经网络模型的输入激活层的数据、非线性转换卷积层的卷积核数量、非线性转换卷积层的特征图尺寸以及非线性转换卷积层的卷积模板参数，创建所述深度神经网络模型的多个非线性转换卷积层以及对应的非线性转换激活层的步骤包括：
    交替使用第一参数矩阵和第二参数矩阵，设置所有非线性转换卷积层的卷积模板参数。
  23. 根据权利要求16所述的计算机设备，其特征在于，所述计算机可读指令还使得所述处理器执行如下步骤：
    采用PReLU算法对所述深度神经网络模型进行参数初始化操作；使用Adam算法对所述深度神经网络模型进行迭代运算，以获取所述深度神经网络 模型中的参数。
  24. 根据权利要求13所述的计算机设备，其特征在于，所述通过所述深度神经网络模型，将所述低分辨率图片转换为高分辨率图片的步骤包括：
    对所述低分辨率图片进行双立方插值放大操作，得到放大操作后的低分辨率图片；
    基于所述放大操作后的低分辨率图片中图片像素的像素亮度值，获取所述低分辨率图片的图片特征点；
    通过所述深度神经网络模型，将所述低分辨率图片的图片特征点转换为高分辨率图片的图片特征点；根据所述高分辨率图片的图片特征点以及所述低分辨率图片的蓝色红色浓度偏移量，获取所述高分辨率图片。
  25. 一个或多个存储有计算机可读指令的非易失性存储介质，所述计算机可读指令被一个或多个处理器执行时，使得一个或多个处理器执行如下步骤：
    获取至少一个深度神经网络模型，其中所述深度神经网络模型是根据对应的高分辨率图片、图片转换算法以及深度神经网络框架生成的；
    获取低分辨率图片，其中所述低分辨率图片是根据对应的高分辨率图片以及图片转换算法生成的；
    根据所述低分辨率图片，确定对应的深度神经网络模型；通过所述深度神经网络模型，将所述低分辨率图片转换为高分辨率图片；
    其中所述深度神经网络模型包括交替使用不同参数矩阵作为卷积模板参数的多个非线性转换卷积层。
  26. 根据权利要求25所述的存储介质，其特征在于，所述根据高分辨率图片、图片转换算法以及深度神经网络框架生成所述深度神经网络模型的步骤包括：
    将所述高分辨率图片分割为多个高分辨率子图片；
    使用所述图片转换算法对所述高分辨率子图片进行图片转换，以得到所述高分辨率子图片对应的低分辨率子图片；
    将所述低分辨率子图片作为所述深度神经网络框架的输入样本，所述高分辨率子图片作为所述深度神经网络框架的输出对比样本，生成对应的深度神经网络模型。
  27. 根据权利要求26所述的存储介质，其特征在于，所述使用所述图片转换算法对所述高分辨率子图片进行图片转换，以得到所述高分辨率子图片对应的低分辨率子图片的步骤包括：
    按设定缩放比例，对所述高分辨率子图片进行图片缩小操作；使用设定压缩算法对所述图片缩小操作后的高分辨率子图片进行图片压缩操作，以得到所述高分辨率子图片对应的低分辨率子图片。
  28. 根据权利要求26所述的存储介质，其特征在于，所述生成对应的深度神经网络模型的步骤包括：
    提取所述低分辨率子图片的图片特征点；
    根据所述低分辨率子图片的图片特征点、输入卷积层的卷积核数量以及输入卷积层的特征图尺寸，创建所述深度神经网络模型的输入卷积层以及对应的输入激活层；
    根据所述深度神经网络模型的输入激活层的数据、非线性转换卷积层的卷积核数量、非线性转换卷积层的特征图尺寸以及非线性转换卷积层的卷积模板参数，创建所述深度神经网络模型的多个非线性转换卷积层以及对应的非线性转换激活层；
    根据所述深度神经网络模型的多个非线性转换激活层的数据、预输出卷积层的卷积核数量以及预输出卷积层的特征图尺寸，创建所述深度神经网络模型的预输出卷积层以及预输出激活层；
    根据所述深度神经网络模型的预输出激活层的数据、输出卷积层的卷积核数量以及输出卷积层的特征图尺寸，创建所述深度神经网络模型的输出卷积层；
    基于所述深度神经网络模型的输出卷积层的数据以及所述高分辨率子图片的图片特征点，生成对应的深度神经网络模型。
  29. 根据权利要求28所述的存储介质，其特征在于，所述提取所述低分辨 率子图片的图片特征点的步骤包括：
    对所述低分辨率子图片进行双立方插值放大操作，以得到放大操作后的低分辨率子图片；基于所述放大操作后的低分辨率子图片中图片像素的像素亮度值，获取所述低分辨率子图片的图片特征点。
  30. 根据权利要求28所述的存储介质，其特征在于，所述计算机可读指令还使得所述处理器执行如下步骤：
    基于所述高分辨率子图片中图片像素的像素亮度值，获取所述高分辨率子图片的图片特征点。
  31. 根据权利要求28所述的存储介质，其特征在于，所述计算机可读指令还使得所述处理器执行如下步骤：
    设置输入卷积层的特征图尺寸与输出卷积层的特征图尺寸；
    其中所述输入卷积层的特征图尺寸与输出卷积层的特征图尺寸相同。
  32. 根据权利要求28所述的存储介质，其特征在于，所述计算机可读指令还使得所述处理器执行如下步骤：
    设置所述输入卷积层的卷积核数量和所述预输出卷积层的卷积核数量；
    其中所述输入卷积层的卷积核数量与所述预输出卷积层的卷积核数量相等。
  33. 根据权利要求32所述的存储介质，其特征在于，所述计算机可读指令还使得所述处理器执行如下步骤：
    设置所述非线性转换卷积层的卷积核数量；
    其中所述输入卷积层的卷积核数量大于所述非线性转换卷积层的卷积核数量。
  34. 根据权利要求28所述的存储介质，其特征在于，所述根据所述深度神经网络模型的输入激活层的数据、非线性转换卷积层的卷积核数量、非线性转换卷积层的特征图尺寸以及非线性转换卷积层的卷积模板参数，创建所述深度神经网络模型的多个非线性转换卷积层以及对应的非线性转换激活层的步骤包括：
    交替使用第一参数矩阵和第二参数矩阵，设置所有非线性转换卷积层的卷积模板参数。
  35. 根据权利要求28所述的存储介质，其特征在于，所述计算机可读指令还使得所述处理器执行如下步骤：
    采用PReLU算法对所述深度神经网络模型进行参数初始化操作；以及
    使用Adam算法对所述深度神经网络模型进行迭代运算，以获取所述深度神经网络模型中的参数。
  36. 根据权利要求25所述的存储介质，其特征在于，所述通过所述深度神经网络模型，将所述低分辨率图片转换为高分辨率图片的步骤包括：
    对所述低分辨率图片进行双立方插值放大操作，得到放大操作后的低分辨率图片；
    基于所述放大操作后的低分辨率图片中图片像素的像素亮度值，获取所述低分辨率图片的图片特征点；
    通过所述深度神经网络模型，将所述低分辨率图片的图片特征点转换为高分辨率图片的图片特征点；根据所述高分辨率图片的图片特征点以及所述低分辨率图片的蓝色红色浓度偏移量，获取所述高分辨率图片。
PCT/CN2018/116987 2017-11-24 2018-11-22 高分辨率图片生成方法、计算机设备及存储介质 WO2019101140A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201711191101.1A CN108022212B (zh) 2017-11-24 2017-11-24 高分辨率图片生成方法、生成装置及存储介质
CN201711191101.1 2017-11-24

Related Child Applications (1)

Application Number Title Priority Date Filing Date
US16/862,194 Continuation US11373275B2 (en) 2017-11-24 2020-04-29 Method for generating high-resolution picture, computer device, and storage medium

Publications (1)

Publication Number Publication Date
WO2019101140A1 true WO2019101140A1 (zh) 2019-05-31

Family

ID=62080053

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2018/116987 WO2019101140A1 (zh) 2017-11-24 2018-11-22 高分辨率图片生成方法、计算机设备及存储介质

Country Status (3)

Country Link
US (1) US11373275B2 (zh)
CN (1) CN108022212B (zh)
WO (1) WO2019101140A1 (zh)

Families Citing this family (23)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104050192B (zh) 2013-03-15 2017-08-04 腾讯科技（深圳）有限公司 一种运行应用的方法、装置和系统
CN108022212B (zh) * 2017-11-24 2022-07-01 腾讯科技（深圳）有限公司 高分辨率图片生成方法、生成装置及存储介质
KR102570562B1 (ko) 2018-07-16 2023-08-24 삼성전자주식회사 영상 처리 장치 및 그 동작방법
KR102312337B1 (ko) * 2018-10-19 2021-10-14 삼성전자주식회사 Ai 부호화 장치 및 그 동작방법, 및 ai 복호화 장치 및 그 동작방법
WO2020080765A1 (en) 2018-10-19 2020-04-23 Samsung Electronics Co., Ltd. Apparatuses and methods for performing artificial intelligence encoding and artificial intelligence decoding on image
WO2020080873A1 (en) 2018-10-19 2020-04-23 Samsung Electronics Co., Ltd. Method and apparatus for streaming data
WO2020080623A1 (ko) * 2018-10-19 2020-04-23 삼성전자 주식회사 영상의 ａｉ 부호화 및 ａｉ 복호화 방법, 및 장치
WO2020080665A1 (en) 2018-10-19 2020-04-23 Samsung Electronics Co., Ltd. Methods and apparatuses for performing artificial intelligence encoding and artificial intelligence decoding on image
US11720997B2 (en) 2018-10-19 2023-08-08 Samsung Electronics Co.. Ltd. Artificial intelligence (AI) encoding device and operating method thereof and AI decoding device and operating method thereof
KR102525578B1 (ko) 2018-10-19 2023-04-26 삼성전자주식회사 부호화 방법 및 그 장치, 복호화 방법 및 그 장치
WO2020080698A1 (ko) 2018-10-19 2020-04-23 삼성전자 주식회사 영상의 주관적 품질을 평가하는 방법 및 장치
CN109409299A (zh) * 2018-10-30 2019-03-01 盯盯拍（深圳）云技术有限公司 图像识别方法及图像识别装置
CN109302587A (zh) * 2018-10-30 2019-02-01 盯盯拍（深圳）云技术有限公司 图像传输方法及图像传输装置
CN110147864B (zh) * 2018-11-14 2022-02-22 腾讯科技（深圳）有限公司 编码图案的处理方法和装置、存储介质、电子装置
CN110992260B (zh) * 2019-10-15 2022-04-22 网宿科技股份有限公司 一种视频超分辨率重建的方法和装置
KR20210056179A (ko) 2019-11-08 2021-05-18 삼성전자주식회사 Ai 부호화 장치 및 그 동작방법, 및 ai 복호화 장치 및 그 동작방법
FR3112009B1 (fr) * 2020-06-30 2022-10-28 St Microelectronics Grenoble 2 Procédé de conversion d’une image numérique
CN112435162B (zh) * 2020-11-13 2024-03-05 中国科学院沈阳自动化研究所 一种基于复数域神经网络的太赫兹图像超分辨重建方法
CN112419151B (zh) * 2020-11-19 2023-07-21 北京有竹居网络技术有限公司 图像退化处理方法、装置、存储介质及电子设备
US11823490B2 (en) * 2021-06-08 2023-11-21 Adobe, Inc. Non-linear latent to latent model for multi-attribute face editing
CN113591798B (zh) * 2021-08-23 2023-11-03 京东科技控股股份有限公司 文档文字的重建方法及装置、电子设备、计算机存储介质
CN115294284B (zh) * 2022-10-09 2022-12-20 南京纯白矩阵科技有限公司 一种保障生成模型唯一性的高分辨率三维模型生成方法
CN116563115A (zh) * 2023-07-07 2023-08-08 深圳市龙之源科技股份有限公司 图像放大方法、装置、电子设备及计算机可读存储介质

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106600538A (zh) * 2016-12-15 2017-04-26 武汉工程大学 一种基于区域深度卷积神经网络的人脸超分辨率算法
CN106910161A (zh) * 2017-01-24 2017-06-30 华南理工大学 一种基于深度卷积神经网络的单幅图像超分辨率重建方法
CN107154021A (zh) * 2017-05-03 2017-09-12 西安电子科技大学 基于深层门限卷积神经网络的图像超分辨方法
CN108022212A (zh) * 2017-11-24 2018-05-11 腾讯科技（深圳）有限公司 高分辨率图片生成方法、生成装置及存储介质

Family Cites Families (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20130028538A1 (en) * 2011-07-29 2013-01-31 Simske Steven J Method and system for image upscaling
CN106204449B (zh) * 2016-07-06 2019-09-10 安徽工业大学 一种基于对称深度网络的单幅图像超分辨率重建方法
CN106791927A (zh) * 2016-12-23 2017-05-31 福建帝视信息科技有限公司 一种基于深度学习的视频增强与传输方法
CN106991646B (zh) * 2017-03-28 2020-05-26 福建帝视信息科技有限公司 一种基于密集连接网络的图像超分辨率方法
CN107240066A (zh) * 2017-04-28 2017-10-10 天津大学 基于浅层和深层卷积神经网络的图像超分辨率重建算法
CN107155110A (zh) * 2017-06-14 2017-09-12 福建帝视信息科技有限公司 一种基于超分辨率技术的图片压缩方法

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106600538A (zh) * 2016-12-15 2017-04-26 武汉工程大学 一种基于区域深度卷积神经网络的人脸超分辨率算法
CN106910161A (zh) * 2017-01-24 2017-06-30 华南理工大学 一种基于深度卷积神经网络的单幅图像超分辨率重建方法
CN107154021A (zh) * 2017-05-03 2017-09-12 西安电子科技大学 基于深层门限卷积神经网络的图像超分辨方法
CN108022212A (zh) * 2017-11-24 2018-05-11 腾讯科技（深圳）有限公司 高分辨率图片生成方法、生成装置及存储介质

Also Published As

Publication number Publication date
CN108022212B (zh) 2022-07-01
CN108022212A (zh) 2018-05-11
US20200258197A1 (en) 2020-08-13
US11373275B2 (en) 2022-06-28

Similar Documents

Publication Publication Date Title
WO2019101140A1 (zh) 高分辨率图片生成方法、计算机设备及存储介质
WO2019192316A1 (zh) 图像相关处理方法及装置、设备、存储介质
US9478006B2 (en) Content aware cropping
US9635246B2 (en) Systems and methods to super resolve a user-selected region of interest
US9692959B2 (en) Image processing apparatus and method
KR102115066B1 (ko) 비디오 안정화를 위한 적응적 경로 평활화
US20150363922A1 (en) Super-resolution from handheld camera
US11704844B2 (en) View synthesis robust to unconstrained image data
JP2017130929A (ja) 撮像装置により取得された文書画像の補正方法及び補正装置
US11367163B2 (en) Enhanced image processing techniques for deep neural networks
US20220130022A1 (en) Image processing method and apparatus, computing device, and storage medium
WO2020001222A1 (zh) 图像处理方法、装置、计算机可读介质及电子设备
WO2017112086A1 (en) Multi-stage image super-resolution with reference merging using personalized dictionaries
AU2017383979A1 (en) Projection image construction method and device
WO2020078102A1 (zh) 图像增强方法、装置和计算机可读存储介质
WO2022016996A1 (zh) 图像处理方法、装置、电子设备及计算机可读存储介质
WO2024041027A1 (zh) 全景图像处理方法、计算机设备和存储介质
US10445921B1 (en) Transferring motion between consecutive frames to a digital image
WO2023151214A1 (zh) 图像生成方法、系统、电子设备、存储介质和产品
WO2023103682A1 (zh) 图像处理方法、装置、设备及介质
US10321164B2 (en) System and method for improving graphics and other signal results through signal transformation and application of dithering
WO2017107605A1 (zh) 一种图像细节处理方法和装置、终端、存储介质
Gutenko et al. Remote volume rendering pipeline for mHealth applications
US9681064B1 (en) Lookup table interpolation in a film emulation camera system
US10735754B2 (en) Chromatic aberration modeling in image compression and enhancement

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 18881983

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 18881983

Country of ref document: EP

Kind code of ref document: A1