WO2023185706A1 - 影像处理方法、影像处理装置、存储介质 - Google Patents

影像处理方法、影像处理装置、存储介质 Download PDF

Info

Publication number
WO2023185706A1
WO2023185706A1 PCT/CN2023/083986 CN2023083986W WO2023185706A1 WO 2023185706 A1 WO2023185706 A1 WO 2023185706A1 CN 2023083986 W CN2023083986 W CN 2023083986W WO 2023185706 A1 WO2023185706 A1 WO 2023185706A1
Authority
WO
WIPO (PCT)
Prior art keywords
image
parameters
compensation
processing
color
Prior art date
Application number
PCT/CN2023/083986
Other languages
English (en)
French (fr)
Inventor
陈冠男
卢运华
朱丹
Original Assignee
京东方科技集团股份有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 京东方科技集团股份有限公司 filed Critical 京东方科技集团股份有限公司
Publication of WO2023185706A1 publication Critical patent/WO2023185706A1/zh

Links

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/50—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding
    • H04N19/503—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding involving temporal prediction
    • H04N19/51—Motion estimation or motion compensation
    • H04N19/53—Multi-resolution motion estimation; Hierarchical motion estimation
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T3/00—Geometric image transformations in the plane of the image
    • G06T3/40—Scaling of whole images or parts thereof, e.g. expanding or contracting
    • G06T3/4007—Scaling of whole images or parts thereof, e.g. expanding or contracting based on interpolation, e.g. bilinear interpolation
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T3/00—Geometric image transformations in the plane of the image
    • G06T3/40—Scaling of whole images or parts thereof, e.g. expanding or contracting
    • G06T3/4046—Scaling of whole images or parts thereof, e.g. expanding or contracting using neural networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00—Image enhancement or restoration
    • G06T5/60—Image enhancement or restoration using machine learning, e.g. neural networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00—Image enhancement or restoration
    • G06T5/70—Denoising; Smoothing
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N5/00—Details of television systems
    • H04N5/222—Studio circuitry; Studio devices; Studio equipment
    • H04N5/262—Studio circuits, e.g. for mixing, switching-over, change of character of image, other special effects ; Cameras specially adapted for the electronic generation of special effects
    • H04N5/265—Mixing
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10016—Video; Image sequence
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20084—Artificial neural networks [ANN]

Definitions

  • Embodiments of the present disclosure relate to an image processing method, an image processing device, and a non-transitory computer-readable storage medium.
  • ultra-high-definition display applications gradually appear in people's lives.
  • the ultra-high-definition display industry chain is constantly improving.
  • the ultra-high-definition display industry chain for the collection end, there are ultra-high-definition cameras, etc., and for the display end, there is HDR (High Dynamic Range Imaging, high dynamic range imaging) TVs, large screens with 4K/8K resolution, etc.
  • HDR High Dynamic Range Imaging, high dynamic range imaging
  • 5G networks ultra-high-definition TV stations, etc., and a large number of enterprises and institutions are laying out their layout in the ultra-high-definition display industry chain.
  • At least one embodiment of the present disclosure provides an image processing method, including: acquiring an input image; performing image quality enhancement processing on the input image to obtain an enhanced image; wherein the enhanced image corresponds to a resolution higher than the desired resolution. Describe the resolution corresponding to the input image.
  • the image processing method provided by at least one embodiment of the present disclosure further includes: performing compensation processing on the enhanced image to obtain a compensated image, wherein the corresponding resolution of the compensated image is higher than that of the input image. resolution.
  • the image processing method is applied to the first display panel to perform compensation processing on the enhanced image to obtain a compensated image, including: from a plurality of compensated images Select a compensation parameter corresponding to the first display panel among the parameters; perform compensation processing on the enhanced image based on the compensation parameter corresponding to the first display panel to obtain the compensated image.
  • Selecting a compensation parameter corresponding to the first display panel includes: obtaining a compensation input parameter corresponding to the first display panel, wherein the compensation input parameter includes a panel parameter corresponding to the first display panel and/ Or environmental parameters of the environment in which the first display panel is located; based on the compensation input parameter, select a compensation parameter corresponding to the compensation input parameter from the plurality of compensation parameters as the compensation parameter corresponding to the first display panel of the compensation parameters.
  • obtaining the compensation input parameters corresponding to the first display panel includes: generating a first array, wherein the first array and the first display panel Corresponding to the panel, the first array includes a plurality of first array elements, the panel parameters are represented by at least one first array element, and the environment parameters are represented by at least one first array element; based on the first array, determine The compensation input parameter corresponding to the first display panel.
  • the panel parameters include the type of the first display panel, the size of the first display panel, and the display mode of the first display panel.
  • the type of the first display panel includes an organic light-emitting display panel and a liquid crystal display panel
  • the display mode of the first display panel includes direct display and projection display.
  • the environmental parameters include ambient light parameters, and the ambient light parameters are determined based on the brightness value of the ambient light of the environment where the first display panel is located.
  • the compensation process includes at least one of the following: brightness adjustment, contrast adjustment, and saturation adjustment.
  • the image processing method provided by at least one embodiment of the present disclosure further includes: generating the multiple compensation parameters, wherein generating the multiple compensation parameters includes: acquiring a color card image obtained by photographing a standard color card; performing image quality enhancement processing on the color card image to obtain an enhanced color card image; performing compensation processing on the enhanced color card image based on initial compensation parameters to obtain a compensated color card image; displaying the information on the second display panel Describe the compensated color card image, and determine whether the display of the compensated color card image meets the preset requirements; in response to the display of the compensated color card image not meeting the preset requirements, the initial compensation parameters are Adjustment is made to obtain the adjusted compensation parameters, and the enhanced color card image is compensated again based on the adjusted compensation parameters until the display of the compensated color card image meets the preset requirements.
  • the compensation parameter corresponding to the compensated color card image that meets the preset requirements is used as one of the plurality of compensation parameters; determine the same as the second display surface Compensation input parameters corresponding to the panel, wherein the compensation input parameters corresponding to the second display panel include panel parameters corresponding to the second display panel and/or environmental parameters of the environment in which the second display panel is located; establishing a system that satisfies The mapping relationship between the compensation parameters corresponding to the compensated color card image required by the preset and the compensation input parameters corresponding to the second display panel.
  • the image quality enhancement processing includes at least one of the following processing: frame interpolation processing, super-resolution processing, noise reduction processing, color correction processing, high dynamic range processing Range transformation processing and detail restoration processing.
  • performing the image quality enhancement processing on the input image to obtain the enhanced image includes: obtaining image quality enhancement parameters; according to the image quality The enhancement parameters perform the image quality enhancement processing on the input image to obtain the enhanced image.
  • the image quality enhancement parameters include at least one of the following parameters: the name of the frame interpolation algorithm and/or the frame interpolation parameters corresponding to the frame interpolation processing, the The super-resolution algorithm name and/or resolution parameters corresponding to the super-resolution processing, the color adjustment algorithm name and/or color adjustment parameters corresponding to the color adjustment process, and the noise reduction algorithm name and/or corresponding to the noise reduction process. or noise reduction parameters, high dynamic range up-conversion algorithms and/or high dynamic range up-conversion parameters corresponding to the high dynamic range up-conversion process, name of the detail restoration algorithm and/or detail restoration parameters corresponding to the detail restoration process.
  • obtaining the image quality enhancement parameters includes: generating a second array, wherein the second array includes a plurality of second array elements, and the frame interpolation processing corresponds to The frame interpolation parameter is represented by at least one second array element, the resolution parameter corresponding to the super-resolution processing is represented by at least one second array element, and the color correction parameter corresponding to the color correction process is represented by at least one second array element.
  • the noise reduction parameter corresponding to the noise reduction process is represented by at least one second array element
  • the high dynamic range up-conversion parameter corresponding to the high dynamic range up-conversion process is represented by at least one second array element
  • the detail repair transformation The detail restoration parameters corresponding to the processing are represented by at least one second array element; based on the second array, the image quality enhancement parameters are determined.
  • obtaining the image quality enhancement parameters includes: obtaining an algorithm string, wherein the algorithm string includes at least one of the following algorithms: frame interpolation algorithm, super-resolution algorithm, color correction algorithm, noise reduction algorithm, high dynamic range up-conversion algorithm, and detail restoration algorithm.
  • the frame interpolation algorithm includes the name of the frame interpolation algorithm and the frame interpolation parameters.
  • the super-resolution algorithm includes the super-resolution algorithm. The name of the resolution algorithm and the super-resolution parameters.
  • the color adjustment algorithm includes The color adjustment algorithm name and the color adjustment parameters
  • the noise reduction algorithm includes the noise reduction algorithm name and the noise reduction parameters
  • the high dynamic range up-conversion algorithm includes the name of the high dynamic range up-conversion algorithm and the high dynamic range up-conversion parameter
  • the detail restoration algorithm includes the name of the detail restoration algorithm and the detail modification parameter; based on the algorithm string, the image quality enhancement parameter is determined.
  • the image used for the frame interpolation processing includes video, and the frame interpolation processing is implemented based on a first deep learning model, and the first deep learning model is Configured to add at least one transition image frame between every two image frames in the video, the super-resolution processing is implemented based on a second deep learning model, and the second deep learning model is configured to perform
  • the super-resolution processed image is subjected to super-resolution processing to improve the spatial resolution of the image used for the super-resolution processing, and the color correction process is implemented based on a third deep learning model;
  • the noise reduction process Implemented based on a fourth deep learning model, the fourth deep learning model is configured to perform noise reduction processing on the image used for the noise reduction processing.
  • the third deep learning model includes a regression sub-model and a plurality of look-up table sub-model groups, and each look-up table sub-model group includes at least one look-up table sub-model.
  • the color correction process includes: preprocessing the image used for the color correction process to obtain normalized data, wherein the preprocessing includes normalization process; using the regression sub-model to Process the normalized data to obtain at least one weight parameter; obtain a color adjustment parameter; select a lookup table submodel corresponding to the color adjustment parameter from the plurality of lookup table submodels according to the color adjustment parameter group; based on the at least one weight parameter and the look-up table sub-model group, determine a target look-up table sub-model; use the target look-up table sub-model to process the normalized data to generate the color grading process Output.
  • the first deep learning model includes a real-time intermediate flow estimation algorithm model.
  • the second deep learning model includes a residual feature distillation network model.
  • the fourth deep learning model includes a Unet network model.
  • the input image includes a video
  • the enhanced image includes an enhanced video corresponding to the video
  • the image quality enhancement is performed on the input image
  • Processing to obtain the enhanced image includes: performing the frame interpolation process on the video to obtain the interpolated frame video; performing the color correction process on the frame interpolated video to obtain The video after color correction; perform the noise reduction processing on the video after color correction to obtain the noise reduction video; perform the super-resolution processing on the noise reduction video to obtain the enhanced video, wherein , the resolution of the enhanced video is higher than the resolution of the noise-reduced video.
  • the color bit depth of the color-adjusted video is higher than the color bit depth corresponding to the frame-interpolated video, and/or the color-adjusted video
  • the corresponding color gamut is higher than the corresponding color gamut of the video after frame insertion.
  • obtaining the input image includes: obtaining an original image file; decoding the original image file to obtain the original image; and performing format conversion processing on the original image. , to obtain the input image, wherein the pixel format of the input image is RGB format.
  • the pixel format of the compensated image is RGB format
  • the image processing method further includes: performing format conversion on the compensated image to obtain an output image.
  • the pixel format of the output image is YUV format; the output image is encoded to obtain an output image file.
  • the color bit depth corresponding to the compensated image is higher than the color bit depth corresponding to the input image, and/or the color bit depth corresponding to the compensated image is The gamut is higher than the color gamut corresponding to the input image.
  • At least one embodiment of the present disclosure provides an image processing apparatus, including: one or more memories non-transiently storing computer-executable instructions; one or more processors configured to run the computer-executable instructions, wherein , when the computer-executable instructions are run by the one or more processors, they implement the image processing method according to any embodiment of the present disclosure.
  • the image processing device provided by at least one embodiment of the present disclosure further includes: an input device, wherein responding to the image processing method includes obtaining a compensation input parameter corresponding to the first display panel and/or obtaining an image quality enhancement parameter. , the compensation input parameters and/or the image quality enhancement parameters are input through the input device.
  • the input device includes a touch screen, a touch pad, a keyboard, a mouse, and a microphone.
  • At least one embodiment of the present disclosure provides a non-transitory computer-readable storage medium, wherein the non-transitory computer-readable storage medium stores computer-executable instructions, and when executed by a processor, the computer-executable instructions implement according to The image processing method according to any embodiment of the present disclosure.
  • At least one embodiment of the present disclosure provides an image processing device, including: an acquisition module, configured to acquire Obtain an input image; an image quality enhancement processing module is used to perform image quality enhancement processing on the input image to obtain an enhanced image; wherein the resolution corresponding to the enhanced image is higher than the resolution corresponding to the input image .
  • the image processing device provided by at least one embodiment of the present disclosure further includes: a compensation processing module for performing compensation processing on the enhanced image to obtain a compensated image; wherein the compensated image corresponds to a high resolution The resolution corresponding to the input image.
  • the image processing device provided by at least one embodiment of the present disclosure further includes a first display panel, wherein the compensation processing module includes: a selection sub-module and a processing sub-module, the selection sub-module is used to select from multiple compensation parameters. Select a compensation parameter corresponding to the first display panel; the processing submodule is used to perform compensation processing on the enhanced image based on the compensation parameter corresponding to the first display panel to obtain the compensation Afterimage.
  • the compensation processing module includes: a selection sub-module and a processing sub-module, the selection sub-module is used to select from multiple compensation parameters. Select a compensation parameter corresponding to the first display panel; the processing submodule is used to perform compensation processing on the enhanced image based on the compensation parameter corresponding to the first display panel to obtain the compensation Afterimage.
  • the selection sub-module includes an acquisition unit and a selection unit, the acquisition unit is used to acquire the compensation input parameter corresponding to the first display panel, wherein,
  • the compensation input parameters include panel parameters corresponding to the first display panel and/or environmental parameters of the environment where the first display panel is located; the selection unit is configured to select from the multiple parameters based on the compensation input parameters.
  • the compensation parameter corresponding to the compensation input parameter is selected as the compensation parameter corresponding to the first display panel among the compensation parameters.
  • the image quality enhancement processing includes at least one of the following processing: frame interpolation processing, super-resolution processing, noise reduction processing, and color correction processing
  • the image quality enhancement processing module includes at least one of the following sub-modules: a frame interpolation sub-module, a super-resolution sub-module, a color correction sub-module and a noise reduction sub-module.
  • the frame interpolation sub-module includes a first deep learning model, and is used The frame interpolation process is performed on the input of the frame interpolation sub-module based on the first deep learning model.
  • the input of the frame interpolation sub-module includes video, and the frame interpolation process includes every two parts of the video.
  • the super-resolution processing is used to improve the spatial resolution of the input of the super-resolution sub-module;
  • the color adjustment sub-module includes a third deep learning model, and is used to adjust the color adjustment based on the third deep learning model.
  • the input of the color sub-module performs the color adjustment process; the noise reduction sub-module includes a fourth deep learning model, and is used to perform the noise reduction on the input of the noise reduction sub-module based on the fourth deep learning model. deal with.
  • the image processing device further includes: a coding module, which , the pixel format of the compensated image is RGB format, and the encoding module is used to: perform format conversion on the compensated image to obtain an output image, wherein the pixel format of the output image is YUV format; for The output image is encoded to obtain an output image file.
  • a coding module which , the pixel format of the compensated image is RGB format, and the encoding module is used to: perform format conversion on the compensated image to obtain an output image, wherein the pixel format of the output image is YUV format; for The output image is encoded to obtain an output image file.
  • Figure 1A is a schematic flow chart of an image processing method provided by at least one embodiment of the present disclosure
  • Figure 1B is a schematic flow chart of another image processing method provided by at least one embodiment of the present disclosure.
  • Figure 2 is an overall flow chart of an image processing method provided by some embodiments of the present disclosure.
  • Figure 3 is a schematic flow chart of a frame insertion process provided by some embodiments of the present disclosure.
  • Figure 4 is a schematic flow chart of super-resolution processing provided by some embodiments of the present disclosure.
  • Figure 5 is a schematic flow chart of a color correction process provided by some embodiments of the present disclosure.
  • Figure 6 is a schematic flow chart of a noise reduction process provided by some embodiments of the present disclosure.
  • Figure 7 is a schematic flow chart of a color correction process and a noise reduction process provided by some embodiments of the present disclosure
  • Figure 8 is a schematic diagram of an image processing device provided by some embodiments of the present disclosure.
  • Figure 9 is a schematic diagram of another image processing device provided by at least one embodiment of the present disclosure.
  • Figure 10 is a schematic diagram of another image processing device provided by at least one embodiment of the present disclosure.
  • Figure 11 is a schematic diagram of a non-transitory computer-readable storage medium provided by at least one embodiment of the present disclosure
  • Figure 12 is a schematic diagram of a hardware environment provided by at least one embodiment of the present disclosure.
  • At least one embodiment of the present disclosure provides an image processing method.
  • the image processing method includes: acquiring an input image; and performing image quality enhancement processing on the input image to obtain an enhanced image.
  • the resolution corresponding to the enhanced image is higher than the resolution corresponding to the input image.
  • the input image is subjected to image quality enhancement processing to realize automatic ultra-high-definition remake of the input image, thereby generating an ultra-high-definition enhanced image and solving the shortage of ultra-high-definition film sources. issues and meet the development needs of the industry.
  • This image processing method can automatically generate ultra-high-definition videos based on low-definition film sources, reducing labor costs and film source production cycles.
  • At least one embodiment of the present disclosure also provides an image processing device and a non-transitory computer-readable storage medium.
  • the image processing method provided by the embodiment of the disclosure can be applied to the image processing device provided by the embodiment of the disclosure, and the image processing device can be configured on an electronic device.
  • the electronic device may be a personal computer, a mobile terminal, etc.
  • the mobile terminal may be a mobile phone, a tablet computer, or other hardware devices with various operating systems. That is to say, the execution subject of the image processing method can be a personal computer, a mobile terminal, etc.
  • Figure 1A is a schematic flow chart of an image processing method provided by at least one embodiment of the present disclosure.
  • Figure 1B is a schematic flow chart of another image processing method provided by at least one embodiment of the present disclosure.
  • Figure 2 is a schematic flow chart of an image processing method provided by at least one embodiment of the present disclosure. Some embodiments provide an overall flow chart of an image processing method.
  • the image processing method can be applied to the first display panel
  • the first display panel can be an organic light emitting diode (OLED) display panel (for example, an active-matrix organic light emitting diode (Active-matrix organic light emitting diode, OLED)).
  • OLED organic light emitting diode
  • AMOLED active-matrix organic light emitting diode
  • QLED quantum dot light-emitting diode
  • liquid crystal display panel etc.
  • the first display panel may have a resolution of 8K (7680 (pixels)*4320 (pixels)), that is, the first display panel may include a pixel array of 7680 (columns)*4320 (rows).
  • the resolution of the first display panel can be set according to the actual situation.
  • the first display panel can have a resolution of 4K (4096 (pixels) * 2160 (pixels)).
  • the embodiment of the present disclosure is for the first display panel.
  • the resolution of the panel is not specifically limited.
  • the first display panel may be a rectangular panel, a circular panel, an oval panel, a polygonal panel, etc.
  • the first display panel can be not only a flat panel, but also a curved panel, or even a spherical panel.
  • the first display panel can be applied to any product or component with a display function, such as mobile phones, tablet computers, televisions, monitors, notebook computers, digital photo frames, and navigators.
  • a display function such as mobile phones, tablet computers, televisions, monitors, notebook computers, digital photo frames, and navigators.
  • the first display panel may also include a projector.
  • the image processing method provided by embodiments of the present disclosure includes steps S10 to S11. As shown in FIG. 1B , in other embodiments, the image processing method provided by the embodiment of the present disclosure includes steps S10 to S12.
  • step S10 an input image is acquired.
  • the input images may include videos and/or pictures.
  • Videos can be various types of videos, and pictures can be various types of pictures. Taking video as an example, if the objects in the video are landscapes, people, etc., then the video is a landscape video, people video, etc.
  • the video can also be surveillance video, animal and plant video, etc.
  • the shape of the picture can be any suitable shape such as a rectangle.
  • the picture can be a static image or a dynamic image.
  • the shape and size of the picture can be set by the user according to the actual situation.
  • the embodiments of the present disclosure do not Specific restrictions.
  • the resolution of the video can be set by the user according to actual conditions, and is not specifically limited in the embodiments of the present disclosure.
  • the embodiments of the present disclosure do not place specific restrictions on the type, resolution and other properties of videos/pictures.
  • the image processing method provided by the present disclosure is a video processing method
  • the image processing method provided by the present disclosure is an image processing method
  • the input image can be acquired through an image acquisition device.
  • the image capture device may include a video camera, a camera, etc.
  • the camera may include a smartphone camera, a tablet camera, a personal computer camera, a digital camera lens, or even a webcam.
  • the input image can be a grayscale image or a color image.
  • the input image may be an image directly collected by the image collection device, or may be an image obtained after preprocessing the image directly collected by the image collection device.
  • the image processing method provided by the embodiment of the present disclosure may also include: The process of preprocessing the collected images directly. Preprocessing can eliminate irrelevant information or noise information in images directly collected by the image acquisition device, thereby facilitating better processing of the input image during subsequent image processing.
  • preprocessing may include performing one or more of data augmentation processing, scaling processing, and gamma correction on images directly collected by the image collection device. Expansion processing includes expanding image data through random cropping, rotation, flipping, skewing, etc.
  • the scaling process includes proportionally scaling the images directly collected by the image acquisition device and cropping them into preset sizes to facilitate subsequent processing operations.
  • step S10 may include: obtaining an original image file; decoding the original image file to obtain an original image; and performing format conversion on the original image to obtain an input image.
  • the original image file is decoded to obtain the original image, and then, the original image is format converted to obtain the input image.
  • the pixel format of the original image is YUV format
  • the pixel format of the input image is RGB format
  • the color depth (color bit depth) of the input image is 8 bits
  • the color gamut of the input image is BT. .709
  • the frame rate of the input image can be 25fps (frames per second, the number of frames transmitted per second)
  • the resolution of the input image can be 1920 (pixels) * 1080 (pixels).
  • the color depth, color gamut, frame rate and resolution of the input image can be set according to the actual situation, and there are no specific restrictions here.
  • the types of pixel formats of the video stream may include YUV420 pixel format, YUV420 10-bit pixel format, YUV422 pixel format, YUV422P10 pixel format, RGB24 pixel format, BGR24 pixel format, etc.
  • the pixel format of the image of the present disclosure can be any of the above pixel formats, and the present disclosure does not limit this.
  • step S11 image quality enhancement processing is performed on the input image to obtain an enhanced image.
  • the enhanced image corresponds to a higher resolution than the input image.
  • the color bit depth corresponding to the enhanced image is higher than the color bit depth corresponding to the input image, and/or the color gamut corresponding to the enhanced image is higher than the color gamut corresponding to the input image.
  • step S11 includes: obtaining image quality enhancement parameters; performing image quality enhancement processing on the input image according to the image quality enhancement parameters to obtain an enhanced image.
  • the image quality enhancement processing includes at least one of the following processing: frame interpolation processing, super-resolution processing, noise reduction processing, color correction processing, high dynamic range (HDR) up-conversion processing, detail restoration processing, etc.
  • the color grading process may be an HDR color grading process.
  • the image quality enhancement parameters include at least one of the following parameters: the frame interpolation algorithm name and/or frame interpolation parameters corresponding to the frame interpolation processing, the super-resolution algorithm name and/or resolution parameters corresponding to the super-resolution processing, adjustment The name of the color correction algorithm and/or color correction parameters corresponding to the color processing, the name of the noise reduction algorithm and/or the noise reduction parameters corresponding to the noise reduction process, etc., the high dynamic range up-conversion algorithm and/or the high dynamic range up-conversion algorithm corresponding to the high dynamic range up-conversion process. Dynamic range up-conversion parameters, detail restoration algorithm names and/or detail restoration parameters corresponding to the detail restoration processing.
  • the image quality enhancement parameters can be set by the user input through the input device, so that they can be set according to the user's needs to meet the different needs of the user and satisfy the needs of different users.
  • various processes in image quality enhancement processing can be flexibly configured, and parameters can be set accordingly according to needs.
  • one or more of the frame interpolation processing, super-resolution processing, noise reduction processing, and color grading processing may not be performed, or the frame interpolation processing, super-resolution processing, noise reduction processing, and color grading processing may all be performed. implement.
  • obtaining the image quality enhancement parameters includes: generating a second array, wherein the second array includes a plurality of second array elements, and the frame interpolation parameter corresponding to the frame interpolation process is composed of at least one first Two array elements are represented, the resolution parameter corresponding to the super-resolution process is represented by at least one second array element, the color correction parameter corresponding to the color correction process is represented by at least one second array element, and the noise reduction parameter corresponding to the noise reduction process is represented by at least one second array element.
  • a second array element represents that the high dynamic range up-conversion parameter corresponding to the high dynamic range up-conversion process is represented by at least one second array element, and the detail repair parameter corresponding to the detail repair transformation process is represented by at least one second array element; based on the first Two arrays to determine the image quality enhancement parameters.
  • the image quality enhancement parameters may be defined in an integer array format, that is, the second array is an integer array, the second array includes a plurality of second array elements, and each second array element corresponds to a meaning. Whether the corresponding processing needs to be executed can be determined based on the value of each second array element. For example, taking the frame interpolation process as an example, if the value of the second array element representing the frame interpolation parameter corresponding to the frame interpolation process is the first value, it means that the frame interpolation process is not performed; if the value of the second array element representing the frame interpolation parameter corresponding to the frame interpolation process is If the value of the second array element is the second numerical value, it means that frame interpolation processing is performed. For example, in some embodiments, the first value is 0 and the second value is 1, however the disclosure is not limited thereto.
  • the second array element 0 in the second array (ie, the first element in the second array) represents the frame interpolation parameter corresponding to the frame interpolation process
  • the second array element 1 in the second array and 2 (that is, the second element and the third element in the second array) represent the resolution parameters corresponding to the super-resolution processing
  • the second array elements 3 and 4 in the second array (that is, the second element in the second array)
  • the four elements and the fifth element) represent the color correction parameters corresponding to the color correction process
  • the second array element 5 i.e., the sixth element in the second array
  • the noise reduction parameters corresponding to the noise reduction process .
  • each second array element in the second array corresponding to the image quality enhancement parameter is as follows:
  • the second array element 0 indicates whether to turn on the frame interpolation process. When the second array element 0 is 0, it means that the frame interpolation process is not turned on. When the second array element 0 is 1, it means that frame interpolation processing is turned on; the second array element 1 means whether super-resolution processing is turned on. When the second array element 1 is 0, it means that super-resolution processing is not turned on. When the second array element 1 is 0, it means that super-resolution processing is not turned on.
  • Array element 1 is 1, which means super-resolution processing is turned on;
  • second array element 2 means super-resolution magnification, when second array element 2 is 0, it means super-resolution magnification is 2 times super-resolution (length and width are increased by two times). times), when the second array element 2 is 1, it means that the super-resolution magnification is 4 times the super-resolution ratio, which is set according to the actual situation;
  • the second array element 3 indicates whether to turn on the color correction process, when the second array element 3 is 0 , it means that the color grading process is not turned on.
  • the second array element 3 is 1, it means that the color grading process is turned on.
  • the second array element 4 means the color grading style.
  • the second array element 4 When the second array element 4 is 0, it means the color grading process.
  • the style is the default style.
  • the second array element 4 When the second array element 4 is 1, it means that the color style is a warm style.
  • the second array element 4 When the second array element 4 is 2, it means that the color style is a cool style, etc., depending on the actual situation.
  • Setting; the second array element 5 indicates whether to enable noise reduction processing. When the second array element 5 is 0, it indicates that the noise reduction processing is not enabled. When the second array element 5 is 1, it indicates that the noise reduction processing is enabled.
  • the number of second array elements in the second array corresponding to the image quality enhancement parameter, the meaning of each second array element, etc. can be set according to specific circumstances, and this disclosure does not limit this.
  • each second array element in the second array can be preset to a default value according to the actual situation.
  • obtaining the image quality enhancement parameters includes: obtaining the image quality enhancement parameters includes: obtaining an algorithm string; and determining the image quality enhancement parameters based on the algorithm string.
  • the algorithm string may include at least one of the following algorithms: frame interpolation algorithm, super-resolution algorithm, color correction algorithm, noise reduction algorithm, high dynamic range up-conversion algorithm, and detail restoration algorithm.
  • the frame interpolation algorithm includes the frame interpolation algorithm name and frame interpolation parameters
  • the super-resolution algorithm includes the super-resolution algorithm name and super-resolution parameters
  • the color correction algorithm includes the color correction algorithm name and color correction parameters
  • the noise reduction algorithm includes the noise reduction algorithm name.
  • the high dynamic range up-conversion algorithm includes the name of the high dynamic range up-conversion algorithm and the high dynamic range up-conversion parameters
  • the detail restoration algorithm includes the name of the detail restoration algorithm and the detail modification parameters.
  • the algorithm string is represented as: ./sr_sdk–i test_video/1080_25.mp4–m frc_1080:denoise_1080:sr1 ⁇ _1080–c 0–p 0–o out.mp4
  • sr_sdk represents the executable file
  • the character test_video/1080_25.mp4 followed by -i represents the video to be tested (for example, the input image in this disclosure)
  • the character followed by -m represents the selected image quality.
  • Algorithms related to enhancement processing Multiple algorithms can be superimposed, and each algorithm is separated by ":”.
  • frc_1080:denoise_1080:sr1 ⁇ _1080 represents three algorithms
  • frc_1080 represents the frame interpolation algorithm.
  • frc represents the name of the frame interpolation algorithm
  • 1080 represents the parameters of the frame interpolation algorithm
  • denoise_1080 represents the denoising algorithm, where denoise represents the name of the denoising algorithm
  • 1080 represents the parameters of the denoising algorithm
  • sr1 ⁇ _1080 represents the super-resolution algorithm, where , sr represents the name of the super-resolution algorithm
  • 1080 represents the parameters of the super-resolution algorithm
  • the character following -c represents the video encoder.
  • the character following -c is 0, which represents h264 encoding
  • -p The characters following represent the pixel format when encoding.
  • the character following -p is 0, which represents the yuv420 pixel format
  • the characters following -o represent the video file obtained after processing the video to be tested.
  • the storage path and file name can support .mp4, .avi and other formats.
  • the parameters of each algorithm are related to the resolution of the video to be tested.
  • the resolution of the video to be tested is 1080
  • the parameters of each algorithm are also 1080.
  • the called algorithm may include: 1) -m sr1 ⁇ _1080/sr1 ⁇ _540, which represents the algorithm for single-frame super-resolution processing; 2) -m sr3 ⁇ _1080/sr3 ⁇ _540, which represents EDVR (Video Restoration with Enhanced Deformable Convolutional Networks) algorithm; 3) -m sr_1080/sr_540, which represents the RFDN (residual feature distillation network) algorithm; 4) -m denoise_1080/denoise_540, which represents the small model noise reduction algorithm; 5) -m denoise3 ⁇ _1080/ denoise3 ⁇ _540, which represents the large model denoising algorithm; 6)-m detail_1080, which Represents the detail enhancement algorithm (for example, detail repair processing); 7) -m hdr_lut_2160/hdr_lut_1080/hdr_lut_540, which represents the small model HDR algorithm
  • the character after -c when the character after -c is 1, it means h265 encoding; when the character after -c is 2, it means mpeg-1 encoding; when the character after -c is 3, it means mpeg-2 encoding; when the character after -c is 3, it means mpeg-2 encoding; when the character after -c is 2, it means mpeg-1 encoding; When the character is 4, it means mpeg-4 encoding; when the character after -c is 5, it means wmv7 encoding; when the character after -c is 6, it means wmv8 encoding.
  • the character after -p when the character after -p is 1, it indicates the yuv422 pixel format. It should be noted that by default, the character after -p is 0.
  • the image used for frame interpolation processing includes a video
  • the frame interpolation processing is implemented based on a first deep learning model
  • the first deep learning model is configured to add between every two image frames in the video. At least one transition image frame.
  • the first deep learning model includes a deep learning-based real-time intermediate flow estimation algorithm (RIFE) model.
  • RIFE real-time intermediate flow estimation algorithm
  • the transition image frames generated by the deep learning-based RIFE model have fewer artifacts, which makes the display effect of the video obtained by interpolating the frames better.
  • the RIFE model runs fast, is suitable for improving the frame rate of videos, and can also enhance visual quality.
  • the RIFE model is capable of end-to-end training and achieves excellent performance.
  • the first deep learning model can also adopt DAIN (Depth-Aware Video Frame Interpolation) model, ToFLOW (Video Enhancement with Task-Oriented Flow) model and other models.
  • DAIN Depth-Aware Video Frame Interpolation
  • ToFLOW Video Enhancement with Task-Oriented Flow
  • the embodiments of the present disclosure do not limit the specific type of the first deep learning model, as long as the first deep learning model can implement frame interpolation processing.
  • Figure 3 is a schematic flow chart of a frame insertion process provided by some embodiments of the present disclosure.
  • the example shown in Figure 3 is a flow chart of frame interpolation processing based on the RIFE model.
  • the two image frames I0 and I1 and the target time t (0 ⁇ t ⁇ 1) are input to the IFNet model.
  • the IFNet module uses To directly estimate the intermediate optical flow, that is, the transition image frame between image frames I0 and I1 is relative to the intermediate optical flow of image frames I0 and I1.
  • the IFNet module can directly and efficiently estimate the intermediate optical flow F_t ⁇ 0, and then use the linear motion assumption to approximate the intermediate optical flow F_t ⁇ 1.
  • the intermediate optical flow F_t ⁇ 1 is expressed as follows:
  • the intermediate optical flows F_t ⁇ 0 and F_t ⁇ 1 and the image frames I0 and I1 are input to the spatial deformation (backwarping) model.
  • the backwarping model is used to estimate the intermediate optical flows F_t ⁇ 0 and F_t ⁇ based on the method based on backward warping. 1 Perform spatial deformation (warp) processing on image frames I0 and I1, thereby obtaining two rough intermediate image frames I_0 ⁇ t and I_1 ⁇ t.
  • the intermediate optical flows F_t ⁇ 0 and F_t ⁇ 1, the image frames I0 and I1, and the intermediate image frames I_0 ⁇ t and I_1 ⁇ t are input to the fusion processing (Fusion processing) model to perform fusion processing to generate the transition image frame It.
  • the fusion processing model is implemented using an encoder-decoder architecture similar to FusionNet.
  • a fusion map and residual terms are first estimated based on the intermediate optical flows F_t ⁇ 0 and F_t ⁇ 1 and image frames I0 and I1, and then, the intermediate image frames I_0 ⁇ t and I_1 ⁇ t are performed based on the fusion map.
  • the linear combination is combined and added to the residual term to obtain the reconstructed transition image frame It.
  • M is the fusion map obtained by fusing the two intermediate image frames I_0 ⁇ t and I_1 ⁇ t
  • ⁇ is the residual term used to refine the image details
  • ⁇ is the element-wise multiplication sign.
  • the image quality enhancement process does not include frame interpolation processing.
  • super-resolution processing is implemented based on a second deep learning model, and the second deep learning model is configured to perform super-resolution processing on images used for super-resolution processing to improve the performance of super-resolution processing.
  • Resolution The spatial resolution of the processed image.
  • the second deep learning model includes a deep learning-based residual feature distillation network (RFDN) model.
  • the RFDN model has a small number of parameters, is fast, and has high accuracy, but can achieve a peak signal-to-noise ratio (PSNR) similar to very large models such as EDVR (Video Restoration with Enhanced Deformable Convolutional Networks, a video restoration network based on deformable convolution). )performance.
  • PSNR peak signal-to-noise ratio
  • the second deep learning model may also adopt a model such as the EDVR model.
  • the embodiments of the present disclosure do not limit the specific type of the second deep learning model, as long as the second deep learning model can implement super-resolution processing.
  • Figure 4 is a schematic flow chart of super-resolution processing provided by some embodiments of the present disclosure.
  • the example shown in Figure 4 is a flow chart for super-resolution processing based on the RFDN model.
  • the RFDN model is configured to perform super-resolution processing on the image input1 used for super-resolution processing to obtain the super-resolution processed image output1.
  • the RFDN model can include a sequentially connected convolution layer Conv11, multiple residual distillation modules RFDB (residual feature distillation block), convolution layer Conv12, convolution layer Conv13, convolution layer Conv14 and pixel shuffle layer.
  • RFDB residual distillation module
  • the multiple residual distillation modules include the first residual distillation module RFDB1, the second residual distillation module RFDB2, and the third residual distillation module RFDB3.
  • the RFDN model does not Limited to the specific structure shown in Figure 4, the RFDN model may include more or less residual distillation modules, and embodiments of the present disclosure do not impose specific limitations on this.
  • each residual distillation module RFDB includes multiple 1*1 convolution layers (1*1 convolution kernel), multiple SRB (shallow residual block, shallow residual block) modules and a 3*3 convolution layer (3*3 convolution kernel), the number of 1*1 convolutional layers is the same as the number of SRB modules, and they correspond one to one.
  • the 1*1 convolutional layer is used for feature distillation, thereby significantly reducing the number of parameters.
  • Multiple SRB modules are used for feature extraction.
  • Each SRB module consists of a 3*3 convolution kernel, residual connection and activation unit (ReLU). )composition.
  • the SRB module can benefit from residual learning without introducing any additional parameters.
  • the SRB module can achieve deeper residual connections and can better utilize the power of residual learning while being lightweight enough.
  • 3*3 convolutional layers can better refine features.
  • the features output by multiple residual distillation modules RFDB are concatenated and processed through the convolution layer Conv12 to reduce the number of features.
  • the features output by the first residual distillation module RFDB1, the features output by the second residual distillation module RFDB2, and the features output by the third residual distillation module RFDB3 are connected (concatenate) and then input to the volume. Layer Conv12 for processing.
  • the convolution layer Conv11 can use a 3*3 convolution kernel for convolution processing
  • the convolution layer Conv12 can use a 1*1 convolution kernel for convolution processing
  • the convolution layer Conv13 can use A 3*3 convolution kernel is used for convolution processing.
  • the convolution layer Conv14 can use a 3*3 convolution kernel for convolution processing.
  • the main function of the pixel reorganization layer PS is to obtain a high-resolution feature map through convolution and multi-channel reorganization of low-resolution feature maps. That is, the pixel reorganization layer PS is used to achieve the operation of improving spatial resolution.
  • Figure 5 is a schematic flow chart of a color correction process provided by some embodiments of the present disclosure.
  • the color grading process is implemented based on a third deep learning model.
  • the third deep learning model includes a regression sub-model and a plurality of look-up table sub-model groups, and each look-up table sub-model group includes at least one look-up table sub-model.
  • color correction processing includes: preprocessing the image used for color correction processing to obtain normalized data, where the preprocessing includes normalization processing; using a regression sub-model to process the normalized data to obtain normalized data.
  • Obtain at least one weight parameter obtain the color adjustment parameter; select a lookup table submodel group corresponding to the color adjustment parameter from multiple lookup table submodels according to the color adjustment parameter; determine based on at least one weight parameter and the lookup table submodel group Target lookup table submodel; the target lookup table submodel is used to process the normalized data to generate the output of the color correction process.
  • multiple lookup table sub-model groups can be trained based on the color grading style of a colorist who has accumulated color grading experience. Multiple lookup table sub-model groups correspond to multiple different color styles, thereby meeting different color matching style requirements.
  • the color grading style can be determined based on the color grading parameters, thereby selecting a lookup table sub-model group corresponding to the color grading style corresponding to the color grading parameter from a plurality of look-up table sub-model groups.
  • the color adjustment parameters can be set by the user according to actual needs and input into the equipment or device that executes the image processing method.
  • the third deep learning model is configured to perform color correction processing on the image input2 used for color correction processing to obtain a color correction processed image output2.
  • the image input2 used for color grading processing may be a standard dynamic range (SDR) image.
  • the image input2 is preprocessed to obtain normalized data.
  • the preprocessing includes normalization processing, that is, the pixel values of all pixels of the image input2 are normalized so that for subsequent model processing.
  • the regression sub-model is used to process the normalized data to obtain at least one weight parameter (W1, W2, ..., Wn, n is a positive integer).
  • W1, W2, ..., Wn, n is a positive integer.
  • a set of lookup table sub-models corresponding to the grading parameters input by the user may be selected from the plurality of look-up table sub-models.
  • the target look-up table sub-model is determined.
  • the lookup table submodel group can include lookup table submodel 3D LUT1, lookup table submodel 3D LUT2,..., lookup table submodel 3D LUTN.
  • each lookup table submodel is multiplied by the corresponding weight parameter to obtain the multiplication result, and then the multiplication results corresponding to all lookup table submodels are added to obtain the target lookup table submodel 3D LUT.
  • the target lookup table sub-model is used to process the normalized data to generate the output output2 of the color correction process.
  • the preprocessing can also include format conversion processing.
  • the image input2 is converted into a nonlinear form by referring to the nonlinear conversion function of the HLG (Hybrid Log Gamma) system in the standard GY/T 351-2018.
  • the RGB data is then normalized on the nonlinear RGB data to obtain normalized data.
  • the target lookup table submodel is used to process the normalized data to obtain the output of the target lookup table submodel.
  • the output of the target lookup table submodel is nonlinear RGB data.
  • the output of the model is post-processed (postprocess) to generate the output output2 of the color correction process.
  • the post-processing can include format conversion processing, that is, the output of the target lookup table sub-model is transformed based on the regulations of GY/T 351-2018 to obtain the color correction process.
  • the output output2 of the color processing can be the YUV format.
  • the output output2 of the color correction process can also be processed to obtain an image with a color depth of 10 bits and a color gamut of BT.2020.
  • the pixel format of this image is also in YUV format.
  • Figure 6 is a schematic flow chart of a noise reduction process provided by some embodiments of the present disclosure.
  • the example shown in Figure 6 is a flow chart of noise reduction processing based on the Unet network model.
  • the noise reduction process is implemented based on a fourth deep learning model, and the fourth deep learning model is configured to perform noise reduction processing on the image used for noise reduction processing.
  • the fourth deep learning model includes a Unet network model, and is used to suppress the noise of the entire image, that is, to perform noise reduction processing on the noise of the entire image.
  • the fourth deep learning model is configured to perform denoising processing on the image input3 used for denoising processing to obtain the denoising processed image output3.
  • the fourth deep learning model may include multiple convolutional layers, multiple residual blocks, multiple deconvolutional layers, and multiple connection layers.
  • the multiple convolutional layers include convolutional layers Conv21, Convolution layer Conv22 and convolution layer Conv23.
  • Multiple deconvolution layers include deconvolution layer Dconv21, deconvolution layer Dconv22 and deconvolution layer Dconv23.
  • Multiple residual blocks include residual block RB1 and residual block RB2.
  • residual block RB3, residual block RB4, residual block RB5, residual block RB6, and the plurality of connection layers include connection layer Ct1 and connection layer Ct2.
  • the fourth deep learning model includes convolution layer Conv21, residual block RB1, convolution layer Conv22, residual block RB2, convolution layer Conv23, residual block RB3, residual block RB4, which are connected in sequence.
  • Deconvolution layer Dconv21, residual block RB5, deconvolution layer Dconv22, residual block RB6, deconvolution layer Dconv23, and connection layer Ct1 are used to map the output of the convolution layer Conv22 and the output of the deconvolution layer Dconv22 connection, and output the result of the mapping connection to the residual block RB6.
  • the connection layer Ct2 is used to map and connect the output of the convolution layer Conv23 and the output of the deconvolution layer Dconv21, and output the result of the mapping connection to the residual block.
  • each convolutional layer can perform a convolution operation, that is, extract features, and the convolution stride of each convolutional layer can be 2 to perform downsampling.
  • Each deconvolution layer is used to perform a deconvolution operation.
  • the deconvolution step (stride) of each deconvolution layer can also be 2 to perform upsampling.
  • the forward propagation process of the convolution layer corresponds to deconvolution.
  • the backpropagation process of the convolutional layer corresponds to the forward propagation process of the deconvolutional layer.
  • the deconvolution operation can deconvolve the feature maps obtained from each layer to obtain a visualized image. In other words, the deconvolution operation can convert the feature information of the image from the feature map space to the pixel space.
  • the deconvolution operation can deconvolve the feature map output by the convolution layer to obtain the deconvolution result.
  • the deconvolution result can display the feature information extracted by the convolution layer.
  • connection layer that is, shallow features and deep features are connected for compensation, thereby reducing the loss of spatial information caused by compression and downsampling.
  • connection can be, for example, a concatenate operation, that is, the feature maps with the same size in the convolution layer and the deconvolution layer are connected through memory mapping (so that the vectors corresponding to the features are merged, and the vectors of the layer where the features are located are merged. The number of channels is doubled).
  • each convolutional/deconvolutional layer is transmitted to a residual block, which ensures that feature information of different scales flows between layers.
  • Figure 7 is a schematic flow chart of a color correction process and a noise reduction process provided by some embodiments of the present disclosure.
  • the image input4 is preprocessed to obtain normalized data.
  • the preprocessing includes normalization processing, that is, the pixel values of all pixels of the image input4 are normalized so that for subsequent model processing.
  • the regression sub-model is used to process the normalized data to obtain at least one weight parameter (W1, W2, ..., Wn, n is a positive integer).
  • W1, W2, ..., Wn, n is a positive integer.
  • the lookup table sub-model group 3D LUT1 ⁇ 3D LUTN corresponding to the color grading parameters input by the user can be selected from multiple look-up table sub-models.
  • the target lookup table submodel 3D LUT is determined.
  • the normalized data is then processed using the target lookup table submodel 3D LUT to generate the output of the color correction process.
  • the fourth deep learning model shown in Figure 6 is used to perform noise reduction processing on the output of the color correction process to obtain the output output4 of the noise reduction processing.
  • the preprocessing may also include format conversion processing.
  • the fourth deep learning model shown in Figure 6 is used to perform noise reduction processing on the output of the color grading process to obtain the output of the fourth deep learning model, and then the fourth deep learning model The output is post-processed to obtain the output output4 of the noise reduction process.
  • the post-processing can include format conversion processing, that is The output of the fourth deep learning model is format transformed to obtain the output output4 of the noise reduction process.
  • image quality enhancement processing is used to sequentially perform frame interpolation processing, color correction processing, noise reduction processing, super-resolution processing, etc. on the input image, so that the resolution of the enhanced image reaches 4K or 8K, etc. , the frame rate of the enhanced image reaches 50-60fps, the color depth (color bit depth) of the enhanced image reaches 10-12 bits, the color gamut of the enhanced image is BT.2020, and the enhanced image has a certain color style.
  • step S11 may include: performing frame interpolation processing on the video (input image) to obtain the interpolated video; performing color correction processing on the interpolated video to obtain the color-corrected video. Video; perform noise reduction processing on the color-corrected video to obtain the noise-reduced video; perform super-resolution processing on the noise-reduced video to obtain the enhanced video.
  • the frame interpolation process can be performed on the video based on the frame interpolation parameters corresponding to the frame interpolation process input by the user to obtain the interpolated frame video;
  • the post-frame interpolation video can be obtained based on the color correction parameters corresponding to the color correction process input by the user.
  • Color correction processing to obtain the color correction video; noise reduction processing can be performed on the color correction video based on the noise reduction parameters corresponding to the noise reduction processing input by the user to obtain the noise reduction video;
  • the super-resolution video can be obtained based on the super resolution input by the user. Perform super-resolution processing on the denoised video using the resolution parameters corresponding to the rate processing to obtain the enhanced video.
  • the resolution of the enhanced video is higher than the resolution of the denoised video.
  • the color bit depth of the video after color correction is higher than the color bit depth of the video after frame insertion, and/or the color gamut of the video after color correction is higher than the color gamut of the video after frame insertion.
  • the resolution of the enhanced video can be 4K or 8K
  • the frame rate of the enhanced video can reach 50-60fps
  • the color depth of the enhanced video can reach 10-12 bits
  • the color gamut of the enhanced video can be BT. 2020.
  • the deep learning models are all implemented under the tensorRT framework.
  • deep learning models need to be initialized before performing relevant processing. Initialization is to set up and import all deep learning models, parameters, drivers and other environments when the program starts to prepare for program running, thereby avoiding repeated calls to relevant configuration modules during program running, resulting in slower processing. This increases processing speed.
  • an enhanced image can be obtained after image quality enhancement processing is performed on the input image.
  • the enhanced image will be sent to, for example, the first display panel for display.
  • the material and display of the first display panel There are various display characteristics, etc., and the ambient light of the environment where the first display panel is located is also different, which will cause the display effect of the first display panel to not reach the expected state.
  • the enhanced image can be compensated to perform secondary enhancement on the content of the enhanced image, so that the display of the image can reach a better state under different conditions and satisfy users. display requirements.
  • step S12 a compensation process is performed on the enhanced image to obtain a compensated image.
  • the resolution corresponding to the compensated image is higher than the resolution corresponding to the input image.
  • the resolution corresponding to the compensated image can be ultra-high-definition resolutions such as 4K and 8K.
  • the frame rate of the compensated image can be 50-60fps
  • the color depth of the compensated image can be 10-12 bits
  • the color gamut of the compensated image can be BT.2020.
  • the input image is subjected to image quality enhancement processing and compensation processing to achieve secondary enhancement of the input image, so that the generated compensated image can adapt to different ambient lights and displays.
  • image quality enhancement processing and compensation processing to achieve secondary enhancement of the input image, so that the generated compensated image can adapt to different ambient lights and displays.
  • Normal display under screen conditions adapts to more application scenarios, and has excellent display effects in different application scenarios.
  • AI Artificial Intelligence
  • the image quality enhancement process is first performed, and after the image quality enhancement process is performed, the compensation process is performed.
  • the color bit depth corresponding to the compensated image is higher than the color bit depth corresponding to the input image, and/or the color gamut corresponding to the compensated image is higher than the color gamut corresponding to the input image.
  • the compensation process includes at least one of the following: brightness adjustment, contrast adjustment, saturation adjustment, etc., that is to say, in the compensation process, the brightness, contrast, saturation, etc. of the enhanced image can be compensated and adjusted to obtain compensation Afterimage.
  • the compensation process includes brightness adjustment, contrast adjustment and saturation adjustment.
  • the brightness adjustment and contrast adjustment may be composed of mapping curves of different parameters, such as exponential function (exp) curve, logarithmic function (log) curve, sigmoid (S-shaped function) curve, polynomial curve, etc.
  • Saturation adjustment is parameter adjusted and calculated by HSV (Hue, Saturation, Value) color space.
  • each compensation curve corresponds to a panel parameter and environmental parameter, which are composed of a combination of brightness, contrast, and saturation algorithms.
  • the compensation parameters may correspond to the compensation curve.
  • step S12 includes: selecting the first display parameter from a plurality of compensation parameters. Compensation parameters corresponding to the display panel; based on the compensation parameters corresponding to the first display panel, compensation processing is performed on the enhanced image to obtain a compensated image.
  • selecting a compensation parameter corresponding to the first display panel from a plurality of compensation parameters includes: obtaining a compensation input parameter corresponding to the first display panel, wherein the compensation input parameter includes Panel parameters corresponding to the first display panel and/or environmental parameters of the environment in which the first display panel is located; based on the compensation input parameter, select the compensation parameter corresponding to the compensation input parameter from a plurality of compensation parameters as the compensation parameter corresponding to the first display panel compensation parameters.
  • the panel parameters include the type of the first display panel, the size of the first display panel, the display mode of the first display panel, and so on.
  • the type of the first display panel includes a light emitting display (LED) panel, a liquid crystal display (LCD) panel, etc.
  • the light emitting display panel may include an organic light emitting diode display panel, a quantum dot light emitting diode display panel, etc.
  • the display modes of the first display panel include direct display and screen projection display.
  • the size of the first display panel may be 15 inches, 17 inches, 19 inches, 20 inches, 24 inches, 32 inches, etc.
  • the environmental parameters include ambient light parameters, etc.
  • the ambient light parameters are determined based on the brightness value of the ambient light of the environment where the first display panel is located.
  • the ambient light parameter may also be determined based on the color of the ambient light of the environment where the first display panel is located.
  • each process in the compensation process (for example, the above-mentioned adjustment of brightness, adjustment of contrast, adjustment of saturation, etc.) can be flexibly configured, and parameters can be set accordingly according to needs.
  • one or more of adjusting brightness, adjusting contrast, and adjusting saturation may not be performed, or all of adjusting brightness, adjusting contrast, and adjusting saturation may be performed.
  • step S12 obtaining the compensation input parameters corresponding to the first display panel includes: generating a first array, wherein the first array corresponds to the first display panel, and the first array includes a plurality of The first array element, the panel parameter is represented by at least one first array element, and the environmental parameter is represented by at least one first array element; based on the first array, the compensation input parameter corresponding to the first display panel is determined.
  • the compensation input parameter may be defined in an integer array format, that is, the first array is an integer array, the first array includes a plurality of first array elements, and each first array element corresponds to a meaning. Whether the corresponding processing needs to be executed can be determined based on the value of each first array element. For example, taking brightness adjustment as an example, if the value of the first array element representing brightness adjustment is the first value, it means that the process of adjusting brightness is not performed; if the value of the first array element representing brightness adjustment is the second value, then Indicates execution of brightness adjustment processing. For example, in some embodiments, the first numerical value is 0 and the second numerical value is 1, however, this disclosure Opening is not limited to this.
  • the first array element 0 in the first array represents the panel parameter
  • the first array element 1 in the first array i.e., the first element in the first array
  • the second element represents the ambient light parameter in the environment parameters.
  • the definition of each first array element in the first array corresponding to the compensation input parameter is as follows:
  • the first array element 0 represents the mode of the display panel (ie, panel parameter). When the first array element 0 is 0, it means that the display panel is LED panel, when the first array element 0 is 1, it means that the size of the display panel is greater than or equal to the size threshold. When the first array element 0 is 2, it means that the display mode of the display panel is projection display.
  • the first array element 1 represents the ambient light parameter. When the first array element 1 is 0, it means that the brightness value of the ambient light is less than 1000 lumens. When the first array element 1 is 1, it means that the brightness value of the ambient light is between 1000 and 2000. between lumens; when the first array element 1 is 2, it means that the brightness value of the ambient light is between 2000 and 3000 lumens; etc., set according to the actual situation.
  • the number of first array elements in the first array corresponding to the compensation input parameter, the meaning of each first array element, etc. can be set according to specific circumstances, and this disclosure does not limit this.
  • each first array element in the first array can be preset with a default value.
  • the default value of the first array element representing the panel parameter may be preset according to the characteristics of the display panel itself.
  • the default value of the first array element representing the ambient light parameter can be set in advance, or a sensor can be set in the display panel to sense the brightness of the ambient light of the environment where the display panel is located, so as to determine the ambient light based on the ambient light parameter. The sensed brightness automatically sets the value of the first array element representing the ambient light parameter.
  • the present disclosure is not limited thereto, and the user can also turn off the function of the sensor, thereby determining the value of the first array element representing the ambient light parameter according to the user's input.
  • the size threshold can be set according to actual needs.
  • the size threshold can be 24 inches.
  • the image processing method may further include: generating multiple compensation parameters.
  • multiple compensation parameters can be generated in advance under various situations, so that during actual application, the corresponding compensation parameters can be selected according to the actual situation.
  • the compensation parameters in each case correspond to one display panel and one ambient light brightness value.
  • generating multiple compensation parameters includes: obtaining a color card image obtained by shooting a standard color card; performing image quality enhancement processing on the color card image to obtain an enhanced color card image; and obtaining an enhanced color card image based on the initial compensation parameters.
  • the parameter is used as one compensation parameter among multiple compensation parameters; the compensation input parameter corresponding to the second display panel is determined, wherein the compensation input parameter corresponding to the second display panel includes the panel parameter corresponding to the second display panel and/or the second display Environmental parameters of
  • the standard color card can be a 24-color card.
  • the preset requirement can indicate whether the color card image is displayed normally after multiple observers observe and compensate.
  • the number of multiple observers can be 4 to 6, and the number of males and females among the multiple observers is multiple observers. half of the number of people.
  • an observer observes that the compensated color card image displays normally it means that the number of color card grids in the compensated color card image that the observer can distinguish is greater than 18 grids.
  • the present disclosure is not limited to this.
  • the number of multiple observers can be more or less.
  • the normal display of the compensated color card image by observers means that the number of color card grids in the compensated color card image that the observer can distinguish can also be greater than 16, 17, 19, 20 grids, etc.
  • the compensation parameters can be matched one-to-one with the corresponding panel parameters and/or environmental parameters, thereby facilitating the selection of required compensation parameters based on the panel parameters and/or environmental parameters during later use.
  • the first display panel and the second display panel may be the same display panel or may be different display panels.
  • multiple different display panels for example, different types, sizes, display methods, etc.
  • Corresponding compensation parameters (these compensation parameters correspond to the same ambient light).
  • the same display panel can also be used to display the compensated color card images under different ambient lights, thereby obtaining compensation parameters corresponding to different ambient lights (these compensation parameters correspond to the same display panel).
  • the panel parameters and environmental parameters of the display panel can also be changed at the same time.
  • the pixel format of the compensated image is RGB format.
  • the image processing method also includes: converting the format of the compensated image to obtain an output image; encoding the output image to obtain an output image file.
  • the pixel format of the output image is YUV format.
  • the resolution of the output image can be the same as the resolution of the compensated image
  • the color depth of the output image can be the same as the color depth of the compensated image
  • the color gamut of the output image can be the same as the color gamut of the compensated image.
  • the output image can be directly used for display on the first display panel.
  • the output image can be encoded to generate an output image file.
  • the output image file can be used for storage in a memory or for transmission between different display devices.
  • the output image includes output video, as shown in Figure 2.
  • compensation processing can be performed on the enhanced video to obtain the compensated video; then, After compensation, the format of the video is converted to obtain the output video; finally, the output video is encoded to obtain the output video file.
  • the pixel format of the output video can be YUV format.
  • FIG. 8 is a schematic diagram of an image processing device provided by some embodiments of the present disclosure.
  • the image processing device can be used to implement the image processing method provided by any embodiment of the present disclosure.
  • the image processing device 100 includes one or more memories 101 and one or more processors 102 .
  • the memory 101 is configured to non-transitoryly store computer-executable instructions; the processor 102 is configured to execute the computer-executable instructions.
  • the computer-executable instructions when executed by the processor, implement the image processing method according to any embodiment of the present disclosure.
  • the specific implementation and related explanations of each step of the image processing method please refer to the above embodiments of the image processing method, and will not be described again here.
  • the image processing device 100 further includes an input device 103 .
  • the compensation input parameters and/or the image quality enhancement parameters are input through the input device 103 .
  • the compensation input parameters and/or the image quality enhancement parameters can be input through physical buttons, virtual buttons (for example, icons on the touch panel/display panel), voice, etc.
  • the input device 103 may include a touch screen, a touch pad, a keyboard, a mouse, a microphone, an electronic pen, and the like.
  • the image processing device 100 further includes a display device.
  • the display device includes a display panel (ie, the first display panel in the image processing method), and the display panel is used to display the output image.
  • the image processing device 100 may further include a communication interface and a communication bus.
  • the memory 101, the processor 102, the input device 103 and the communication interface communicate with each other through a communication bus.
  • Memory 101, Components such as the processor 102, the input device 103 and the communication interface may also communicate through network connections. This disclosure does not limit the type and function of the network.
  • the image processing device 100 includes multiple processors 102, the multiple processors 102 may also communicate through a communication bus or network.
  • the communication bus may be a Peripheral Component Interconnect Standard (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, or the like.
  • PCI Peripheral Component Interconnect Standard
  • EISA Extended Industry Standard Architecture
  • the communication bus can be divided into address bus, data bus, control bus, etc.
  • the communication interface is used to implement communication between the image processing apparatus 100 and other devices.
  • the processor 102 and the memory 101 can be provided on the server side (or cloud) or on the client side (for example, a mobile device such as a mobile phone).
  • the processor 102 can control other components in the image processing device 100 to perform desired functions.
  • the processor 102 may be a central processing unit (CPU), a network processor (NP), etc.; it may also be a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable Logic devices, discrete gate or transistor logic devices, discrete hardware components.
  • the central processing unit (CPU) can be X86 or ARM architecture, etc.
  • the GPU can be integrated directly into the motherboard separately or built into the motherboard's Northbridge chip. GPUs can also be built into a central processing unit (CPU).
  • memory 101 may include any combination of one or more computer program products, which may include various forms of computer-readable storage media, such as volatile memory and/or non-volatile memory.
  • Volatile memory may include, for example, random access memory (RAM) and/or cache memory (cache), etc.
  • Non-volatile memory may include, for example, read-only memory (ROM), hard disk, erasable programmable read-only memory (EPROM), portable compact disk read-only memory (CD-ROM), USB memory, flash memory, and the like.
  • One or more computer-executable instructions may be stored on the computer-readable storage medium, and the processor 102 may execute the computer-executable instructions to implement various functions of the image processing apparatus 100 .
  • Various application programs, various data, etc. can also be stored in the memory 101 .
  • image processing device 100 can achieve similar technical effects to the foregoing image processing method, and repeated details will not be described again.
  • FIG. 9 is a schematic diagram of another image processing device provided by at least one embodiment of the present disclosure.
  • FIG. 10 is a schematic diagram of another image processing device provided by at least one embodiment of the present disclosure.
  • the image processing device shown in Figure 9 can be used to implement the image processing method provided by any embodiment of the present disclosure.
  • the image processing device 200 may include: an acquisition module 201 and an image quality enhancement processing module 202 .
  • the acquisition module 201 is used to acquire input images.
  • the acquisition module 201 is used to implement step S10 shown in FIG. 1A and FIG. 1B.
  • step S10 shown in FIG. 1A and FIG. 1B in the embodiment of the above image processing method. Relevant descriptions will not be repeated here.
  • the image quality enhancement processing module 202 is used to perform image quality enhancement processing on the input image to obtain an enhanced image.
  • the enhanced image corresponds to a higher resolution than the input image.
  • the image quality enhancement processing module 202 is used to implement step S11 shown in Figures 1A and 1B.
  • step S11 shown in Figures 1A and 1B.
  • the image quality enhancement processing includes at least one of the following processing: frame interpolation processing, super-resolution processing, noise reduction processing, color correction processing, HDR up-conversion processing, detail restoration processing, etc.
  • the image quality enhancement processing module 202 includes at least one of the following sub-modules: frame insertion sub-module 2021, super-resolution sub-module 2022, color correction sub-module 2023, noise reduction sub-module 2024, HDR up-conversion sub-module Modules and details fix submodules etc.
  • the frame insertion sub-module 2021 is used to implement frame insertion processing
  • the super-resolution sub-module 2022 is used to implement super-resolution processing
  • the color correction sub-module 2023 is used to implement color correction processing
  • the noise reduction sub-module 2024 is used to implement noise reduction processing.
  • the HDR upconversion submodule is used to implement HDR upconversion processing
  • the detail repair submodule is used to implement detail repair processing.
  • the frame interpolation sub-module 2021 includes a first deep learning model, and is used to perform frame interpolation processing on the input of the frame interpolation sub-module 2021 based on the first deep learning model.
  • the input of the frame interpolation sub-module 2021 includes video, and the frame interpolation processing includes The process of adding at least one transition image frame between every two image frames of the video.
  • the super-resolution sub-module 2022 includes a second deep learning model and is configured to perform super-resolution processing on the input of the super-resolution sub-module 2022 based on the second deep learning model to improve the efficiency of the input of the super-resolution sub-module 2022. spatial resolution.
  • the color correction sub-module 2023 includes a third deep learning model, and is configured to perform color correction processing on the input of the color correction sub-module 2023 based on the third deep learning model.
  • the noise reduction sub-module 2024 includes a fourth deep learning model, and is used to perform noise reduction processing on the input of the noise reduction sub-module 2024 based on the fourth deep learning model.
  • frame insertion sub-module 2021 super-resolution sub-module 2022
  • color correction sub-module 2024 For detailed description of the functions implemented by block 2023 and noise reduction sub-module 2024, please refer to the relevant descriptions of frame interpolation processing, super-resolution processing, noise reduction processing and color correction processing in the embodiments of the above image processing method, and there will be no duplication. Again.
  • the image processing device 200 may further include a compensation processing module 203 .
  • the compensation processing module 203 is used to perform compensation processing on the enhanced image to obtain a compensated image.
  • the resolution corresponding to the compensated image is higher than the resolution corresponding to the input image.
  • the compensation processing module 203 is used to implement step S12 shown in Figure 1B.
  • step S12 shown in Figure 1B For a specific description of the functions implemented by the compensation processing module 203, please refer to the relevant description of step S12 shown in Figure 1B in the embodiment of the above image processing method. Repeat No further details will be given.
  • the pixel format of the compensated image is RGB format
  • the image processing device also includes an encoding module 205.
  • the encoding module 205 is used to perform format conversion on the compensated image to obtain an output image; and to encode the output image to obtain an output image file.
  • the pixel format of the output image is YUV format.
  • the image processing device 200 further includes a display device.
  • the display device includes a first display panel, and the first display panel is used to display the output image.
  • the compensation processing module 203 includes a selection sub-module 2031 and a processing sub-module 2032.
  • the selection sub-module 2031 is used to select the compensation parameter corresponding to the first display panel from multiple compensation parameters; the processing sub-module 2032 is used to perform compensation processing on the enhanced image based on the compensation parameter corresponding to the first display panel to obtain Compensated image.
  • the selection sub-module 2031 includes an acquisition unit and a selection unit.
  • the acquisition unit is configured to acquire compensation input parameters corresponding to the first display panel, where the compensation input parameters include panel parameters corresponding to the first display panel and/or environmental parameters of the environment in which the first display panel is located.
  • the selection unit is configured to select a compensation parameter corresponding to the compensation input parameter from a plurality of compensation parameters as a compensation parameter corresponding to the first display panel based on the compensation input parameter.
  • the image processing device 200 further includes an input device 204 .
  • the image quality enhancement parameters are input to the image quality enhancement processing module 202 through the input device 204
  • the compensation input parameters are input to the compensation processing module 203 through the input device 204 .
  • the acquisition unit may be connected with the input device 204 to receive the compensation input parameter input from the input device 204 .
  • the acquisition module 201 the image quality enhancement processing module 202, the compensation processing module 203, Data communication can occur between the input device 204 and the encoding module 205.
  • the acquisition module 201, the image quality enhancement processing module 202, the compensation processing module 203 and/or the encoding module 205 include codes and programs stored in the memory; the processor can execute the codes and programs to implement the acquisition module 201 as described above. , some or all functions of the image quality enhancement processing module 202, the compensation processing module 203 and/or the encoding module 205.
  • the acquisition module 201, the image quality enhancement processing module 202, the compensation processing module 203, and/or the encoding module 205 can be dedicated hardware devices used to implement the acquisition module 201, the image quality enhancement processing module 202, and the compensation processing module as described above. 203 and/or some or all of the functionality of encoding module 205.
  • the acquisition module 201, the image quality enhancement processing module 202, the compensation processing module 203 and/or the encoding module 205 may be one circuit board or a combination of multiple circuit boards, used to implement the functions described above.
  • the one circuit board or a combination of multiple circuit boards may include: (1) one or more processors; (2) one or more non-transitory memories connected to the processors; and (3) Firmware stored in memory that is executable by the processor.
  • the input device 204 may include a touch screen, a touch pad, a keyboard, a mouse, a microphone, an electronic pen, or the like.
  • the image processing device can achieve similar technical effects to the foregoing image processing method, which will not be described again here.
  • FIG. 11 is a schematic diagram of a non-transitory computer-readable storage medium provided by at least one embodiment of the present disclosure.
  • one or more computer-executable instructions 1101 may be non-transitory stored on a non-transitory computer-readable storage medium 1100.
  • the computer-executable instructions 1101 are executed by a processor (or computer), one or more steps of the image processing method according to any embodiment of the present disclosure may be performed.
  • the non-transitory computer-readable storage medium 1100 can be applied in the above-mentioned image processing device 100.
  • it can include the memory 101 in the image processing device 100.
  • the description of the non-transitory computer-readable storage medium 1100 may refer to the description of the memory 101 in the embodiment of the image processing apparatus 100, and repeated descriptions will not be repeated.
  • Figure 12 is a schematic diagram of a hardware environment provided by at least one embodiment of the present disclosure.
  • the electronic device provided by the present disclosure can be applied in an Internet system.
  • the computer system provided in FIG. 12 can be used to realize the functions of the image processing device involved in the present disclosure.
  • Such computer systems can include personal computers, laptops, tablets, mobile phones, personal digital assistants, smart glasses, smart watches, smart rings, smart helmets, and any smart portable or wearable devices.
  • the specific system in this example utilizes a functional block diagram to illustrate a user interface hardware platform.
  • Such computer equipment may be a general purpose computer equipment, or a special purpose computer equipment. Both computer devices can be used to implement the image processing apparatus in embodiments of the present disclosure.
  • a computer system may include any component that implements the information currently described needed to implement image processing.
  • a computer system can be implemented by a computer device through its hardware devices, software programs, firmware, and combinations thereof.
  • Only one computer device is shown in Figure 12, but the related computer functions described in this embodiment to implement the information required for image processing can be implemented in a distributed manner by a group of similar platforms. Disperse the processing load on a computer system.
  • the computer system may include a communication port 250, connected thereto is a network that implements data communication ("from/to the network" in Figure 12), for example, the computer system may send and receive through the communication port 250 Information and data, that is, the communication port 250 can realize wireless or wired communication between the computer system and other electronic devices to exchange data.
  • the computer system may also include a processor set 220 (ie, the processors described above) for executing program instructions.
  • the processor group 220 may be composed of at least one processor (eg, CPU).
  • the computer system may include an internal communications bus 210.
  • the computer system may include different forms of program storage units and data storage units (i.e., the memory or storage media described above), such as hard disk 270, read-only memory (ROM) 230, and random access memory (RAM) 240, which can be used to store Various data files used by the computer for processing and/or communications, and possibly program instructions executed by the processor set 220.
  • the computer system may also include an input/output 260 for enabling input/output data flow between the computer system and other components (eg, user interface 280, etc.).
  • input devices including, for example, touch screens, touch pads, keyboards, mice, cameras, microphones, accelerometers, gyroscopes, etc.; including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.
  • output devices including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.
  • storage devices including tapes, hard disks, etc.; and communication interfaces.
  • FIG. 12 illustrates a computer system having various devices, it should be understood that the computer system is not required to have all of the devices shown and may instead have more or fewer devices.

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Artificial Intelligence (AREA)
  • Evolutionary Computation (AREA)
  • Image Processing (AREA)

Abstract

一种影像处理方法、影像处理装置和非瞬时性计算机可读存储介质。影像处理方法包括：获取输入影像；对输入影像进行画质增强处理，以得到增强后影像。增强后影像对应的分辨率高于输入影像对应的分辨率。

Description

影像处理方法、影像处理装置、存储介质
本申请要求于2022年03月31日递交的中国专利申请第202210343186.5号的优先权，在此全文引用上述中国专利申请公开的内容以作为本申请的一部分。
技术领域
本公开的实施例涉及一种影像处理方法、影像处理装置、非瞬时性计算机可读存储介质。
背景技术
随着社会技术的发展和国家政策的扶持，超高清显示应用逐渐出现在人们的生活中。目前，超高清显示的产业链在不断的完善中，在超高清显示的产业链中，对于采集端，具有超高清摄像机等，对于显示端，具有HDR(High Dynamic Range Imaging，高动态范围成像)电视、4K/8K分辨率的大屏幕等，对于传输端，具有5G网络、超高清电视台等，大量的企事业单位在超高清显示的产业链中进行布局。
发明内容
本公开至少一个实施例提供一种影像处理方法，包括：获取输入影像；对所述输入影像进行画质增强处理，以得到增强后影像；其中，所述增强后影像对应的分辨率高于所述输入影像对应的分辨率。
例如，本公开至少一个实施例提供的影像处理方法还包括：对所述增强后影像进行补偿处理，以得到补偿后影像，其中，所述补偿后影像对应的分辨率高于所述输入影像对应的分辨率。
例如，在本公开至少一个实施例提供的影像处理方法中，所述影像处理方法应用于第一显示面板，对所述增强后影像进行补偿处理，以得到补偿后影像，包括：从多个补偿参数中选择与所述第一显示面板对应的补偿参数；基于与所述第一显示面板对应的所述补偿参数，对所述增强后影像进行补偿处理，以得到所述补偿后影像。
例如，在本公开至少一个实施例提供的影像处理方法中，从多个补偿参数 中选择与所述第一显示面板对应的补偿参数，包括：获取与所述第一显示面板对应的补偿输入参数，其中，所述补偿输入参数包括所述第一显示面板对应的面板参数和/或所述第一显示面板所处的环境的环境参数；基于所述补偿输入参数，从所述多个补偿参数中选择与所述补偿输入参数对应的补偿参数作为与所述第一显示面板对应的所述补偿参数。
例如，在本公开至少一个实施例提供的影像处理方法中，获取与所述第一显示面板对应的补偿输入参数，包括：生成第一数组，其中，所述第一数组与所述第一显示面板对应，所述第一数组包括多个第一数组元素，所述面板参数由至少一个第一数组元素表示，所述环境参数由至少一个第一数组元素表示；基于所述第一数组，确定与所述第一显示面板对应的所述补偿输入参数。
例如，在本公开至少一个实施例提供的影像处理方法中，所述面板参数包括所述第一显示面板的类型、所述第一显示面板的尺寸以及所述第一显示面板的显示方式。
例如，在本公开至少一个实施例提供的影像处理方法中，所述第一显示面板的类型包括有机发光显示面板和液晶显示面板，所述第一显示面板的显示方式包括直接显示和投屏显示。
例如，在本公开至少一个实施例提供的影像处理方法中，所述环境参数包括环境光参数，所述环境光参数基于所述第一显示面板所在的环境的环境光的亮度值确定。
例如，在本公开至少一个实施例提供的影像处理方法中，所述补偿处理包括以下至少一项：亮度调整、对比度调整和饱和度调整。
例如，本公开至少一个实施例提供的影像处理方法还包括：生成所述多个补偿参数，其中，生成所述多个补偿参数包括：获取对标准色卡进行拍摄得到的色卡影像；对所述色卡影像进行画质增强处理，以得到增强后色卡影像；基于初始补偿参数对所述增强后色卡影像进行补偿处理，以得到补偿后色卡影像；在第二显示面板上显示所述补偿后色卡影像，并确定所述补偿后色卡影像的显示是否满足预设要求；响应于所述补偿后色卡影像的显示未达到所述预设要求，则对所述初始补偿参数进行调整以得到调整后的补偿参数，并基于所述调整后的补偿参数再次对所述增强后色卡影像进行补偿处理，直至得到的补偿后色卡影像的显示满足所述预设要求，将满足所述预设要求的补偿后色卡影像所对应的补偿参数作为所述多个补偿参数中的一个补偿参数；确定与所述第二显示面 板对应的补偿输入参数，其中，所述第二显示面板对应的补偿输入参数包括所述第二显示面板对应的面板参数和/或所述第二显示面板所处的环境的环境参数；建立满足所述预设要求的补偿后色卡影像所对应的补偿参数与所述第二显示面板对应的补偿输入参数之间的映射关系。
例如，在本公开至少一个实施例提供的影像处理方法中，所述画质增强处理包括以下处理中的至少一项：插帧处理、超分辨率处理、降噪处理、调色处理、高动态范围上变换处理、细节修复处理。
例如，在本公开至少一个实施例提供的影像处理方法中，对所述输入影像进行所述画质增强处理，以得到所述增强后影像，包括：获取画质增强参数；根据所述画质增强参数对所述输入影像进行所述画质增强处理，以得到所述增强后影像。
例如，在本公开至少一个实施例提供的影像处理方法中，所述画质增强参数包括以下参数中的至少一项：所述插帧处理对应的插帧算法名称和/或插帧参数、所述超分辨率处理对应的超分辨率算法名称和/或分辨率参数、所述调色处理对应的调色算法名称和/或调色参数和所述降噪处理对应的降噪算法名称和/或降噪参数、所述高动态范围上变换处理对应的高动态范围上变换算法和/或高动态范围上变换参数、所述细节修复处理对应的细节修复算法名称和/或细节修复参数。
例如，在本公开至少一个实施例提供的影像处理方法中，获取画质增强参数包括：生成第二数组，其中，所述第二数组包括多个第二数组元素，所述插帧处理对应的插帧参数由至少一个第二数组元素表示，所述超分辨率处理对应的分辨率参数由至少一个第二数组元素表示，所述调色处理对应的调色参数由至少一个第二数组元素表示，所述降噪处理对应的降噪参数由至少一个第二数组元素表示，所述高动态范围上变换处理对应的高动态范围上变换参数由至少一个第二数组元素表示，所述细节修复变换处理对应的细节修复参数由至少一个第二数组元素表示；基于所述第二数组，确定所述画质增强参数。
例如，在本公开至少一个实施例提供的影像处理方法中，获取画质增强参数包括：获取算法字符串，其中，所述算法字符串包括以下算法中的至少一个：插帧算法、超分辨率算法、调色算法、降噪算法、高动态范围上变换算法、细节修复算法，所述插帧算法包括所述插帧算法名称和所述插帧参数，所述超分辨率算法包括所述超分辨率算法名称和所述超分辨率参数，所述调色算法包括 所述调色算法名称和所述调色参数，所述降噪算法包括所述降噪算法名称和所述降噪参数，所述高动态范围上变换算法包括所述高动态范围上变算法名称和所述高动态范围上变换参数，所述细节修复算法包括所述细节修复算法名称和所述细节修改参数；基于所述算法字符串，确定所述画质增强参数。
例如，在本公开至少一个实施例提供的影像处理方法中，用于进行所述插帧处理的影像包括视频，所述插帧处理基于第一深度学习模型实现，所述第一深度学习模型被配置为在所述视频中的每两个图像帧之间增加至少一个过渡图像帧，所述超分辨率处理基于第二深度学习模型实现，所述第二深度学习模型被配置为对用于进行所述超分辨率处理的影像进行超分辨率处理，以提高用于进行所述超分辨率处理的影像的空间分辨率，所述调色处理基于第三深度学习模型实现；所述降噪处理基于第四深度学习模型实现，所述第四深度学习模型被配置为对用于进行所述降噪处理的影像进行降噪处理。
例如，在本公开至少一个实施例提供的影像处理方法中，所述第三深度学习模型包括回归子模型和多个查找表子模型组，每个查找表子模型组包括至少一个查找表子模型；所述调色处理包括：对用于进行所述调色处理的影像进行预处理，以得到归一化数据，其中，所述预处理包括归一化处理；利用所述回归子模型对所述归一化数据进行处理，以得到至少一个权重参数；获取调色参数；根据所述调色参数，从所述多个查找表子模型中选择与所述调色参数对应的查找表子模型组；基于所述至少一个权重参数和所述查找表子模型组，确定目标查找表子模型；利用所述目标查找表子模型对所述归一化数据进行处理，以生成所述调色处理的输出。
例如，在本公开至少一个实施例提供的影像处理方法中，所述第一深度学习模型包括实时中间流估计算法模型。
例如，在本公开至少一个实施例提供的影像处理方法中，所述第二深度学习模型包括残差特征蒸馏网络模型。
例如，在本公开至少一个实施例提供的影像处理方法中，所述第四深度学习模型包括Unet网络模型。
例如，在本公开至少一个实施例提供的影像处理方法中，所述输入影像包括视频，所述增强后影像包括与所述视频对应的增强后视频，对所述输入影像进行所述画质增强处理，以得到所述增强后影像，包括：对所述视频进行所述插帧处理，以得到插帧后视频；对所述插帧后视频进行所述调色处理，以得到 调色后视频；对所述调色后视频进行所述降噪处理，以得到降噪后视频；对所述降噪后视频进行所述超分辨率处理，以得到所述增强后视频，其中，所述增强后视频的分辨率高于所述降噪后视频的分辨率。
例如，在本公开至少一个实施例提供的影像处理方法中，所述调色后视频的色位深度高于所述插帧后视频对应的色位深度，和/或，所述调色后视频对应的色域高于所述插帧后视频对应的色域。
例如，在本公开至少一个实施例提供的影像处理方法中，获取输入影像包括：获取原始影像文件；对所述原始影像文件进行解码处理，以得到原始影像；对所述原始影像进行格式转换处理，以得到所述输入影像，其中，所述输入影像的像素格式为RGB格式。
例如，在本公开至少一个实施例提供的影像处理方法中，所述补偿后影像的像素格式为RGB格式，所述影像处理方法还包括：对所述补偿后影像进行格式转换，以得到输出影像，其中，所述输出影像的像素格式为YUV格式；对所述输出影像进行编码处理，以得到输出影像文件。
例如，在本公开至少一个实施例提供的影像处理方法中，所述补偿后影像对应的色位深度高于所述输入影像对应的色位深度，和/或，所述补偿后影像对应的色域高于所述输入影像对应的色域。
本公开至少一个实施例提供一种影像处理装置，包括：一个或多个存储器，非瞬时性地存储有计算机可执行指令；一个或多个处理器，配置为运行所述计算机可执行指令，其中，所述计算机可执行指令被所述一个或多个处理器运行时实现根据本公开任一实施例所述的影像处理方法。
例如，本公开至少一个实施例提供的影像处理装置还包括：输入装置，其中，响应于所述影像处理方法包括获取与所述第一显示面板对应的补偿输入参数和/或获取画质增强参数，所述补偿输入参数和/或所述画质增强参数通过所述输入装置输入。
例如，在本公开至少一个实施例提供的影像处理装置中，所述输入装置包括触摸屏、触摸板、键盘、鼠标、麦克风。
本公开至少一个实施例提供一种非瞬时性计算机可读存储介质，其中，所述非瞬时性计算机可读存储介质存储有计算机可执行指令，所述计算机可执行指令被处理器执行时实现根据本公开任一实施例所述的影像处理方法。
本公开至少一个实施例提供一种影像处理装置，包括：获取模块，用于获 取输入影像；画质增强处理模块，用于对所述输入影像进行画质增强处理，以得到增强后影像；其中，所述增强后影像对应的分辨率高于所述输入影像对应的分辨率。
例如，本公开至少一个实施例提供的影像处理装置还包括：补偿处理模块，用于对所述增强后影像进行补偿处理，以得到补偿后影像；其中，所述补偿后影像对应的分辨率高于所述输入影像对应的分辨率。
例如，本公开至少一个实施例提供的影像处理装置还包括第一显示面板，其中，所述补偿处理模块包括：选择子模块和处理子模块，所述选择子模块用于从多个补偿参数中选择与所述第一显示面板对应的补偿参数；所述处理子模块用于基于与所述第一显示面板对应的所述补偿参数，对所述增强后影像进行补偿处理，以得到所述补偿后影像。
例如，在本公开至少一个实施例提供的影像处理装置中，所述选择子模块包括获取单元和选择单元，所述获取单元用于获取与所述第一显示面板对应的补偿输入参数，其中，所述补偿输入参数包括所述第一显示面板对应的面板参数和/或所述第一显示面板所处的环境的环境参数；所述选择单元用于基于所述补偿输入参数，从所述多个补偿参数中选择与所述补偿输入参数对应的补偿参数作为与所述第一显示面板对应的所述补偿参数。
例如，在本公开至少一个实施例提供的影像处理装置中，所述画质增强处理包括以下处理中的至少一项：插帧处理、超分辨率处理、降噪处理和调色处理，所述画质增强处理模块包括以下子模块中的至少一个：插帧子模块、超分辨率子模块、调色子模块和降噪子模块，所述插帧子模块包括第一深度学习模型，且用于基于所述第一深度学习模型对所述插帧子模块的输入进行所述插帧处理，所述插帧子模块的输入包括视频，所述插帧处理包括在所述视频的每两个图像帧之间增加至少一个过渡图像帧的处理；所述超分辨率子模块包括第二深度学习模型，且用于基于所述第二深度学习模型对所述超分辨率子模块的输入进行所述超分辨率处理，以提高所述超分辨率子模块的输入的空间分辨率；所述调色子模块包括第三深度学习模型，且用于基于所述第三深度学习模型对所述调色子模块的输入进行所述调色处理；所述降噪子模块包括第四深度学习模型，且用于基于所述第四深度学习模型对所述降噪子模块的输入进行所述降噪处理。
例如，本公开至少一个实施例提供的影像处理装置还包括：编码模块，其 中，所述补偿后影像的像素格式为RGB格式，所述编码模块用于：对所述补偿后影像进行格式转换，以得到输出影像，其中，所述输出影像的像素格式为YUV格式；对所述输出影像进行编码处理，以得到输出影像文件。
附图说明
为了更清楚地说明本公开实施例的技术方案，下面将对实施例的附图作简单地介绍，显而易见地，下面描述中的附图仅仅涉及本公开的一些实施例，而非对本公开的限制。
图1A为本公开至少一个实施例提供的一种影像处理方法的示意性流程图；
图1B为本公开至少一个实施例提供的另一种影像处理方法的示意性流程图；
图2为本公开一些实施例提供的一种影像处理方法的整体流程图；
图3为本公开一些实施例提供的一种插帧处理的示意性流程图；
图4为本公开一些实施例提供的一种超分辨率处理的示意性流程图；
图5为本公开一些实施例提供的一种调色处理的示意性流程图；
图6为本公开一些实施例提供的一种降噪处理的示意性流程图；
图7为本公开一些实施例提供的一种调色处理和降噪处理的示意性流程图；
图8为本公开一些实施例提供的一种影像处理装置的示意图；
图9为本公开至少一个实施例提供的另一种影像处理装置的示意图；
图10为本公开至少一个实施例提供的另一种影像处理装置的示意图；
图11为本公开至少一个实施例提供的一种非瞬时性计算机可读存储介质的示意图；
图12为本公开至少一个实施例提供的一种硬件环境的示意图。
具体实施方式
为了使得本公开实施例的目的、技术方案和优点更加清楚，下面将结合本公开实施例的附图，对本公开实施例的技术方案进行清楚、完整地描述。显然，所描述的实施例是本公开的一部分实施例，而不是全部的实施例。基于所描述的本公开的实施例，本领域普通技术人员在无需创造性劳动的前提下所获得的所有其他实施例，都属于本公开保护的范围。
除非另外定义，本公开使用的技术术语或者科学术语应当为本公开所属领域内具有一般技能的人士所理解的通常意义。本公开中使用的“第一”、“第二”以及类似的词语并不表示任何顺序、数量或者重要性，而只是用来区分不同的组成部分。“包括”或者“包含”等类似的词语意指出现该词前面的元件或者物件涵盖出现在该词后面列举的元件或者物件及其等同，而不排除其他元件或者物件。“连接”或者“相连”等类似的词语并非限定于物理的或者机械的连接，而是可以包括电性的连接，不管是直接的还是间接的。“上”、“下”、“左”、“右”等仅用于表示相对位置关系，当被描述对象的绝对位置改变后，则该相对位置关系也可能相应地改变。
为了保持本公开实施例的以下说明清楚且简明，本公开省略了部分已知功能和已知部件的详细说明。
现在超高清行业所面临的矛盾是，超高清的设备平台的技术发展较快，而超高清视频内容的生产制作却远远落后。4K/8K的片源存量远远不能满足超高清播放需求，但大量的标清、高清视频的库存无法在超高清的设备平台上播放。目前，对标清片源、高清片源等进行重制以得到超高清片源是可以解决超高清片源不足的最快速直接的手段。然而，低清片源的超高清重制大都依靠人工处理，所以片源生产周期长，人力成本高。
本公开至少一个实施例提供一种影像处理方法，该影像处理方法包括：获取输入影像；对输入影像进行画质增强处理，以得到增强后影像。增强后影像对应的分辨率高于输入影像对应的分辨率。
在本公开的实施例提供的影像处理方法中，通过对输入影像进行画质增强处理，以实现自动对输入影像进行超高清重制，从而生成超高清的增强后影像，解决超高清片源不足的问题，满足行业发展需求。该影像处理方法可以实现自动基于低清片源生成超高清视频，降低人力成本和片源生产周期。
本公开至少一个实施例还提供一种影像处理装置和非瞬时性计算机可读存储介质。
本公开实施例提供的影像处理方法可应用于本公开实施例提供的影像处理装置，该影像处理装置可被配置于电子设备上。该电子设备可以是个人计算机、移动终端等，该移动终端可以是手机、平板电脑等具有各种操作系统的硬件设备。也就是说，影像处理方法的执行主体可以为个人计算机、移动终端等。
下面对本公开的实施例进行详细说明，但是本公开并不限于这些具体的实 施例。
图1A为本公开至少一个实施例提供的一种影像处理方法的示意性流程图，图1B为本公开至少一个实施例提供的另一种影像处理方法的示意性流程图，图2为本公开一些实施例提供的一种影像处理方法的整体流程图。
例如，影像处理方法可以应用于第一显示面板，第一显示面板可以为有机发光二极管(organic light emitting diode，OLED)显示面板(例如，有源矩阵有机发光二极管(Active-matrix organic light emitting diode，AMOLED)显示面板)、量子点发光二极管(QLED)显示面板、液晶显示面板等。
例如，第一显示面板可以具有8K(7680(像素)*4320(像素))分辨率，即第一显示面板可以包括7680(列)*4320(行)的像素阵列。需要说明的是，第一显示面板的分辨率可以根据实际情况设置，例如，第一显示面板可以具有4K(4096(像素)*2160(像素))分辨率，本公开的实施例对第一显示面板的分辨率不作具体限制。
例如，第一显示面板可以为矩形面板、圆形面板、椭圆形面板或多边形面板等。另外，第一显示面板不仅可以为平面面板，也可以为曲面面板，甚至球面面板。
例如，第一显示面板可以应用于手机、平板电脑、电视机、显示器、笔记本电脑、数码相框、导航仪等任何具有显示功能的产品或部件中。
需要说明的是，第一显示面板也可以包括投影仪。
如图1A所示，在一些实施例中，本公开的实施例提供的影像处理方法包括步骤S10至步骤S11。如图1B所示，在另一些实施例中，本公开的实施例提供的影像处理方法包括步骤S10至步骤S12。
首先，如图1A和图1B所示，在步骤S10，获取输入影像。
例如，输入影像可以包括视频和/或图片等。视频可以为各种类型的视频，图片可以为各种类型的图片。以视频为例，若视频中的对象为风景、人物等，则该视频为风景视频、人物视频等。视频还可以为监控视频、动植物视频等。例如，对于图片，图片的形状可以为矩形等各种合适的形状，图片可以为静态图像也可以为动态图像，图片的形状和尺寸等可以由用户根据实际情况设定，本公开的实施例不作具体限制。对于视频，视频的分辨率可以由用户根据实际情况设定，本公开的实施例不作具体限制。本公开的实施例对视频/图片的类型、分辨率等性质不作具体限制。
例如，当输入影像为视频，该本公开提供的影像处理方法即为视频处理方法；当输入影像为图片，该本公开提供的影像处理方法即为图像处理方法。需要说明的是，在本公开的实施例中，除非特别说明，以输入影像为视频为例进行说明。
例如，输入影像可以通过影像采集装置获取。影像采集装置可以包括摄像机、照相机等，摄像机可以包括智能手机的摄像头、平板电脑的摄像头、个人计算机的摄像头、数码照相机的镜头、或者甚至可以是网络摄像头。
例如，输入影像可以为灰度影像，也可以为彩色影像。
例如，输入影像可以是影像采集装置直接采集到的影像，也可以是对影像采集装置直接采集到的影像进行预处理之后获得的影像。例如，为了避免影像采集装置直接采集得到的影像的数据质量、数据不均衡等对于后续影像处理过程的影响，在执行步骤S10之前，本公开实施例提供的影像处理方法还可以包括对影像采集装置直接采集得到的影像进行预处理的过程。预处理可以消除影像采集装置直接采集到的影像中的无关信息或噪声信息，从而在后续影像处理过程中便于更好地对输入影像进行处理。预处理例如可以包括对影像采集装置直接采集到的影像进行扩充(Data Augment)处理、缩放处理和伽玛(Gamma)校正等中的一项或多项。扩充处理包括通过随机裁剪、旋转、翻转、偏斜等方式扩充影像数据。缩放处理包括对影像采集装置直接采集到的影像进行等比例缩放并剪裁为预设尺寸，以便于后续进行处理操作。
例如，在一些实施例中，步骤S10可以包括：获取原始影像文件；对原始影像文件进行解码处理，以得到原始影像；对原始影像进行格式转换处理，以得到输入影像。
例如，如图2所示，首先，对原始影像文件进行解码处理，以得到原始影像，然后，对原始影像进行格式转换处理，以得到输入影像。例如，在一些实施例中，原始影像的像素格式为YUV格式，输入影像的像素格式为RGB格式，输入影像的色深(色位深度)为8比特(bits)，输入影像的色域为BT.709，输入影像的帧率可以为25fps(frames per second，画面每秒传输帧数)，输入影像的分辨率可以为1920(像素)*1080(像素)。需要说明的是，输入影像的色深、色域、帧率和分辨率等均可以根据实际情况设置，在此不作具体限制。
例如，视频流的像素格式的类型可以包括YUV420像素格式、YUV420 10位像素格式、YUV422像素格式、YUV422P10像素格式、RGB24像素格式、 BGR24像素格式等等，本公开的影像的像素格式可以为上述任一一种像素格式，本公开对此不作限制。
如图1A和图1B所示，在步骤S11，对输入影像进行画质增强处理，以得到增强后影像。例如，增强后影像对应的分辨率高于输入影像对应的分辨率。
例如，增强后影像对应的色位深度高于输入影像对应的色位深度，和/或，增强后影像对应的色域高于输入影像对应的色域。
例如，在一些实施例中，步骤S11包括：获取画质增强参数；根据画质增强参数对输入影像进行画质增强处理，以得到增强后影像。
例如，画质增强处理包括以下处理中的至少一项：插帧处理、超分辨率处理、降噪处理、调色处理、高动态范围(HDR)上变换处理和细节修复处理等。例如，调色处理可以为HDR调色处理。
例如，画质增强参数包括以下参数中的至少一项：插帧处理对应的插帧算法名称和/或插帧参数、超分辨率处理对应的超分辨率算法名称和/或分辨率参数、调色处理对应的调色算法名称和/或调色参数和降噪处理对应的降噪算法名称和/或降噪参数等、高动态范围上变换处理对应的高动态范围上变换算法和/或高动态范围上变换参数、细节修复处理对应的细节修复算法名称和/或细节修复参数。
例如，画质增强参数可以由用户通过输入装置输入而设置，从而可以根据用户的需求进行设置，以满足用户的不同需求和满足不同用户的需求。
例如，画质增强处理中的各项处理(例如，上述插帧处理、超分辨率处理、降噪处理和调色处理等)均可灵活配置，而且可以根据需求对应设置参数。例如，插帧处理、超分辨率处理、降噪处理和调色处理等中的一项或多项可以不执行，或者，插帧处理、超分辨率处理、降噪处理和调色处理可以均执行。
例如，在一些实施例中，在步骤S11中，获取画质增强参数包括：生成第二数组，其中，第二数组包括多个第二数组元素，插帧处理对应的插帧参数由至少一个第二数组元素表示，超分辨率处理对应的分辨率参数由至少一个第二数组元素表示，调色处理对应的调色参数由至少一个第二数组元素表示，降噪处理对应的降噪参数由至少一个第二数组元素表示，高动态范围上变换处理对应的高动态范围上变换参数由至少一个第二数组元素表示，细节修复变换处理对应的细节修复参数由至少一个第二数组元素表示；基于第二数组，确定画质增强参数。
例如，画质增强参数可被定义为整型数组格式，即第二数组为整型数组，第二数组包括多个第二数组元素，每个第二数组元素对应一个含义。根据每个第二数组元素的数值即可确定其对应的处理是否需要执行。例如，以插帧处理为例，若表示插帧处理对应的插帧参数的第二数组元素的值为第一数值，则表示不执行插帧处理；若表示插帧处理对应的插帧参数的第二数组元素的值为第二数值，则表示执行插帧处理。例如，在一些实施例中，第一数值为0，第二数值为1，然而本公开不限于此。
例如，在一些实施例中，第二数组中的第二数组元素0(即第二数组中的第一个元素)表示插帧处理对应的插帧参数，第二数组中的第二数组元素1和2(即第二数组中的第二个元素和第三个元素)表示超分辨率处理对应的分辨率参数，第二数组中的第二数组元素3和4(即第二数组中的第四个元素和第五个元素)表示调色处理对应的调色参数，第二数组中的第二数组元素5(即第二数组中的第六个元素)表示降噪处理对应的降噪参数。对画质增强参数对应的第二数组中的各个第二数组元素的定义如下：第二数组元素0表示是否开启插帧处理，当第二数组元素0为0，则表示不开启插帧处理，当第二数组元素0为1，则表示开启插帧处理；第二数组元素1表示是否开启超分辨率处理，当第二数组元素1为0，则表示不开启超分辨率处理，当第二数组元素1为1，则表示开启超分辨率处理；第二数组元素2表示超分倍率，当第二数组元素2为0，则表示超分倍率为2倍超分(长度和宽度均提升两倍)，当第二数组元素2为1，则表示超分倍率为4倍超分等，具体根据实际情况设置；第二数组元素3表示是否开启调色处理，当第二数组元素3为0，则表示不开启调色处理，当第二数组元素3为1，则表示开启调色处理；第二数组元素4表示是调色风格，当第二数组元素4为0时，则表示调色风格为默认风格，当第二数组元素4为1时，则表示调色风格为暖色风格，当第二数组元素4为2时，则表示调色风格为冷色风格，等等，具体根据实际情况设置；第二数组元素5表示是否开启降噪处理，当第二数组元素5为0，则表示不开启降噪处理，当第二数组元素5为1，则表示开启降噪处理。画质增强参数对应的第二数组中的第二数组元素的数量、各个第二数组元素表示的含义等可以根据具体情况设置，本公开对此不作限制。
需要说明的是，第二数组中的各个第二数组元素的值可以根据实际情况预先设定默认值。
例如，在一些实施例中，在步骤S11中，获取画质增强参数包括：获取画质增强参数包括：获取算法字符串；基于算法字符串，确定画质增强参数。
例如，算法字符串可以包括以下算法中的至少一个：插帧算法、超分辨率算法、调色算法、降噪算法、高动态范围上变换算法、细节修复算法。插帧算法包括插帧算法名称和插帧参数，超分辨率算法包括超分辨率算法名称和超分辨率参数，调色算法包括调色算法名称和调色参数，降噪算法包括降噪算法名称和降噪参数，高动态范围上变换算法包括所述高动态范围上变算法名称和所述高动态范围上变换参数，细节修复算法包括所述细节修复算法名称和所述细节修改参数。
例如，在一些实施例中，算法字符串表示为：./sr_sdk–i test_video/1080_25.mp4–m frc_1080:denoise_1080:sr1×_1080–c 0–p 0–o out.mp4
其中，sr_sdk表示可执行文件；-i后面所跟的字符test_video/1080_25.mp4表示待测试的视频(例如，本公开中的输入影像)；-m后面所跟的字符表示所选用的与画质增强处理相关的算法，多个算法可以叠加，且各个算法之间采用“:”分隔开，例如，在上述例子中，frc_1080:denoise_1080:sr1×_1080表示三种算法，frc_1080表示插帧算法，其中，frc表示插帧算法名称，1080表示插帧算法的参数，denoise_1080表示降噪算法，其中，denoise表示降噪算法名称，1080表示降噪算法的参数，sr1×_1080表示超分辨率算法，其中，sr表示超分辨率算法名称，1080表示超分辨率算法的参数；-c后面所跟的字符表示视频编码器，在上述示例中，-c后面的字符为0，其表示h264编码；-p后面所跟的字符表示编码时的像素格式，在上述示例中，-p后面的字符为0，其表示yuv420像素格式；-o后面所跟的字符表示对待测试的视频进行处理之后得到的视频文件的存储路径及文件名，可以支持.mp4，.avi等格式。
需要说明的是，各个算法的参数与待测试的视频的分辨率相关，例如，在上述例子中，待测试的视频的分辨率为1080，则各个算法的参数也为1080。
例如，调用的算法可以包括：1)-m sr1×_1080/sr1×_540，其表示单帧超分辨率处理的算法；2)-m sr3×_1080/sr3×_540，其表示EDVR(Video Restoration with Enhanced Deformable Convolutional Networks)算法；3)-m sr_1080/sr_540，其表示RFDN(residual feature distillation network)算法；4)-m denoise_1080/denoise_540，其表示小模型降噪算法；5)-m denoise3×_1080/denoise3×_540，其表示大模型降噪算法；6)-m detail_1080，其 表示细节增强算法(例如，细节修复处理)；7)-m hdr_lut_2160/hdr_lut_1080/hdr_lut_540，其表示小模型HDR算法；8)-m hdr_2160/hdr_1080/hdr_540，其表示大模型HDR算法；9)-m frc_2160/frc_1080/frc_540，其表示插帧算法。需要说明的是，算法的表现形式可以根据实际情况设置，本公开对此不作具体限制。
例如，当-c后面的字符为1时表示h265编码；当-c后面的字符为2时表示mpeg-1编码；当-c后面的字符为3时表示mpeg-2编码；当-c后面的字符为4时表示mpeg-4编码；当-c后面的字符为5时表示wmv7编码；当-c后面的字符为6时表示wmv8编码。
例如，当-p后面的字符为1时表示yuv422像素格式，需要说明的是，默认情况下，-p后面的字符为0。
例如，在一些实施例中，用于进行插帧处理的影像包括视频，插帧处理基于第一深度学习模型实现，第一深度学习模型被配置为在视频中的每两个图像帧之间增加至少一个过渡图像帧。
例如，插帧处理用于给视频的每相邻两个图像帧中间增加新的过渡图像帧，使视频的播放更加流畅。在一些实施例中，第一深度学习模型包括基于深度学习的实时中间流估计算法(RIFE)模型。该基于深度学习的RIFE模型生成的过渡图像帧的伪影少，从而使得插帧得到的视频的显示效果更好。而且，RIFE模型的运行速度快，适用于对视频的帧率提升，还可以增强视觉质量。RIFE模型能够进行端到端训练并获得出色的性能。
需要说明的是，在另一些实施例中，第一深度学习模型也可以采用DAIN(Depth-Aware Video Frame Interpolation)模型、ToFLOW(Video Enhancement with Task-Oriented Flow)模型等模型。本公开的实施例对第一深度学习模型的具体类型不作限制，只要第一深度学习模型能够实现插帧处理即可。
图3为本公开一些实施例提供的一种插帧处理的示意性流程图。图3所示的示例是基于RIFE模型进行插帧处理的流程图。
例如，如图3所示，对于视频中的相邻两个图像帧I0和I1，该两个图像帧I0和I1以及目标时间t(0≤t≤1)被输入到IFNet模型，IFNet模块用于直接估算中间光流，即图像帧I0和I1之间的过渡图像帧相对于图像帧I0和I1的中间光流。IFNet模块可以直接高效的估计中间光流F_t→0，然后使用线性运动假设近似得到中间光流F_t→1，中间光流F_t→1表示如下：
然后，将中间光流F_t→0和F_t→1以及图像帧I0和I1输入至空间形变(backwarping)模型，backwarping模型用于对基于backward warping的方法根据估算的中间光流F_t→0和F_t→1对图像帧I0和I1进行空间形变(warp)处理，从而得到两个粗略的中间图像帧I_0→t和I_1→t。此外，为了降低仿射结果的伪影问题，该中间光流F_t→0和F_t→1、图像帧I0和I1以及中间图像帧I_0→t和I_1→t被输入至融合处理(Fusion processing)模型以进行融合处理，从而生成过渡图像帧It。融合处理模型采用了类似FusionNet的编码器-解码器架构实现。在融合处理模型中，首先基于中间光流F_t→0和F_t→1以及图像帧I0和I1估计一个融合图和残差项，然后，根据融合图对中间图像帧I_0→t和I_1→t进行线性组合并与残差项相加，得到重建的过渡图像帧It。过渡图像帧It表示如下：
It＝M⊙I_0→t+(1-M)⊙I_1→t+Δ，
其中，M是用于融合两个中间图像帧I_0→t和I_1→t得到的融合图，Δ是用于细化图像细节的残差项，⊙是逐元素乘法符号。
需要说明的是，当输入影像为图片时，画质增强处理不包括插帧处理。
例如，在一些实施例中，超分辨率处理基于第二深度学习模型实现，第二深度学习模型被配置为对用于进行超分辨率处理的影像进行超分辨率处理，以提高用于进行超分辨率处理的影像的空间分辨率。
例如，超分辨率处理可提升影像的空间分辨率(像素数据)，且不损失细节。在一些实施例中，第二深度学习模型包括基于深度学习的残差特征蒸馏网络(residual feature distillation network，RFDN)模型。该RFDN模型的参数量小，速度快，准确性高，但可以达到与EDVR(Video Restoration with Enhanced Deformable Convolutional Networks，基于可形变卷积的视频恢复网络)等超大模型相似的峰值信噪比(PSNR)性能。
需要说明的是，在另一些实施例中，第二深度学习模型也可以采用EDVR模型等模型。本公开的实施例对第二深度学习模型的具体类型不作限制，只要第二深度学习模型能够实现超分辨率处理即可。
图4为本公开一些实施例提供的一种超分辨率处理的示意性流程图。图4所示的示例是基于RFDN模型进行超分辨率处理的流程图。
例如，如图4所示，RFDN模型被配置为对用于进行超分辨率处理的影像input1进行超分辨率处理，以得到超分辨率处理后的影像output1。
例如，RFDN模型可以包括依次连接的卷积层Conv11、多个残差蒸馏模块RFDB(residual feature distillation block)、卷积层Conv12、卷积层Conv13、卷积层Conv14和像素重组(Pixel shuffle)层PS。
例如，在图4所示的示例中，多个残差蒸馏模块包括第一残差蒸馏模块RFDB1、第二残差蒸馏模块RFDB2和第三残差蒸馏模块RFDB3，需要说明的是，RFDN模型不限于图4所示的具体结构，RFDN模型可以包括更多或更少的残差蒸馏模块，本公开的实施例对此不作具体限制。
例如，每个残差蒸馏模块RFDB包括多个1*1卷积层(1*1的卷积核)、多个SRB(shallow residual block，浅层残差块)模块和一个3*3卷积层(3*3的卷积核)，1*1卷积层的数量和SRB模块的数量相同，且一一对应。1*1卷积层用于进行特征蒸馏，从而显著减少了参数数量，多个SRB模块用于进行特征提取，每个SRB模块由3*3的卷积核、残差连接和激活单元(ReLU)组成。SRB模块可以在不引入任何额外参数的情况下受益于残差学习。SRB模块可以实现更深的残差连接，并且可以更好地利用残差学习的能力，同时也足够轻量。3*3卷积层可以更好地细化特征。
例如，多个残差蒸馏模块RFDB输出的特征进行连接(concatenate)操作，并通过卷积层Conv12进行处理，以减少特征数。如图4所示，第一残差蒸馏模块RFDB1输出的特征、第二残差蒸馏模块RFDB2输出的特征和第三残差蒸馏模块RFDB3输出的特征进行连接(concatenate)操作，然后被输入至卷积层Conv12进行处理。
例如，在一些实施例中，卷积层Conv11可以采用3*3的卷积核进行卷积处理，卷积层Conv12可以采用1*1的卷积核进行卷积处理，卷积层Conv13可以采用3*3的卷积核进行卷积处理，卷积层Conv14可以采用3*3的卷积核进行卷积处理。
例如，像素重组层PS的主要功能是将低分辨率的特征图，通过卷积和多通道间的重组得到高分辨率的特征图，即像素重组层PS用于实现提升空间分辨率的操作。
图5为本公开一些实施例提供的一种调色处理的示意性流程图。
例如，在一些实施例中，调色处理基于第三深度学习模型实现。
例如，第三深度学习模型包括回归子模型和多个查找表子模型组，每个查找表子模型组包括至少一个查找表子模型。
例如，调色处理包括：对用于进行调色处理的影像进行预处理，以得到归一化数据，其中，预处理包括归一化处理；利用回归子模型对归一化数据进行处理，以得到至少一个权重参数；获取调色参数；根据调色参数，从多个查找表子模型中选择与调色参数对应的查找表子模型组；基于至少一个权重参数和查找表子模型组，确定目标查找表子模型；利用目标查找表子模型对归一化数据进行处理，以生成调色处理的输出。
例如，多个查找表子模型组可以根据积累有调色经验的调色师的调色风格进行训练得到。多个查找表子模型组分别对应多种不同的色彩风格，从而应对不同的调色风格需求。基于调色参数可以确定调色风格，从而从多个查找表子模型组选择与该调色参数对应的调色风格对应的查找表子模型组。调色参数可以由用户根据实际需求设置，且输入到执行该影像处理方法的设备或装置中。
例如，如图5所示，第三深度学习模型被配置为对用于进行调色处理的影像input2进行调色处理，以得到调色处理后的影像output2。例如，用于进行调色处理的影像input2可以为标准动态范围(SDR)影像。
例如，如图5所示，首先，对影像input2进行预处理(preprocess)以得到归一化数据，预处理包括归一化处理，即将影像input2的所有像素的像素值进行归一化处理，以便于后续模型的处理。然后，利用回归子模型对归一化数据进行处理，以得到至少一个权重参数(W1、W2、…、Wn，n为正整数)。然后，基于用户输入的调色参数，可以从多个查找表子模型中选择与用户输入的调色参数对应的查找表子模型组。然后，基于至少一个权重参数和查找表子模型组，确定目标查找表子模型，例如，至少一个权重参数与查找表子模型组中的至少一个查找表子模型一一对应，如图5所示，查找表子模型组可以包括查找表子模型3D LUT1、查找表子模型3D LUT2、…、查找表子模型3D LUTN。然后，每个查找表子模型与对应的权重参数相乘，以得到乘法结果，接着将所有查找表子模型对应的乘法结果相加，以得到目标查找表子模型3D LUT。例如，3D LUT表示为：3D LUT＝3D LUT1*W1+3D LUT2*W2+…+3D LUTN*Wn。最后，利用目标查找表子模型对归一化数据进行处理，以生成调色处理的输出output2。
需要说明的是，若影像input2的像素格式不是RGB格式，例如，影像input2 的像素格式为YUV格式，则预处理还可以包括格式转换处理，将影像input2参照标准GY/T 351-2018中，关于HLG(Hybrid Log Gamma)系统的非线性转换函数进行转换，变换成非线性RGB数据，然后在非线性RGB数据进行归一化处理，以得到归一化数据。在此情况下，利用目标查找表子模型对归一化数据进行处理，以得到目标查找表子模型的输出，目标查找表子模型的输出为非线性RGB数据，最后，需要对目标查找表子模型的输出进行后处理(postprocess)，以生成调色处理的输出output2，后处理可以包括格式转换处理，即将目标查找表子模型的输出基于GY/T 351-2018的规定进行变换，以得到调色处理的输出output2，例如，该调色处理的输出output2的像素格式可以为YUV格式。例如，还可以对调色处理的输出output2进行处理以得到色深10bits、色域BT.2020的影像，该影像的像素格式也为YUV格式。
图6为本公开一些实施例提供的一种降噪处理的示意性流程图。图6所示的示例是基于Unet网络模型进行降噪处理的流程图。
例如，在一些实施例中，降噪处理基于第四深度学习模型实现，第四深度学习模型被配置为对用于进行降噪处理的影像进行降噪处理。
例如，如图6所示，第四深度学习模型包括Unet网络模型，且用于对影像整体的噪声进行抑制，即对影像整体的噪声进行降噪处理。例如，第四深度学习模型被配置为对用于进行降噪处理的影像input3进行降噪处理，以得到降噪处理后的影像output3。
例如，如图6所示，第四深度学习模型可以包括多个卷积层、多个残差块、多个反卷积层和多个连接层，多个卷积层包括卷积层Conv21、卷积层Conv22和卷积层Conv23，多个反卷积层包括反卷积层Dconv21、反卷积层Dconv22和反卷积层Dconv23，多个残差块包括残差块RB1、残差块RB2、残差块RB3、残差块RB4、残差块RB5、残差块RB6，多个连接层包括连接层Ct1、连接层Ct2。如图6所示，第四深度学习模型包括依次连接的卷积层Conv21、残差块RB1、卷积层Conv22、残差块RB2、卷积层Conv23、残差块RB3、残差块RB4、反卷积层Dconv21、残差块RB5、反卷积层Dconv22、残差块RB6、反卷积层Dconv23，连接层Ct1用于将卷积层Conv22的输出和反卷积层Dconv22的输出进行映射连接，并将映射连接的结果输出至残差块RB6，连接层Ct2用于将卷积层Conv23的输出和反卷积层Dconv21的输出进行映射连接，并将映射连接的结果输出至残差块RB5。
例如，每个卷积层可以进行卷积操作，即提取特征，每个卷积层的卷积步长(stride)可以为2，从而进行下采样。每个反卷积层用于执行反卷积操作，每个反卷积层的反卷积步长(stride)也可以为2，从而进行上采样，卷积层的前向传播过程对应反卷积层的反向传播过程，卷积层的反向传播过程对应反卷积层的前向传播过程。反卷积操作可以将各层得到的特征图进行反卷积以得到可视化的图像，也就是说，反卷积操作可以将图像的特征信息从特征图空间转化到像素空间。反卷积操作可以对卷积层输出的特征图进行反卷积，以得到反卷积结果。该反卷积结果可以显示卷积层提取到的特征信息。
例如，通过连接层将相同尺度的卷积层的输出和反卷积层的输出进行连接，即连接浅层特征和深层特征以进行补偿，从而减少压缩和下采样导致的空间信息丢失。此处，所称的连接，例如可以是合并concatenate操作，即通过内存映射的方式将卷积层和反卷积层中具有相同大小的特征映射连接(使得特征对应的向量合并，特征所在层的通道数加倍)。
例如，每个卷积层/反卷积层的输出均被传输至一个残差块，这样可保证不同尺度的特征信息在各层之间的流通。
图7为本公开一些实施例提供的一种调色处理和降噪处理的示意性流程图。
例如，如图7所示，首先，对影像input4进行预处理(preprocess)以得到归一化数据，预处理包括归一化处理，即将影像input4的所有像素的像素值进行归一化处理，以便于后续模型的处理。然后，利用回归子模型对归一化数据进行处理，以得到至少一个权重参数(W1、W2、…、Wn，n为正整数)。然后，基于用户输入的调色参数，可以从多个查找表子模型中选择与用户输入的调色参数对应的查找表子模型组3D LUT1～3D LUTN。然后，基于至少一个权重参数W1～Wn和查找表子模型组3D LUT1～3D LUTN，确定目标查找表子模型3D LUT。然后，利用目标查找表子模型3D LUT对归一化数据进行处理，以生成调色处理的输出。然后，利用图6所示第四深度学习模型对调色处理的输出进行降噪处理，以得到降噪处理的输出output4。类似地，若影像input2的像素格式不是RGB格式，则预处理还可以包括格式转换处理。此时，在得到调色处理的输出之后，利用图6所示第四深度学习模型对调色处理的输出进行降噪处理，以得到第四深度学习模型的输出，然后对第四深度学习模型的输出进行后处理以得到降噪处理的输出output4，后处理可以包括格式转换处理，即 将第四深度学习模型的输出进行格式变换，以得到降噪处理的输出output4。
例如，在一些实施例中，画质增强处理用于对输入影像依次进行插帧处理、调色处理、降噪处理、超分辨率处理等，从而使得增强后影像的分辨率达到4K或8K等，增强后影像的帧率达到50～60fps，增强后影像的色深(色位深度)达到10～12比特(bits)，增强后影像的色域为BT.2020，并使得增强后影像具有一定的色彩风格。
例如，在一些实施例中，输入影像包括视频，增强后影像包括与视频对应的增强后视频。如图2所示，在一些实施例中，步骤S11可以包括：对视频(输入影像)进行插帧处理，以得到插帧后视频；对插帧后视频进行调色处理，以得到调色后视频；对调色后视频进行降噪处理，以得到降噪后视频；对降噪后视频进行超分辨率处理，以得到增强后视频。
例如，可以基于由用户输入的插帧处理对应的插帧参数对视频进行插帧处理，以得到插帧后视频；可以基于由用户输入的调色处理对应的调色参数对插帧后视频进行调色处理，以得到调色后视频；可以基于由用户输入的降噪处理对应的降噪参数对调色后视频进行降噪处理，以得到降噪后视频；可以基于由用户输入的超分辨率处理对应的分辨率参数对降噪后视频进行超分辨率处理，以得到增强后视频。
例如，增强后视频的分辨率高于降噪后视频的分辨率。
例如，调色后视频的色位深度高于插帧后视频对应的色位深度，和/或，调色后视频对应的色域高于插帧后视频对应的色域。
例如，增强后视频的分辨率可以为4K或8K，增强后视频的帧率达到50～60fps，增强后视频的色深达到10～12比特(bits)，增强后视频的色域可以为BT.2020。
需要说明的是，在本公开的实施例中，深度学习模型(上述第一深度学习模型至第四深度学习模型)均在tensorRT框架下实现。例如，深度学习模型在执行相关处理之前，需要进行初始化。初始化是为了在程序启动时，把所有的深度学习模型、参数、驱动等环境进行设置和导入，为程序运行做准备，进而避免程序运行过程中重复调用相关配置模块而导致处理速度变慢，由此增加处理速度。
例如，对输入影像进行画质增强处理之后可以得到的增强后影像，增强后影像会发送到例如第一显示面板中进行显示。然而，第一显示面板的材质、显 示特性等多种多样，第一显示面板所处的环境的环境光也有差别，这些都会导致第一显示面板的显示效果达不到预期的状态。为了补偿不同显示面板和环境光的影响，可以对增强后影像进行补偿处理，以对增强后影像的内容进行二次增强，以期在不同条件下影像的显示均可达到较好的状态，满足用户的显示需求。
如图1B所示，在步骤S12，对增强后影像进行补偿处理，以得到补偿后影像。
例如，补偿后影像对应的分辨率高于输入影像对应的分辨率。例如，补偿后影像对应的分辨率可以为4K、8K等超高清分辨率。补偿后影像的帧率可以为50～60fps，补偿后影像的色深可以为10～12比特(bits)，补偿后影像的色域可以为BT.2020。
在本公开的实施例提供的影像处理方法中，通过对输入影像进行画质增强处理和补偿处理，以实现对输入影像进行二次增强，从而使得生成的补偿后影像能够适应不同环境光和显示屏条件下的正常显示，适应更多应用场景，且在不同应用场景均具有优异的显示效果。而且，通过集成人工智能(Artificial Intelligence，AI)技术，设计应用模式，使影像处理过程变得更加自动化、智能化。
在本公开提供的影像处理方法中，首先执行画质增强处理，并在执行画质增强处理之后，执行补偿处理。
例如，在一些实施例中，补偿后影像对应的色位深度高于输入影像对应的色位深度，和/或，补偿后影像对应的色域高于输入影像对应的色域。
例如，补偿处理包括以下至少一项：亮度调整、对比度调整和饱和度调整等，也就是说，在补偿处理中，可以对增强后影像的亮度、对比度、饱和度等进行补偿调整，从而得到补偿后影像。
例如，在一些示例中，补偿处理包括亮度调整、对比度调整和饱和度调整，亮度调整和对比度调整可以由不同参数的映射曲线构成，如指数函数(exp)曲线、对数函数(log)曲线、sigmoid(S型函数)曲线、多项式曲线等。饱和度调整则由HSV(Hue，Saturation，Value)色彩空间进行参数调整和计算。
例如，每条补偿曲线对应一种面板参数和环境参数，均由亮度、对比度、饱和度算法结合构成。补偿参数可以与该补偿曲线对应。
例如，在一些实施例中，步骤S12包括：从多个补偿参数中选择与第一显 示面板对应的补偿参数；基于与第一显示面板对应的补偿参数，对增强后影像进行补偿处理，以得到补偿后影像。
例如，在一些实施例中，在步骤S12中，从多个补偿参数中选择与第一显示面板对应的补偿参数，包括：获取与第一显示面板对应的补偿输入参数，其中，补偿输入参数包括第一显示面板对应的面板参数和/或第一显示面板所处的环境的环境参数；基于补偿输入参数，从多个补偿参数中选择与补偿输入参数对应的补偿参数作为与第一显示面板对应的补偿参数。
例如，面板参数包括第一显示面板的类型、第一显示面板的尺寸以及第一显示面板的显示方式等。
例如，第一显示面板的类型包括发光显示(LED)面板和液晶显示(LCD)面板等，发光显示面板可以包括有机发光二极管显示面板、量子点发光二极管显示面板等。第一显示面板的显示方式包括直接显示和投屏显示等。第一显示面板的尺寸可以为15英寸、17英寸、19英寸、20英寸、24英寸、32英寸等。
例如，环境参数包括环境光参数等，环境光参数基于第一显示面板所在的环境的环境光的亮度值确定。又例如，环境光参数还可以基于第一显示面板所在的环境的环境光的颜色等确定。
例如，补偿处理中的各项处理(例如，上述调整亮度、调整对比度和调整饱和度等)均可灵活配置，且可以根据需求对应设置参数。例如，调整亮度、调整对比度和调整饱和度中的一个或多个可以不执行，调整亮度、调整对比度和调整饱和度也可以均执行。
例如，在一些实施例中，在步骤S12中，获取与第一显示面板对应的补偿输入参数，包括：生成第一数组，其中，第一数组与第一显示面板对应，第一数组包括多个第一数组元素，面板参数由至少一个第一数组元素表示，环境参数由至少一个第一数组元素表示；基于第一数组，确定与第一显示面板对应的补偿输入参数。
例如，补偿输入参数可被定义为整型数组格式，即第一数组为整型数组，第一数组包括多个第一数组元素，每个第一数组元素对应一个含义。根据每个第一数组元素的数值即可确定其对应的处理是否需要执行。例如，以调整亮度为例，若表示调整亮度的第一数组元素的值为第一数值，则表示不执行调整亮度的处理；若表示调整亮度的第一数组元素的值为第二数值，则表示执行调整亮度的处理。例如，在一些实施例中，第一数值为0，第二数值为1，然而本公 开不限于此。
例如，在一些实施例中，第一数组中的第一数组元素0(即第一数组中的第一个元素)表示面板参数，第一数组中的第一数组元素1(即第一数组中的第二个元素)表示环境参数中的环境光参数。对补偿输入参数对应的第一数组中的各个第一数组元素的定义如下：第一数组元素0表示显示面板的模式(即面板参数)，当第一数组元素0为0，则表示显示面板为LED面板，当第一数组元素0为1，则表示显示面板的尺寸大于等于尺寸阈值，当第一数组元素0为2，则表示显示面板的显示方式为投屏显示，当第一数组元素0为3，则表示显示面板为LCD面板，等等，具体根据实际情况设置。第一数组元素1表示环境光参数，当第一数组元素1为0，则表示环境光的亮度值小于1000流明，当第一数组元素1为1，则表示环境光的亮度值位于1000至2000流明之间；当第一数组元素1为2，则表示环境光的亮度值位于2000至3000流明之间；等等，具体根据实际情况设置。补偿输入参数对应的第一数组中的第一数组元素的数量、各个第一数组元素表示的含义等可以根据具体情况设置，本公开对此不作限制。
需要说明的是，第一数组中的各个第一数组元素的值可以预先设定默认值。例如，可以根据显示面板自身的特性预先设定表示面板参数的第一数组元素的默认值。此外，对于环境光参数，可以预先设置表示环境光参数的第一数组元素的默认值，也可以在显示面板中设置传感器，以感测显示面板所处的环境的环境光的亮度，从而基于该感测的亮度自动设置表示环境光参数的第一数组元素的值。本公开不限于此，用户也可以关闭传感器的功能，从而根据用户的输入确定表示环境光参数的第一数组元素的值。
例如，尺寸阈值可以根据实际需求设置，例如，在一些示例中，尺寸阈值可以为24英寸。
例如，在一些实施例中，影像处理方法还可以包括：生成多个补偿参数。例如，可以预先在各种不同的情况下生成多个补偿参数，从而使得在实际应用过程中，可以根据实际情况选择对应的补偿参数。每个情况下的补偿参数对应一种显示面板和一种环境光的亮度值。
例如，生成多个补偿参数包括：获取对标准色卡进行拍摄得到的色卡影像；对色卡影像进行画质增强处理，以得到增强后色卡影像；基于初始补偿参数对增强后色卡影像进行补偿处理，以得到补偿后色卡影像；在第二显示面板上显 示补偿后色卡影像，并确定补偿后色卡影像的显示是否满足预设要求；响应于补偿后色卡影像的显示未达到预设要求，则对初始补偿参数进行调整以得到调整后的补偿参数，并基于调整后的补偿参数再次对增强后色卡影像进行补偿处理，直至得到的补偿后色卡影像的显示满足预设要求，将满足预设要求的补偿后色卡影像所对应的补偿参数作为多个补偿参数中的一个补偿参数；确定与第二显示面板对应的补偿输入参数，其中，第二显示面板对应的补偿输入参数包括第二显示面板对应的面板参数和/或第二显示面板所处的环境的环境参数；建立满足预设要求的补偿后色卡影像所对应的补偿参数与第二显示面板对应的补偿输入参数之间的映射关系。
例如，标准色卡可以为24色卡。例如，预设要求可以表示多个观察员观察补偿后色卡影像是否正常显示，例如，多个观察员的人数可以为4～6，多个观察员中的男性的数量和女性的数量均为多个观察员的人数的一半。例如，观察员观察补偿后色卡影像正常显示表示观察员可以分辨的补偿后色卡影像中的色卡格数大于18格。本公开不限于此，例如，多个观察员的人数可以更多或更少，观察员观察补偿后色卡影像正常显示表示观察员可以分辨的补偿后色卡影像中的色卡格数也可以大于16、17、19、20格等。
例如，在生成多个补偿参数后，可将补偿参数与对应的面板参数和/或环境参数进行一一对应，从而便于在后期使用过程中基于面板参数和/或环境参数选择需要的补偿参数。
例如，第一显示面板和第二显示面板可以为同一显示面板，也可以为不同显示面板。
例如，在生成多个补偿参数时，可以采用多个不同的显示面板(例如，类型、尺寸、显示方式等不同)在同一环境光下显示补偿后色卡影像，从而得到与不同的显示面板分别对应的补偿参数(这些补偿参数与同一环境光对应)。也可以利用同一显示面板在不同的环境光下显示补偿后色卡影像，从而得到与不同环境光分别对应的补偿参数(这些补偿参数与同一显示面板对应)。需要说明的是，也可以同时改变显示面板的面板参数和环境参数。
例如，在一些实施例中，补偿后影像的像素格式为RGB格式。影像处理方法还包括：对补偿后影像进行格式转换，以得到输出影像；对输出影像进行编码处理，以得到输出影像文件。
例如，输出影像的像素格式为YUV格式。
例如，输出影像的分辨率可以与补偿后影像的分辨率相同，输出影像的色深可以与补偿后影像的色深相同，输出影像的色域可以与补偿后影像的色域相同。
例如，输出影像可以直接用于在第一显示面板上进行显示。而为了便于传输和存储，可以对输出影像进行编码，以生成输出影像文件，该输出影像文件可以用于在存储器中进行存储，也可以用于在不同显示装置之间进行传输。
例如，在一个实施例中，输出影像包括输出视频，如图2所示，在一个示例中，在得到增强后视频之后，可以对增强后视频进行补偿处理，以得到补偿后视频；然后，对补偿后视频进行格式转换，以得到输出视频；最后，对输出视频进行编码处理，以得到输出视频文件。此时，输出视频的像素格式可以为YUV格式。
图8为本公开一些实施例提供的一种影像处理装置的示意图。该影像处理装置可以用于实现本公开任一实施例提供的影像处理方法。
例如，如图8所示，影像处理装置100包括一个或多个存储器101和一个或多个处理器102。存储器101被配置为非瞬时性地存储有计算机可执行指令；处理器102被配置为运行计算机可执行指令。计算机可执行指令被处理器运行时实现根据本公开任一实施例所述的影像处理方法。关于该影像处理方法的各个步骤的具体实现以及相关解释内容可以参见上述影像处理方法的实施例，在此不做赘述。
例如，如图8所示，影像处理装置100还包括输入装置103。响应于影像处理方法包括获取与第一显示面板对应的补偿输入参数和/或获取画质增强参数，补偿输入参数和/或画质增强参数通过输入装置103输入。例如，补偿输入参数和/或画质增强参数可以通过实体按钮、虚拟按钮(例如，触控面板/显示面板上的图标)、语音等方式输入。
例如，输入装置103可以包括触摸屏、触摸板、键盘、鼠标、麦克风、电子笔等。
例如，在一些实施例中，影像处理装置100还包括显示装置，显示装置包括显示面板(即影像处理方法中的第一显示面板)，显示面板用于显示输出影像。
例如，影像处理装置100还可以包括通信接口和通信总线。存储器101、处理器102、输入装置103和通信接口通过通信总线实现相互通信。存储器101、 处理器102、输入装置103和通信接口等组件之间也可以通过网络连接进行通信。本公开对网络的类型和功能在此不作限制。例如，当影像处理装置100包括多个处理器102时，多个处理器102之间也可以通过通信总线或网络进行通信。
例如，处理器102执行存储器101上所存放的计算机可执行指令而实现的影像处理方法的其他实现方式，与前述影像处理方法的实施例中所提及的实现方式相同，这里也不再赘述。
例如，通信总线可以是外设部件互连标准(PCI)总线或扩展工业标准结构(EISA)总线等。该通信总线可以分为地址总线、数据总线、控制总线等。
例如，通信接口用于实现影像处理装置100与其他设备之间的通信。
例如，处理器102和存储器101可以设置在服务器端(或云端)，也可以设置在客户端(例如，手机等移动设备)。
例如，处理器102可以控制影像处理装置100中的其它组件以执行期望的功能。处理器102可以是中央处理器(CPU)、网络处理器(NP)等；还可以是数字信号处理器(DSP)、专用集成电路(ASIC)、现场可编程门阵列(FPGA)或者其他可编程逻辑器件、分立门或者晶体管逻辑器件、分立硬件组件。中央处理元(CPU)可以为X86或ARM架构等。GPU可以单独地直接集成到主板上，或者内置于主板的北桥芯片中。GPU也可以内置于中央处理器(CPU)上。
例如，存储器101可以包括一个或多个计算机程序产品的任意组合，计算机程序产品可以包括各种形式的计算机可读存储介质，例如易失性存储器和/或非易失性存储器。易失性存储器例如可以包括随机存取存储器(RAM)和/或高速缓冲存储器(cache)等。非易失性存储器例如可以包括只读存储器(ROM)、硬盘、可擦除可编程只读存储器(EPROM)、便携式紧致盘只读存储器(CD-ROM)、USB存储器、闪存等。在所述计算机可读存储介质上可以存储一个或多个计算机可执行指令，处理器102可以运行计算机可执行指令，以实现影像处理装置100的各种功能。在存储器101中还可以存储各种应用程序和各种数据等。
需要说明的是，影像处理装置100可以实现与前述影像处理方法相似的技术效果，重复之处不再赘述。
图9为本公开至少一个实施例提供的另一种影像处理装置的示意图，图10为本公开至少一个实施例提供的另一种影像处理装置的示意图。图9所示的影像处理装置可以用于实现本公开任一实施例提供的影像处理方法。
例如，如图9所示，影像处理装置200可以包括：获取模块201和画质增强处理模块202。
例如，获取模块201用于获取输入影像。获取模块201用于实现图1A和图1B所示的步骤S10，关于获取模块201所实现的功能的具体说明可以参考上述影像处理方法的实施例中对于图1A和图1B所示的步骤S10的相关描述，重复之处不再赘述。
例如，画质增强处理模块202用于对输入影像进行画质增强处理，以得到增强后影像。例如，增强后影像对应的分辨率高于输入影像对应的分辨率。画质增强处理模块202用于实现图1A和图1B所示的步骤S11，关于画质增强处理模块202所实现的功能的具体说明可以参考上述影像处理方法的实施例中对于图1A和图1B所示的步骤S11的相关描述，重复之处不再赘述。
例如，画质增强处理包括以下处理中的至少一项：插帧处理、超分辨率处理、降噪处理、调色处理、HDR上变换处理、细节修复处理等。如图10所示，画质增强处理模块202包括以下子模块中的至少一个：插帧子模块2021、超分辨率子模块2022、调色子模块2023、降噪子模块2024、HDR上变换子模块和细节修复子模块等。插帧子模块2021用于实现插帧处理，超分辨率子模块2022用于实现超分辨率处理，调色子模块2023用于实现调色处理，降噪子模块2024用于实现降噪处理，HDR上变换子模块用于实现HDR上变换处理，细节修复子模块用于实现细节修复处理。
例如，插帧子模块2021包括第一深度学习模型，且用于基于第一深度学习模型对插帧子模块2021的输入进行插帧处理，插帧子模块2021的输入包括视频，插帧处理包括在视频的每两个图像帧之间增加至少一个过渡图像帧的处理。
例如，超分辨率子模块2022包括第二深度学习模型，且用于基于第二深度学习模型对超分辨率子模块2022的输入进行超分辨率处理，以提高超分辨率子模块2022的输入的空间分辨率。
例如，调色子模块2023包括第三深度学习模型，且用于基于第三深度学习模型对调色子模块2023的输入进行调色处理。
例如，降噪子模块2024包括第四深度学习模型，且用于基于第四深度学习模型对降噪子模块2024的输入进行降噪处理。
需要说明的是，关于插帧子模块2021、超分辨率子模块2022、调色子模 块2023和降噪子模块2024所实现的功能的具体说明可以参考上述影像处理方法的实施例中对于插帧处理、超分辨率处理、降噪处理和调色处理的相关描述，重复之处不再赘述。
例如，如图9所示，影像处理装置200还可以包括补偿处理模块203。例如，补偿处理模块203用于对增强后影像进行补偿处理，以得到补偿后影像。补偿后影像对应的分辨率高于输入影像对应的分辨率。补偿处理模块203用于实现图1B所示的步骤S12，关于补偿处理模块203所实现的功能的具体说明可以参考上述影像处理方法的实施例中对于图1B所示的步骤S12的相关描述，重复之处不再赘述。
例如，在一些实施例中，补偿后影像的像素格式为RGB格式，
如图9所示，影像处理装置还包括编码模块205，编码模块205用于对补偿后影像进行格式转换，以得到输出影像；对输出影像进行编码处理，以得到输出影像文件。
例如，输出影像的像素格式为YUV格式。
例如，在一些实施例中，影像处理装置200还包括显示装置，显示装置包括第一显示面板，第一显示面板用于显示输出影像。
例如，在一些实施例中，如图10所示，补偿处理模块203包括选择子模块2031和处理子模块2032。选择子模块2031用于从多个补偿参数中选择与第一显示面板对应的补偿参数；处理子模块2032用于基于与第一显示面板对应的补偿参数，对增强后影像进行补偿处理，以得到补偿后影像。
例如，在一些实施例中，选择子模块2031包括获取单元和选择单元。例如，获取单元用于获取与第一显示面板对应的补偿输入参数，补偿输入参数包括第一显示面板对应的面板参数和/或第一显示面板所处的环境的环境参数。例如，选择单元用于基于补偿输入参数，从多个补偿参数中选择与补偿输入参数对应的补偿参数作为与第一显示面板对应的补偿参数。
例如，如图9和图10所示，影像处理装置200还包括输入装置204。如图10所示，画质增强参数通过输入装置204被输入至画质增强处理模块202，补偿输入参数通过输入装置204被输入至补偿处理模块203。
例如，获取单元可以与输入装置204连接，以接收从输入装置204输入的补偿输入参数。
例如，例如，获取模块201、画质增强处理模块202、补偿处理模块203、 输入装置204和编码模块205之间可以进行数据通信。
例如，获取模块201、画质增强处理模块202、补偿处理模块203和/或编码模块205包括存储在存储器中的代码和程序；处理器可以执行该代码和程序以实现如上所述的获取模块201、画质增强处理模块202、补偿处理模块203和/或编码模块205的一些功能或全部功能。例如，获取模块201、画质增强处理模块202、补偿处理模块203和/或编码模块205可以是专用硬件器件，用来实现如上所述的获取模块201、画质增强处理模块202、补偿处理模块203和/或编码模块205的一些或全部功能。例如，获取模块201、画质增强处理模块202、补偿处理模块203和/或编码模块205可以是一个电路板或多个电路板的组合，用于实现如上所述的功能。在本申请实施例中，该一个电路板或多个电路板的组合可以包括：(1)一个或多个处理器；(2)与处理器相连接的一个或多个非暂时的存储器；以及(3)处理器可执行的存储在存储器中的固件。
例如，输入装置204可以包括触摸屏、触摸板、键盘、鼠标、麦克风、电子笔等。
需要说明的是，影像处理装置可以实现与前述影像处理方法相似的技术效果，在此不再赘述。
图11为本公开至少一个实施例提供的一种非瞬时性计算机可读存储介质的示意图。例如，如图11所示，在非瞬时性计算机可读存储介质1100上可以非暂时性地存储一个或多个计算机可执行指令1101。例如，当计算机可执行指令1101由处理器(或计算机)执行时可以执行根据本公开的任一实施例所述的影像处理方法中的一个或多个步骤。
例如，该非瞬时性计算机可读存储介质1100可以应用于上述影像处理装置100中，例如，其可以包括影像处理装置100中的存储器101。
例如，关于非瞬时性计算机可读存储介质1100的说明可以参考影像处理装置100的实施例中对于存储器101的描述，重复之处不再赘述。
图12为本公开至少一个实施例提供的一种硬件环境的示意图。本公开提供的电子设备可以应用在互联网系统。
利用图12中提供的计算机系统可以实现本公开中涉及的影像处理装置的功能。这类计算机系统可以包括个人电脑、笔记本电脑、平板电脑、手机、个人数码助理、智能眼镜、智能手表、智能指环、智能头盔及任何智能便携设备或可穿戴设备等。本实施例中的特定系统利用功能框图解释了一个包含用户界 面的硬件平台。这种计算机设备可以是一个通用目的的计算机设备，或一个有特定目的的计算机设备。两种计算机设备都可以被用于实现本公开的实施例中的影像处理装置。计算机系统可以包括实施当前描述的实现影像处理所需要的信息的任何组件。例如，计算机系统能够被计算机设备通过其硬件设备、软件程序、固件以及它们的组合所实现。为了方便起见，图12中只绘制了一台计算机设备，但是本实施例所描述的实现影像处理所需要的信息的相关计算机功能是可以以分布的方式、由一组相似的平台所实施的，分散计算机系统的处理负荷。
如图12所示，计算机系统可以包括通信端口250，与之相连的是实现数据通信的网络(图12中的“来自/去往网络”)，例如，计算机系统可以通过通信端口250发送和接收信息及数据，即通信端口250可以实现计算机系统与其他电子设备进行无线或有线通信以交换数据。计算机系统还可以包括一个处理器组220(即上面描述的处理器)，用于执行程序指令。处理器组220可以由至少一个处理器(例如，CPU)组成。计算机系统可以包括一个内部通信总线210。计算机系统可以包括不同形式的程序储存单元以及数据储存单元(即上面描述的存储器或存储介质)，例如硬盘270、只读存储器(ROM)230、随机存取存储器(RAM)240，能够用于存储计算机处理和/或通信使用的各种数据文件，以及处理器组220所执行的可能的程序指令。计算机系统还可以包括一个输入/输出260，输入/输出260用于实现计算机系统与其他组件(例如，用户界面280等)之间的输入/输出数据流。
通常，以下装置可以连接输入/输出260：包括例如触摸屏、触摸板、键盘、鼠标、摄像头、麦克风、加速度计、陀螺仪等的输入装置；包括例如液晶显示器(LCD)、扬声器、振动器等的输出装置；包括例如磁带、硬盘等的存储装置；以及通信接口。
虽然图12示出了具有各种装置的计算机系统，但应理解的是，并不要求计算机系统具备所有示出的装置，可以替代地，计算机系统可以具备更多或更少的装置。
对于本公开，还有以下几点需要说明：
(1)本公开实施例附图只涉及到与本公开实施例涉及到的结构，其他结构可参考通常设计。
(2)为了清晰起见，在用于描述本发明的实施例的附图中，层或结构的厚 度和尺寸被放大。可以理解，当诸如层、膜、区域或基板之类的元件被称作位于另一元件“上”或“下”时，该元件可以“直接”位于另一元件“上”或“下”，或者可以存在中间元件。
(3)在不冲突的情况下，本公开的实施例及实施例中的特征可以相互组合以得到新的实施例。
以上所述仅为本公开的具体实施方式，但本公开的保护范围并不局限于此，本公开的保护范围应以所述权利要求的保护范围为准。

Claims (29)

  1. 一种影像处理方法，包括：
    获取输入影像；
    对所述输入影像进行画质增强处理，以得到增强后影像；
    其中，所述增强后影像对应的分辨率高于所述输入影像对应的分辨率。
  2. 根据权利要求1所述的影像处理方法，还包括：
    对所述增强后影像进行补偿处理，以得到补偿后影像，
    其中，所述补偿后影像对应的分辨率高于所述输入影像对应的分辨率。
  3. 根据权利要求2所述的影像处理方法，其中，所述影像处理方法应用于第一显示面板，
    对所述增强后影像进行补偿处理，以得到补偿后影像，包括：
    从多个补偿参数中选择与所述第一显示面板对应的补偿参数；
    基于与所述第一显示面板对应的所述补偿参数，对所述增强后影像进行补偿处理，以得到所述补偿后影像。
  4. 根据权利要求3所述的影像处理方法，其中，从多个补偿参数中选择与所述第一显示面板对应的补偿参数，包括：
    获取与所述第一显示面板对应的补偿输入参数，其中，所述补偿输入参数包括所述第一显示面板对应的面板参数和/或所述第一显示面板所处的环境的环境参数；
    基于所述补偿输入参数，从所述多个补偿参数中选择与所述补偿输入参数对应的补偿参数作为与所述第一显示面板对应的所述补偿参数。
  5. 根据权利要求4所述的影像处理方法，其中，获取与所述第一显示面板对应的补偿输入参数，包括：
    生成第一数组，其中，所述第一数组与所述第一显示面板对应，所述第一数组包括多个第一数组元素，所述面板参数由至少一个第一数组元素表示，所述环境参数由至少一个第一数组元素表示；
    基于所述第一数组，确定与所述第一显示面板对应的所述补偿输入参数。
  6. 根据权利要求4或5所述的影像处理方法，其中，所述面板参数包括所述第一显示面板的类型、所述第一显示面板的尺寸以及所述第一显示面板的显示方式。
  7. 根据权利要求6所述的影像处理方法，其中，所述第一显示面板的类型包括有机发光显示面板和液晶显示面板，所述第一显示面板的显示方式包括直接显示和投屏显示。
  8. 根据权利要求4～7任一项所述的影像处理方法，其中，所述环境参数包括环境光参数，
    所述环境光参数基于所述第一显示面板所在的环境的环境光的亮度值确定。
  9. 根据权利要求2-8任一项所述的影像处理方法，其中，所述补偿处理包括以下至少一项：亮度调整、对比度调整和饱和度调整。
  10. 根据权利要求3～9任一项所述的影像处理方法，还包括：生成所述多个补偿参数，
    其中，生成所述多个补偿参数包括：
    获取对标准色卡进行拍摄得到的色卡影像；
    对所述色卡影像进行画质增强处理，以得到增强后色卡影像；
    基于初始补偿参数对所述增强后色卡影像进行补偿处理，以得到补偿后色卡影像；
    在第二显示面板上显示所述补偿后色卡影像，并确定所述补偿后色卡影像的显示是否满足预设要求；
    响应于所述补偿后色卡影像的显示未达到所述预设要求，则对所述初始补偿参数进行调整以得到调整后的补偿参数，并基于所述调整后的补偿参数再次对所述增强后色卡影像进行补偿处理，直至得到的补偿后色卡影像的显示满足所述预设要求，将满足所述预设要求的补偿后色卡影像所对应的补偿参数作为所述多个补偿参数中的一个补偿参数；
    确定与所述第二显示面板对应的补偿输入参数，其中，所述第二显示面板对应的补偿输入参数包括所述第二显示面板对应的面板参数和/或所述第二显示面板所处的环境的环境参数；
    建立满足所述预设要求的补偿后色卡影像所对应的补偿参数与所述第二显示面板对应的补偿输入参数之间的映射关系。
  11. 根据权利要求1～10任一项所述的影像处理方法，其中，所述画质增强处理包括以下处理中的至少一项：插帧处理、超分辨率处理、降噪处理、调色处理、高动态范围上变换处理、细节修复处理。
  12. 根据权利要求11所述的影像处理方法，其中，对所述输入影像进行所述画质增强处理，以得到所述增强后影像，包括：
    获取画质增强参数；
    根据所述画质增强参数对所述输入影像进行所述画质增强处理，以得到所述增强后影像。
  13. 根据权利要求12所述的影像处理方法，其中，所述画质增强参数包括以下参数中的至少一项：所述插帧处理对应的插帧算法名称和/或插帧参数、所述超分辨率处理对应的超分辨率算法名称和/或分辨率参数、所述调色处理对应的调色算法名称和/或调色参数和所述降噪处理对应的降噪算法名称和/或降噪参数、所述高动态范围上变换处理对应的高动态范围上变换算法和/或高动态范围上变换参数、所述细节修复处理对应的细节修复算法名称和/或细节修复参数。
  14. 根据权利要求13所述的影像处理方法，其中，获取画质增强参数包括：
    生成第二数组，其中，所述第二数组包括多个第二数组元素，所述插帧处理对应的插帧参数由至少一个第二数组元素表示，所述超分辨率处理对应的分辨率参数由至少一个第二数组元素表示，所述调色处理对应的调色参数由至少一个第二数组元素表示，所述降噪处理对应的降噪参数由至少一个第二数组元素表示，所述高动态范围上变换处理对应的高动态范围上变换参数由至少一个第二数组元素表示，所述细节修复变换处理对应的细节修复参数由至少一个第二数组元素表示；
    基于所述第二数组，确定所述画质增强参数。
  15. 根据权利要求13所述的影像处理方法，其中，获取画质增强参数包括：
    获取算法字符串，其中，所述算法字符串包括以下算法中的至少一个：插帧算法、超分辨率算法、调色算法、降噪算法、高动态范围上变换算法、细节修复算法，所述插帧算法包括所述插帧算法名称和所述插帧参数，所述超分辨率算法包括所述超分辨率算法名称和所述超分辨率参数，所述调色算法包括所述调色算法名称和所述调色参数，所述降噪算法包括所述降噪算法名称和所述降噪参数，所述高动态范围上变换算法包括所述高动态范围上变算法名称和所述高动态范围上变换参数，所述细节修复算法包括所述细节修复算法名称和所 述细节修改参数；
    基于所述算法字符串，确定所述画质增强参数。
  16. 根据权利要求11～15任一项所述的影像处理方法，其中，用于进行所述插帧处理的影像包括视频，
    所述插帧处理基于第一深度学习模型实现，所述第一深度学习模型被配置为在所述视频中的每两个图像帧之间增加至少一个过渡图像帧，
    所述超分辨率处理基于第二深度学习模型实现，所述第二深度学习模型被配置为对用于进行所述超分辨率处理的影像进行超分辨率处理，以提高用于进行所述超分辨率处理的影像的空间分辨率，
    所述调色处理基于第三深度学习模型实现；
    所述降噪处理基于第四深度学习模型实现，所述第四深度学习模型被配置为对用于进行所述降噪处理的影像进行降噪处理。
  17. 根据权利要求16所述的影像处理方法，其中，所述第三深度学习模型包括回归子模型和多个查找表子模型组，每个查找表子模型组包括至少一个查找表子模型；
    所述调色处理包括：
    对用于进行所述调色处理的影像进行预处理，以得到归一化数据，其中，所述预处理包括归一化处理；
    利用所述回归子模型对所述归一化数据进行处理，以得到至少一个权重参数；
    获取调色参数；
    根据所述调色参数，从所述多个查找表子模型中选择与所述调色参数对应的查找表子模型组；
    基于所述至少一个权重参数和所述查找表子模型组，确定目标查找表子模型；
    利用所述目标查找表子模型对所述归一化数据进行处理，以生成所述调色处理的输出。
  18. 根据权利要求16或17所述的影像处理方法，其中，所述第一深度学习模型包括实时中间流估计算法模型。
  19. 根据权利要求16～18任一项所述的影像处理方法，其中，所述第二深度学习模型包括残差特征蒸馏网络模型。
  20. 根据权利要求16～19任一项所述的影像处理方法，其中，所述第四深度学习模型包括Unet网络模型。
  21. 根据权利要求11～20任一项所述的影像处理方法，其中，所述输入影像包括视频，所述增强后影像包括与所述视频对应的增强后视频，
    对所述输入影像进行所述画质增强处理，以得到所述增强后影像，包括：
    对所述视频进行所述插帧处理，以得到插帧后视频；
    对所述插帧后视频进行所述调色处理，以得到调色后视频；
    对所述调色后视频进行所述降噪处理，以得到降噪后视频；
    对所述降噪后视频进行所述超分辨率处理，以得到所述增强后视频，
    其中，所述增强后视频的分辨率高于所述降噪后视频的分辨率。
  22. 根据权利要求21所述的影像处理方法，其中，所述调色后视频的色位深度高于所述插帧后视频对应的色位深度，和/或，所述调色后视频对应的色域高于所述插帧后视频对应的色域。
  23. 根据权利要求1～22任一项所述的影像处理方法，其中，获取输入影像包括：
    获取原始影像文件；
    对所述原始影像文件进行解码处理，以得到原始影像；
    对所述原始影像进行格式转换处理，以得到所述输入影像，
    其中，所述输入影像的像素格式包括RGB格式。
  24. 根据权利要求2～23任一项所述的影像处理方法，其中，所述补偿后影像的像素格式为RGB格式，
    所述影像处理方法还包括：
    对所述补偿后影像进行格式转换，以得到输出影像，其中，所述输出影像的像素格式为YUV格式；
    对所述输出影像进行编码处理，以得到输出影像文件。
  25. 根据权利要求2～24任一项所述的影像处理方法，其中，所述补偿后影像对应的色位深度高于所述输入影像对应的色位深度，和/或，所述补偿后影像对应的色域高于所述输入影像对应的色域。
  26. 一种影像处理装置，包括：
    一个或多个存储器，非瞬时性地存储有计算机可执行指令；
    一个或多个处理器，配置为运行所述计算机可执行指令，
    其中，所述计算机可执行指令被所述一个或多个处理器运行时实现根据权利要求1～25任一项所述的影像处理方法。
  27. 根据权利要求26所述的影像处理装置，还包括：输入装置，
    其中，响应于所述影像处理方法包括获取与所述第一显示面板对应的补偿输入参数和/或获取画质增强参数，
    所述补偿输入参数和/或所述画质增强参数通过所述输入装置输入。
  28. 根据权利要求27所述的影像处理装置，其中，所述输入装置包括触摸屏、触摸板、键盘、鼠标、麦克风。
  29. 一种非瞬时性计算机可读存储介质，其中，所述非瞬时性计算机可读存储介质存储有计算机可执行指令，所述计算机可执行指令被处理器执行时实现根据权利要求1～25中任一项所述的影像处理方法。
PCT/CN2023/083986 2022-03-31 2023-03-27 影像处理方法、影像处理装置、存储介质 WO2023185706A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202210343186.5A CN116939225A (zh) 2022-03-31 2022-03-31 影像处理方法、影像处理装置、存储介质
CN202210343186.5 2022-03-31

Publications (1)

Publication Number Publication Date
WO2023185706A1 true WO2023185706A1 (zh) 2023-10-05

Family

ID=88199389

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2023/083986 WO2023185706A1 (zh) 2022-03-31 2023-03-27 影像处理方法、影像处理装置、存储介质

Country Status (2)

Country Link
CN (1) CN116939225A (zh)
WO (1) WO2023185706A1 (zh)

Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2007329678A (ja) * 2006-06-07 2007-12-20 Matsushita Electric Ind Co Ltd 輝度マスキング及び輝度補償を用いた超解像処理方法及び装置
CN108391060A (zh) * 2018-03-26 2018-08-10 华为技术有限公司 一种图像处理方法、图像处理装置和终端
CN109889800A (zh) * 2019-02-28 2019-06-14 深圳市商汤科技有限公司 图像增强方法和装置、电子设备、存储介质
CN111476719A (zh) * 2020-05-06 2020-07-31 Oppo广东移动通信有限公司 图像处理方法、装置、计算机设备及存储介质
CN112184548A (zh) * 2020-09-07 2021-01-05 中国科学院深圳先进技术研究院 图像超分辨率方法、装置、设备及存储介质
CN112529775A (zh) * 2019-09-18 2021-03-19 华为技术有限公司 一种图像处理的方法和装置
WO2021164731A1 (zh) * 2020-02-19 2021-08-26 华为技术有限公司 图像增强方法以及图像增强装置
CN113986177A (zh) * 2021-11-05 2022-01-28 Oppo广东移动通信有限公司 投屏方法、投屏装置、存储介质与电子设备

Patent Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2007329678A (ja) * 2006-06-07 2007-12-20 Matsushita Electric Ind Co Ltd 輝度マスキング及び輝度補償を用いた超解像処理方法及び装置
CN108391060A (zh) * 2018-03-26 2018-08-10 华为技术有限公司 一种图像处理方法、图像处理装置和终端
CN109889800A (zh) * 2019-02-28 2019-06-14 深圳市商汤科技有限公司 图像增强方法和装置、电子设备、存储介质
CN112529775A (zh) * 2019-09-18 2021-03-19 华为技术有限公司 一种图像处理的方法和装置
WO2021164731A1 (zh) * 2020-02-19 2021-08-26 华为技术有限公司 图像增强方法以及图像增强装置
CN111476719A (zh) * 2020-05-06 2020-07-31 Oppo广东移动通信有限公司 图像处理方法、装置、计算机设备及存储介质
CN112184548A (zh) * 2020-09-07 2021-01-05 中国科学院深圳先进技术研究院 图像超分辨率方法、装置、设备及存储介质
CN113986177A (zh) * 2021-11-05 2022-01-28 Oppo广东移动通信有限公司 投屏方法、投屏装置、存储介质与电子设备

Also Published As

Publication number Publication date
CN116939225A (zh) 2023-10-24

Similar Documents

Publication Publication Date Title
US9984446B2 (en) Video tone mapping for converting high dynamic range (HDR) content to standard dynamic range (SDR) content
US20200258197A1 (en) Method for generating high-resolution picture, computer device, and storage medium
US8711248B2 (en) Global alignment for high-dynamic range image generation
CN110996170B (zh) 视频文件播放方法及相关设备
JP7359521B2 (ja) 画像処理方法および装置
US20210233210A1 (en) Method and system of real-time super-resolution image processing
US20180005358A1 (en) A method and apparatus for inverse-tone mapping a picture
WO2019101005A1 (zh) 像素补偿方法、装置和终端设备
WO2021104079A1 (zh) 视频处理方法、装置、存储介质及计算机设备
US11070705B2 (en) System and method for image dynamic range adjusting
WO2020001176A1 (zh) 显示方法及装置、可见光通信传输方法及装置
US20220237754A1 (en) Image processing method and apparatus
US8860806B2 (en) Method, device, and system for performing color enhancement on whiteboard color image
US11010879B2 (en) Video image processing method and apparatus thereof, display device, computer readable storage medium and computer program product
US20180048914A1 (en) Image processing method and related apparatus
US20230300475A1 (en) Image processing method and apparatus, and electronic device
WO2021184931A1 (zh) 适用于光学穿透式头戴显示器的颜色对比增强绘制方法、装置以及系统
WO2023185706A1 (zh) 影像处理方法、影像处理装置、存储介质
US9286655B2 (en) Content aware video resizing
WO2017107605A1 (zh) 一种图像细节处理方法和装置、终端、存储介质
WO2021217428A1 (zh) 图像处理方法、装置、摄像设备和存储介质
JP2010213356A (ja) 画像データを処理する方法、及び装置
CN115063327B (zh) 图像处理方法和装置、以及视频处理方法和装置
CN113194267B (zh) 图像处理方法及装置、拍照方法及装置
WO2022228368A1 (zh) 一种图像处理方法、设备及系统

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 23778073

Country of ref document: EP

Kind code of ref document: A1