WO2023010755A1 - 一种hdr视频转换方法、装置、设备及计算机存储介质 - Google Patents
一种hdr视频转换方法、装置、设备及计算机存储介质 Download PDFInfo
- Publication number
- WO2023010755A1 WO2023010755A1 PCT/CN2021/138138 CN2021138138W WO2023010755A1 WO 2023010755 A1 WO2023010755 A1 WO 2023010755A1 CN 2021138138 W CN2021138138 W CN 2021138138W WO 2023010755 A1 WO2023010755 A1 WO 2023010755A1
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- frames
- frame
- sdr video
- video
- sdr
- Prior art date
Links
- 238000006243 chemical reaction Methods 0.000 title claims abstract description 63
- 238000000034 method Methods 0.000 title claims abstract description 57
- 238000012545 processing Methods 0.000 claims abstract description 28
- 238000000605 extraction Methods 0.000 claims abstract description 11
- 238000004590 computer program Methods 0.000 claims description 29
- 230000006870 function Effects 0.000 description 11
- 238000010586 diagram Methods 0.000 description 7
- 230000008569 process Effects 0.000 description 5
- 238000012549 training Methods 0.000 description 5
- 238000013528 artificial neural network Methods 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 2
- 239000003086 colorant Substances 0.000 description 2
- 238000013135 deep learning Methods 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 230000000717 retained effect Effects 0.000 description 2
- 230000004913 activation Effects 0.000 description 1
- 238000004891 communication Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000009432 framing Methods 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000003062 neural network model Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 230000002123 temporal effect Effects 0.000 description 1
- 238000012360 testing method Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/90—Dynamic range modification of images or parts thereof
- G06T5/92—Dynamic range modification of images or parts thereof based on global image properties
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/50—Image enhancement or restoration using two or more images, e.g. averaging or subtraction
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N7/00—Television systems
- H04N7/01—Conversion of standards, e.g. involving analogue television standards or digital television standards processed at pixel level
- H04N7/0127—Conversion of standards, e.g. involving analogue television standards or digital television standards processed at pixel level by changing the field or frame frequency of the incoming video signal, e.g. frame rate converter
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10016—Video; Image sequence
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20081—Training; Learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20084—Artificial neural networks [ANN]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20172—Image enhancement details
- G06T2207/20208—High dynamic range [HDR] image processing
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20212—Image combination
- G06T2207/20221—Image fusion; Image merging
Definitions
- the present application belongs to the technical field of video processing, and in particular relates to an HDR video conversion method, device, equipment and computer storage medium.
- High Dynamic Range Compared with standard dynamic range (Standard Dynamic Range, SDR) video, high dynamic range (High Dynamic Range, HDR) video has a larger dynamic range and a wider color gamut, and can show higher contrast and richer colors. color. With the gradual maturity of HDR video shooting and display technology, more and more playback devices support HDR video playback. Therefore, how to convert SDR video to HDR video has become a current hot issue.
- SDR Standard Dynamic Range
- HDR High Dynamic Range
- the currently proposed HDR video conversion method is to first convert each frame of SDR video frame in the SDR video into an HDR video frame by using a neural network, and then combine the frame way to get the converted HDR video. But the quality of the HDR video obtained in this way is poor.
- Embodiments of the present application provide an HDR video conversion method, device, terminal device, and storage medium, which can solve the problem of low quality of HDR video converted from SDR video.
- the embodiment of the present application provides a HDR video conversion method, the method comprising: performing frame extraction processing on the SDR video to be processed, to obtain M frames of SDR video frames contained in the SDR video;
- the trained HDR video conversion model to convert M frames of SDR video frames into M frames of HDR video frames, wherein the tth frame of HDR video frame in the M frames of HDR video frames is based on the tth frame contained in the SDR video K frame SDR video frame conversion including SDR video frame is obtained, 0 ⁇ t ⁇ M, t, K, M are all positive integers;
- each HDR video frame is converted from K frames of SDR video frames in the SDR video. It is understandable that in a video, due to changes in lighting, there may be some inter-frame information between frames, so that the information lost in the previous frame may be retained in the next frame. Therefore, using the HDR video conversion method provided by this application, when converting HDR video frames, the information between K frames and SDR video frames is fully considered, reducing the information loss during the conversion of SDR video frames, so that the processed HDR The video frame also preserves the inter-frame information as much as possible, which improves the conversion quality of HDR video.
- the K frames of SDR video frames are continuous SDR video frames.
- the K frames of SDR video frames include k-n+1 frames The first SDR video frame in the above SDR video.
- the K frames of SDR video frames include k-p+1 frames The Mth frame of the SDR video frame in the SDR video.
- an embodiment of the present application provides an HDR video conversion device, the device comprising: a frame extraction unit, configured to perform frame extraction processing on the SDR video to be processed, to obtain M frames of SDR video frames contained in the SDR video ;
- a conversion unit configured to convert M frames of SDR video frames into M frames of HDR video frames using the trained HDR video conversion model, wherein the tth frame of HDR video frame in the M frames of HDR video frames is based on the SDR video
- the conversion of K frames of SDR video frames including the t-th frame of SDR video frames is obtained, 0 ⁇ t ⁇ M, and t, K, and M are all positive integers;
- a frame merging unit configured to perform frame merging processing on the M frames of HDR video frames to obtain an HDR video corresponding to the SDR video.
- the K frames of SDR video frames are continuous SDR video frames.
- the embodiment of the present application provides a terminal device, including a memory, a processor, and a computer program stored in the memory and operable on the processor.
- a terminal device including a memory, a processor, and a computer program stored in the memory and operable on the processor.
- the processor executes the computer program, any of the above-mentioned first aspect one of the methods described.
- an embodiment of the present application provides a computer-readable storage medium, where a computer program is stored in the computer-readable storage medium, and when the computer program is executed by a processor, the method described in any one of the above-mentioned first aspects is implemented.
- an embodiment of the present application provides a computer program product, which, when the computer program product is run on a terminal device, causes the terminal device to execute the method described in any one of the foregoing first aspects.
- Figure 1 is a schematic diagram of the HDR and SDR color gamut ranges provided by an embodiment of the present application
- Fig. 2 is a flow chart of an embodiment of an HDR video conversion method provided by an embodiment of the present application
- FIG. 3 is a structural diagram of an HDR video conversion method provided by an embodiment of the present application.
- FIG. 4 is a schematic diagram of an HDR video conversion device provided by an embodiment of the present application.
- Fig. 5 is a schematic structural diagram of a terminal device provided by an embodiment of the present application.
- High Dynamic Range Compared with the standard dynamic range (Standard Dynamic Range, SDR), the high dynamic range (High Dynamic Range, HDR) has a larger dynamic range and a wider color gamut. Due to the wide color gamut and large dynamic range of HDR, therefore, HDR video can show video with higher contrast and richer colors.
- SDR Standard Dynamic Range
- HDR High Dynamic Range
- FIG 1 shows a schematic diagram of the range of HDR and SDR color gamuts.
- BT.709 and BT.2020 are TV parameter standards issued by the ITU (International Telecommunication Union).
- the color gamut standard formulated is mostly used to test the color range that the projector can cover.
- BT.2020 has the largest range, followed by the color gamut range of DCI-P3, and the color range represented by BT.709
- the domain range is the smallest.
- HDR video adopts BT.709 color gamut
- HDR video adopts BT.2020 color gamut with wider color gamut.
- HDR video will also adopt DCI-P3 color gamut.
- the contrast and color of the HDR video are better than the SDR video.
- the currently proposed HDR video conversion method is to first convert each frame of SDR video frame in the SDR video into an HDR video frame by using a neural network, and then combine the frame way to get the converted HDR video. But the quality of the HDR video obtained in this way is poor.
- the application provides a HDR video conversion method, so that each frame of HDR video frame is composed of K frames of SDR video frames in the SDR video to be processed
- the converted HDR video frame takes into account the information between the SDR video frames, reducing the information loss during the HDR video conversion process of the SDR video frame, so that the converted HDR video frame also retains the inter-frame information as much as possible, thereby improving HDR.
- the conversion quality of the video is a HDR video conversion method, so that each frame of HDR video frame is composed of K frames of SDR video frames in the SDR video to be processed
- the converted HDR video frame takes into account the information between the SDR video frames, reducing the information loss during the HDR video conversion process of the SDR video frame, so that the converted HDR video frame also retains the inter-frame information as much as possible, thereby improving HDR.
- FIG. 2 it is a flowchart of an embodiment of an HDR video conversion method provided by the present application.
- the subject of execution of the method may be a video processing device.
- the video processing device may be a mobile terminal device such as a smart phone, a tablet computer, or a video camera, or may be a terminal device capable of processing video data such as a desktop computer, a robot, or a server.
- the method includes:
- S101 Perform frame extraction processing on the SDR video to be processed to obtain M frames of SDR video frames contained in the SDR video.
- the SDR video to be processed may be a complete video taken, downloaded or read from a local storage area, or an SDR video segment intercepted from a completed video.
- a video conversion tool may be used to extract frames of the SDR video to be processed.
- FFmpeg Fast Forward Mpeg
- t is a variable with a value range between 1 and M, that is to say, when each frame of SDR video frame is converted to the corresponding HDR video frame, K frames of SDR video frames including the SDR video frame can be input It is processed in the HDR video conversion model, and the HDR video frame corresponding to the SDR video frame is obtained as output.
- the above 8 SDR video frames are converted into 8 HDR video frames by using the HDR video conversion model.
- the 3rd frame HDR video frame corresponding to the 3rd frame SDR video frame is 5 frames of SDR video including the 3rd frame SDR video frame in the SDR video frame conversion.
- the aforementioned K frames of SDR video frames may be continuous SDR video frames.
- the tth HDR video frame corresponding to the tth SDR video frame may be obtained by converting the tth frame to the t+K-1th SDR video frame in the SDR video.
- the HDR video frame corresponding to each SDR video frame can be obtained by converting the last K frames of SDR video frames.
- the second HDR video frame corresponding to the second SDR video frame may be converted from the 2nd to 6th SDR video frames in the SDR video.
- the t-th HDR video frame corresponding to the t-th SDR video frame may be obtained by converting the t-K+1-th frame to the t-th SDR video frame in the SDR video.
- the HDR video frame corresponding to each SDR video frame can be converted from the previous K frames of SDR video frames.
- the 1st to 5th HDR video frames corresponding to the 1st to 5th SDR video frames can be converted from the 1st to 5th video frames in the SDR video, and the 4th frame corresponding to the 6th video frame
- the frame HDR video frame can be obtained by converting the 2nd-6th video frame in the SDR video
- the 7th HDR video frame corresponding to the 7th frame of video can be obtained by converting the 3rd-7th video frame in the SDR video.
- the t-th HDR video frame corresponding to the t-th SDR video frame may also be converted from K frames of SDR video frames adjacent to the t-th SDR video frame in the SDR video.
- the output of the model processing is to obtain the tth frame of HDR video frame, that is, the K frame of SDR video frame includes the t-kth frame of SDR video frame to the t+kth frame of SDR video frame in the SDR video.
- the K frames of SDR video frames include the first SDR video frame of k-n+1 SDR video frames.
- the number of SDR video frames before the first frame and the second frame in the SDR video frame is less than two frames
- the 5 SDR video frames include the first SDR video frame in the 3 SDR video frames
- the 5 SDR video frames used to convert the first HDR video frame include the first SDR video frame in the SDR video frame Frame, 1st frame, 1st frame, 2nd frame and 3rd frame
- SDR video frame means 3 frames 1st frame SDR video frame, 1 frame 2nd SDR video frame and 1 frame 3rd SDR video frame.
- the first SDR video frame in the 2 SDR video frames is included in the 5 SDR video frames
- the above-mentioned 5 frames of SDR video frames used to convert the 2nd frame of HDR video frame include the 1st frame, the 1st frame, the 2nd frame, the 3rd frame and the 4th frame of the SDR video frame in the SDR video frame, that is The 1st SDR video frame of 2 frames, the 2nd SDR video frame of 1 frame, the 3rd SDR video frame of 1 frame, and the 4th SDR video frame of 1 frame.
- the K frames of SDR video frames include the M-th SDR video frame of the k-p+1 SDR video frames.
- the number of SDR video frames after the 7th frame and the 8th frame in the SDR video frame is less than two frames
- the 5 SDR video frames include the 8th SDR video frame in the 2 SDR video frames, that is to say, the 5 SDR video frames used to convert the 7th HDR video frame include the 8th SDR video frame in the SDR video frame
- the 5th frame, the 6th frame, the 7th frame, the 8th frame and the 8th frame SDR video frame are the 1st frame 5th SDR video frame, the 1st frame 6th SDR video frame, the 1st frame 7th SDR video frame and 2 Frame 8th SDR video frame.
- the 5 SDR video frames include the 8th SDR video frame in the 3 SDR video frames, that is Said
- the above-mentioned 5 frames of SDR video frame used to convert the 8th frame of HDR video frame include the 6th frame, the 7th frame, the 8th frame, the 8th frame and the 8th frame of the SDR video frame, that is, the 1st frame of the SDR video frame 6 SDR video frames, 1st frame 7th SDR video frame and 3rd frame 8th SDR video frame.
- the second frame of the HDR video frame in the 5 frames of HDR video frames is based on The conversion of 9 SDR video frames including the second SDR video frame in the SDR video is obtained.
- the 9 SDR video frames used to convert the 2nd HDR video frame include: 1st SDR video frame of 4 SDR video frames, 2nd SDR video frame of 1 SDR video frame, 1 frame The 3rd SDR video frame in the SDR video, the 4th SDR video frame in 1 SDR video frame, and the 5th SDR video frame in 2 SDR video frames.
- the M frame SDR video frame obtained in step S101 is input into the trained HDR video conversion model for processing, and the output is obtained corresponding to the SDR video frame M-frame HDR video frames.
- the HDR video conversion model may be any neural network model that can realize the task of converting SDR video to HDR video.
- it could be a fully convolutional model.
- the fully convolutional model includes 3 convolutional layers with a convolution kernel size of 1 ⁇ 1, and 2 Rectified Linear Unit (ReLU) activation functions interspersed among the 3 convolutional layers.
- ReLU Rectified Linear Unit
- the training method of the HDR video conversion model includes: using a preset training set and a preset loss function to iteratively train the HDR video conversion initial model to obtain the above-mentioned HDR video conversion model; wherein the training set includes a plurality of SDR video frame samples and HDR video frame samples corresponding to the above SDR video frame samples.
- an SDR video sample and its corresponding HDR video sample are acquired first.
- SDR video samples and corresponding HDR video samples can be obtained from public video websites. It is also possible to perform SDR and HDR processing on videos in the same RAW data format, respectively, to obtain SDR video samples and corresponding HDR video samples. It is also possible to use the SDR camera and the HDR camera respectively to shoot corresponding SDR video samples and HDR video samples in the same scene.
- the SDR video samples and their corresponding HDR video samples are frame-drawn to obtain a plurality of SDR video frame samples and the temporal and spatial connections between multiple SDR video samples.
- L2 is used as the loss function for HDR video conversion model training.
- the loss function is used to describe the loss between the predicted HDR video frame and the HDR video frame sample, where the predicted HDR video frame is obtained by processing the SDR video frame sample by the full convolution model.
- the M frames of HDR video frames processed by the HDR video conversion model may be combined by using a video conversion tool, for example, the M frames of HDR video frames may be combined by using the FFmpeg tool.
- the video conversion tool used for the SDR video frame extraction to be processed is the same as the video conversion tool used for the M-frame HDR video frame combination, it can not only save the cost of video processing and improve the efficiency of video processing, but also The processing accuracy of the SDR video to be processed and the obtained HDR video corresponding to the SDR video to be processed can be kept consistent.
- each HDR video frame is converted from K frames of SDR video frames in the SDR video. It is understandable that in a video, due to changes in lighting, there may be some inter-frame information between frames, so that the information lost in the previous frame may be retained in the next frame. Therefore, using the HDR video conversion method provided by this application, when converting HDR video frames, the information between K frames and SDR video frames is fully considered, reducing the information loss during the conversion of SDR video frames, so that the processed HDR The video frame also preserves the inter-frame information as much as possible, which improves the conversion quality of HDR video.
- an embodiment of the present application provides an HDR video conversion device.
- the embodiment of the device corresponds to the embodiment of the method described above.
- the embodiment of the device does not implement the method described above.
- the details in the examples are described one by one, but it should be clear that the device in this embodiment can correspondingly implement all the content in the foregoing method embodiments.
- the present application provides an HDR video conversion device, and the above-mentioned device 200 includes:
- the frame extraction unit 201 is used to perform frame extraction processing on the SDR video to be processed, so as to obtain M frames of SDR video frames contained in the SDR video;
- the converting unit 202 is configured to convert the M frames of SDR video frames into M frames of HDR video frames using the trained HDR video conversion model, wherein the tth frame of the HDR video frame in the M frames of HDR video frames is based on the information contained in the SDR video.
- K frames of SDR video frames including t frames of SDR video frames are converted to obtain, 0 ⁇ t ⁇ M, t, K, and M are all positive integers;
- the framing unit 203 is configured to perform frame merging processing on M frames of HDR video frames to obtain HDR video corresponding to the SDR video.
- the K frames of SDR video frames are continuous SDR video frames.
- K frames of SDR video frames include the first frame in k-n+1 frames of SDR video SDR video frame.
- K frames of SDR video frames include the Mth frame in k-p+1 frames of SDR video SDR video frame.
- FIG. 5 is a schematic diagram of a terminal device provided in an embodiment of the present application.
- the terminal device 300 provided in this embodiment includes: a memory 302 and a processor 301, the memory 302 is used to store computer programs; the processor 301 is used to The methods described in the above method embodiments are executed when the computer program is called, for example, steps S101 to S103 shown in FIG. 2 .
- the processor 301 executes the computer program, it realizes the functions of the modules/units in the above-mentioned device embodiments, for example, the functions of the units 201 to 203 shown in FIG. 4 .
- the computer program may be divided into one or more modules/units, and the one or more modules/units are stored in the memory 302 and executed by the processor 301 to complete this Apply.
- the one or more modules/units may be a series of computer program instruction segments capable of accomplishing specific functions, and the instruction segments are used to describe the execution process of the computer program in the terminal device.
- FIG. 5 is only an example of a terminal device, and does not constitute a limitation on the terminal device. It may include more or less components than those shown in the figure, or combine certain components, or different components, such as
- the terminal device may also include an input and output device, a network access device, a bus, and the like.
- the processor 301 may be a central processing unit (Central Processing Unit, CPU), or other general-purpose processors, digital signal processors (Digital Signal Processor, DSP), application specific integrated circuits (Application Specific Integrated Circuit, ASIC), Field-Programmable Gate Array (Field-Programmable Gate Array, FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc.
- a general-purpose processor may be a microprocessor, or the processor may be any conventional processor, or the like.
- the storage 302 may be an internal storage unit of the terminal device, for example, a hard disk or memory of the terminal device.
- the memory 302 may also be an external storage device of the terminal device, such as a plug-in hard disk equipped on the terminal device, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, Flash Card (Flash Card), etc. Further, the memory 302 may also include both an internal storage unit of the terminal device and an external storage device.
- the memory 302 is used to store the computer program and other programs and data required by the terminal device.
- the memory 302 can also be used to temporarily store data that has been output or will be output.
- the terminal device provided in this embodiment can execute the foregoing method embodiment, and its implementation principle and technical effect are similar, and details are not repeated here.
- the embodiment of the present application also provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the method described in the foregoing method embodiment is implemented.
- the embodiment of the present application further provides a computer program product, which, when the computer program product runs on a terminal device, enables the terminal device to implement the method described in the foregoing method embodiments when executed.
- An embodiment of the present application further provides a chip system, including a processor, the processor is coupled to a memory, and the processor executes a computer program stored in the memory, so as to implement the method described in the above method embodiment.
- the chip system may be a single chip, or a chip module composed of multiple chips.
- the above integrated units are realized in the form of software function units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, all or part of the procedures in the methods of the above embodiments in the present application can be completed by instructing related hardware through computer programs, and the computer programs can be stored in a computer-readable storage medium.
- the computer program When executed by a processor, the steps in the above-mentioned various method embodiments can be realized.
- the computer program includes computer program code, and the computer program code may be in the form of source code, object code, executable file or some intermediate form.
- the computer-readable storage medium may at least include: any entity or device capable of carrying computer program codes to a photographing device/terminal device, a recording medium, a computer memory, a read-only memory (Read-Only Memory, ROM), a random access Memory (Random Access Memory, RAM), electrical carrier signals, telecommunication signals, and software distribution media.
- a photographing device/terminal device a recording medium
- a computer memory a read-only memory (Read-Only Memory, ROM), a random access Memory (Random Access Memory, RAM), electrical carrier signals, telecommunication signals, and software distribution media.
- ROM read-only memory
- RAM random access Memory
- electrical carrier signals telecommunication signals
- software distribution media such as U disk, mobile hard disk, magnetic disk or optical disk, etc.
- computer readable media may not be electrical carrier signals and telecommunication signals under legislation and patent practice.
- references to "one embodiment” or “some embodiments” or the like in this application means that a particular feature, structure, or characteristic described in connection with the embodiment is included in one or more embodiments of the present application.
- appearances of the phrases “in one embodiment,” “in some embodiments,” “in other embodiments,” “in other embodiments,” etc. in various places in this specification are not necessarily All refer to the same embodiment, but mean “one or more but not all embodiments” unless specifically stated otherwise.
- the terms “including”, “comprising”, “having” and variations thereof mean “including but not limited to”, unless specifically stated otherwise.
- first and second are used for description purposes only, and cannot be interpreted as indicating or implying relative importance or implicitly indicating the quantity of indicated technical features. Thus, the features defined as “first” and “second” may explicitly or implicitly include at least one of these features. It should also be understood that the term “and/or” used in the description of the present application and the appended claims refers to any combination and all possible combinations of one or more of the associated listed items, and includes these combinations.
- connection and “connected” should be understood in a broad sense, for example, it can be mechanical connection or electrical connection; it can be direct connection or through An intermediate medium is indirectly connected, which can be the internal communication of two elements or the interaction relationship between two elements. Unless otherwise clearly defined, those of ordinary skill in the art can understand the above terms in this application according to the specific situation. specific meaning.
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Two-Way Televisions, Distribution Of Moving Picture Or The Like (AREA)
Abstract
本申请提供一种HDR视频转换方法、装置、设备及计算机存储介质。涉及视频处理技术领域,上述方法包括:对待处理的SDR视频进行抽帧处理,得到SDR视频中包含的M帧SDR视频帧;利用已训练的HDR视频转换模型将M帧SDR视频帧转换为M帧HDR视频帧,其中,M帧HDR视频帧中的第t帧HDR视频帧是根据SDR视频中包含第t帧SDR视频帧在内的K帧SDR视频帧转换得到,0<t≤M,t、K、M均为正整数;将M帧HDR视频帧进行合帧处理,得到与SDR视频对应的HDR视频。本申请在SDR视频帧转换过程中尽可能保留了SDR视频帧之间的信息,避免了SDR视频帧转换过程中信息的损失,提高了HDR视频的转换质量。
Description
本申请属于视频处理技术领域,尤其涉及一种HDR视频转换方法、装置、设备及计算机存储介质。
相比于标准动态范围(Standard Dynamic Range,SDR)视频,高动态范围(High Dynamic Range,HDR)视频具有更大的动态范围以及更宽广的色域,能够展现出更高的对比度以及更丰富的色彩。随着HDR视频的拍摄及显示技术的逐渐成熟,越来越多的播放设备支持HDR视频的播放。因此,如何将SDR视频转换为HDR视频,成为当前的热门问题。
由于深度学习方法在图像处理和计算机视觉领域的广泛应用,目前提出的HDR视频转换方法是,首先利用神经网络将SDR视频中的每一帧SDR视频帧转换为HDR视频帧,然后通过合帧的方式获取转换后的HDR视频。但通过这种方式得到的HDR视频的质量较差。
本申请实施例提供一种HDR视频转换方法、装置、终端设备及存储介质,可以解决由SDR视频转换得到的HDR视频质量较低的问题。
第一方面,本申请实施例提供了一种HDR视频转换方法,该方法包括:对待处理的SDR视频进行抽帧处理,得到所述SDR视频中包含的M帧SDR视频帧;
利用已训练的HDR视频转换模型将M帧SDR视频帧转换为M帧HDR视频帧,其中,所述M帧HDR视频帧中的第t帧HDR视频帧是根据所述SDR视频中包含第t帧SDR视频帧在内的K帧SDR视频帧转换得到,0<t≤M,t、K、M均为正整数;
将所述M帧HDR视频帧进行合帧处理,得到与所述SDR视频对应的HDR视频。
在本申请提供的HDR视频转换方法中,每一帧HDR视频帧均是由SDR视频中的K帧SDR视频帧转换得到的。可以理解的是,在视频中,由于光照的变化,帧与帧之间可能会存在一些帧间信息,使得前一帧损失的信息可能在后一帧中有所保留。因此,采用本申请提供的HDR视频转换方法,在转换HDR视频帧时,充分考虑了K帧SDR视频帧之间的信息,减小了SDR视频帧转换过程中的信息损失,使得处理后的HDR视频帧也尽可能保留了帧间信息,从而提高了HDR视频的转换质量。
可选地,所述K帧SDR视频帧是连续的SDR视频帧。
可选地,所述K帧SDR视频帧包括所述SDR视频中的第t-k帧SDR视频帧到第t+k帧SDR视频帧,其中,2k+1=K。
可选地,若所述SDR视频中位于所述第t帧SDR视频帧之前的SDR视频帧数为n,0≤n<k,则所述K帧SDR视频帧包括k-n+1帧所述SDR视频中的第1帧SDR视频帧。
可选地,若所述SDR视频中位于所述第t帧SDR视频帧之后的SDR视频帧数为p,0≤p<k,则所述K帧SDR视频帧包括k-p+1帧所述SDR视频中的第M帧SDR视频帧。
第二方面,本申请实施例提供了一种HDR视频转换装置,该装置包括:抽帧单元,用于对待处理的SDR视频进行抽帧处理,得到所述SDR视频中包含的M帧SDR视频帧;
转换单元,用于利用已训练的HDR视频转换模型将M帧SDR视频帧转换为M帧HDR视频帧,其中,所述M帧HDR视频帧中的第t帧HDR视频帧是根据所述SDR视频中包含第t帧SDR视频帧在内的K帧SDR视频帧转换得到,0<t≤M,t、K、M均为正整数;
合帧单元,用于将所述M帧HDR视频帧进行合帧处理,得到与所述SDR视频对应的HDR视频。
可选地,所述K帧SDR视频帧是连续的SDR视频帧。
可选地,所述K帧SDR视频帧包括所述SDR视频中的第t-k帧SDR视频帧到第t+k帧SDR视频帧,其中,2k+1=K。
第三方面,本申请实施例提供了一种终端设备,包括存储器、处理器以及存储在存储器中并可在处理器上运行的计算机程序,处理器执行计算机程序时实现如上述第一方面中任一项所述的方法。
第四方面,本申请实施例提供了一种计算机可读存储介质,计算机可读存储介质存储有计算机程序,计算机程序被处理器执行时实现如上述第一方面中任一项所述的方法。
第五方面,本申请实施例提供了一种计算机程序产品,当计算机程序产品在终端设备上运行时,使得终端设备执行上述第一方面中任一项所述的方法。
可以理解的是,上述第二方面至第五方面的有益效果可以参见上述第一方面和第一方面的各可能的实施方式所带来的有益效果的相关描述,在此不再赘述。
为了更清楚地说明本申请实施例中的技术方案,下面将对实施例或现有技术描述中所需要使用的附图作简单地介绍,显而易见地,下面描述中的附图仅仅是本申请的一些实施例,对于本领域普通技术人员来讲,在不付出创造性劳动性的前提下,还可以根据这些附图获得其他的附图。
图1是本申请一实施例提供的HDR和SDR色域表示范围的示意图;
图2是本申请一实施例提供的一种HDR视频转换方法的一个实施例的流程图;
图3是本申请一实施例提供的一种HDR视频转换方法的架构图;
图4是本申请一实施例提供的一种HDR视频转换装置的示意图;
图5是本申请一实施例提供的一种终端设备的结构示意图。
为使本申请实施例的目的、技术方案和优点更加清楚,下面将结合本申请实施例中的附图,对本申请实施例中的技术方案进行清楚、完整地描述,显然,所描述的实施例是本申请一部分实施例,而不是全部的实施例。基于本申请中的实施例,本领域普通技术人员在没有作出创造性劳动前提下所获得的所有其他实施例,都属于本申请保护的范围。
相较于标准动态范围(Standard Dynamic Range,SDR),高动态范围(High Dynamic Range,HDR)的动态范围更大且色域范围具有更宽广,由于HDR的色域广、动态范围大,因此,HDR视频能够展现出对比度更高和色彩更加丰富的视频。
如图1所示为HDR和SDR色域表示范围的示意图,其中,BT.709和BT.2020都是ITU(国际电信联盟)发布的电视参数标准,DCI-P3是美国电影工业为数字电影院所制定的色域标准,多用来测试投影机所能覆盖的色彩范围。就色域范围来讲,在图1所示的DCI-P3、BT.709和BT.2020中范围最大的是BT.2020,DCI-P3的色域范围次之,BT.709所表示的色域范围最小。
通常SDR视频采用BT.709色域,而HDR视频采用的是色域更为宽广的BT.2020色域,在实际应用HDR视频也会采用DCI-P3色域。就同一视频而言,无论HDR视频采用BT.2020色域还是DCI-P3色域,HDR视频展现出对比度和色彩都要优于SDR视频。
随着HDR视频的拍摄及显示技术的逐渐成熟,越来越多的播放设备支持HDR视频的播放。因此,如何将SDR视频转换为HDR视频,成为当前的热门问题。
由于深度学习方法在图像处理和计算机视觉领域的广泛应用,目前提出的HDR视频转换方法是,首先利用神经网络将SDR视频中的每一帧SDR视频帧转换为HDR视频帧,然后通过合帧的方式获取转换后的HDR视频。但通过这种方式得到的HDR视频的质量较差。
针对现有技术中SDR视频转换得到的HDR视频质量较低的问题,本申请提供一种HDR视频转换方法,使得每一帧HDR视频帧均是由待处理的SDR视频中的K帧SDR视频帧转换得到的,考虑了SDR视频帧之间的信息,减少了SDR视频帧进行HDR视频转换过程中的信息损失,使得转换得到后的HDR视频帧也尽可能保留了帧间信息,进而提高了HDR视频的转换质量。
下面以具体地实施例对本申请的技术方案进行详细说明。下面这几个具体的实施例可以相互结合,对于相同或相似的概念或过程可能在某些实施例不再赘述。
参见图2,为本申请提供的一种HDR视频转换方法的一个实施例的流程图。该方法的执行主体可以是视频处理设备。其中,视频处理设备可以是智能手机、平板电脑、摄像机等移动终端设备,还可以是台式电脑、机器人、服务器等能够处理视频数据的终端设备。如图2所示,该方法包括:
S101,对待处理的SDR视频进行抽帧处理,得到SDR视频中包含的M帧SDR视频帧。
其中,待处理的SDR视频可以是拍摄、下载或者是从本地存储区域中读取的完整的视频,也可以是从完成的视频中截取的SDR视频片段。
示例性的,可以采用视频转换工具对待处理的SDR视频进行抽帧。例如,采用FFmpeg(Fast Forward Mpeg)工具对待处理的SDR视频进行抽帧。
S102,利用已训练的HDR视频转换模型将M帧SDR视频帧转换为M帧HDR视频帧,其中,M帧HDR视频帧中的第t帧HDR视频帧是根据SDR视频中包含第t帧SDR视频帧在内的K帧SDR视频帧转换得到,0<t≤M,t、K、M均为正整数。
其中,t是取值范围在1至M之间的变量,也就是说每一帧SDR视频帧在转换对应的HDR视频帧时,可以将包含该SDR视频帧在内的K帧SDR视频帧输入至HDR视频转换模型中处理,输出得到该SDR视频帧对应的HDR视频帧。
例如,假设待处理的SDR视频中包含8(即M=8)帧SDR视频帧,那么利用HDR视频转换模型将上述8帧SDR视频帧转换为8帧HDR视频帧。当K=5时,上述转换后的8帧HDR视频帧中的每一帧HDR视频帧是根据5帧SDR视频帧转换得到。例如,对于第3(即t=3时)帧SDR视频帧,与第3帧SDR视频帧对应的第3帧HDR视频帧是SDR视频中包含第3帧SDR视频帧在内的5帧SDR视频帧转换得到。
在一个示例中,上述K帧SDR视频帧可以是连续的SDR视频帧。
例如,第t帧SDR视频帧对应的第t帧HDR视频帧可以是由SDR视频中的第t帧至第t+K-1帧SDR视频帧转换得到。对于SDR视频中的最后K帧SDR视频帧,每一SDR视频帧对应的HDR视频帧都可以是由最后K帧SDR视频帧转换得到。以K=5为例,第1帧SDR视频帧分别对应的第1帧HDR视频帧可以是SDR视频中第1-5帧SDR视频帧转换得到。第2帧SDR视频帧分别对应的第2帧HDR视频帧可以是SDR视频中第2-6帧SDR视频帧转换得到。
又例如,第t帧SDR视频帧对应的第t帧HDR视频帧可以是由SDR视频中的第t-K+1帧至第t帧SDR视频帧转换得到。当然,对于SDR视频中的前K帧SDR视频帧,每一SDR视频帧对应的HDR视频帧都可以是由前K帧SDR视频帧转换得到。以K=5为例,第1至5帧SDR视频帧分别对应的第1至5帧HDR视频帧可以是由SDR视频中第1至5帧视频帧转换得到,第6帧视频对应的第4帧HDR视频帧可以是由SDR视频中第2-6帧视频帧转换得到,第7帧视频对应的第7帧HDR视频帧可以是由SDR视频中第3-7帧视频帧转换得到。
示例性的,第t帧SDR视频帧对应的第t帧HDR视频帧还可以是由SDR视频中与第t帧SDR视频帧相邻的K帧SDR视频帧转换得到。以K=5为例,第4帧视频对应的第4帧HDR视频帧可以是由SDR视频中第3-7帧视频帧转换得到,第5帧视频对应的第5帧HDR视频帧可以是由SDR视频中第4-8帧视频帧转换得到。
在一个示例中,为了充分考虑SDR视频帧的帧间信息,可以取第t帧SDR视频帧前后各k(2k+1=K)帧SDR视频帧作为HDR视频转换模型的输入,经HDR视频转换模型处理输出得到第t帧HDR视频帧,即K帧SDR视频帧包括SDR视频中的第t-k帧SDR视频帧到第t+k帧SDR视频帧。
示例性的,针对k=2,K=5,假设SDR视频中包含SDR视频帧的帧数M的取值为8,即M=8,例如,第4帧视频对应的第4帧HDR视频帧可以是由SDR视频中第2-6帧视频帧转换得到,第5帧视频对应的第5帧HDR视频帧可以是由SDR视频中第3-7帧视频帧转换得到。
若SDR视频中位于第t帧SDR视频帧之前的SDR视频帧数为n,n<k,则K帧SDR视频帧包括k-n+1帧SDR视频中的第1帧SDR视频帧。
基于上述示例,SDR视频帧中的第1帧和第2帧之前的SDR视频帧数不足两帧,那么SDR视频中位于第1帧SDR视频帧之前的SDR视频帧数为n=0,n<k,则5帧SDR视频帧中包括3帧SDR视频中的第1帧SDR视频帧,也就是说,用于转换第1帧HDR视频帧的5帧SDR视频帧包括SDR视频帧中的第1帧、第1帧、第1帧、第2帧和第3帧SDR视频帧即3帧第1帧SDR视频帧、1帧第2帧SDR视频帧和1帧第3帧SDR视频帧。又例如,SDR视频中位于第2帧SDR视频帧之前的SDR视频帧数为n=1,n<k,则5帧SDR视频帧中包括2帧SDR视频中的第1帧SDR视频帧,也就是说,上述用于转换第2帧HDR视频帧的5帧SDR视频帧则包括SDR视频帧中的第1帧、第1帧、第2帧、第3帧和第4帧SDR视频帧,即2帧第1帧SDR视频帧、1帧第2帧SDR视频帧、1帧第3帧SDR视频帧和1帧第4帧SDR视频帧。
若SDR视频中位于第t帧SDR视频帧之后的SDR视频帧数为p,p<k,则K帧SDR视频帧包括k-p+1帧SDR视频中的第M帧SDR视频帧。
基于上述示例,SDR视频帧中的第7帧和第8帧之后的SDR视频帧数不足两帧,那么SDR视频中位于第7帧SDR视频帧之后的SDR视频帧数为p=1,p<k,则5帧SDR视频帧包括2帧SDR视频中的第8帧SDR视频帧,也就是说,上述用于转换第7帧HDR视频帧的5帧SDR视频帧则包括SDR视频帧中的第5帧、第6帧、第7帧、第8帧和第8帧SDR视频帧即1帧第5帧SDR视频帧、1帧第6帧SDR视频帧、1帧第7帧SDR视频帧和2帧第8帧SDR视频帧。又例如,SDR视频中位于第8帧SDR视频帧之后的SDR视频帧数为p=0,p<k,则5帧SDR视频帧包括3帧SDR视频中的第8帧SDR视频帧,也就是说,上述用于转换第8帧HDR视频帧的5帧SDR视频帧包括SDR视频帧中的第6帧、第7帧、第8帧、第8帧和第8帧SDR视频帧即1帧第6帧SDR视频帧、1帧第7帧SDR视频帧和3帧第8帧SDR视频帧。
值得说明的是,如果SDR视频帧数较少,假设SDR视频中包含5帧SDR视频帧,即M=5;针对k=3,K=7,以第3帧为例,5帧HDR视频帧中的第3帧HDR视频帧是根据SDR视频中包含第3帧SDR视频帧在内的7帧SDR视频帧转换得到,SDR视频中位于第3帧SDR视频帧之前的SDR视频帧数为n=2,且SDR视频中位于第3帧SDR视频帧之后的SDR视频帧数为p=2,那么7帧SDR视频帧中包括2帧SDR视频中的第1帧SDR视频帧,且7帧SDR视频帧包括2帧SDR视频中的第5帧SDR视频帧;因此,用于转换第3帧HDR视频帧的7帧SDR视频帧包括:2帧SDR视频中的第1帧SDR视频帧、1帧SDR视频中的第2帧SDR视频帧、1帧SDR视频中的第3帧SDR视频帧、1帧SDR视频中的第4帧SDR视频帧以及2帧SDR视频中的第5帧SDR视频帧。
又例如,针对k=4,K=9,假设SDR视频中包含5帧SDR视频帧,即M=5,以第2帧为例,5帧HDR视频帧中的第2帧HDR视频帧是根据SDR视频中包含第2帧SDR视频帧在内的9帧SDR视频帧转换得到,SDR视频中位于第2帧SDR视频帧之前的SDR视频帧数为n=1,且SDR视频中位于第2帧SDR视频帧之后的SDR视频帧数为p=3,那么9帧SDR视频帧包括4帧SDR视频中的第1帧SDR视频帧,9帧SDR视频帧包括2帧SDR视频中的第5帧SDR视频帧,因此,用于转换第2帧HDR视频帧的9帧SDR视频帧包括:4帧SDR视频中的第1帧SDR视频帧、1帧SDR视频中的第2帧SDR视频帧、1帧SDR视频中的第3帧SDR视频帧、1帧SDR视频中的第4帧SDR视频帧以及2帧SDR视频中的第5帧SDR视频帧。
如图3所示为本申请提供的一种HDR视频转换方法的架构图,将步骤S101得到的M帧SDR视频帧输入至已训练的HDR视频转换模型中处理,输出得到与SDR视频帧对应的M帧HDR视频帧。
其中,HDR视频转换模型可以是任意能够实现SDR视频向HDR视频转换的任务的神经网络模型。例如,可以是全卷积模型。示例性的,全卷积模型包括3个卷积核大小为1×1的卷积层,及3个卷积层中穿插设置有2个线性整流函数(Rectified Linear Unit,ReLU)激活函数。
HDR视频转换模型的训练方式包括:利用预设的训练集和预设的损失函数对HDR视频转换初始模型进行迭代训练,得到上述HDR视频转换模型;其中,训练集包括多个SDR视频帧样本以及与上述SDR视频帧样本对应的HDR视频帧样本。
具体的,首先获取SDR视频样本及其对应的HDR视频样本。示例性的,可以从公开的视频网站中获取SDR视频样本及对应的HDR视频样本。也可以对同一RAW数据格式的视频分别进行SDR和HDR处理,得到SDR视频样本及其对应的HDR视频样本。还可以分别利用SDR相机和HDR相机在同一场景下,分别拍摄对应的SDR视频样本和HDR视频样本。
在获取到SDR视频样本及其对应的HDR视频样本之后,分别对SDR视频样本及其对应的HDR视频样本进行抽帧处理,得到多个SDR视频帧样本以及在时序上和空间上与多个SDR视频帧样本一一对应的HDR视频帧样本。
其中,采用L2作为HDR视频转换模型训练的损失函数。损失函数用于描述预测的HDR视频帧和HDR视频帧样本之间的损失,其中预测的HDR视频帧为全卷积模型对SDR视频帧样本进行处理得到的。
可以理解的是,针对不同的HDR视频转换模型可以设计对应的训练方式和损失函数来训练初始模型,从而得到不同HDR视频转换模型。
S103,将M帧HDR视频帧进行合帧处理,得到与SDR视频对应的HDR视频。
应该理解的,可以采用视频转换工具对经HDR视频转换模型处理后得到的M帧HDR视频帧进行合帧,例如,采用FFmpeg工具对M帧HDR视频帧进行合帧。
值得说明的是,当对待处理的SDR视频抽帧采用的视频转换工具和对M帧HDR视频帧合帧采用的视频转换工具相同时,不仅能够节省视频处理的成本,提高视频处理的效率,而且能够使待处理的SDR视频和得到的与待处理的SDR视频对应的HDR视频在处理精度上保持一致。
本申请提供的HDR视频转换方法,每一帧HDR视频帧均是由SDR视频中的K帧SDR视频帧转换得到的。可以理解的是,在视频中,由于光照的变化,帧与帧之间可能会存在一些帧间信息,使得前一帧损失的信息可能在后一帧中有所保留。因此,采用本申请提供的HDR视频转换方法,在转换HDR视频帧时,充分考虑了K帧SDR视频帧之间的信息,减小了SDR视频帧转换过程中的信息损失,使得处理后的HDR视频帧也尽可能保留了帧间信息,从而提高了HDR视频的转换质量。
基于同一发明构思,作为对上述方法的实现,本申请实施例提供了一种HDR视频转换装置,该装置实施例与前述方法实施例对应,为便于阅读,本装置实施例不再对前述方法实施例中的细节内容进行逐一赘述,但应当明确,本实施例中的装置能够对应实现前述方法实施例中的全部内容。
如图4所示,本申请提供一种HDR视频转换装置,上述装置200包括:
抽帧单元201,用于对待处理的SDR视频进行抽帧处理,得到SDR视频中包含的M帧SDR视频帧;
转换单元202,用于利用已训练的HDR视频转换模型将M帧SDR视频帧转换为M帧HDR视频帧,其中,M帧HDR视频帧中的第t帧HDR视频帧是根据SDR视频中包含第t帧SDR视频帧在内的K帧SDR视频帧转换得到,0<t≤M,t、K、M均为正整数;
合帧单元203,用于将M帧HDR视频帧进行合帧处理,得到与SDR视频对应的HDR视频。
可选地,K帧SDR视频帧是连续的SDR视频帧。
可选地,K帧SDR视频帧包括SDR视频中的第t-k帧SDR视频帧到第t+k帧SDR视频帧,其中,2k+1=K。
可选地,若SDR视频中位于第t帧SDR视频帧之前的SDR视频帧数为n,0≤n<k,则K帧SDR视频帧包括k-n+1帧SDR视频中的第1帧SDR视频帧。
可选地,若SDR视频中位于第t帧SDR视频帧之后的SDR视频帧数为p,0≤p<k,则K帧SDR视频帧包括k-p+1帧SDR视频中的第M帧SDR视频帧。
所属领域的技术人员可以清楚地了解到,为了描述的方便和简洁,仅以上述各功能单元、模块的划分进行举例说明,实际应用中,可以根据需要而将上述功能分配由不同的功能单元、模块完成,即将所述装置的内部结构划分成不同的功能单元或模块,以完成以上描述的全部或者部分功能。实施例中的各功能单元、模块可以集成在一个处理单元中,也可以是各个单元单独物理存在,也可以两个或两个以上单元集成在一个单元中,上述集成的单元既可以采用硬件的形式实现,也可以采用软件功能单元的形式实现。另外,各功能单元、模块的具体名称也只是为了便于相互区分,并不用于限制本申请的保护范围。上述系统中单元、模块的具体工作过程,可以参考前述方法实施例中的对应过程,在此不再赘述。
基于同一发明构思,本申请实施例还提供了一种终端设备。图5为本申请实施例提供的终端设备的示意图,如图5所示,本实施例提供的终端设备300包括:存储器302和处理器301,存储器302用于存储计算机程序;处理器301用于在调用计算机程序时执行上述方法实施例所述的方法,例如图2所示的步骤S101至步骤S103。或者,所述处理器301执行所述计算机程序时实现上述各装置实施例中各模块/单元的功能,例如图4所示单元201至单元203的功能。
示例性的,所述计算机程序可以被分割成一个或多个模块/单元,所述一个或者多个模块/单元被存储在所述存储器302中,并由所述处理器301执行,以完成本申请。所述一个或多个模块/单元可以是能够完成特定功能的一系列计算机程序指令段,该指令段用于描述所述计算机程序在所述终端设备中的执行过程。
本领域技术人员可以理解,图5仅仅是终端设备的示例,并不构成对终端设备的限定,可以包括比图示更多或更少的部件,或者组合某些部件,或者不同的部件,例如所述终端设备还可以包括输入输出设备、网络接入设备、总线等。
所述处理器301可以是中央处理单元(Central Processing Unit,CPU),还可以是其它通用处理器、数字信号处理器(Digital Signal Processor,DSP)、专用集成电路(Application Specific Integrated Circuit,ASIC)、现场可编程门阵列(Field-Programmable Gate Array,FPGA)或者其它可编程逻辑器件、分立门或者晶体管逻辑器件、分立硬件组件等。通用处理器可以是微处理器或者该处理器也可以是任何常规的处理器等。
所述存储器302可以是所述终端设备的内部存储单元,例如终端设备的硬盘或内存。所述存储器302也可以是所述终端设备的外部存储设备,例如所述终端设备上配备的插接式硬盘,智能存储卡(Smart Media Card,SMC),安全数字(Secure Digital,SD)卡,闪存卡(Flash Card)等。进一步地,所述存储器302还可以既包括所述终端设备的内部存储单元也包括外部存储设备。所述存储器302用于存储所述计算机程序以及所述终端设备所需的其它程序和数据。所述存储器302还可以用于暂时地存储已经输出或者将要输出的数据。
本实施例提供的终端设备可以执行上述方法实施例,其实现原理与技术效果类似,此处不再赘述。
本申请实施例还提供一种计算机可读存储介质,其上存储有计算机程序,计算机程序被处理器执行时实现上述方法实施例所述的方法。
本申请实施例还提供一种计算机程序产品,当计算机程序产品在终端设备上运行时,使得终端设备执行时实现上述方法实施例所述的方法。
本申请实施例还提供一种芯片系统,包括处理器,所述处理器与存储器耦合,所述处理器执行存储器中存储的计算机程序,以实现上述方法实施例所述的方法。其中,所述芯片系统可以为单个芯片,或者多个芯片组成的芯片模组。
上述集成的单元如果以软件功能单元的形式实现并作为独立的产品销售或使用时,可以存储在一个计算机可读取存储介质中。基于这样的理解,本申请实现上述实施例方法中的全部或部分流程,可以通过计算机程序来指令相关的硬件来完成,所述的计算机程序可存储于一计算机可读存储介质中,该计算机程序在被处理器执行时,可实现上述各个方法实施例的步骤。其中,所述计算机程序包括计算机程序代码,所述计算机程序代码可以为源代码形式、对象代码形式、可执行文件或某些中间形式等。所述计算机可读存储介质至少可以包括:能够将计算机程序代码携带到拍照装置/终端设备的任何实体或装置、记录介质、计算机存储器、只读存储器(Read-Only Memory ,ROM)、随机存取存储器(Random Access Memory,RAM)、电载波信号、电信信号以及软件分发介质。例如U盘、移动硬盘、磁碟或者光盘等。在某些司法管辖区,根据立法和专利实践,计算机可读介质不可以是电载波信号和电信信号。
在本申请中描述的参考“一个实施例”或“一些实施例”等意味着在本申请的一个或多个实施例中包括结合该实施例描述的特定特征、结构或特点。由此,在本说明书中的不同之处出现的语句“在一个实施例中”、“在一些实施例中”、“在其他一些实施例中”、“在另外一些实施例中”等不是必然都参考相同的实施例,而是意味着“一个或多个但不是所有的实施例”,除非是以其他方式另外特别强调。术语“包括”、“包含”、“具有”及它们的变形都意味着“包括但不限于”,除非是以其他方式另外特别强调。
在本申请的描述中,需要理解的是,术语“第一”、“第二”仅用于描述目的,而不能理解为指示或暗示相对重要性或者隐含指明所指示的技术特征的数量。由此,限定有“第一”、“第二”的特征可以明示或者隐含地包括至少一个该特征。还应当理解,在本申请说明书和所附权利要求书中使用的术语“和/或”是指相关联列出的项中的一个或多个的任何组合以及所有可能组合,并且包括这些组合。
此外,在本申请中,除非另有明确的规定和限定,术语“连接”、“相连”等应做广义理解,例如可以是机械连接,也可以是电连接;可以是直接连接,也可以通过中间媒介间接相连,可以是两个元件内部的连通或两个元件的相互作用关系,除非另有明确的限定、对于本领域的普通技术人员而言,可以根据具体情况理解上述术语在本申请中的具体含义。
以上各实施例仅用以说明本申请的技术方案,而非对其限制;尽管参照前述各实施例对本申请进行了详细的说明,本领域的普通技术人员应当理解:其依然可以对前述各实施例所记载的技术方案进行修改,或者对其中部分或者全部技术特征进行等同替换;而这些修改或者替换,并不使相应技术方案的本质脱离本申请各实施例技术方案的范围。
Claims (10)
- 一种HDR视频转换方法,其特征在于,包括:对待处理的SDR视频进行抽帧处理,得到所述SDR视频中包含的M帧SDR视频帧;利用已训练的HDR视频转换模型将M帧SDR视频帧转换为M帧HDR视频帧,其中,所述M帧HDR视频帧中的第t帧HDR视频帧是根据所述SDR视频中包含第t帧SDR视频帧在内的K帧SDR视频帧转换得到,0<t≤M,t、K、M均为正整数;将所述M帧HDR视频帧进行合帧处理,得到与所述SDR视频对应的HDR视频。
- 根据权利要求1所述的方法,其特征在于,所述K帧SDR视频帧是连续的SDR视频帧。
- 根据权利要求1所述的方法,其特征在于,所述K帧SDR视频帧包括所述SDR视频中的第t-k帧SDR视频帧到第t+k帧SDR视频帧,其中,2k+1=K。
- 根据权利要求3所述的方法,其特征在于,若所述SDR视频中位于所述第t帧SDR视频帧之前的SDR视频帧数为n,0≤n<k,则所述K帧SDR视频帧包括k-n+1帧所述SDR视频中的第1帧SDR视频帧。
- 根据权利要求3或4所述的方法,其特征在于,若所述SDR视频中位于所述第t帧SDR视频帧之后的SDR视频帧数为p,0≤p<k,则所述K帧SDR视频帧包括k-p+1帧所述SDR视频中的第M帧SDR视频帧。
- 一种HDR视频转换装置,其特征在于,包括:抽帧单元,用于对待处理的SDR视频进行抽帧处理,得到所述SDR视频中包含的M帧SDR视频帧;转换单元,用于利用已训练的HDR视频转换模型将M帧SDR视频帧转换为M帧HDR视频帧,其中,所述M帧HDR视频帧中的第t帧HDR视频帧是根据所述SDR视频中包含第t帧SDR视频帧在内的K帧SDR视频帧转换得到,0<t≤M,t、K、M均为正整数;合帧单元,用于将所述M帧HDR视频帧进行合帧处理,得到与所述SDR视频对应的HDR视频。
- 根据权利要求6所述的装置,其特征在于,所述K帧SDR视频帧是连续的SDR视频帧。
- 根据权利要求6所述的装置,其特征在于,所述K帧SDR视频帧包括所述SDR视频中的第t-k帧SDR视频帧到第t+k帧SDR视频帧,其中,2k+1=K。
- 一种终端设备,包括存储器、处理器以及存储在所述存储器中并可在所述处理器上运行的计算机程序,其特征在于,所述处理器执行所述计算机程序时实现如权利要求1至5任一项所述的方法。
- 一种计算机可读存储介质,所述计算机可读存储介质存储有计算机程序,其特征在于,所述计算机程序被处理器执行时实现如权利要求1至5任一项所述的方法。
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN202110882184.9 | 2021-08-02 | ||
CN202110882184.9A CN113781319A (zh) | 2021-08-02 | 2021-08-02 | 一种hdr视频转换方法、装置、设备及计算机存储介质 |
Publications (1)
Publication Number | Publication Date |
---|---|
WO2023010755A1 true WO2023010755A1 (zh) | 2023-02-09 |
Family
ID=78836527
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
PCT/CN2021/138138 WO2023010755A1 (zh) | 2021-08-02 | 2021-12-15 | 一种hdr视频转换方法、装置、设备及计算机存储介质 |
Country Status (2)
Country | Link |
---|---|
CN (1) | CN113781319A (zh) |
WO (1) | WO2023010755A1 (zh) |
Families Citing this family (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN113781319A (zh) * | 2021-08-02 | 2021-12-10 | 中国科学院深圳先进技术研究院 | 一种hdr视频转换方法、装置、设备及计算机存储介质 |
CN114422718B (zh) * | 2022-01-19 | 2022-12-13 | 北京百度网讯科技有限公司 | 一种视频转换方法、装置、电子设备及存储介质 |
Citations (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20190080440A1 (en) * | 2017-09-08 | 2019-03-14 | Interdigital Vc Holdings, Inc. | Apparatus and method to convert image data |
CN110310231A (zh) * | 2018-03-27 | 2019-10-08 | 天开数码媒体有限公司 | 一种将第一动态范围视频转换为第二动态范围视频的设备及其方法 |
CN111784570A (zh) * | 2019-04-04 | 2020-10-16 | Tcl集团股份有限公司 | 一种视频图像超分辨率重建方法及设备 |
CN113781319A (zh) * | 2021-08-02 | 2021-12-10 | 中国科学院深圳先进技术研究院 | 一种hdr视频转换方法、装置、设备及计算机存储介质 |
CN113784175A (zh) * | 2021-08-02 | 2021-12-10 | 中国科学院深圳先进技术研究院 | 一种hdr视频转换方法、装置、设备及计算机存储介质 |
-
2021
- 2021-08-02 CN CN202110882184.9A patent/CN113781319A/zh active Pending
- 2021-12-15 WO PCT/CN2021/138138 patent/WO2023010755A1/zh active Application Filing
Patent Citations (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20190080440A1 (en) * | 2017-09-08 | 2019-03-14 | Interdigital Vc Holdings, Inc. | Apparatus and method to convert image data |
CN110310231A (zh) * | 2018-03-27 | 2019-10-08 | 天开数码媒体有限公司 | 一种将第一动态范围视频转换为第二动态范围视频的设备及其方法 |
CN111784570A (zh) * | 2019-04-04 | 2020-10-16 | Tcl集团股份有限公司 | 一种视频图像超分辨率重建方法及设备 |
CN113781319A (zh) * | 2021-08-02 | 2021-12-10 | 中国科学院深圳先进技术研究院 | 一种hdr视频转换方法、装置、设备及计算机存储介质 |
CN113784175A (zh) * | 2021-08-02 | 2021-12-10 | 中国科学院深圳先进技术研究院 | 一种hdr视频转换方法、装置、设备及计算机存储介质 |
Also Published As
Publication number | Publication date |
---|---|
CN113781319A (zh) | 2021-12-10 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
KR102149187B1 (ko) | 전자 장치와, 그의 제어 방법 | |
WO2023010749A1 (zh) | 一种hdr视频转换方法、装置、设备及计算机存储介质 | |
WO2023010750A1 (zh) | 一种图像颜色映射方法、装置、终端设备及存储介质 | |
WO2023010755A1 (zh) | 一种hdr视频转换方法、装置、设备及计算机存储介质 | |
KR102617258B1 (ko) | 이미지 프로세싱 방법 및 장치 | |
WO2023010754A1 (zh) | 一种图像处理方法、装置、终端设备及存储介质 | |
US20180005358A1 (en) | A method and apparatus for inverse-tone mapping a picture | |
CN107155093B (zh) | 一种视频预览方法、装置及设备 | |
US11915058B2 (en) | Video processing method and device, electronic equipment and storage medium | |
US20220261961A1 (en) | Method and device, electronic equipment, and storage medium | |
WO2020215180A1 (zh) | 图像处理方法、装置和电子设备 | |
CN113487524B (zh) | 图像格式转换方法、装置、设备、存储介质及程序产品 | |
US6446155B1 (en) | Resource bus interface | |
WO2023010751A1 (zh) | 图像高亮区域的信息补偿方法、装置、设备及存储介质 | |
CN110225316B (zh) | 一种软硬协同的多路视频处理装置及系统 | |
CN108174084A (zh) | 全景视频处理方法及终端设备 | |
WO2012112341A1 (en) | Timing generator and method of generating timing signals | |
CN111447360A (zh) | 应用程序控制方法及装置、存储介质、电子设备 | |
US11057587B2 (en) | Compositing video signals and stripping composite video signal | |
WO2023010753A1 (zh) | 一种色域映射方法、装置、终端设备及存储介质 | |
CN113873141A (zh) | 电子设备 | |
TWI825410B (zh) | 影像處理方法、裝置、攝影設備和儲存介質 | |
CN115967784A (zh) | 基于mipi csi c-phy协议的图像传输处理系统及处理方法 | |
WO2022061723A1 (zh) | 一种图像处理方法、设备、终端及存储介质 | |
US20220207671A1 (en) | Conversion of image |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
121 | Ep: the epo has been informed by wipo that ep was designated in this application |
Ref document number: 21952618 Country of ref document: EP Kind code of ref document: A1 |
|
NENP | Non-entry into the national phase |
Ref country code: DE |
|
122 | Ep: pct application non-entry in european phase |
Ref document number: 21952618 Country of ref document: EP Kind code of ref document: A1 |