WO2018040599A1 - Method and device for processing video frame, and computer-readable storage medium - Google Patents

Method and device for processing video frame, and computer-readable storage medium Download PDF

Info

Publication number
WO2018040599A1
WO2018040599A1 PCT/CN2017/082524 CN2017082524W WO2018040599A1 WO 2018040599 A1 WO2018040599 A1 WO 2018040599A1 CN 2017082524 W CN2017082524 W CN 2017082524W WO 2018040599 A1 WO2018040599 A1 WO 2018040599A1
Authority
WO
WIPO (PCT)
Prior art keywords
component
interval
video frame
mean
boundary line
Prior art date
Application number
PCT/CN2017/082524
Other languages
French (fr)
Chinese (zh)
Inventor
文锦松
顾国华
徐科
Original Assignee
深圳市中兴微电子技术有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 深圳市中兴微电子技术有限公司 filed Critical 深圳市中兴微电子技术有限公司
Publication of WO2018040599A1 publication Critical patent/WO2018040599A1/en

Links

Images

Classifications

    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N9/00Details of colour television systems
    • H04N9/64Circuits for processing colour signals
    • H04N9/68Circuits for processing colour signals for controlling the amplitude of colour signals, e.g. automatic chroma control circuits
    • H04N9/69Circuits for processing colour signals for controlling the amplitude of colour signals, e.g. automatic chroma control circuits for modifying the colour signals by gamma correction
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N9/00Details of colour television systems
    • H04N9/77Circuits for processing the brightness signal and the chrominance signal relative to each other, e.g. adjusting the phase of the brightness signal relative to the colour signal, correcting differential gain or differential phase

Definitions

  • the present invention relates to video processing technologies, and in particular, to a video frame processing method, apparatus, and computer readable storage medium.
  • HDR High-Dynamic Range
  • the brightness range that the human eye can feel is about 10 -3 -10 6 nits, and the instantaneous contrast range can be as much as 10,000:1; while the brightness of consumer LCDs is generally around 300 to 400 nit.
  • the contrast range is typically 2000:1. It can be seen that the brightness resolution of the human eye is much higher than that of the current mainstream display devices. Therefore, the HDR display technology is to increase the brightness range of the display. In general, it makes the display brightness brighter and makes the black appear darker.
  • the current display contrast limit is determined by the ITU BT.709 standard issued by the International Telecommunication Union (ITU) in 1990.
  • the Electro-Optical Transfer Function (EOTF) in this standard is based on the characteristics of a cathode ray tube (CRT) display.
  • CRT cathode ray tube
  • the brightness of CRT monitors generally does not exceed 100 nit.
  • the BT.709 standard has bound the best display. Therefore, many organizations in the broadcasting and television industry have proposed new HDR display technology to play through HDR displays. HDR video to achieve a higher dynamic range of display.
  • embodiments of the present invention are directed to a method and apparatus for processing a video frame and a computer readable storage medium.
  • an embodiment of the present invention provides a method for processing a video frame, where the method includes:
  • the converted C component is further enhanced in color saturation according to a preset color interval to obtain a processed C component.
  • the determining, according to the Y component of the HDR video frame, the adaptation curve corresponding to the HDR video frame including:
  • the upper boundary line and the lower boundary line are determined according to the preset number of intervals and the output values of the upper boundary line and the lower boundary line at the end points of each interval;
  • the adaptive curve is obtained according to a preset fitting algorithm and the upper boundary line and the lower boundary line.
  • the metadata of the HDR video frame is the luminance average of the HDR frame.
  • the obtaining, according to the metadata of the HDR video frame and the preset number of intervals, the output values of the upper boundary and the lower boundary at the end points of each interval including:
  • input is the brightness average of the input HDR frame
  • output is the gamma output value
  • is the gamma distortion coefficient
  • n is the point number within the interval
  • the output value of the end point of the highlight interval is obtained according to the symmetrical relationship between the highlight interval and the low light interval and the output value at the end of the low light interval.
  • the determining the upper boundary line and the lower boundary line according to the preset number of intervals and the output values of the upper boundary line and the lower boundary line at the end points of each interval including:
  • ⁇ y 1 represents the difference of the output of the low light interval
  • ⁇ y 2 represents the difference of the output of the medium light interval
  • the midpoint of the medium light interval is the intersection of the upper boundary and the lower boundary
  • the lower limit line down_line is set to a straight line with a slope of 1.
  • the adaptive curve is obtained according to a preset fitting algorithm and the upper boundary line and the lower boundary line, including:
  • mean_up is 1024; and the expression clip3(mean, 0, mean_up) means: when mean ⁇ 0, the value of the expression is 0, when mean>mean_up, the value of the expression is mean_up; when 0 ⁇ mean ⁇ When mean_up, the value of the expression is mean.
  • the converted C component is enhanced according to a preset color interval, and the processed C is obtained.
  • Component including:
  • the converted C component is subjected to saturation enhancement according to the preset color interval and the enhancement coefficient corresponding to each color interval, and the processed C component is obtained.
  • an embodiment of the present invention provides a processing apparatus for a video frame, where the apparatus includes: a determining unit, a Y component adjusting unit, and a C component processing unit;
  • the determining unit is configured to determine an adaptive curve corresponding to the HDR video frame according to a Y component of the HDR video frame;
  • the Y component adjustment unit is configured to adjust a Y component of the HDR video frame according to the adaptive curve, and output the processed Y component;
  • the C component processing unit is configured to convert the C component of the HDR video frame according to a preset mapping relationship, and then perform the color saturation enhancement according to the preset color interval by the converted C component, and obtain the processed C component.
  • the determining unit is configured to acquire metadata of the HDR video frame according to a Y component of the HDR video frame, where the metadata is used to indicate feature information of the HDR video frame;
  • the metadata of the HDR video frame is the luminance average of the HDR frame.
  • the determining unit is configured to:
  • input is the brightness average of the input HDR frame
  • output is the gamma output value
  • is the gamma distortion coefficient
  • n is the point number within the interval
  • the output value of the end point of the highlight interval is obtained.
  • the determining unit is configured to:
  • ⁇ y 1 represents the difference of the output of the low light interval
  • ⁇ y 2 represents the difference of the output of the medium light interval
  • the midpoint of the medium light interval is the intersection of the upper boundary and the lower boundary
  • the lower limit line down_line is set to a straight line having a slope of 1.
  • the determining unit is configured to:
  • mean_up is 1024; and the expression clip3(mean, 0, mean_up) means: when mean ⁇ 0, the value of the expression is 0, when mean>mean_up, the value of the expression is mean_up; when 0 ⁇ mean ⁇ When mean_up, the value of the expression is mean.
  • the C component processing unit is configured to:
  • an embodiment of the present invention further provides a processing apparatus for a video frame, including: a processor and a memory configured to store a computer program executable on the processor,
  • the processor is configured to perform the steps of the processing method of the video frame when the computer program is executed.
  • an embodiment of the present invention provides a computer readable storage medium having stored thereon a set of instructions that, when executed, cause at least one processor to perform the processing method of the video frame described above.
  • Embodiments of the present invention provide a method, an apparatus, and a computer readable storage medium for processing a video frame, and the Y component and the C component of the HDR video frame are respectively processed by an adaptive curve and a color saturation enhancement process, so that the processed video frame is processed.
  • the HDR effect is produced when the SDR display is played.
  • FIG. 1 is a comparison diagram of a photoelectric conversion curve of HDR and a photoelectric conversion curve of an SDR display according to an embodiment of the present invention
  • FIG. 2 is a schematic flowchart of a method for processing a video frame according to an embodiment of the present invention
  • FIG. 3 is a schematic flowchart of determining an adaptive curve according to an embodiment of the present invention.
  • FIG. 4 is a schematic diagram of a curve according to an embodiment of the present invention.
  • FIG. 5 is a schematic flowchart of obtaining an output value of an upper boundary line and a lower boundary line at an end point of each interval according to an embodiment of the present disclosure
  • FIG. 6 is a schematic diagram of a curve base according to an embodiment of the present invention.
  • FIG. 7 is a schematic diagram of another curve base according to an embodiment of the present invention.
  • FIG. 8 is a schematic flowchart of obtaining a processed C component according to an embodiment of the present disclosure.
  • FIG. 9 is a schematic structural diagram of a device for processing a video frame according to an embodiment of the present invention.
  • the embodiment of the invention firstly briefly describes the problems brought by the HDR video playing directly on the SDR display.
  • the HDR photoelectric conversion curve on the left is taken as an example of the ST2084 standard, and the normalized optical signal is represented by 0 to 1. It is 0nit to 10000nit. It can be seen that the curve is steep in the low light portion and smooth in the middle and high light portions; the SDR photoelectric conversion curve in the right figure is exemplified by the BT709 standard, and the normalized optical signal from 0 to 1 indicates 0 nit to 100 nit. You can see the curve on the right The steepness in the low light portion is lower than that in the left image; and the curve in the right image is also flatter than the left image in the middle and high light portions.
  • HDR video with the horizontal axis coordinate of about 0.5 is interpreted according to the SDR signal, then the value of the electrical signal is significantly higher, and since the left curve is in the middle part of the middle part of the curve. Gentle, causing a significant difference in the difference between optical signals. Therefore, when no HDR video is played directly through the SDR display, the medium light portion will be significantly brighter and the contrast is significantly lower. As a result, HDR video is generally over-bright and whitened on SDR displays.
  • the HDR video content is the BT2020 domain and the SDR display is the BT709 domain, there is a problem that the HDR video is directly played through the SDR display, and the color is not bright enough.
  • the embodiments of the present invention are directed to the above problems and the causes of the problems.
  • the following embodiments provide a method and a device for processing video frames to implement an HDR effect when an HDR video is played on an SDR display.
  • the method may include:
  • S201 Determine an adaptation curve corresponding to the HDR video frame according to a Y component of the HDR video frame.
  • S203 Convert the C component of the HDR video frame according to a preset mapping relationship, and then perform the color saturation enhancement on the converted C component according to a preset color interval to obtain a processed C component.
  • the Y component and the C component of the HDR video frame are respectively subjected to an adaptive curve and color saturation enhancement processing, so that the processed video frame produces an HDR effect when played on the SDR display.
  • determining the adaptive curve corresponding to the HDR video frame according to the Y component of the HDR video frame, as described in step S201, may specifically include:
  • S2011 Acquire metadata of the HDR video frame according to a Y component of the HDR video frame, where the metadata is used to indicate feature information of the HDR video frame;
  • the metadata of the HDR video frame is the average value of the brightness of the HDR frame.
  • S2012 Obtain an output value of an upper boundary line and a lower boundary line at an end point of each interval according to metadata of the HDR video frame and a preset number of intervals;
  • S2013 determining an upper boundary line and a lower boundary line according to the preset number of intervals and the output values of the upper boundary line and the lower boundary line at the end points of each interval;
  • S2014 Acquire the adaptive curve according to a preset fitting algorithm and the upper boundary line and the lower boundary line.
  • the dotted line is an adaptive curve, and it can be seen that the adaptive curve adaptively changes between the upper boundary line up_line and the lower boundary line down_line.
  • the abscissa in Fig. 4 represents the input value of HDR 10bit, which can be regarded as the value of the Y component of the current HDR video frame; the ordinate is the output value, which can be regarded as the adjusted Y component value.
  • the video frame can include three regions of low light, medium light, and high light.
  • the end point B of the low-light area of the upper line up_line determines an increase in the low-light contrast pull-up point.
  • the position of the B point determines the low-light pull-up range and the pull-down amplitude of the light between the BCs;
  • the start point D of the highlight area of the upper line up_line It can be directly set to the symmetry point of point B for the promotion of highlights and determines the extent of the light increase between points C and D.
  • the adjustment of the low-light area and the highlight area of the video frame at points B and D is the focus of realizing the effect of generating HDR when the HDR video frame is played in the SDR display. Therefore, referring to FIG. 5, according to the HDR video described in step S2012 The metadata of the frame and the preset number of intervals obtain the output values of the upper and lower bounds at the endpoints of each interval, including:
  • input is the brightness average of the input HDR frame
  • output is the gamma output value
  • is the gamma distortion coefficient
  • S20123 obtaining an output value at an end point of the low light interval according to a derivative of the gamma output value and a preset low light division threshold;
  • S20124 The output value of the end point of the highlight section is obtained according to the symmetrical relationship between the highlight interval and the low light interval and the output value at the end of the low light interval.
  • B_split is the output value of B at the end of the low-light interval
  • g_thr is the B-point division threshold, and the larger the g_thr, the more the value is set to low light
  • the B-point is the first derivative with the derivative greater than g_thr a point.
  • Equation 4 For the symmetrical relationship between the highlight interval and the low light interval and the output value at the end of the low light interval described in S20124, the output value of the end point of the highlight interval is obtained, and in the specific implementation process, the calculation can be performed by Equation 4. :
  • the output value at the end of the interval is obtained, and accordingly, the upper boundary and the lower boundary are determined according to the preset number of intervals and the output values of the upper and lower boundaries at the end points of the intervals, as described in step S2013, and may include:
  • the lower limit line down_line is set to a straight line having a slope of 1.
  • the upper boundary line up_line and the lower boundary line down_line each include four intervals, as shown in Equation 6:
  • Each segment can be designed using a curve, a quadratic curve, or even an N-time curve, but it must be ensured that the derivative is positive.
  • the upper boundary line up_line is designed as a curve base with a ⁇ value equal to 1.5 and a g_thr of 0.66, and the coordinates of points B and D are determined. Therefore, the AB segment, the BC segment and the CD segment are straight lines, and the DE segment is a curve. As shown in Equation 6. Point C is the intersection of the upper boundary line up_line and the lower boundary line down_line, which is 512 in this embodiment.
  • obtaining the adaptive curve according to the preset fitting algorithm and the upper boundary line and the lower boundary line described in step S2014 may include:
  • mean_up is 1024; and the expression clip3(mean, 0, mean_up) means: when mean ⁇ 0, the value of the expression is 0, when mean>mean_up, the value of the expression is mean_up; when 0 ⁇ mean ⁇ When mean_up, the value of the expression is mean.
  • the adaptive curve LUT here is a broken line shown in FIG. 4 for indicating the correspondence between the Y component of the HDR video frame and the Y component after processing, and therefore, the Y component of the HDR video frame can be As the input value of the broken line, the corresponding output value is the Y component of the HDR video frame adjusted according to the adaptive curve described in step S102, and the processed Y component is output.
  • the converted C component is enhanced according to a preset color interval.
  • the processed C component which may specifically include:
  • S2031 Convert the C component from the BT2020 domain to the BT709 domain according to a preset conversion matrix
  • S2032 Perform saturation enhancement on the converted C component according to a preset color interval and an enhancement coefficient corresponding to each color interval, to obtain a processed C component.
  • step S2031 the conversion of the YUV BT2020 domain to the BT709 domain can be performed by Equation 8:
  • the conversion matrix Can be It can be understood that a person skilled in the art can select different conversion matrices according to the actual situation and needs, and this embodiment does not describe the details.
  • the preset color interval may be 6 color intervals, which may include red, yellow, green, blue-green, blue, and magenta; the saturation enhancement coefficients corresponding to different color intervals are respectively corresponding to: 164 , 164, 196, 164, 196, 164.
  • the converted C component is enhanced in accordance with Equation 9, and the processed C component is obtained.
  • the embodiment provides a method for processing a video frame, which adopts an adaptive curve and a color saturation enhancement process for the Y component and the C component of the HDR video frame, respectively, so that the processed video frame generates HDR when played on the SDR display. effect.
  • FIG. 9 a processing device 90 for a video frame according to an embodiment of the present invention is shown.
  • the device 90 includes: a determining unit 901, a Y component adjusting unit 902, and a C component.
  • Processing unit 903 wherein
  • the determining unit 901 is configured to determine an adaptive curve corresponding to the HDR video frame according to a Y component of the high dynamic range HDR video frame;
  • the Y component adjustment unit 902 is configured to adjust a Y component of the HDR video frame according to the adaptive curve, and output the processed Y component;
  • the C component processing unit 903 is configured to convert the C component of the HDR video frame according to a preset mapping relationship, and then perform the color saturation enhancement according to the preset color interval to obtain the processed C component.
  • the latter C component is configured to convert the C component of the HDR video frame according to a preset mapping relationship, and then perform the color saturation enhancement according to the preset color interval to obtain the processed C component. The latter C component.
  • the determining unit 901 is configured to acquire metadata of the HDR video frame according to a Y component of the HDR video frame, where the metadata is used to indicate feature information of the HDR video frame;
  • the out value determines the upper and lower boundaries
  • the metadata of the HDR video frame is the luminance average of the HDR frame.
  • the determining unit 901 is configured to:
  • input is the brightness average of the input HDR frame
  • output is the gamma output value
  • is the gamma distortion coefficient
  • n is the point number within the interval
  • the output value of the end point of the highlight interval is obtained.
  • the determining unit 901 is configured to:
  • ⁇ y 1 represents the difference of the output of the low light interval
  • ⁇ y 2 represents the difference of the output of the medium light interval
  • the midpoint of the medium light interval is the intersection of the upper boundary and the lower boundary
  • the lower limit line down_line is set to a straight line having a slope of 1.
  • the determining unit 901 is configured to:
  • mean_up is 1024; and the expression clip3(mean, 0, mean_up) means: when mean ⁇ 0, the value of the expression is 0, when mean>mean_up, the value of the expression is mean_up; when 0 ⁇ mean ⁇ When mean_up, the value of the expression is mean.
  • the C component processing unit 903 is configured to:
  • the determining unit 901, the Y component adjusting unit 902, and the C component processing unit 903 may be processed by a processor (such as a central processing unit (CPU), a microprocessor (MCU) in the processing device 90 of the video frame. , Micro Control Unit), digital signal processor (DSP, Digital Signal Processor) or programmable logic array (FPGA, Field-Programmable Gate Array), etc.
  • a processor such as a central processing unit (CPU), a microprocessor (MCU) in the processing device 90 of the video frame.
  • Micro Control Unit Micro Control Unit
  • DSP Digital Signal Processor
  • FPGA Field-Programmable Gate Array
  • the processing device for a video frame includes: a processor and a memory configured to store a computer program capable of running on a processor, wherein the processor is configured to run the computer program, carried out:
  • the processor is configured to execute the computer program, executing:
  • the upper boundary line and the lower boundary line are determined according to the preset number of intervals and the output values of the upper boundary line and the lower boundary line at the end points of each interval;
  • the adaptive curve is obtained according to a preset fitting algorithm and the upper boundary line and the lower boundary line.
  • the metadata of the HDR video frame is a luminance average of the HDR frame.
  • the processor is configured to execute the computer program, executing:
  • input is the brightness average of the input HDR frame
  • output is the gamma output value
  • is the gamma distortion coefficient
  • n is the point number within the interval
  • the output value of the end point of the highlight interval is obtained according to the symmetrical relationship between the highlight interval and the low light interval and the output value at the end of the low light interval.
  • the processor is configured to execute the computer program, executing:
  • ⁇ y 1 represents the difference of the output of the low light interval
  • ⁇ y 2 represents the difference of the output of the medium light interval
  • the midpoint of the medium light interval is the intersection of the upper boundary and the lower boundary
  • the lower limit line down_line is set to a straight line with a slope of 1.
  • the processor is configured to execute the computer program, executing:
  • mean_up is 1024; and the expression clip3(mean, 0, mean_up) means: when mean ⁇ 0, the value of the expression is 0, when mean>mean_up, the value of the expression is mean_up; when 0 ⁇ mean ⁇ When mean_up, the value of the expression is mean.
  • the processor is configured to execute the computer program, executing:
  • the converted C component is subjected to saturation enhancement according to the preset color interval and the enhancement coefficient corresponding to each color interval, and the processed C component is obtained.
  • the present embodiment provides a video frame processing apparatus 90 that passes the Y component and the C component of the HDR video frame through adaptive curve and color saturation enhancement processing, respectively, so that the processed video frame generates HDR when played on the SDR display. Effect.
  • embodiments of the present invention can be provided as a method, system, or computer program product. Accordingly, the present invention may employ hardware embodiments, software embodiments, or junctions. In the form of an embodiment of the software and hardware aspects. Moreover, the invention can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage and optical storage, etc.) including computer usable program code.
  • the computer program instructions can also be stored in a computer readable memory that can direct a computer or other programmable data processing device to operate in a particular manner, such that the instructions stored in the computer readable memory produce an article of manufacture comprising the instruction device.
  • the apparatus implements the functions specified in one or more blocks of a flow or a flow and/or block diagram of the flowchart.
  • These computer program instructions can also be loaded onto a computer or other programmable data processing device such that a series of operational steps are performed on a computer or other programmable device to produce computer-implemented processing for execution on a computer or other programmable device.
  • the instructions provide steps for implementing the functions specified in one or more of the flow or in a block or blocks of a flow diagram.
  • an embodiment of the present invention further provides a computer readable storage medium, where a set of instructions are stored, and when executed, at least one processor is caused to perform a processing method of a video frame according to an embodiment of the present invention.
  • the Y component and the C component of the HDR video frame are respectively processed by an adaptive curve and a color saturation enhancement process, so that the processed video frame is in the SDR display. When playing, it produces an HDR effect.

Abstract

Disclosed in an embodiment of the invention are a method and device for processing a video frame, and a computer-readable storage medium. The method comprises: determining, according to a Y component of a high dynamic range (HDR) video frame, a self-adaptive curve corresponding to the HDR video frame; adjusting, according to the self-adaptive curve, the Y component of the HDR video frame, and outputting a processed Y component; and after performing, according to a preset mapping relationship, conversion on a C component of the HDR video frame, performing, according to a preset color range, color saturation enhancement on the converted C component to obtain a processed C component.

Description

一种视频帧的处理方法、装置和计算机可读存储介质Method, device and computer readable storage medium for processing video frames
相关申请的交叉引用Cross-reference to related applications
本申请基于申请号为201610798824.7、申请日为2016年08月31日的中国专利申请提出,并要求该中国专利申请的优先权,该中国专利申请的全部内容在此引入本申请作为参考。The present application is filed on the basis of the Chinese Patent Application Serial No. No. No. No. No. No. No. No. No No No No No No No No No No No No No No No No No No No No No No No No No No No No No No No No No No No No No No No No No No No No No No No No No No No No No No No No No No No No No No No No No No No No No No No
技术领域Technical field
本发明涉及视频处理技术,尤其涉及一种视频帧的处理方法、装置和计算机可读存储介质。The present invention relates to video processing technologies, and in particular, to a video frame processing method, apparatus, and computer readable storage medium.
背景技术Background technique
高动态范围(HDR,High-Dynamic Range)可以拓展显示的亮度范围,展现更多的亮部和暗部细节,为画面带来更丰富的色彩和更生动自然的细节表现,从而使得画面更加接近人眼所见。HDR (High-Dynamic Range) can expand the brightness range of the display, showing more highlights and shadow details, bringing richer colors and more vivid and natural details to the picture, making the picture closer to people. Seeing what you see.
人眼所能感受的亮度范围约为10-3-106尼特(nit),所能感受的瞬时对比度范围可达10000:1;而目前消费级液晶显示器的亮度一般在300至400nit左右,其对比度范围一般为2000:1。可见,人眼的亮度分辨能力远高于当前主流的显示设备。所以HDR显示技术就是要提升显示的亮度范围,通俗来说就是使得显示亮度更亮的同时,又让呈现的黑色更黑。The brightness range that the human eye can feel is about 10 -3 -10 6 nits, and the instantaneous contrast range can be as much as 10,000:1; while the brightness of consumer LCDs is generally around 300 to 400 nit. The contrast range is typically 2000:1. It can be seen that the brightness resolution of the human eye is much higher than that of the current mainstream display devices. Therefore, the HDR display technology is to increase the brightness range of the display. In general, it makes the display brightness brighter and makes the black appear darker.
目前显示器对比度的限制是由1990年国际电信联盟(ITU,International Telecommunication Union)发布的ITU BT.709标准决定的。该标准中的电光转换函数(EOTF,Electro-Optical Transfer Function)是以过去阴极射线管(CRT,Cathode Ray Tube)显示器的特性为基础设定的。但CRT显示器的亮度一般不超过100nit,对于现在亮度可达400nit甚至1000nit的显示器而言,BT.709标准已经束缚了最佳显示效果的呈现。因此,当前不少广播电视界的机构组织均提出了新的HDR显示技术,通过HDR显示器播放 HDR视频,以在实现更高动态范围的显示。The current display contrast limit is determined by the ITU BT.709 standard issued by the International Telecommunication Union (ITU) in 1990. The Electro-Optical Transfer Function (EOTF) in this standard is based on the characteristics of a cathode ray tube (CRT) display. However, the brightness of CRT monitors generally does not exceed 100 nit. For displays with brightness up to 400 nit or even 1000 nit, the BT.709 standard has bound the best display. Therefore, many organizations in the broadcasting and television industry have proposed new HDR display technology to play through HDR displays. HDR video to achieve a higher dynamic range of display.
但是现在大部分的存量电视都是基于ITU BT.709标准的标准动态范围(SDR,Standard Dynamic Range)显示器。因此,会造成这些显示器在播放HDR内容时,用户也无法体会到HDR的效果。However, most of the existing TVs are now based on the Standard Dynamic Range (SDR) display of the ITU BT.709 standard. As a result, users will not be able to experience the effects of HDR when playing HDR content on these displays.
发明内容Summary of the invention
为解决上述技术问题,本发明实施例期望提供一种视频帧的处理方法/装置和计算机可读存储介质。In order to solve the above technical problem, embodiments of the present invention are directed to a method and apparatus for processing a video frame and a computer readable storage medium.
本发明实施例的技术方案是这样实现的:The technical solution of the embodiment of the present invention is implemented as follows:
第一方面,本发明实施例提供了一种视频帧的处理方法,所述方法包括:In a first aspect, an embodiment of the present invention provides a method for processing a video frame, where the method includes:
根据HDR视频帧的Y分量确定所述HDR视频帧对应的自适应曲线;Determining an adaptive curve corresponding to the HDR video frame according to a Y component of the HDR video frame;
根据所述自适应曲线对所述HDR视频帧的Y分量进行调整,输出处理后的Y分量;Adjusting a Y component of the HDR video frame according to the adaptive curve, and outputting the processed Y component;
将所述HDR视频帧的C分量按照预设的映射关系进行转换后,再将转换后的C分量按照预设的颜色区间进行颜色饱和度增强,得到处理后的C分量。After the C component of the HDR video frame is converted according to a preset mapping relationship, the converted C component is further enhanced in color saturation according to a preset color interval to obtain a processed C component.
在上述方案中,所述根据HDR视频帧的Y分量确定所述HDR视频帧对应的自适应曲线,包括:In the above solution, the determining, according to the Y component of the HDR video frame, the adaptation curve corresponding to the HDR video frame, including:
根据所述HDR视频帧的Y分量获取所述HDR视频帧的元数据;其中,所述元数据用于指示所述HDR视频帧的特征信息;Obtaining metadata of the HDR video frame according to a Y component of the HDR video frame; wherein the metadata is used to indicate feature information of the HDR video frame;
根据所述HDR视频帧的元数据以及预设的区间数目获取上界线和下界线在各区间端点处的输出值;Obtaining an output value of the upper boundary line and the lower boundary line at the end points of each interval according to the metadata of the HDR video frame and the preset number of intervals;
根据预设的区间数目以及上界线和下界线在各区间端点处的输出值确定上界线和下界线;The upper boundary line and the lower boundary line are determined according to the preset number of intervals and the output values of the upper boundary line and the lower boundary line at the end points of each interval;
按照预设的拟合算法以及所述上界线和所述下界线获取所述自适应曲线。The adaptive curve is obtained according to a preset fitting algorithm and the upper boundary line and the lower boundary line.
在上述方案中,所述HDR视频帧的元数据为所述HDR帧的亮度均值。 In the above solution, the metadata of the HDR video frame is the luminance average of the HDR frame.
在上述方案中,所述根据所述HDR视频帧的元数据以及预设的区间数目获取上界线和下界线在各区间端点处的输出值,包括:In the above solution, the obtaining, according to the metadata of the HDR video frame and the preset number of intervals, the output values of the upper boundary and the lower boundary at the end points of each interval, including:
根据所述HDR帧的亮度均值以及式1进行伽马(gamma)转换,获得gamma输出值;Obtaining a gamma output value according to a luminance average of the HDR frame and a gamma conversion of Equation 1;
Figure PCTCN2017082524-appb-000001
Figure PCTCN2017082524-appb-000001
其中,input为输入的HDR帧的亮度均值,output为gamma输出值,γ为gamma扭曲系数;Wherein, input is the brightness average of the input HDR frame, output is the gamma output value, and γ is the gamma distortion coefficient;
根据式2获取gamma输出值的导数gradient[n];Obtaining the derivative gradient[n] of the gamma output value according to Equation 2;
Figure PCTCN2017082524-appb-000002
Figure PCTCN2017082524-appb-000002
其中,n为区间内的点序号;Where n is the point number within the interval;
根据gamma输出值的导数以及预设的低光划分门限获取低光区间的端点处的输出值;Obtaining an output value at an end point of the low light interval according to a derivative of the gamma output value and a preset low light division threshold;
根据高光区间与低光区间之间的对称关系以及低光区间的端点处的输出值,获取高光区间的端点出的输出值。The output value of the end point of the highlight interval is obtained according to the symmetrical relationship between the highlight interval and the low light interval and the output value at the end of the low light interval.
在上述方案中,所述根据预设的区间数目以及上界线和下界线在各区间端点处的输出值确定上界线和下界线,包括:In the above solution, the determining the upper boundary line and the lower boundary line according to the preset number of intervals and the output values of the upper boundary line and the lower boundary line at the end points of each interval, including:
在获取到低光区间的端点处的输出值和高光区间的端点出的输出值之后,根据式3获取上界线up_line在各区间的曲线;After acquiring the output value at the end point of the low light interval and the output value at the end point of the highlight interval, the curve of the upper boundary line up_line in each interval is obtained according to Equation 3;
Figure PCTCN2017082524-appb-000003
Figure PCTCN2017082524-appb-000003
其中,Δy1表示的是低光区间的output的差值;Δy2表示中光区间的output的差值;中光区间的中点就是上界线和下界线的交汇点;Where Δy 1 represents the difference of the output of the low light interval; Δy 2 represents the difference of the output of the medium light interval; the midpoint of the medium light interval is the intersection of the upper boundary and the lower boundary;
所述下界线down_line设置为斜率为1的直线。 The lower limit line down_line is set to a straight line with a slope of 1.
在上述方案中,按照预设的拟合算法以及所述上界线和所述下界线获取所述自适应曲线,包括:In the above solution, the adaptive curve is obtained according to a preset fitting algorithm and the upper boundary line and the lower boundary line, including:
根据所述上界线up_line和所述下界线down_line以及式4获取所述自适应曲线LUT;Obtaining the adaptive curve LUT according to the upper boundary line up_line and the lower boundary line down_line and Equation 4;
Figure PCTCN2017082524-appb-000004
Figure PCTCN2017082524-appb-000004
其中,mean_up为1024;并且表达式clip3(mean,0,mean_up)表示:当mean<0时,表达式的值为0,当mean>mean_up时,表达式的值为mean_up;当0<mean<mean_up时,表达式的值为mean。Where mean_up is 1024; and the expression clip3(mean, 0, mean_up) means: when mean<0, the value of the expression is 0, when mean>mean_up, the value of the expression is mean_up; when 0<mean< When mean_up, the value of the expression is mean.
在上述方案中,所述将所述HDR视频帧的C分量按照预设的映射关系进行转换后,再将转换后的C分量按照预设的颜色区间进行颜色饱和度增强,得到处理后的C分量,包括:In the above solution, after converting the C component of the HDR video frame according to a preset mapping relationship, the converted C component is enhanced according to a preset color interval, and the processed C is obtained. Component, including:
按照预设的转换矩阵将C分量由BT2020域转换到BT709域;Converting the C component from the BT2020 domain to the BT709 domain according to a preset conversion matrix;
根据预设的颜色区间以及各颜色区间对应的增强系数对转换后的C分量进行饱和度增强,得到处理后的C分量。The converted C component is subjected to saturation enhancement according to the preset color interval and the enhancement coefficient corresponding to each color interval, and the processed C component is obtained.
第二方面,本发明实施例提供了一种视频帧的处理装置,所述装置包括:确定单元、Y分量调整单元和C分量处理单元;其中,In a second aspect, an embodiment of the present invention provides a processing apparatus for a video frame, where the apparatus includes: a determining unit, a Y component adjusting unit, and a C component processing unit;
所述确定单元,配置为根据HDR视频帧的Y分量确定所述HDR视频帧对应的自适应曲线;The determining unit is configured to determine an adaptive curve corresponding to the HDR video frame according to a Y component of the HDR video frame;
所述Y分量调整单元,配置为根据所述自适应曲线对所述HDR视频帧的Y分量进行调整,输出处理后的Y分量;The Y component adjustment unit is configured to adjust a Y component of the HDR video frame according to the adaptive curve, and output the processed Y component;
所述C分量处理单元,配置为将所述HDR视频帧的C分量按照预设的映射关系进行转换后,再将转换后的C分量按照预设的颜色区间进行颜色饱和度增强,得到处理后的C分量。The C component processing unit is configured to convert the C component of the HDR video frame according to a preset mapping relationship, and then perform the color saturation enhancement according to the preset color interval by the converted C component, and obtain the processed C component.
在上述方案中,所述确定单元,配置为根据所述HDR视频帧的Y分量获取所述HDR视频帧的元数据;其中,所述元数据用于指示所述HDR视频帧的特征信息;In the foregoing solution, the determining unit is configured to acquire metadata of the HDR video frame according to a Y component of the HDR video frame, where the metadata is used to indicate feature information of the HDR video frame;
以及,根据所述HDR视频帧的元数据以及预设的区间数目获取上界线 和下界线在各区间端点处的输出值;And obtaining an upper boundary according to the metadata of the HDR video frame and the preset number of intervals And the output value of the lower boundary at the end of each interval;
以及,根据预设的区间数目以及上界线和下界线在各区间端点处的输出值确定上界线和下界线;And determining an upper boundary line and a lower boundary line according to the preset number of intervals and the output values of the upper boundary line and the lower boundary line at the end points of each interval;
以及,按照预设的拟合算法以及所述上界线和所述下界线获取所述自适应曲线。And acquiring the adaptive curve according to a preset fitting algorithm and the upper boundary line and the lower boundary line.
在上述方案中,所述HDR视频帧的元数据为所述HDR帧的亮度均值。In the above solution, the metadata of the HDR video frame is the luminance average of the HDR frame.
在上述方案中,所述确定单元,配置为:In the above solution, the determining unit is configured to:
根据所述HDR帧的亮度均值以及式5进行gamma转换,获得gamma输出值;Obtaining a gamma output value according to a brightness average of the HDR frame and a gamma conversion of Equation 5;
Figure PCTCN2017082524-appb-000005
Figure PCTCN2017082524-appb-000005
其中,input为输入的HDR帧的亮度均值,output为gamma输出值,γ为gamma扭曲系数;Wherein, input is the brightness average of the input HDR frame, output is the gamma output value, and γ is the gamma distortion coefficient;
以及,根据式6获取gamma输出值的导数gradient[n];And obtaining a derivative of the gamma output value according to Equation 6 gradient[n];
Figure PCTCN2017082524-appb-000006
Figure PCTCN2017082524-appb-000006
其中,n为区间内的点序号;Where n is the point number within the interval;
以及,根据gamma输出值的导数以及预设的低光划分门限获取低光区间的端点处的输出值;And obtaining an output value at an end point of the low light interval according to a derivative of the gamma output value and a preset low light division threshold;
以及,根据高光区间与低光区间之间的对称关系以及低光区间的端点处的输出值,获取高光区间的端点出的输出值。And, according to the symmetrical relationship between the highlight interval and the low light interval and the output value at the end of the low light interval, the output value of the end point of the highlight interval is obtained.
在上述方案中,所述确定单元,配置为:In the above solution, the determining unit is configured to:
在获取到低光区间的端点处的输出值和高光区间的端点出的输出值之后,根据式7获取上界线up_line在各区间的曲线; After acquiring the output value at the end point of the low light interval and the output value at the end point of the highlight interval, obtaining a curve of the upper boundary line up_line in each interval according to Equation 7;
Figure PCTCN2017082524-appb-000007
Figure PCTCN2017082524-appb-000007
其中,Δy1表示的是低光区间的output的差值;Δy2表示中光区间的output的差值;中光区间的中点就是上界线和下界线的交汇点;Where Δy 1 represents the difference of the output of the low light interval; Δy 2 represents the difference of the output of the medium light interval; the midpoint of the medium light interval is the intersection of the upper boundary and the lower boundary;
以及,所述下界线down_line设置为斜率为1的直线。And, the lower limit line down_line is set to a straight line having a slope of 1.
在上述方案中,所述确定单元,配置为:In the above solution, the determining unit is configured to:
根据所述上界线up_line和所述下界线down_line以及式8获取所述自适应曲线LUT;Obtaining the adaptive curve LUT according to the upper boundary line up_line and the lower boundary line down_line and Equation 8;
Figure PCTCN2017082524-appb-000008
Figure PCTCN2017082524-appb-000008
其中,mean_up为1024;并且表达式clip3(mean,0,mean_up)表示:当mean<0时,表达式的值为0,当mean>mean_up时,表达式的值为mean_up;当0<mean<mean_up时,表达式的值为mean。Where mean_up is 1024; and the expression clip3(mean, 0, mean_up) means: when mean<0, the value of the expression is 0, when mean>mean_up, the value of the expression is mean_up; when 0<mean< When mean_up, the value of the expression is mean.
在上述方案中,所述C分量处理单元,配置为:In the above solution, the C component processing unit is configured to:
按照预设的转换矩阵将C分量由BT2020域转换到BT709域;Converting the C component from the BT2020 domain to the BT709 domain according to a preset conversion matrix;
以及,根据预设的颜色区间以及各颜色区间对应的增强系数对转换后的C分量进行饱和度增强,得到处理后的C分量。And performing saturation enhancement on the converted C component according to the preset color interval and the enhancement coefficient corresponding to each color interval, to obtain the processed C component.
第三方面,本发明实施例还提供了一种视频帧的处理装置,包括:处理器和配置为存储能够在处理器上运行的计算机程序的存储器,In a third aspect, an embodiment of the present invention further provides a processing apparatus for a video frame, including: a processor and a memory configured to store a computer program executable on the processor,
其中,所述处理器配置为运行所述计算机程序时,执行上述视频帧的处理方法的步骤。The processor is configured to perform the steps of the processing method of the video frame when the computer program is executed.
第四方面,本发明实施例提供了一种计算机可读存储介质,其上存储有一组指令,当执行所述指令时,引起至少一个处理器执行上述的视频帧的处理方法。 In a fourth aspect, an embodiment of the present invention provides a computer readable storage medium having stored thereon a set of instructions that, when executed, cause at least one processor to perform the processing method of the video frame described above.
本发明实施例提供了一种视频帧的处理方法、装置和计算机可读存储介质,将HDR视频帧的Y分量和C分量分别通过自适应曲线和颜色饱和度增强处理,使得处理后的视频帧在SDR显示器播放时,产生HDR的效果。Embodiments of the present invention provide a method, an apparatus, and a computer readable storage medium for processing a video frame, and the Y component and the C component of the HDR video frame are respectively processed by an adaptive curve and a color saturation enhancement process, so that the processed video frame is processed. The HDR effect is produced when the SDR display is played.
附图说明DRAWINGS
图1为本发明实施例提供的HDR的光电转换曲线和SDR显示器的光电转换曲线的对比图;1 is a comparison diagram of a photoelectric conversion curve of HDR and a photoelectric conversion curve of an SDR display according to an embodiment of the present invention;
图2为本发明实施例提供的一种视频帧的处理方法流程示意图;2 is a schematic flowchart of a method for processing a video frame according to an embodiment of the present invention;
图3为本发明实施例提供的一种确定自适应曲线的流程示意图;FIG. 3 is a schematic flowchart of determining an adaptive curve according to an embodiment of the present invention;
图4为本发明实施例提供的一种曲线示意图;4 is a schematic diagram of a curve according to an embodiment of the present invention;
图5为本发明实施例提供的一种获取上界线和下界线在各区间端点处的输出值的流程示意图;FIG. 5 is a schematic flowchart of obtaining an output value of an upper boundary line and a lower boundary line at an end point of each interval according to an embodiment of the present disclosure;
图6为本发明实施例提供的一种曲线基的示意图;FIG. 6 is a schematic diagram of a curve base according to an embodiment of the present invention; FIG.
图7为本发明实施例提供的另一种曲线基的示意图;FIG. 7 is a schematic diagram of another curve base according to an embodiment of the present invention; FIG.
图8为本发明实施例提供的一种获得处理后的C分量的流程示意图;FIG. 8 is a schematic flowchart of obtaining a processed C component according to an embodiment of the present disclosure;
图9为本发明实施例提供的一种视频帧的处理装置结构示意图。FIG. 9 is a schematic structural diagram of a device for processing a video frame according to an embodiment of the present invention.
具体实施方式detailed description
下面将结合本发明实施例中的附图,对本发明实施例中的技术方案进行清楚、完整地描述。The technical solutions in the embodiments of the present invention will be clearly and completely described in the following with reference to the accompanying drawings.
本发明实施例首先对HDR视频直接在SDR显示器上进行播放所带来的问题进行简要的阐述。The embodiment of the invention firstly briefly describes the problems brought by the HDR video playing directly on the SDR display.
如图1所示的HDR的光电转换曲线和SDR显示器的光电转换曲线的对比图,可以看出,左图的HDR光电转换曲线以ST2084标准为例,归一化的光信号从0到1表示的是0nit到10000nit。可以看到曲线在低光部分很陡峭,在中光和高光部分很平缓;右图的SDR光电转换曲线以BT709标准为例,归一化的光信号从0到1表示的是0nit到100nit,可以看到右图曲线 在低光部分的陡峭程度低于左图;并且右图曲线在中光及高光部分的平缓程度也低于左图。As shown in the comparison between the HDR photoelectric conversion curve and the photoelectric conversion curve of the SDR display shown in FIG. 1, it can be seen that the HDR photoelectric conversion curve on the left is taken as an example of the ST2084 standard, and the normalized optical signal is represented by 0 to 1. It is 0nit to 10000nit. It can be seen that the curve is steep in the low light portion and smooth in the middle and high light portions; the SDR photoelectric conversion curve in the right figure is exemplified by the BT709 standard, and the normalized optical signal from 0 to 1 indicates 0 nit to 100 nit. You can see the curve on the right The steepness in the low light portion is lower than that in the left image; and the curve in the right image is also flatter than the left image in the middle and high light portions.
而在实际拍摄时,无论是HDR设备拍摄还是SDR设备拍摄;所以大部分拍摄的视频还是集中在中光部分,也就是横轴坐标为0.5附近的范围。In actual shooting, whether it is HDR device shooting or SDR device shooting; so most of the captured video is concentrated in the middle part, that is, the horizontal axis coordinates are around 0.5.
但是,横轴坐标为0.5左右的HDR视频如果按照SDR信号来解读的话,那么会出现电信号取值明显偏高的情况,而且由于左图曲线在中光部分相对右图曲线的中光部分较为平缓,造成光信号之间的差异明显降低。所以当不做任何处理,直接通过SDR显示器播放HDR视频时,中光部分就会明显过亮,且对比度明显偏低。从而造成HDR视频在SDR显示器上播放普遍出现过亮,泛白的原因。However, if the HDR video with the horizontal axis coordinate of about 0.5 is interpreted according to the SDR signal, then the value of the electrical signal is significantly higher, and since the left curve is in the middle part of the middle part of the curve. Gentle, causing a significant difference in the difference between optical signals. Therefore, when no HDR video is played directly through the SDR display, the medium light portion will be significantly brighter and the contrast is significantly lower. As a result, HDR video is generally over-bright and whitened on SDR displays.
另外,由于HDR视频内容是BT2020域的,SDR显示器是BT709域的,因此,直接通过SDR显示器播放HDR视频还会出现颜色不够艳丽的问题。In addition, since the HDR video content is the BT2020 domain and the SDR display is the BT709 domain, there is a problem that the HDR video is directly played through the SDR display, and the color is not bright enough.
本发明实施例针对上述问题及问题出现的原因,通过下面的实施例提出了一种视频帧的处理方法和装置,以实现在SDR显示器播放HDR视频时,产生HDR的效果。The embodiments of the present invention are directed to the above problems and the causes of the problems. The following embodiments provide a method and a device for processing video frames to implement an HDR effect when an HDR video is played on an SDR display.
实施例一 Embodiment 1
参见图2,其示出了本发明实施例提供的一种视频帧的处理方法,该方法可以包括:Referring to FIG. 2, a method for processing a video frame according to an embodiment of the present invention is provided. The method may include:
S201:根据HDR视频帧的Y分量确定所述HDR视频帧对应的自适应曲线;S201: Determine an adaptation curve corresponding to the HDR video frame according to a Y component of the HDR video frame.
S202:根据所述自适应曲线对所述HDR视频帧的Y分量进行调整,输出处理后的Y分量;S202: Adjust a Y component of the HDR video frame according to the adaptive curve, and output the processed Y component.
S203:将所述HDR视频帧的C分量按照预设的映射关系进行转换后,再将转换后的C分量按照预设的颜色区间进行颜色饱和度增强,得到处理后的C分量。S203: Convert the C component of the HDR video frame according to a preset mapping relationship, and then perform the color saturation enhancement on the converted C component according to a preset color interval to obtain a processed C component.
通过上述方案,将HDR视频帧的Y分量和C分量分别通过自适应曲线和颜色饱和度增强处理,使得处理后的视频帧在SDR显示器播放时,产生HDR的效果。 Through the above scheme, the Y component and the C component of the HDR video frame are respectively subjected to an adaptive curve and color saturation enhancement processing, so that the processed video frame produces an HDR effect when played on the SDR display.
示例性地,参见图3,对于步骤S201所述的根据HDR视频帧的Y分量确定所述HDR视频帧对应的自适应曲线,具体可以包括:Illustratively, referring to FIG. 3, determining the adaptive curve corresponding to the HDR video frame according to the Y component of the HDR video frame, as described in step S201, may specifically include:
S2011:根据HDR视频帧的Y分量获取所述HDR视频帧的元数据;其中,所述元数据用于指示所述HDR视频帧的特征信息;S2011: Acquire metadata of the HDR video frame according to a Y component of the HDR video frame, where the metadata is used to indicate feature information of the HDR video frame;
在本实施例中,所述HDR视频帧的元数据为所述HDR帧的亮度均值。In this embodiment, the metadata of the HDR video frame is the average value of the brightness of the HDR frame.
S2012:根据HDR视频帧的元数据以及预设的区间数目获取上界线和下界线在各区间端点处的输出值;S2012: Obtain an output value of an upper boundary line and a lower boundary line at an end point of each interval according to metadata of the HDR video frame and a preset number of intervals;
S2013:根据预设的区间数目以及上界线和下界线在各区间端点处的输出值确定上界线和下界线;S2013: determining an upper boundary line and a lower boundary line according to the preset number of intervals and the output values of the upper boundary line and the lower boundary line at the end points of each interval;
S2014:按照预设的拟合算法以及所述上界线和所述下界线获取所述自适应曲线。S2014: Acquire the adaptive curve according to a preset fitting algorithm and the upper boundary line and the lower boundary line.
对于上述示例,需要说明的是,如图4中的标注,虚线为自适应曲线,可以看出,自适应曲线在上界线up_line和下界线down_line之间自适应变化。图4中的横坐标表示HDR 10bit的输入值,可以认为是当前HDR视频帧的Y分量的值;纵坐标为输出值,可以认为是调整后的Y分量值。For the above example, it should be noted that, as indicated in FIG. 4, the dotted line is an adaptive curve, and it can be seen that the adaptive curve adaptively changes between the upper boundary line up_line and the lower boundary line down_line. The abscissa in Fig. 4 represents the input value of HDR 10bit, which can be regarded as the value of the Y component of the current HDR video frame; the ordinate is the output value, which can be regarded as the adjusted Y component value.
由于视频帧可以包括低光、中光和高光三个区域。上界线up_line的低光区域的终点B确定一个增加低光对比度拉升点,B点位置决定了低光拉升幅度以及BC之间中光的下拉幅度;上界线up_line的高光区域的起点D点可以直接设定为B点的对称点,用于对高光的提升,并且决定了C、D点之间中光的提升幅度。Since the video frame can include three regions of low light, medium light, and high light. The end point B of the low-light area of the upper line up_line determines an increase in the low-light contrast pull-up point. The position of the B point determines the low-light pull-up range and the pull-down amplitude of the light between the BCs; the start point D of the highlight area of the upper line up_line It can be directly set to the symmetry point of point B for the promotion of highlights and determines the extent of the light increase between points C and D.
B点和D点对于视频帧低光区域和高光区域的调整是实现HDR视频帧在SDR显示器播放时产生HDR的效果的重点,因此,参见图5,对于步骤S2012所述的根据所述HDR视频帧的元数据以及预设的区间数目获取上界线和下界线在各区间端点处的输出值,具体包括:The adjustment of the low-light area and the highlight area of the video frame at points B and D is the focus of realizing the effect of generating HDR when the HDR video frame is played in the SDR display. Therefore, referring to FIG. 5, according to the HDR video described in step S2012 The metadata of the frame and the preset number of intervals obtain the output values of the upper and lower bounds at the endpoints of each interval, including:
S20121:根据所述HDR帧的亮度均值以及式1进行gamma转换,获得gamma输出值;S20121: performing gamma conversion according to the brightness average value of the HDR frame and Equation 1, to obtain a gamma output value;
Figure PCTCN2017082524-appb-000009
Figure PCTCN2017082524-appb-000009
其中,input为输入的HDR帧的亮度均值,output为gamma输出值,γ为gamma扭曲系数;Wherein, input is the brightness average of the input HDR frame, output is the gamma output value, and γ is the gamma distortion coefficient;
需要说明的是,参见图6,表示γ从1.0遍历到2.2的时候,gamma转换的曲线基,图6所示的为上界线up_line的曲线基;可以看出γ为1.0的时候表示1:1输出,不做任何压缩;γ越大,对低光的压缩越明显,效果会越倾向于黑色更黑。同时次低光部分细节对比度会越明显。It should be noted that, referring to FIG. 6, the curve base of gamma conversion when γ is traversed from 1.0 to 2.2, and the curve base of upper line up_line shown in FIG. 6; it can be seen that γ is 1.0 when indicating 1:1. Output, do not do any compression; the larger γ, the more obvious the compression of low light, the more inclined the effect will be blacker and darker. At the same time, the detail contrast of the sub-low light part will be more obvious.
相应地,当γ从1.0遍历到2.2的时候,以γ的倒数作为gamma扭曲系数的gamma转换的曲线基,可以作为下界线down_line的曲线基,如图7所示。Correspondingly, when γ is traversed from 1.0 to 2.2, the reciprocal of γ is used as the curve base of the gamma conversion of the gamma distortion coefficient, which can be used as the curve base of the lower limit line down_line, as shown in FIG.
S20122:根据式2获取gamma输出值的导数;S20122: Obtain a derivative of a gamma output value according to Equation 2;
Figure PCTCN2017082524-appb-000010
Figure PCTCN2017082524-appb-000010
其中,gradient[n]为gamma输出值的导数,n为区间内的点序号;Where gradient[n] is the derivative of the gamma output value and n is the point number within the interval;
S20123:根据gamma输出值的导数以及预设的低光划分门限获取低光区间的端点处的输出值;S20123: obtaining an output value at an end point of the low light interval according to a derivative of the gamma output value and a preset low light division threshold;
S20124:根据高光区间与低光区间之间的对称关系以及低光区间的端点处的输出值,获取高光区间的端点出的输出值。S20124: The output value of the end point of the highlight section is obtained according to the symmetrical relationship between the highlight interval and the low light interval and the output value at the end of the low light interval.
需要说明的是,对于S20123所述的根据gamma输出值的导数以及预设的低光划分门限获取低光区间的端点处的输出值,具体可以通过式3获取:It should be noted that, for the derivative according to the gamma output value and the preset low light division threshold described in S20123, the output value at the end point of the low light interval is obtained, which can be obtained by using Equation 3:
Figure PCTCN2017082524-appb-000011
Figure PCTCN2017082524-appb-000011
其中,B_split为低光区间的端点处B的输出值;g_thr为B点划分门限值,g_thr越大,那么会有更多的取值定为低光;B点就是导数大于g_thr的第一个点。Where B_split is the output value of B at the end of the low-light interval; g_thr is the B-point division threshold, and the larger the g_thr, the more the value is set to low light; the B-point is the first derivative with the derivative greater than g_thr a point.
对于S20124所述的根据高光区间与低光区间之间的对称关系以及低光区间的端点处的输出值,获取高光区间的端点出的输出值,在具体实现过程中,可以通过式4进行计算:For the symmetrical relationship between the highlight interval and the low light interval and the output value at the end of the low light interval described in S20124, the output value of the end point of the highlight interval is obtained, and in the specific implementation process, the calculation can be performed by Equation 4. :
Figure PCTCN2017082524-appb-000012
Figure PCTCN2017082524-appb-000012
由此获取到了区间端点处的输出值,相应地,对于步骤S2013所述的根据预设的区间数目以及上界线和下界线在各区间端点处的输出值确定上界线和下界线,可以包括:The output value at the end of the interval is obtained, and accordingly, the upper boundary and the lower boundary are determined according to the preset number of intervals and the output values of the upper and lower boundaries at the end points of the intervals, as described in step S2013, and may include:
在获取到低光区间的端点处的输出值和高光区间的端点出的输出值之后,根据式5获取上界线up_line在各区间的曲线;After acquiring the output value at the end point of the low light interval and the output value at the end point of the highlight interval, obtaining a curve of the upper boundary line up_line in each interval according to Equation 5;
Figure PCTCN2017082524-appb-000013
Figure PCTCN2017082524-appb-000013
以及,所述下界线down_line设置为斜率为1的直线。And, the lower limit line down_line is set to a straight line having a slope of 1.
具体地,参见图4,上界线up_line和下界线down_line均包含四个区间,如式6所示:Specifically, referring to FIG. 4, the upper boundary line up_line and the lower boundary line down_line each include four intervals, as shown in Equation 6:
Figure PCTCN2017082524-appb-000014
Figure PCTCN2017082524-appb-000014
每一段都可以使用一次曲线,二次曲线,甚至N次曲线来设计,但是一定需要保证导数为正。Each segment can be designed using a curve, a quadratic curve, or even an N-time curve, but it must be ensured that the derivative is positive.
在本实施例中,下界线down_line设计为γ值等于1.0的gamma曲线,也就是斜率为1的直线,所以A-E段都是一个表达式:y=x;In this embodiment, the lower boundary line down_line is designed as a gamma curve with a gamma value equal to 1.0, that is, a straight line with a slope of 1, so the A-E segment is an expression: y=x;
上界线up_line设计为γ值等于1.5,g_thr为0.66的曲线基,并确定定下B、D点坐标。因此,AB段,BC段和CD段均为直线,DE段为曲线。如公式6所示。C点为上界线up_line和下界线down_line的交汇点,本实施例中取512。The upper boundary line up_line is designed as a curve base with a γ value equal to 1.5 and a g_thr of 0.66, and the coordinates of points B and D are determined. Therefore, the AB segment, the BC segment and the CD segment are straight lines, and the DE segment is a curve. As shown in Equation 6. Point C is the intersection of the upper boundary line up_line and the lower boundary line down_line, which is 512 in this embodiment.
在得到上界线up_line和下界线down_line之后,对于步骤S2014所述的按照预设的拟合算法以及所述上界线和所述下界线获取所述自适应曲线,可以包括:After obtaining the upper bound line up_line and the lower bound line down_line, obtaining the adaptive curve according to the preset fitting algorithm and the upper boundary line and the lower boundary line described in step S2014 may include:
根据所述上界线up_line和所述下界线down_line以及式7获取所述自适 应曲线LUT;Obtaining the self-adaptation according to the upper boundary line up_line and the lower boundary line down_line and Equation 7 Should curve LUT;
Figure PCTCN2017082524-appb-000015
Figure PCTCN2017082524-appb-000015
其中,mean_up为1024;并且表达式clip3(mean,0,mean_up)表示:当mean<0时,表达式的值为0,当mean>mean_up时,表达式的值为mean_up;当0<mean<mean_up时,表达式的值为mean。Where mean_up is 1024; and the expression clip3(mean, 0, mean_up) means: when mean<0, the value of the expression is 0, when mean>mean_up, the value of the expression is mean_up; when 0<mean< When mean_up, the value of the expression is mean.
可以理解地,这里的自适应曲线LUT就是图4中所示的虚线,用于表示HDR视频帧的Y分量与处理之后的Y分量之间的对应关系,因此,可以将HDR视频帧的Y分量作为虚线的输入值,那么相应的输出值就是步骤S102所述的根据自适应曲线对HDR视频帧的Y分量进行调整,输出处理后的Y分量。It can be understood that the adaptive curve LUT here is a broken line shown in FIG. 4 for indicating the correspondence between the Y component of the HDR video frame and the Y component after processing, and therefore, the Y component of the HDR video frame can be As the input value of the broken line, the corresponding output value is the Y component of the HDR video frame adjusted according to the adaptive curve described in step S102, and the processed Y component is output.
示例性地,参见图8,对于S203所述的将所述HDR视频帧的C分量按照预设的映射关系进行转换后,再将转换后的C分量按照预设的颜色区间进行颜色饱和度增强,得到处理后的C分量,具体可以包括:For example, referring to FIG. 8, after converting the C component of the HDR video frame according to a preset mapping relationship, the converted C component is enhanced according to a preset color interval. And obtaining the processed C component, which may specifically include:
S2031:按照预设的转换矩阵将C分量由BT2020域转换到BT709域;S2031: Convert the C component from the BT2020 domain to the BT709 domain according to a preset conversion matrix;
S2032:根据预设的颜色区间以及各颜色区间对应的增强系数对转换后的C分量进行饱和度增强,得到处理后的C分量。S2032: Perform saturation enhancement on the converted C component according to a preset color interval and an enhancement coefficient corresponding to each color interval, to obtain a processed C component.
具体地,对于步骤S2031,可以通过式8进行YUV BT2020域到BT709域的转换:Specifically, for step S2031, the conversion of the YUV BT2020 domain to the BT709 domain can be performed by Equation 8:
Figure PCTCN2017082524-appb-000016
Figure PCTCN2017082524-appb-000016
在本实施例中,转换矩阵
Figure PCTCN2017082524-appb-000017
可以为
Figure PCTCN2017082524-appb-000018
可以理解地,本领域技术人员可以根据实际情况及需求选取不同的转换矩阵,本实施例不做赘述。
In this embodiment, the conversion matrix
Figure PCTCN2017082524-appb-000017
Can be
Figure PCTCN2017082524-appb-000018
It can be understood that a person skilled in the art can select different conversion matrices according to the actual situation and needs, and this embodiment does not describe the details.
对于步骤S2032,预设的颜色区间可以为6个颜色区间,可以包括红、黄、绿、蓝绿、蓝、品红;不同的颜色区间所对应的饱和度增强系数gain依次对应分别为:164,164,196,164,196,164。For the step S2032, the preset color interval may be 6 color intervals, which may include red, yellow, green, blue-green, blue, and magenta; the saturation enhancement coefficients corresponding to different color intervals are respectively corresponding to: 164 , 164, 196, 164, 196, 164.
因此,将转换后的C分量按照式9进行饱和度增强,得到处理后的C分量。Therefore, the converted C component is enhanced in accordance with Equation 9, and the processed C component is obtained.
Figure PCTCN2017082524-appb-000019
Figure PCTCN2017082524-appb-000019
其中,cb_in为经过式8转换后的cb;cr_in为经过式8转换后的cr。Where cb_in is cb after the conversion of Equation 8, and cr_in is cr after the conversion of Equation 8.
本实施例提供了一种视频帧的处理方法,将HDR视频帧的Y分量和C分量分别通过自适应曲线和颜色饱和度增强处理,使得处理后的视频帧在SDR显示器播放时,产生HDR的效果。The embodiment provides a method for processing a video frame, which adopts an adaptive curve and a color saturation enhancement process for the Y component and the C component of the HDR video frame, respectively, so that the processed video frame generates HDR when played on the SDR display. effect.
实施例二Embodiment 2
基于前述实施例相同的技术构思,参见图9,其示出了本发明实施例提供的一种视频帧的处理装置90,所述装置90包括:确定单元901、Y分量调整单元902和C分量处理单元903;其中,Based on the same technical concept of the foregoing embodiment, referring to FIG. 9, a processing device 90 for a video frame according to an embodiment of the present invention is shown. The device 90 includes: a determining unit 901, a Y component adjusting unit 902, and a C component. Processing unit 903; wherein
所述确定单元901,配置为根据高动态范围HDR视频帧的Y分量确定所述HDR视频帧对应的自适应曲线;The determining unit 901 is configured to determine an adaptive curve corresponding to the HDR video frame according to a Y component of the high dynamic range HDR video frame;
所述Y分量调整单元902,配置为根据所述自适应曲线对所述HDR视频帧的Y分量进行调整,输出处理后的Y分量;The Y component adjustment unit 902 is configured to adjust a Y component of the HDR video frame according to the adaptive curve, and output the processed Y component;
所述C分量处理单元903,配置为将所述HDR视频帧的C分量按照预设的映射关系进行转换后,再将转换后的C分量按照预设的颜色区间进行颜色饱和度增强,得到处理后的C分量。The C component processing unit 903 is configured to convert the C component of the HDR video frame according to a preset mapping relationship, and then perform the color saturation enhancement according to the preset color interval to obtain the processed C component. The latter C component.
在上述方案中,所述确定单元901,配置为根据所述HDR视频帧的Y分量获取所述HDR视频帧的元数据;其中,所述元数据用于指示所述HDR视频帧的特征信息;In the foregoing solution, the determining unit 901 is configured to acquire metadata of the HDR video frame according to a Y component of the HDR video frame, where the metadata is used to indicate feature information of the HDR video frame;
以及,根据所述HDR视频帧的元数据以及预设的区间数目获取上界线和下界线在各区间端点处的输出值;And obtaining an output value of the upper boundary line and the lower boundary line at the end points of each interval according to the metadata of the HDR video frame and the preset number of intervals;
以及,根据预设的区间数目以及上界线和下界线在各区间端点处的输 出值确定上界线和下界线;And, according to the preset number of intervals and the upper and lower bounds at the end of each interval The out value determines the upper and lower boundaries;
以及,按照预设的拟合算法以及所述上界线和所述下界线获取所述自适应曲线。And acquiring the adaptive curve according to a preset fitting algorithm and the upper boundary line and the lower boundary line.
在上述方案中,所述HDR视频帧的元数据为所述HDR帧的亮度均值。In the above solution, the metadata of the HDR video frame is the luminance average of the HDR frame.
在上述方案中,所述确定单元901,配置为:In the above solution, the determining unit 901 is configured to:
根据所述HDR帧的亮度均值以及式10进行gamma转换,获得gamma输出值;Obtaining a gamma output value according to a luminance average of the HDR frame and a gamma conversion of Equation 10;
Figure PCTCN2017082524-appb-000020
Figure PCTCN2017082524-appb-000020
其中,input为输入的HDR帧的亮度均值,output为gamma输出值,γ为gamma扭曲系数;Wherein, input is the brightness average of the input HDR frame, output is the gamma output value, and γ is the gamma distortion coefficient;
以及,根据式11获取gamma输出值的导数gradient[n];And obtaining a derivative of the gamma output value according to Equation 11 gradient[n];
Figure PCTCN2017082524-appb-000021
Figure PCTCN2017082524-appb-000021
其中,n为区间内的点序号;Where n is the point number within the interval;
以及,根据gamma输出值的导数以及预设的低光划分门限获取低光区间的端点处的输出值;And obtaining an output value at an end point of the low light interval according to a derivative of the gamma output value and a preset low light division threshold;
以及,根据高光区间与低光区间之间的对称关系以及低光区间的端点处的输出值,获取高光区间的端点出的输出值。And, according to the symmetrical relationship between the highlight interval and the low light interval and the output value at the end of the low light interval, the output value of the end point of the highlight interval is obtained.
在上述方案中,所述确定单元901,配置为:In the above solution, the determining unit 901 is configured to:
在获取到低光区间的端点处的输出值和高光区间的端点出的输出值之后,根据式12获取上界线up_line在各区间的曲线;After acquiring the output value at the end point of the low light interval and the output value at the end point of the highlight interval, obtaining a curve of the upper boundary line up_line in each interval according to Equation 12;
Figure PCTCN2017082524-appb-000022
Figure PCTCN2017082524-appb-000022
其中,Δy1表示的是低光区间的output的差值;Δy2表示中光区间的output的差值;中光区间的中点就是上界线和下界线的交汇点;Where Δy 1 represents the difference of the output of the low light interval; Δy 2 represents the difference of the output of the medium light interval; the midpoint of the medium light interval is the intersection of the upper boundary and the lower boundary;
以及,所述下界线down_line设置为斜率为1的直线。And, the lower limit line down_line is set to a straight line having a slope of 1.
在上述方案中,所述确定单元901,配置为:In the above solution, the determining unit 901 is configured to:
根据所述上界线up_line和所述下界线down_line以及式13获取所述自适应曲线LUT;Obtaining the adaptive curve LUT according to the upper boundary line up_line and the lower boundary line down_line and Equation 13;
Figure PCTCN2017082524-appb-000023
Figure PCTCN2017082524-appb-000023
其中,mean_up为1024;并且表达式clip3(mean,0,mean_up)表示:当mean<0时,表达式的值为0,当mean>mean_up时,表达式的值为mean_up;当0<mean<mean_up时,表达式的值为mean。Where mean_up is 1024; and the expression clip3(mean, 0, mean_up) means: when mean<0, the value of the expression is 0, when mean>mean_up, the value of the expression is mean_up; when 0<mean< When mean_up, the value of the expression is mean.
在上述方案中,所述C分量处理单元903,配置为:In the above solution, the C component processing unit 903 is configured to:
按照预设的转换矩阵将C分量由BT2020域转换到BT709域;Converting the C component from the BT2020 domain to the BT709 domain according to a preset conversion matrix;
以及,根据预设的颜色区间以及各颜色区间对应的增强系数对转换后的C分量进行饱和度增强,得到处理后的C分量。And performing saturation enhancement on the converted C component according to the preset color interval and the enhancement coefficient corresponding to each color interval, to obtain the processed C component.
实际应用时,所述确定单元901、Y分量调整单元902和C分量处理单元903可由视频帧的处理装置90中的处理器(比如中央处理器(CPU,Central Processing Unit)、微处理器(MCU,Micro Control Unit)、数字信号处理器(DSP,Digital Signal Processor)或可编程逻辑阵列(FPGA,Field-Programmable Gate Array)等)实现。In practical applications, the determining unit 901, the Y component adjusting unit 902, and the C component processing unit 903 may be processed by a processor (such as a central processing unit (CPU), a microprocessor (MCU) in the processing device 90 of the video frame. , Micro Control Unit), digital signal processor (DSP, Digital Signal Processor) or programmable logic array (FPGA, Field-Programmable Gate Array), etc.
基于此,本发明实施例提供的视频帧的处理装置,包括:处理器和配置为存储能够在处理器上运行的计算机程序的存储器,其中,所述处理器配置为运行所述计算机程序时,执行:Based on this, the processing device for a video frame provided by the embodiment of the present invention includes: a processor and a memory configured to store a computer program capable of running on a processor, wherein the processor is configured to run the computer program, carried out:
根据HDR视频帧的Y分量确定所述HDR视频帧对应的自适应曲线;Determining an adaptive curve corresponding to the HDR video frame according to a Y component of the HDR video frame;
根据所述自适应曲线对所述HDR视频帧的Y分量进行调整,输出处理后的Y分量;Adjusting a Y component of the HDR video frame according to the adaptive curve, and outputting the processed Y component;
将所述HDR视频帧的C分量按照预设的映射关系进行转换后,再将转 换后的C分量按照预设的颜色区间进行颜色饱和度增强,得到处理后的C分量。Converting the C component of the HDR video frame according to a preset mapping relationship, and then turning The changed C component is subjected to color saturation enhancement according to a preset color interval to obtain a processed C component.
在一具体实施例中,所述处理器配置为运行所述计算机程序时,执行:In a specific embodiment, the processor is configured to execute the computer program, executing:
根据所述HDR视频帧的Y分量获取所述HDR视频帧的元数据;其中,所述元数据用于指示所述HDR视频帧的特征信息;Obtaining metadata of the HDR video frame according to a Y component of the HDR video frame; wherein the metadata is used to indicate feature information of the HDR video frame;
根据所述HDR视频帧的元数据以及预设的区间数目获取上界线和下界线在各区间端点处的输出值;Obtaining an output value of the upper boundary line and the lower boundary line at the end points of each interval according to the metadata of the HDR video frame and the preset number of intervals;
根据预设的区间数目以及上界线和下界线在各区间端点处的输出值确定上界线和下界线;The upper boundary line and the lower boundary line are determined according to the preset number of intervals and the output values of the upper boundary line and the lower boundary line at the end points of each interval;
按照预设的拟合算法以及所述上界线和所述下界线获取所述自适应曲线。The adaptive curve is obtained according to a preset fitting algorithm and the upper boundary line and the lower boundary line.
其中,所述HDR视频帧的元数据为所述HDR帧的亮度均值。The metadata of the HDR video frame is a luminance average of the HDR frame.
在一具体实施例中,所述处理器配置为运行所述计算机程序时,执行:In a specific embodiment, the processor is configured to execute the computer program, executing:
根据所述HDR帧的亮度均值以及式1进行伽马gamma转换,获得gamma输出值;Obtaining a gamma output value according to a luminance mean value of the HDR frame and a gamma gamma conversion of Equation 1;
Figure PCTCN2017082524-appb-000024
Figure PCTCN2017082524-appb-000024
其中,input为输入的HDR帧的亮度均值,output为gamma输出值,γ为gamma扭曲系数;Wherein, input is the brightness average of the input HDR frame, output is the gamma output value, and γ is the gamma distortion coefficient;
根据式2获取gamma输出值的导数gradient[n];Obtaining the derivative gradient[n] of the gamma output value according to Equation 2;
Figure PCTCN2017082524-appb-000025
Figure PCTCN2017082524-appb-000025
其中,n为区间内的点序号;Where n is the point number within the interval;
根据gamma输出值的导数以及预设的低光划分门限获取低光区间的端点处的输出值;Obtaining an output value at an end point of the low light interval according to a derivative of the gamma output value and a preset low light division threshold;
根据高光区间与低光区间之间的对称关系以及低光区间的端点处的输出值,获取高光区间的端点出的输出值。The output value of the end point of the highlight interval is obtained according to the symmetrical relationship between the highlight interval and the low light interval and the output value at the end of the low light interval.
在一具体实施例中,所述处理器配置为运行所述计算机程序时,执行: In a specific embodiment, the processor is configured to execute the computer program, executing:
在获取到低光区间的端点处的输出值和高光区间的端点出的输出值之后,根据式3获取上界线up_line在各区间的曲线;After acquiring the output value at the end point of the low light interval and the output value at the end point of the highlight interval, the curve of the upper boundary line up_line in each interval is obtained according to Equation 3;
Figure PCTCN2017082524-appb-000026
Figure PCTCN2017082524-appb-000026
其中,Δy1表示的是低光区间的output的差值;Δy2表示中光区间的output的差值;中光区间的中点就是上界线和下界线的交汇点;Where Δy 1 represents the difference of the output of the low light interval; Δy 2 represents the difference of the output of the medium light interval; the midpoint of the medium light interval is the intersection of the upper boundary and the lower boundary;
所述下界线down_line设置为斜率为1的直线。The lower limit line down_line is set to a straight line with a slope of 1.
在一具体实施例中,所述处理器配置为运行所述计算机程序时,执行:In a specific embodiment, the processor is configured to execute the computer program, executing:
根据所述上界线up_line和所述下界线down_line以及式4获取所述自适应曲线LUT;Obtaining the adaptive curve LUT according to the upper boundary line up_line and the lower boundary line down_line and Equation 4;
Figure PCTCN2017082524-appb-000027
Figure PCTCN2017082524-appb-000027
其中,mean_up为1024;并且表达式clip3(mean,0,mean_up)表示:当mean<0时,表达式的值为0,当mean>mean_up时,表达式的值为mean_up;当0<mean<mean_up时,表达式的值为mean。Where mean_up is 1024; and the expression clip3(mean, 0, mean_up) means: when mean<0, the value of the expression is 0, when mean>mean_up, the value of the expression is mean_up; when 0<mean< When mean_up, the value of the expression is mean.
在一具体实施例中,所述处理器配置为运行所述计算机程序时,执行:In a specific embodiment, the processor is configured to execute the computer program, executing:
按照预设的转换矩阵将C分量由BT2020域转换到BT709域;Converting the C component from the BT2020 domain to the BT709 domain according to a preset conversion matrix;
根据预设的颜色区间以及各颜色区间对应的增强系数对转换后的C分量进行饱和度增强,得到处理后的C分量。The converted C component is subjected to saturation enhancement according to the preset color interval and the enhancement coefficient corresponding to each color interval, and the processed C component is obtained.
本实施例提供了一种视频帧的处理装置90,将HDR视频帧的Y分量和C分量分别通过自适应曲线和颜色饱和度增强处理,使得处理后的视频帧在SDR显示器播放时,产生HDR的效果。The present embodiment provides a video frame processing apparatus 90 that passes the Y component and the C component of the HDR video frame through adaptive curve and color saturation enhancement processing, respectively, so that the processed video frame generates HDR when played on the SDR display. Effect.
本领域内的技术人员应明白,本发明的实施例可提供为方法、系统、或计算机程序产品。因此,本发明可采用硬件实施例、软件实施例、或结 合软件和硬件方面的实施例的形式。而且,本发明可采用在一个或多个其中包含有计算机可用程序代码的计算机可用存储介质(包括但不限于磁盘存储器和光学存储器等)上实施的计算机程序产品的形式。Those skilled in the art will appreciate that embodiments of the present invention can be provided as a method, system, or computer program product. Accordingly, the present invention may employ hardware embodiments, software embodiments, or junctions. In the form of an embodiment of the software and hardware aspects. Moreover, the invention can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage and optical storage, etc.) including computer usable program code.
本发明是参照根据本发明实施例的方法、设备(系统)、和计算机程序产品的流程图和/或方框图来描述的。应理解可由计算机程序指令实现流程图和/或方框图中的每一流程和/或方框、以及流程图和/或方框图中的流程和/或方框的结合。可提供这些计算机程序指令到通用计算机、专用计算机、嵌入式处理机或其他可编程数据处理设备的处理器以产生一个机器,使得通过计算机或其他可编程数据处理设备的处理器执行的指令产生用于实现在流程图一个流程或多个流程和/或方框图一个方框或多个方框中指定的功能的装置。The present invention has been described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (system), and computer program products according to embodiments of the invention. It will be understood that each flow and/or block of the flowchart illustrations and/or FIG. These computer program instructions can be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing device to produce a machine for the execution of instructions for execution by a processor of a computer or other programmable data processing device. Means for implementing the functions specified in one or more of the flow or in a block or blocks of the flow chart.
这些计算机程序指令也可存储在能引导计算机或其他可编程数据处理设备以特定方式工作的计算机可读存储器中,使得存储在该计算机可读存储器中的指令产生包括指令装置的制造品,该指令装置实现在流程图一个流程或多个流程和/或方框图一个方框或多个方框中指定的功能。The computer program instructions can also be stored in a computer readable memory that can direct a computer or other programmable data processing device to operate in a particular manner, such that the instructions stored in the computer readable memory produce an article of manufacture comprising the instruction device. The apparatus implements the functions specified in one or more blocks of a flow or a flow and/or block diagram of the flowchart.
这些计算机程序指令也可装载到计算机或其他可编程数据处理设备上,使得在计算机或其他可编程设备上执行一系列操作步骤以产生计算机实现的处理,从而在计算机或其他可编程设备上执行的指令提供用于实现在流程图一个流程或多个流程和/或方框图一个方框或多个方框中指定的功能的步骤。These computer program instructions can also be loaded onto a computer or other programmable data processing device such that a series of operational steps are performed on a computer or other programmable device to produce computer-implemented processing for execution on a computer or other programmable device. The instructions provide steps for implementing the functions specified in one or more of the flow or in a block or blocks of a flow diagram.
基于此,本发明实施例还提供了一种计算机可读存储介质,其上存储有一组指令,当执行所述指令时,引起至少一个处理器执行本发明实施例的视频帧的处理方法。Based on this, an embodiment of the present invention further provides a computer readable storage medium, where a set of instructions are stored, and when executed, at least one processor is caused to perform a processing method of a video frame according to an embodiment of the present invention.
以上所述,仅为本发明的较佳实施例而已,并非用于限定本发明的保护范围。The above is only the preferred embodiment of the present invention and is not intended to limit the scope of the present invention.
工业实用性Industrial applicability
本发明实施例提供的方案,将HDR视频帧的Y分量和C分量分别通过自适应曲线和颜色饱和度增强处理,使得处理后的视频帧在SDR显示器 播放时,产生HDR的效果。 According to the solution provided by the embodiment of the present invention, the Y component and the C component of the HDR video frame are respectively processed by an adaptive curve and a color saturation enhancement process, so that the processed video frame is in the SDR display. When playing, it produces an HDR effect.

Claims (16)

  1. 一种视频帧的处理方法,所述方法包括:A method for processing a video frame, the method comprising:
    根据高动态范围HDR视频帧的Y分量确定所述HDR视频帧对应的自适应曲线;Determining an adaptive curve corresponding to the HDR video frame according to a Y component of the high dynamic range HDR video frame;
    根据所述自适应曲线对所述HDR视频帧的Y分量进行调整,输出处理后的Y分量;Adjusting a Y component of the HDR video frame according to the adaptive curve, and outputting the processed Y component;
    将所述HDR视频帧的C分量按照预设的映射关系进行转换后,再将转换后的C分量按照预设的颜色区间进行颜色饱和度增强,得到处理后的C分量。After the C component of the HDR video frame is converted according to a preset mapping relationship, the converted C component is further enhanced in color saturation according to a preset color interval to obtain a processed C component.
  2. 根据权利要求1所述的方法,其中,所述根据HDR视频帧的Y分量确定所述HDR视频帧对应的自适应曲线,包括:The method of claim 1, wherein the determining an adaptation curve corresponding to the HDR video frame according to a Y component of the HDR video frame comprises:
    根据所述HDR视频帧的Y分量获取所述HDR视频帧的元数据;其中,所述元数据用于指示所述HDR视频帧的特征信息;Obtaining metadata of the HDR video frame according to a Y component of the HDR video frame; wherein the metadata is used to indicate feature information of the HDR video frame;
    根据所述HDR视频帧的元数据以及预设的区间数目获取上界线和下界线在各区间端点处的输出值;Obtaining an output value of the upper boundary line and the lower boundary line at the end points of each interval according to the metadata of the HDR video frame and the preset number of intervals;
    根据预设的区间数目以及上界线和下界线在各区间端点处的输出值确定上界线和下界线;The upper boundary line and the lower boundary line are determined according to the preset number of intervals and the output values of the upper boundary line and the lower boundary line at the end points of each interval;
    按照预设的拟合算法以及所述上界线和所述下界线获取所述自适应曲线。The adaptive curve is obtained according to a preset fitting algorithm and the upper boundary line and the lower boundary line.
  3. 根据权利要求2所述的方法,其中,所述HDR视频帧的元数据为所述HDR帧的亮度均值。The method of claim 2 wherein the metadata of the HDR video frame is a luminance mean of the HDR frame.
  4. 根据权利要求3所述的方法,其中,所述根据所述HDR视频帧的元数据以及预设的区间数目获取上界线和下界线在各区间端点处的输出值,包括:The method according to claim 3, wherein the obtaining the output values of the upper and lower boundaries at the endpoints of the intervals according to the metadata of the HDR video frame and the preset number of intervals comprises:
    根据所述HDR帧的亮度均值以及式1进行伽马gamma转换,获得gamma输出值; Obtaining a gamma output value according to a luminance mean value of the HDR frame and a gamma gamma conversion of Equation 1;
    Figure PCTCN2017082524-appb-100001
    Figure PCTCN2017082524-appb-100001
    其中,input为输入的HDR帧的亮度均值,output为gamma输出值,γ为gamma扭曲系数;Wherein, input is the brightness average of the input HDR frame, output is the gamma output value, and γ is the gamma distortion coefficient;
    根据式2获取gamma输出值的导数gradient[n];Obtaining the derivative gradient[n] of the gamma output value according to Equation 2;
    Figure PCTCN2017082524-appb-100002
    Figure PCTCN2017082524-appb-100002
    其中,n为区间内的点序号;Where n is the point number within the interval;
    根据gamma输出值的导数以及预设的低光划分门限获取低光区间的端点处的输出值;Obtaining an output value at an end point of the low light interval according to a derivative of the gamma output value and a preset low light division threshold;
    根据高光区间与低光区间之间的对称关系以及低光区间的端点处的输出值,获取高光区间的端点出的输出值。The output value of the end point of the highlight interval is obtained according to the symmetrical relationship between the highlight interval and the low light interval and the output value at the end of the low light interval.
  5. 根据权利要求4所述的方法,其中,所述根据预设的区间数目以及上界线和下界线在各区间端点处的输出值确定上界线和下界线,包括:The method according to claim 4, wherein said determining an upper boundary and a lower boundary according to a preset number of intervals and an output value of an upper boundary and a lower boundary at each end of each interval, comprising:
    在获取到低光区间的端点处的输出值和高光区间的端点出的输出值之后,根据式3获取上界线up_line在各区间的曲线;After acquiring the output value at the end point of the low light interval and the output value at the end point of the highlight interval, the curve of the upper boundary line up_line in each interval is obtained according to Equation 3;
    Figure PCTCN2017082524-appb-100003
    Figure PCTCN2017082524-appb-100003
    其中,Δy1表示的是低光区间的output的差值;Δy2表示中光区间的output的差值;中光区间的中点就是上界线和下界线的交汇点;Where Δy 1 represents the difference of the output of the low light interval; Δy 2 represents the difference of the output of the medium light interval; the midpoint of the medium light interval is the intersection of the upper boundary and the lower boundary;
    所述下界线down_line设置为斜率为1的直线。The lower limit line down_line is set to a straight line with a slope of 1.
  6. 根据权利要求2所述的方法,其中,按照预设的拟合算法以及所述上界线和所述下界线获取所述自适应曲线,包括:The method of claim 2, wherein the adaptive curve is obtained according to a preset fitting algorithm and the upper boundary line and the lower boundary line, comprising:
    根据所述上界线up_line和所述下界线down_line以及式4获取所述自适应曲线LUT; Obtaining the adaptive curve LUT according to the upper boundary line up_line and the lower boundary line down_line and Equation 4;
    Figure PCTCN2017082524-appb-100004
    Figure PCTCN2017082524-appb-100004
    其中,mean_up为1024;并且表达式clip3(mean,0,mean_up)表示:当mean<0时,表达式的值为0,当mean>mean_up时,表达式的值为mean_up;当0<mean<mean_up时,表达式的值为mean。Where mean_up is 1024; and the expression clip3(mean, 0, mean_up) means: when mean<0, the value of the expression is 0, when mean>mean_up, the value of the expression is mean_up; when 0<mean< When mean_up, the value of the expression is mean.
  7. 根据权利要求1所述的方法,其中,所述将所述HDR视频帧的C分量按照预设的映射关系进行转换后,再将转换后的C分量按照预设的颜色区间进行颜色饱和度增强,得到处理后的C分量,包括:The method according to claim 1, wherein the C component of the HDR video frame is converted according to a preset mapping relationship, and then the converted C component is subjected to color saturation enhancement according to a preset color interval. , get the processed C component, including:
    按照预设的转换矩阵将C分量由BT2020域转换到BT709域;Converting the C component from the BT2020 domain to the BT709 domain according to a preset conversion matrix;
    根据预设的颜色区间以及各颜色区间对应的增强系数对转换后的C分量进行饱和度增强,得到处理后的C分量。The converted C component is subjected to saturation enhancement according to the preset color interval and the enhancement coefficient corresponding to each color interval, and the processed C component is obtained.
  8. 一种视频帧的处理装置,所述装置包括:确定单元、Y分量调整单元和C分量处理单元;其中,A processing device for a video frame, the device comprising: a determining unit, a Y component adjusting unit, and a C component processing unit; wherein
    所述确定单元,配置为根据高动态范围HDR视频帧的Y分量确定所述HDR视频帧对应的自适应曲线;The determining unit is configured to determine an adaptive curve corresponding to the HDR video frame according to a Y component of the high dynamic range HDR video frame;
    所述Y分量调整单元,配置为根据所述自适应曲线对所述HDR视频帧的Y分量进行调整,输出处理后的Y分量;The Y component adjustment unit is configured to adjust a Y component of the HDR video frame according to the adaptive curve, and output the processed Y component;
    所述C分量处理单元,配置为将所述HDR视频帧的C分量按照预设的映射关系进行转换后,再将转换后的C分量按照预设的颜色区间进行颜色饱和度增强,得到处理后的C分量。The C component processing unit is configured to convert the C component of the HDR video frame according to a preset mapping relationship, and then perform the color saturation enhancement according to the preset color interval by the converted C component, and obtain the processed C component.
  9. 根据权利要求8所述的装置,其中,所述确定单元,配置为根据所述HDR视频帧的Y分量获取所述HDR视频帧的元数据;其中,所述元数据用于指示所述HDR视频帧的特征信息;The apparatus according to claim 8, wherein the determining unit is configured to acquire metadata of the HDR video frame according to a Y component of the HDR video frame; wherein the metadata is used to indicate the HDR video Characteristic information of the frame;
    以及,根据所述HDR视频帧的元数据以及预设的区间数目获取上界线和下界线在各区间端点处的输出值;And obtaining an output value of the upper boundary line and the lower boundary line at the end points of each interval according to the metadata of the HDR video frame and the preset number of intervals;
    以及,根据预设的区间数目以及上界线和下界线在各区间端点处的输出值确定上界线和下界线;And determining an upper boundary line and a lower boundary line according to the preset number of intervals and the output values of the upper boundary line and the lower boundary line at the end points of each interval;
    以及,按照预设的拟合算法以及所述上界线和所述下界线获取所述自 适应曲线。And acquiring the self according to a preset fitting algorithm and the upper boundary line and the lower boundary line Adapt to the curve.
  10. 根据权利要求9所述的装置,其中,所述HDR视频帧的元数据为所述HDR帧的亮度均值。The apparatus of claim 9, wherein the metadata of the HDR video frame is a luminance mean of the HDR frame.
  11. 根据权利要求10所述的装置,其中,所述确定单元,配置为:The apparatus according to claim 10, wherein the determining unit is configured to:
    根据所述HDR帧的亮度均值以及式5进行伽马gamma转换,获得gamma输出值;Obtaining a gamma output value according to a luminance average of the HDR frame and a gamma gamma conversion of Equation 5;
    Figure PCTCN2017082524-appb-100005
    Figure PCTCN2017082524-appb-100005
    其中,input为输入的HDR帧的亮度均值,output为gamma输出值,γ为gamma扭曲系数;Wherein, input is the brightness average of the input HDR frame, output is the gamma output value, and γ is the gamma distortion coefficient;
    以及,根据式6获取gamma输出值的导数gradient[n];And obtaining a derivative of the gamma output value according to Equation 6 gradient[n];
    Figure PCTCN2017082524-appb-100006
    Figure PCTCN2017082524-appb-100006
    其中,n为区间内的点序号;Where n is the point number within the interval;
    以及,根据gamma输出值的导数以及预设的低光划分门限获取低光区间的端点处的输出值;And obtaining an output value at an end point of the low light interval according to a derivative of the gamma output value and a preset low light division threshold;
    以及,根据高光区间与低光区间之间的对称关系以及低光区间的端点处的输出值,获取高光区间的端点出的输出值。And, according to the symmetrical relationship between the highlight interval and the low light interval and the output value at the end of the low light interval, the output value of the end point of the highlight interval is obtained.
  12. 根据权利要求11所述的装置,其中,所述确定单元,配置为:The apparatus according to claim 11, wherein the determining unit is configured to:
    在获取到低光区间的端点处的输出值和高光区间的端点出的输出值之后,根据式7获取上界线up_line在各区间的曲线;After acquiring the output value at the end point of the low light interval and the output value at the end point of the highlight interval, obtaining a curve of the upper boundary line up_line in each interval according to Equation 7;
    Figure PCTCN2017082524-appb-100007
    Figure PCTCN2017082524-appb-100007
    其中,Δy1表示的是低光区间的output的差值;Δy2表示中光区间的output 的差值;中光区间的中点就是上界线和下界线的交汇点;Where Δy 1 represents the difference of the output of the low light interval; Δy 2 represents the difference of the output of the medium light interval; the midpoint of the medium light interval is the intersection of the upper boundary and the lower boundary;
    以及,所述下界线down_line设置为斜率为1的直线。And, the lower limit line down_line is set to a straight line having a slope of 1.
  13. 根据权利要求9所述的装置,其中,所述确定单元,配置为:The apparatus according to claim 9, wherein the determining unit is configured to:
    根据所述上界线up_line和所述下界线down_line以及式8获取所述自适应曲线LUT;Obtaining the adaptive curve LUT according to the upper boundary line up_line and the lower boundary line down_line and Equation 8;
    Figure PCTCN2017082524-appb-100008
    Figure PCTCN2017082524-appb-100008
    其中,mean_up为1024;并且表达式clip3(mean,0,mean_up)表示:当mean<0时,表达式的值为0,当mean>mean_up时,表达式的值为mean_up;当0<mean<mean_up时,表达式的值为mean。Where mean_up is 1024; and the expression clip3(mean, 0, mean_up) means: when mean<0, the value of the expression is 0, when mean>mean_up, the value of the expression is mean_up; when 0<mean< When mean_up, the value of the expression is mean.
  14. 根据权利要求8所述的装置,其中,所述C分量处理单元,配置为:The apparatus of claim 8, wherein the C component processing unit is configured to:
    按照预设的转换矩阵将C分量由BT2020域转换到BT709域;Converting the C component from the BT2020 domain to the BT709 domain according to a preset conversion matrix;
    以及,根据预设的颜色区间以及各颜色区间对应的增强系数对转换后的C分量进行饱和度增强,得到处理后的C分量。And performing saturation enhancement on the converted C component according to the preset color interval and the enhancement coefficient corresponding to each color interval, to obtain the processed C component.
  15. 一种视频帧的处理装置,包括:处理器和配置为存储能够在处理器上运行的计算机程序的存储器,A processing device for a video frame, comprising: a processor and a memory configured to store a computer program executable on the processor,
    其中,所述处理器配置为运行所述计算机程序时,执行权利要求1至7任一项所述方法的步骤。Wherein the processor is configured to perform the steps of the method of any one of claims 1 to 7 when the computer program is run.
  16. 一种计算机可读存储介质,其上存储有一组指令,当执行所述指令时,引起至少一个处理器执行如权利要求1至7任一项所述的视频帧的处理方法。 A computer readable storage medium having stored thereon a set of instructions which, when executed, cause at least one processor to perform a method of processing a video frame as claimed in any one of claims 1 to 7.
PCT/CN2017/082524 2016-08-31 2017-04-28 Method and device for processing video frame, and computer-readable storage medium WO2018040599A1 (en)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201610798824.7 2016-08-31
CN201610798824.7A CN107786865B (en) 2016-08-31 2016-08-31 A kind for the treatment of method and apparatus of video frame

Publications (1)

Publication Number Publication Date
WO2018040599A1 true WO2018040599A1 (en) 2018-03-08

Family

ID=61299953

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2017/082524 WO2018040599A1 (en) 2016-08-31 2017-04-28 Method and device for processing video frame, and computer-readable storage medium

Country Status (2)

Country Link
CN (1) CN107786865B (en)
WO (1) WO2018040599A1 (en)

Families Citing this family (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108882028B (en) 2018-07-05 2019-06-14 华为技术有限公司 The processing method and processing device of vision signal
CN109327734B (en) * 2018-11-27 2020-11-13 成都索贝数码科技股份有限公司 Method for down-converting HDR video to SDR video based on dynamic photometry
EP3742432A1 (en) * 2019-05-24 2020-11-25 InterDigital CE Patent Holdings Device and method for transition between luminance levels

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101082992A (en) * 2007-07-06 2007-12-05 浙江大学 Drawing of real time high dynamic range image and display process
CN101901475A (en) * 2010-06-12 2010-12-01 北京理工大学 High dynamic range image tone mapping method based on retina adaptive model
CN103295194A (en) * 2013-05-15 2013-09-11 中山大学 Brightness-controllable and detail-preservation tone mapping method
US20130241931A1 (en) * 2012-03-14 2013-09-19 Dolby Laboratiories Licensing Corporation Efficient Tone-Mapping Of High-Bit-Depth Video To Low-Bit-Depth Display
CN103747225A (en) * 2014-01-23 2014-04-23 福州大学 High-dynamic-range image dual-screen display method based on color space switching
CN104408752A (en) * 2014-12-11 2015-03-11 西安电子科技大学 High dynamic range image compression method based on mixed tone mapping algorithm

Family Cites Families (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101707666A (en) * 2009-11-26 2010-05-12 北京中星微电子有限公司 Adjusting method and device with high dynamic range
CN101916555B (en) * 2010-07-26 2012-08-08 武汉大学 Method for displaying and reproducing high dynamic range images
CN103024300B (en) * 2012-12-25 2015-11-25 华为技术有限公司 A kind of method for high dynamic range image display and device
CN104618700B (en) * 2015-01-29 2017-01-18 电子科技大学 Enhanced display method for color high dynamic range image
CN105744378A (en) * 2016-02-15 2016-07-06 深圳Tcl数字技术有限公司 Video playing method and device

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101082992A (en) * 2007-07-06 2007-12-05 浙江大学 Drawing of real time high dynamic range image and display process
CN101901475A (en) * 2010-06-12 2010-12-01 北京理工大学 High dynamic range image tone mapping method based on retina adaptive model
US20130241931A1 (en) * 2012-03-14 2013-09-19 Dolby Laboratiories Licensing Corporation Efficient Tone-Mapping Of High-Bit-Depth Video To Low-Bit-Depth Display
CN103295194A (en) * 2013-05-15 2013-09-11 中山大学 Brightness-controllable and detail-preservation tone mapping method
CN103747225A (en) * 2014-01-23 2014-04-23 福州大学 High-dynamic-range image dual-screen display method based on color space switching
CN104408752A (en) * 2014-12-11 2015-03-11 西安电子科技大学 High dynamic range image compression method based on mixed tone mapping algorithm

Also Published As

Publication number Publication date
CN107786865B (en) 2019-11-26
CN107786865A (en) 2018-03-09

Similar Documents

Publication Publication Date Title
CN107154059B (en) High dynamic range video processing method
KR102157032B1 (en) Display management for high dynamic range video
JP6663214B2 (en) Display method and display device
CN109325922B (en) Image self-adaptive enhancement method and device and image processing equipment
JP2003230160A (en) Color picture saturation adjustment apparatus and method therefor
CN107820069B (en) Video monitoring equipment ISP debugging method
US10332481B2 (en) Adaptive display management using 3D look-up table interpolation
WO2016058226A1 (en) Image colour enhancement method and system
US10223778B2 (en) Image contrast enhancement method and apparatus thereof
WO2018040599A1 (en) Method and device for processing video frame, and computer-readable storage medium
CN110706172B (en) Low-illumination color image enhancement method based on adaptive chaotic particle swarm optimization
CN108022223B (en) Tone mapping method based on logarithm mapping function blocking processing fusion
CN111064942B (en) Image processing method and apparatus
TW201444336A (en) Method and device of skin tone optimization in color gamut mapping system
JP5878586B2 (en) Image color adjustment method and electronic apparatus therefor
CN112488968B (en) Image enhancement method for hierarchical histogram equalization fusion
CN107358592B (en) Iterative global adaptive image enhancement method
JP2002536920A (en) Histogram equalization method
TWI697873B (en) Image saturation adjusting method and device
US8284316B2 (en) Real-time image processing circuit capable of enhancing brightness contrast and color saturation
KR101634652B1 (en) Method and apparatus for intensificating contrast in image
JP2009010636A (en) Adaption histogram equalization method, and adaption histogram equalization apparatus
TWI587245B (en) Image enhancement method
JP2023525284A (en) Encoders, Decoders, Systems, and Methods for Determining Tone Mapping Curve Parameters
CN111031301A (en) Method for adjusting color gamut space, storage device and display terminal

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 17844907

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 17844907

Country of ref document: EP

Kind code of ref document: A1