WO2016155125A1 - 一种图像处理方法及装置 - Google Patents
一种图像处理方法及装置 Download PDFInfo
- Publication number
- WO2016155125A1 WO2016155125A1 PCT/CN2015/081798 CN2015081798W WO2016155125A1 WO 2016155125 A1 WO2016155125 A1 WO 2016155125A1 CN 2015081798 W CN2015081798 W CN 2015081798W WO 2016155125 A1 WO2016155125 A1 WO 2016155125A1
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- bit rate
- frame image
- encoder
- mean difference
- threshold
- Prior art date
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N7/00—Television systems
- H04N7/18—Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast
Definitions
- the present application provides an image processing method and apparatus for solving the video mosaic effect caused by camera shaking and affecting the user experience.
- An image processing method that performs the following process for an image sent to an encoder:
- the encoding bit rate calculated by the encoder is adjusted according to the first preset adjustment mode.
- the method further comprises:
- the coding bit rate calculated by the encoder is lowered.
- the encoding the bit rate calculated by the encoder is up-regulated according to the first preset adjustment manner, including:
- the encoding bit rate calculated by the encoder is adjusted upward according to the preset adjustment value corresponding to the numerical interval.
- the encoding bit rate calculated by the encoder is adjusted according to the second preset adjustment manner, including:
- the encoding bit rate calculated by the encoder is adjusted according to the preset adjustment value corresponding to the numerical interval.
- An image processing apparatus for processing an image transmitted to an encoder, the apparatus comprising:
- a calculating unit configured to calculate a brightness mean difference between the current frame image and the previous frame image, and a ratio of pixels that generate a brightness change in the current frame image to all pixels in the current frame image compared to the previous frame image;
- a first determining unit configured to determine whether the brightness mean difference is greater than a first shaking threshold
- a second determining unit configured to determine whether the ratio is greater than a second shaking threshold
- the bit rate up-scaling unit is configured to: when determining that the brightness mean difference is greater than the first shaking threshold, and the ratio is greater than the second shaking threshold, adjust the encoding bit rate calculated by the encoder according to the first preset adjustment manner.
- the method further comprises:
- a third determining unit configured to determine whether the brightness mean difference is less than a first static threshold
- a fourth determining unit configured to determine whether the ratio is less than a second static threshold
- bit rate down-scaling unit configured to calculate an encoder according to a second preset adjustment manner when determining that the brightness mean difference is less than a first static threshold, and the ratio is less than a second static threshold The encoding bit rate is lowered.
- bit rate up-scaling unit comprises:
- a first bit rate up-modulation sub-unit configured to up-regulate an encoding bit rate calculated by the encoder by a first preset value
- a second bit rate up-modulation sub-unit configured to determine a preset value interval to which the brightness mean difference belongs; and perform an upward adjustment on an encoding bit rate calculated by the encoder according to the preset adjustment value corresponding to the value interval.
- bit rate down-scaling unit comprises:
- a first bit rate down-modulation sub-unit configured to: downgrade an encoding bit rate calculated by an encoder to a second preset value
- the second bit rate down-modulation sub-unit is configured to determine a preset value interval to which the brightness mean difference belongs; and to down-modulate the coded bit rate calculated by the encoder according to the preset adjustment value corresponding to the value interval.
- the image processing method performs the following process for the image sent to the encoder: calculating the brightness mean difference between the current frame image and the previous frame image, and comparing the upper limit a frame image, a ratio of a pixel point in the current frame image that produces a change in brightness to a ratio of all pixels in the current frame image, determining whether the brightness mean difference is greater than a first shaking threshold, and determining whether the ratio is greater than a second shaking threshold If the brightness mean difference is greater than the first shaking threshold, and the ratio is greater than the second shaking threshold, the encoding bit rate calculated by the encoder is adjusted according to the first preset adjustment mode.
- the present application compares the brightness mean difference of the two frames before and after the image, and when the brightness mean difference is found to be too large, further determines whether the pixel point where the brightness change occurs in the current frame image accounts for the proportion of all the pixels in the image is too large, and if so, It is determined that the camera is shaken when the current frame image is captured. Therefore, in order to avoid the mosaic effect of the video, the encoding bit rate calculated by the encoder is adjusted upward to ensure the quality of the output video and improve the user experience.
- FIG. 1 is a flowchart of an image processing method according to an embodiment of the present application
- FIG. 2 is a flowchart of another image processing method disclosed in an embodiment of the present application.
- FIG. 3 is a schematic structural diagram of an image processing apparatus according to an embodiment of the present disclosure.
- FIG. 4 is a schematic structural diagram of a bit rate up-scaling unit disclosed in an embodiment of the present application.
- FIG. 5 is a schematic structural diagram of another image processing apparatus according to an embodiment of the present disclosure.
- FIG. 6 is a schematic structural diagram of a bit rate down-scaling unit disclosed in an embodiment of the present application.
- the embodiment of the present application provides an image processing method, which is processed in an image sent to an encoder, and the specific processing process is shown in FIG. 1 .
- the method includes:
- Step S100 calculating a brightness mean difference between the current frame image and the previous frame image, and comparing a pixel point of the current frame image that produces a change in brightness to a ratio of all pixels in the current frame image compared to the previous frame image;
- the brightness average value is first calculated, and then the difference between the brightness and the mean value of the image of the previous frame is calculated to obtain a brightness mean value.
- the brightness mean of the current frame image specifically calculating the mean value of the brightness values of the respective pixel points in the current frame image.
- Step S110 Determine whether the brightness mean difference is greater than a first shaking threshold
- Step S120 determining whether the ratio is greater than a second shaking threshold
- step S130 If the brightness mean difference is greater than the first shaking threshold, and the ratio is greater than the second shaking threshold, Then executing step S130;
- the first sway threshold and the second sway threshold are both preset values. If it is determined that the luminance mean difference between the current frame image and the previous frame image exceeds the first shaking threshold, it means that the two frames before and after the image greatly changed. Further, it is determined whether the ratio of the pixel points that generate the brightness change in the current frame image exceeds the second shaking threshold, and if so, it is determined that the camera does have a large shake when the current frame image is acquired.
- Step S130 The encoding bit rate calculated by the encoder is adjusted according to the first preset adjustment mode.
- the first preset adjustment manner includes multiple manners, and the coding bit rate calculated by the encoder is adjusted upward according to the first preset adjustment manner, so that the quality of the output video is improved, and the mosaic effect is avoided.
- the image processing method provided by the embodiment of the present application performs a process of calculating an average value of brightness of a current frame image and a previous frame image, and generating a current frame image compared to a previous frame image.
- the encoding bit rate calculated by the encoder is adjusted according to the first preset adjustment manner.
- the present application compares the brightness mean difference of the two frames before and after the image, and when the brightness mean difference is found to be too large, further determines whether the pixel point where the brightness change occurs in the current frame image accounts for the proportion of all the pixels in the image is too large, and if so, It is determined that the camera is shaken when the current frame image is captured. Therefore, in order to avoid the mosaic effect of the video, the encoding bit rate calculated by the encoder is adjusted upward to ensure the quality of the output video and improve the user experience.
- the camera is aimed at the teacher who is teaching, to collect the teacher lecture video. If at some point, the student accidentally touched the camera, causing the camera to shake a bit. It is understandable that if the frame image captured at the current time is not processed, the mosaic will appear after the code is displayed. effect.
- the processing of this application is:
- For the current frame image calculate the brightness mean difference ⁇ AVG from the previous frame image, and the ratio of the pixels that produce the brightness change in the current frame image to all the pixels in the current frame image ⁇ RATIO compared to the previous frame image. . Determine whether it is satisfied: And, among them, with The first shaking threshold and the second shaking threshold are respectively. When it is determined that the above conditions are satisfied, it is determined that the camera has shaken when the current frame image is acquired. Thus, the encoding bit rate calculated by the encoder is adjusted to ensure the quality of the output video.
- we introduce several first preset adjustment methods that is, how to adjust the encoding bit rate calculated by the encoder after determining the mosaic effect of the current frame image.
- the first preset value is preset, and when it is determined that the brightness mean difference is greater than the first shaking threshold, and the ratio is greater than the second threshold, the encoded bit rate calculated by the encoder is adjusted upward by the first preset value.
- the first adjustment method is a relatively simple and convenient adjustment method, but the adjustment accuracy is not very accurate. In some scenarios where the output video quality is not too high, this adjustment can be used directly.
- the correspondence between the numerical interval of the luminance mean difference and the adjustment value is set in advance.
- determining a numerical interval to which the brightness mean difference between the current frame image and the previous frame image belongs, according to the numerical interval corresponding to the numerical interval Adjust the value to increase the encoding bit rate calculated by the encoder.
- the encoding bit rate calculated by the encoder can be adjusted by 30 QP.
- the second adjustment method is more accurate than the first adjustment method and can be applied to In scenes with high output video quality.
- the present application provides another image processing method.
- the image processing method can further include:
- Step S140 determining whether the brightness mean difference is less than a first static threshold
- Step S150 determining whether the ratio is less than a second static threshold
- step S160 is performed
- the first static threshold and the second static threshold are both preset values. If it is determined that the luminance mean difference between the current frame image and the previous frame image is less than the first stationary threshold, it means that the changes of the two frames before and after are not large. Further determining whether the ratio of the pixel points that produce the change in brightness in the current frame image is less than the second resting threshold, and if so, determining that the current frame image is relatively stationary with the image content of the previous frame.
- step S160 the encoding bit rate calculated by the encoder is lowered according to the second preset adjustment mode.
- the second preset adjustment manner includes multiple manners, and the coding bit rate calculated by the encoder is lowered according to the second preset adjustment manner, so that the output video content quality is slightly reduced, and the bandwidth resource is reduced without affecting the viewing. usage of.
- the camera is aligned with the blackboard on the podium to collect the teacher lecture video. If the teacher temporarily goes out at a certain moment, the camera is always on the blackboard. No change has taken place.
- the processing of this application is:
- For the current frame image calculate the brightness mean difference ⁇ AVG from the previous frame image, and the ratio of the pixels that produce the brightness change in the current frame image to all the pixels in the current frame image ⁇ RATIO compared to the previous frame image. . Determine whether it is satisfied: And, among them, with The first stationary threshold and the second stationary threshold are respectively. When it is determined that the above conditions are satisfied, it is determined that the acquired current frame image is almost unchanged from the previous frame image. Thus, in order to reduce the use of bandwidth resources, the encoding bit rate calculated by the encoder is down-regulated.
- we introduce several second preset adjustment methods that is, how to adjust the encoding bit rate calculated by the encoder when determining that the current frame image is relatively stationary with the previous frame image content.
- the second preset value is preset, and when it is determined that the brightness mean difference is less than the first static threshold, and the ratio is less than the second static threshold, the encoded bit rate calculated by the encoder is lowered by the second preset value.
- the first adjustment method is a relatively simple and convenient adjustment method, but the adjustment accuracy is not very accurate. In some scenarios where the bandwidth requirements are not too high, this adjustment method can be used directly.
- the numerical interval corresponding to the luminance mean difference is used to down-modulate the encoding bit rate calculated by the encoder according to the adjustment value corresponding to the numerical interval.
- the encoder can be The calculated coding bit rate is lowered by 35QP.
- the second adjustment method is more accurate than the first adjustment method, and can be applied to a scene with high bandwidth requirements.
- the image processing apparatus provided by the embodiment of the present application is described below, and the image processing apparatus described below and the image processing apparatus described above can refer to each other.
- FIG. 3 is a schematic structural diagram of an image processing apparatus according to an embodiment of the present application.
- the device processes the image sent to the encoder. As shown in Figure 3, the device comprises:
- the calculating unit 31 is configured to calculate a brightness mean difference between the current frame image and the previous frame image, and compare a pixel point of the current frame image that produces a change in brightness to a ratio of all pixels in the current frame image compared to the previous frame image;
- the brightness average value is first calculated, and then the difference between the brightness and the mean value of the image of the previous frame is calculated to obtain a brightness mean value.
- the brightness mean of the current frame image specifically calculating the mean value of the brightness values of the respective pixel points in the current frame image.
- the first determining unit 32 is configured to determine whether the brightness mean difference is greater than a first shaking threshold
- a second determining unit 33 configured to determine whether the ratio is greater than a second shaking threshold
- the first sway threshold and the second sway threshold are both preset values. If it is determined that the luminance mean difference between the current frame image and the previous frame image exceeds the first shaking threshold, it means that the two frames before and after the image greatly changed. Further, it is determined whether the ratio of the pixel points that generate the brightness change in the current frame image exceeds the second shaking threshold, and if so, it is determined that the camera does have a large shake when the current frame image is acquired.
- the bit rate up-adjusting unit 34 is configured to: when determining that the brightness mean difference is greater than the first shaking threshold, and the ratio is greater than the second shaking threshold, adjusting the encoding bit rate calculated by the encoder according to the first preset adjustment manner .
- the first preset adjustment manner includes multiple manners, and the coding bit rate calculated by the encoder is adjusted upward according to the first preset adjustment manner, so that the quality of the output video is improved, and the mosaic effect is avoided.
- the image processing apparatus performs a process of calculating an average value of luminance of a current frame image and a previous frame image, and generating a current frame image compared to a previous frame image, for an image transmitted to the encoder.
- the encoding bit rate calculated by the encoder is adjusted according to the first preset adjustment manner.
- the present application compares the brightness mean difference of the two frames before and after the image, and when the brightness mean difference is found to be too large, further determines whether the pixel point where the brightness change occurs in the current frame image accounts for the proportion of all the pixels in the image is too large, and if so, It is determined that the camera is shaken when the current frame image is captured. Therefore, in order to avoid the mosaic effect of the video, the encoding bit rate calculated by the encoder is adjusted upward to ensure the quality of the output video and improve the user experience.
- the embodiment of the present application discloses an optional structure of a bit rate up-scaling unit.
- the bit rate up-scaling unit 34 may include:
- a first bit rate up-modulation sub-unit 341, configured to up-regulate an encoding bit rate calculated by the encoder by a first preset value
- the second bit rate up-modulation sub-unit 342 is configured to determine a preset value interval to which the brightness mean difference belongs, and up-modulate the coded bit rate calculated by the encoder according to the preset adjustment value corresponding to the value interval.
- the adjustment method of the first bit rate up-regulating subunit is a relatively simple and convenient adjustment method, but the adjustment accuracy is not very accurate. In some scenarios where the output video quality is not too high, this adjustment can be used directly.
- the adjustment method of the second bit rate up-regulating sub-unit is more accurate than the first adjustment mode, and can be applied to a scene with high output video quality.
- the embodiment of the present application further provides another image processing apparatus. As shown in FIG. 3 and FIG. 5, the apparatus may further include:
- the third determining unit 35 is configured to determine whether the brightness mean difference is less than a first static threshold
- a fourth determining unit 36 configured to determine whether the ratio is less than a second static threshold
- the first static threshold and the second static threshold are both preset values. If it is determined that the luminance mean difference between the current frame image and the previous frame image is less than the first stationary threshold, it means that the changes of the two frames before and after are not large. Further determining whether the ratio of the pixel points that produce the change in brightness in the current frame image is less than the second resting threshold, and if so, determining that the current frame image is relatively stationary with the image content of the previous frame.
- bit rate down-scaling unit 37 configured to: when determining that the brightness mean difference is less than a first static threshold, and the ratio is less than a second static threshold, lowering an encoding bit rate calculated by an encoder according to a second preset adjustment manner .
- the second preset adjustment manner includes multiple manners, and the coding bit rate calculated by the encoder is lowered according to the second preset adjustment manner, so that the output video content quality is slightly reduced, and the bandwidth resource is reduced without affecting the viewing. usage of.
- the embodiment of the present application discloses an optional structure of the bit rate up-scaling unit.
- the bit rate up-scaling unit 37 may include:
- a first bit rate down-modulation sub-unit 371, configured to: downgrade an encoding bit rate calculated by an encoder to a second preset value
- the second bit rate down-sub-sub-unit 372 is configured to determine a preset value interval to which the brightness mean difference belongs, and to down-modulate the coded bit rate calculated by the encoder according to the preset adjustment value corresponding to the value interval.
- the adjustment method of the first bit rate down-sub-unit is a relatively simple and convenient adjustment method, but the adjustment accuracy is not very accurate. In some scenarios where the bandwidth requirements are not too high, this adjustment method can be used directly.
- the adjustment method of the second bit rate down-sub-unit is more accurate than the first adjustment mode, and can be applied to a scene with high bandwidth requirements.
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Compression Or Coding Systems Of Tv Signals (AREA)
Abstract
本申请公开了一种图像处理方法及装置，针对发送给编码器的图像执行下述过程：计算当前帧图像与上一帧图像的亮度均值差，以及相比于上一帧图像，当前帧图像中产生亮度变化的像素点占当前帧图像中所有像素点的比例，在确定所述亮度均值差大于第一晃动阈值，且所述比例大于第二晃动阈值时，按照第一预置调整方式，将编码器计算出的编码比特率上调。本申请在判断拍摄当前帧图像时摄像头发生了晃动后，将编码器计算出的编码比特率上调，从而避免视频出现马赛克效应，保证了输出视频的质量。
Description
本申请要求于2015年3月30日提交中国专利局、申请号为201510144740.7、发明名称为“一种图像处理方法及装置”的国内申请的优先权，其全部内容通过引用结合在本申请中。
随着网络的普及，网络视频通话等也得到了广泛的应用。在进行网络视频通话时，首先需要摄像头采集图像信息，并经过编码器进行编码后，发给对方客户端进行播放。
但是，如果摄像头在采集图像信息时突然发生大幅度的晃动，则采集的图像经编码后播放时，会出现严重的马赛克效应，大大降低了视频质量，影响用户的体验。
发明内容
有鉴于此，本申请提供了一种图像处理方法及装置，用于解决由于摄像头晃动所造成的视频马赛克效应，影响用户体验的问题。
为了实现上述目的，现提出的方案如下：
一种图像处理方法，针对发送给编码器的图像，执行下述过程：
计算当前帧图像与上一帧图像的亮度均值差，以及相比于上一帧图像，当前帧图像中产生亮度变化的像素点占当前帧图像中所有像素点的比例；
判断所述亮度均值差是否大于第一晃动阈值；以及，
判断所述比例是否大于第二晃动阈值；
若所述亮度均值差大于第一晃动阈值，且所述比例大于第二晃动阈值，则按照第一预置调整方式，将编码器计算出的编码比特率上调。
优选地，还包括：
判断所述亮度均值差是否小于第一静止阈值；以及，
判断所述比例是否小于第二静止阈值；
若所述亮度均值差小于第一静止阈值，且所述比例小于第二静止阈值，
则按照第二预置调整方式，将编码器计算出的编码比特率下调。
优选地，所述按照第一预置调整方式，将编码器计算出的编码比特率上调，包括：
将编码器计算出的编码比特率上调第一预置值；
或者，
确定所述亮度均值差所属的预置的数值区间；
按照预置的与该数值区间对应的调整值，对编码器计算出的编码比特率进行上调。
优选地，所述按照第二预置调整方式，将编码器计算出的编码比特率下调，包括：
将编码器计算出的编码比特率下调第二预置值；
或者，
确定所述亮度均值差所属的预置的数值区间；
按照预置的与该数值区间对应的调整值，对编码器计算出的编码比特率进行下调。
一种图像处理装置，针对发送给编码器的图像进行处理，该装置包括：
计算单元，用于计算当前帧图像与上一帧图像的亮度均值差，以及相比于上一帧图像，当前帧图像中产生亮度变化的像素点占当前帧图像中所有像素点的比例；
第一判断单元，用于判断所述亮度均值差是否大于第一晃动阈值；
第二判断单元，用于判断所述比例是否大于第二晃动阈值；
比特率上调单元，用于在确定所述亮度均值差大于第一晃动阈值，且所述比例大于第二晃动阈值时，按照第一预置调整方式，将编码器计算出的编码比特率上调。
优选地，还包括：
第三判断单元，用于判断所述亮度均值差是否小于第一静止阈值；
第四判断单元，用于判断所述比例是否小于第二静止阈值；
比特率下调单元，用于在确定所述亮度均值差小于第一静止阈值，且所述比例小于第二静止阈值时，按照第二预置调整方式，将编码器计算出
的编码比特率下调。
优选地，所述比特率上调单元包括：
第一比特率上调子单元，用于将编码器计算出的编码比特率上调第一预置值；
第二比特率上调子单元，用于确定所述亮度均值差所属的预置的数值区间；按照预置的与该数值区间对应的调整值，对编码器计算出的编码比特率进行上调。
优选地，所述比特率下调单元包括：
第一比特率下调子单元，用于将编码器计算出的编码比特率下调第二预置值；
第二比特率下调子单元，用于确定所述亮度均值差所属的预置的数值区间；按照预置的与该数值区间对应的调整值，对编码器计算出的编码比特率进行下调。
从上述的技术方案可以看出，本申请实施例提供的图像处理方法，针对发送给编码器的图像执行下述过程：计算当前帧图像与上一帧图像的亮度均值差，以及相比于上一帧图像，当前帧图像中产生亮度变化的像素点占当前帧图像中所有像素点的比例，判断所述亮度均值差是否大于第一晃动阈值，以及，判断所述比例是否大于第二晃动阈值，若所述亮度均值差大于第一晃动阈值，且所述比例大于第二晃动阈值，则按照第一预置调整方式，将编码器计算出的编码比特率上调。本申请通过对比前后两帧图像的亮度均值差，在发现亮度均值差过大时，进一步确定当前帧图像中发生亮度变化的像素点占图像中所有像素点的比例是否过大，如果是，则确定出在拍摄当前帧图像时摄像头发生了晃动，因而为了避免视频出现马赛克效应，将编码器计算出的编码比特率上调，以保证输出视频的质量，提升了用户的体验度。
为了更清楚地说明本申请实施例或现有技术中的技术方案，下面将对
实施例或现有技术描述中所需要使用的附图作简单地介绍，显而易见地，下面描述中的附图仅仅是本申请的实施例，对于本领域普通技术人员来讲，在不付出创造性劳动的前提下，还可以根据提供的附图获得其他的附图。
图1为本申请实施例公开的一种图像处理方法流程图；
图2为本申请实施例公开的另一种图像处理方法流程图；
图3为本申请实施例公开的一种图像处理装置的结构示意图；
图4为本申请实施例公开的一种比特率上调单元的结构示意图；
图5为本申请实施例公开的另一种图像处理装置的结构示意图；
图6为本申请实施例公开的一种比特率下调单元的结构示意图。
下面将结合本申请实施例中的附图，对本申请实施例中的技术方案进行清楚、完整地描述，显然，所描述的实施例仅仅是本申请一部分实施例，而不是全部的实施例。基于本申请中的实施例，本领域普通技术人员在没有做出创造性劳动前提下所获得的所有其他实施例，都属于本申请保护的范围。
本申请实施例提供了一种图像处理方法，该方法针对发送给编码器的图像中进行处理，具体处理过程参见图1。
如图1所示，该方法包括：
步骤S100、计算当前帧图像与上一帧图像的亮度均值差，以及相比于上一帧图像，当前帧图像中产生亮度变化的像素点占当前帧图像中所有像素点的比例；
具体地，对于当前帧图像，首先计算出其亮度均值，进而计算其与上一帧图像的亮度均值的差值，得到亮度均差值。在计算当前帧图像的亮度均值时，具体为计算当前帧图像中各个像素点的亮度值的均值。
此外，还需要计算相比于上一帧图像，当前帧图像中产生亮度变化的像素点占当前帧图像中所有像素点的比例值。
步骤S110、判断所述亮度均值差是否大于第一晃动阈值；
步骤S120、判断所述比例是否大于第二晃动阈值；
若所述亮度均值差大于第一晃动阈值，且所述比例大于第二晃动阈值，
则执行步骤S130；
具体地，上述第一晃动阈值和第二晃动阈值均为预先设定的值。如果确定当前帧图像与上一帧图像的亮度均值差超过第一晃动阈值，则意味着前后两帧图像发生了很大的变动。进一步判断当前帧图像中，产生亮度变化的像素点所占的比例是否超过第二晃动阈值，如果是，则确定在采集当前帧图像时，摄像头确实发生了大幅度的晃动。
步骤S130、按照第一预置调整方式，将编码器计算出的编码比特率上调。
具体地，第一预置调整方式包括多种方式，按照第一预置调整方式将编码器计算出的编码比特率上调，使得输出视频的质量得到提升，避免马赛克效应。
本申请实施例提供的图像处理方法，针对发送给编码器的图像执行下述过程：计算当前帧图像与上一帧图像的亮度均值差，以及相比于上一帧图像，当前帧图像中产生亮度变化的像素点占当前帧图像中所有像素点的比例，判断所述亮度均值差是否大于第一晃动阈值，以及，判断所述比例是否大于第二晃动阈值，若所述亮度均值差大于第一晃动阈值，且所述比例大于第二晃动阈值，则按照第一预置调整方式，将编码器计算出的编码比特率上调。本申请通过对比前后两帧图像的亮度均值差，在发现亮度均值差过大时，进一步确定当前帧图像中发生亮度变化的像素点占图像中所有像素点的比例是否过大，如果是，则确定出在拍摄当前帧图像时摄像头发生了晃动，因而为了避免视频出现马赛克效应，将编码器计算出的编码比特率上调，以保证输出视频的质量，提升了用户的体验度。
为了便于理解本申请的方案，我们以一个具体场景进行解释。
在本实施例提供的场景下，摄像头对准正在授课的老师，以采集老师讲课视频。假如某一时刻，同学不小心碰了下摄像头，造成摄像头大幅度晃动了一下，可以理解的是，对于当前时刻采集的这一帧图像，如果不进行任何处理的话，经编码显示后会出现马赛克效应。而本申请的处理方式是：
对于当前帧图像，计算其与上一帧图像的亮度均值差△AVG，以及相比
于上一帧图像，当前帧图像中产生亮度变化的像素点占当前帧图像中所有像素点的比例△RATIO。判断是否满足：
且，
其中，
和
分别为第一晃动阈值和第二晃动阈值。在确定满足上述条件时，确定当采集当前帧图像时摄像头发生了晃动。因而，对于编码器计算出的编码比特率上调，以保证输出视频的质量。
在本申请的另一个实施例中，我们介绍几种第一预置调整方式，也即在确定当前帧图像会出现马赛克效应后，如何调整编码器计算出的编码比特率。
第一种调节方式：
预先设定第一预置值，在判断所述亮度均值差大于第一晃动阈值，且所述比例大于第二晃大阈值时，将编码器计算出的编码比特率上调第一预置值。
显然，第一种调节方式是一种比较简单、方便的调节方式，但是其调整准确度不是非常的精确。在某些对输出视频质量要求不是太高的场景下，可以直接使用这种调节方式。
第二种调节方式：
预先设定亮度均值差的数值区间与调整值的对应关系。在判断所述亮度均值差大于第一晃动阈值，且所述比例大于第二晃大阈值时，确定出当前帧图像与上一帧图像的亮度均值差所属的数值区间，按照该数值区间对应的调整值，对编码器计算出的编码比特率进行上调。
举例如：预先设定如下对应关系：
表1
在确定亮度均值差为7时，则确定出调整值为30。因而可以对编码器计算出的编码比特率上调30QP。
显然，第二种调节方式相比于第一种调节方式更加精确，能够适用于
对输出视频质量较高的场景中。
此外，还可以预先设定相比于上一帧图像，当前帧图像中产生亮度变化的像素点占当前帧图像中所有像素点的比例，与调整值之间的对应关系。并参考上述第二种调节方式进行编码比特率的调节，对此不再详述。
此外，在摄像头采集视频的过程中，现有技术还面临这样一种问题：当摄像头镜头内容相对静止时，用户一般会对镜头中的内容细节质量要求降低，而更偏重于音频内容，因此花费大部分带宽在无意义的背景细节上，是一种资源的浪费。基于此，本申请提供了另一种图像处理方法。结合图1和图2可知，图像处理方法还可以进一步包括：
步骤S140、判断所述亮度均值差是否小于第一静止阈值；
步骤S150、判断所述比例是否小于第二静止阈值；
若所述亮度均值差小于第一静止阈值，且所述比例小于第二静止阈值，则执行步骤S160；
具体地，上述第一静止阈值和第二静止阈值均为预先设定的值。如果确定当前帧图像与上一帧图像的亮度均值差小于第一静止阈值，则意味着前后两帧图像发生的变动不大。进一步判断当前帧图像中，产生亮度变化的像素点所占的比例是否小于第二静止阈值，如果是，则确定当前帧图像与上一帧图像内容相对静止。
步骤S160、则按照第二预置调整方式，将编码器计算出的编码比特率下调。
具体地，第二预置调整方式包括多种方式，按照第二预置调整方式将编码器计算出的编码比特率下调，使得输出视频内容质量稍微降低，在不影响观看的情况下降低带宽资源的使用。
需要说明的是，上述步骤S140-S160的执行顺序可以与步骤S110-S130并列执行，或者先后执行，对此本申请不进行限定。
为了便于理解本申请的方案，我们以一个具体场景进行解释。
在本实施例提供的场景下，摄像头对准讲台上的黑板，以采集老师讲课视频。假如某一时刻老师临时出去一下，因而摄像头对准的黑板上一直
没有发生变化。对此，本申请的处理方式是：
对于当前帧图像，计算其与上一帧图像的亮度均值差△AVG，以及相比于上一帧图像，当前帧图像中产生亮度变化的像素点占当前帧图像中所有像素点的比例△RATIO。判断是否满足：
且，
其中，
和
分别为第一静止阈值和第二静止阈值。在确定满足上述条件时，确定采集的当前帧图像相比于上一帧图像，内容几乎不变。因而，为了降低带宽资源的使用，对于编码器计算出的编码比特率进行下调。
在本申请的另一个实施例中，我们介绍几种第二预置调整方式，也即在确定当前帧图像与上一帧图像内容相对静止时，如何调整编码器计算出的编码比特率。
第一种调节方式：
预先设定第二预置值，在判断所述亮度均值差小于第一静止阈值，且所述比例小于第二静止阈值时，将编码器计算出的编码比特率下调第二预置值。
显然，第一种调节方式是一种比较简单、方便的调节方式，但是其调整准确度不是非常的精确。在某些对带宽资源要求不太高的场景下，可以直接使用这种调节方式。
第二种调节方式：
预先设定亮度均值差的数值区间与调整值的对应关系，在判断所述亮度均值差小于第一静止阈值，且所述比例小于第二静止阈值时，确定出当前帧图像与上一帧图像的亮度均值差所属的数值区间，按照该数值区间对应的调整值，对编码器计算出的编码比特率进行下调。
举例如：预先设定如下对应关系：
表2
在确定亮度均值差为7时，则确定出调整值为35。因而可以对编码器
计算出的编码比特率下调35QP。
显然，第二种调节方式相比于第一种调节方式更加精确，能够适用于对带宽要求较高的场景中。
此外，还可以预先设定相比于上一帧图像，当前帧图像中产生亮度变化的像素点占当前帧图像中所有像素点的比例，与调整值之间的对应关系。并参考上述第二种调节方式进行编码比特率的调节，对此不再详述。
下面对本申请实施例提供的图像处理装置进行描述，下文描述的图像处理装置与上文描述的图像处理装置可相互对应参照。
参见图3，图3为本申请实施例公开的一种图像处理装置的结构示意图。
该装置针对发送给编码器的图像进行处理。如图3所示，该装置包括：
计算单元31，用于计算当前帧图像与上一帧图像的亮度均值差，以及相比于上一帧图像，当前帧图像中产生亮度变化的像素点占当前帧图像中所有像素点的比例；
具体地，对于当前帧图像，首先计算出其亮度均值，进而计算其与上一帧图像的亮度均值的差值，得到亮度均差值。在计算当前帧图像的亮度均值时，具体为计算当前帧图像中各个像素点的亮度值的均值。
此外，还需要计算相比于上一帧图像，当前帧图像中产生亮度变化的像素点占当前帧图像中所有像素点的比例值。
第一判断单元32，用于判断所述亮度均值差是否大于第一晃动阈值；
第二判断单元33，用于判断所述比例是否大于第二晃动阈值；
具体地，上述第一晃动阈值和第二晃动阈值均为预先设定的值。如果确定当前帧图像与上一帧图像的亮度均值差超过第一晃动阈值，则意味着前后两帧图像发生了很大的变动。进一步判断当前帧图像中，产生亮度变化的像素点所占的比例是否超过第二晃动阈值，如果是，则确定在采集当前帧图像时，摄像头确实发生了大幅度的晃动。
比特率上调单元34，用于在确定所述亮度均值差大于第一晃动阈值，且所述比例大于第二晃动阈值时，按照第一预置调整方式，将编码器计算出的编码比特率上调。
具体地，第一预置调整方式包括多种方式，按照第一预置调整方式将编码器计算出的编码比特率上调，使得输出视频的质量得到提升，避免马赛克效应。
本申请实施例提供的图像处理装置，针对发送给编码器的图像执行下述过程：计算当前帧图像与上一帧图像的亮度均值差，以及相比于上一帧图像，当前帧图像中产生亮度变化的像素点占当前帧图像中所有像素点的比例，判断所述亮度均值差是否大于第一晃动阈值，以及，判断所述比例是否大于第二晃动阈值，若所述亮度均值差大于第一晃动阈值，且所述比例大于第二晃动阈值，则按照第一预置调整方式，将编码器计算出的编码比特率上调。本申请通过对比前后两帧图像的亮度均值差，在发现亮度均值差过大时，进一步确定当前帧图像中发生亮度变化的像素点占图像中所有像素点的比例是否过大，如果是，则确定出在拍摄当前帧图像时摄像头发生了晃动，因而为了避免视频出现马赛克效应，将编码器计算出的编码比特率上调，以保证输出视频的质量，提升了用户的体验度。
可选的，本申请实施例公开了一种比特率上调单元的可选结构，如图4所示，比特率上调单元34可以包括：
第一比特率上调子单元341，用于将编码器计算出的编码比特率上调第一预置值；
第二比特率上调子单元342，用于确定所述亮度均值差所属的预置的数值区间；按照预置的与该数值区间对应的调整值，对编码器计算出的编码比特率进行上调。
显然，第一比特率上调子单元的调节方式是一种比较简单、方便的调节方式，但是其调整准确度不是非常的精确。在某些对输出视频质量要求不是太高的场景下，可以直接使用这种调节方式。而第二比特率上调子单元的调节方式相比于第一种调节方式更加精确，能够适用于对输出视频质量较高的场景中。
此外，在摄像头采集视频的过程中，现有技术还面临这样一种问题：当摄像头镜头内容相对静止时，用户一般会对镜头中的内容细节质量要求降低，而更偏重于音频内容，因此花费大部分带宽在无意义的背景细节上，
是一种资源的浪费。基于此，本申请实施例还提供了另外一种图像处理装置，结合图3和图5可知，该装置还可以包括：
第三判断单元35，用于判断所述亮度均值差是否小于第一静止阈值；
第四判断单元36，用于判断所述比例是否小于第二静止阈值；
具体地，上述第一静止阈值和第二静止阈值均为预先设定的值。如果确定当前帧图像与上一帧图像的亮度均值差小于第一静止阈值，则意味着前后两帧图像发生的变动不大。进一步判断当前帧图像中，产生亮度变化的像素点所占的比例是否小于第二静止阈值，如果是，则确定当前帧图像与上一帧图像内容相对静止。
比特率下调单元37，用于在确定所述亮度均值差小于第一静止阈值，且所述比例小于第二静止阈值时，按照第二预置调整方式，将编码器计算出的编码比特率下调。
具体地，第二预置调整方式包括多种方式，按照第二预置调整方式将编码器计算出的编码比特率下调，使得输出视频内容质量稍微降低，在不影响观看的情况下降低带宽资源的使用。
可选的，本申请实施例公开了比特率上调单元的一种可选结构，如图6所示，比特率上调单元37可以包括：
第一比特率下调子单元371，用于将编码器计算出的编码比特率下调第二预置值；
第二比特率下调子单元372，用于确定所述亮度均值差所属的预置的数值区间；按照预置的与该数值区间对应的调整值，对编码器计算出的编码比特率进行下调。
显然，第一比特率下调子单元的调节方式是一种比较简单、方便的调节方式，但是其调整准确度不是非常的精确。在某些对带宽资源要求不太高的场景下，可以直接使用这种调节方式。而第二比特率下调子单元的调节方式相比于第一种调节方式更加精确，能够适用于对带宽要求较高的场景中。
最后，还需要说明的是，在本文中，诸如第一和第二等之类的关系术
语仅仅用来将一个实体或者操作与另一个实体或操作区分开来，而不一定要求或者暗示这些实体或操作之间存在任何这种实际的关系或者顺序。而且，术语“包括”、“包含”或者其任何其他变体意在涵盖非排他性的包含，从而使得包括一系列要素的过程、方法、物品或者设备不仅包括那些要素，而且还包括没有明确列出的其他要素，或者是还包括为这种过程、方法、物品或者设备所固有的要素。在没有更多限制的情况下，由语句“包括一个……”限定的要素，并不排除在包括所述要素的过程、方法、物品或者设备中还存在另外的相同要素。
本说明书中各个实施例采用递进的方式描述，每个实施例重点说明的都是与其他实施例的不同之处，各个实施例之间相同相似部分互相参见即可。
对所公开的实施例的上述说明，使本领域专业技术人员能够实现或使用本申请。对这些实施例的多种修改对本领域的专业技术人员来说将是显而易见的，本文中所定义的一般原理可以在不脱离本申请的精神或范围的情况下，在其它实施例中实现。因此，本申请将不会被限制于本文所示的这些实施例，而是要符合与本文所公开的原理和新颖特点相一致的最宽的范围。
Claims (8)
- 一种图像处理方法，其特征在于，针对发送给编码器的图像，执行下述过程：计算当前帧图像与上一帧图像的亮度均值差，以及相比于上一帧图像，当前帧图像中产生亮度变化的像素点占当前帧图像中所有像素点的比例；判断所述亮度均值差是否大于第一晃动阈值；以及，判断所述比例是否大于第二晃动阈值；若所述亮度均值差大于第一晃动阈值，且所述比例大于第二晃动阈值，则按照第一预置调整方式，将编码器计算出的编码比特率上调。
- 根据权利要求1所述的方法，其特征在于，还包括：判断所述亮度均值差是否小于第一静止阈值；以及，判断所述比例是否小于第二静止阈值；若所述亮度均值差小于第一静止阈值，且所述比例小于第二静止阈值，则按照第二预置调整方式，将编码器计算出的编码比特率下调。
- 根据权利要求1所述的方法，其特征在于，所述按照第一预置调整方式，将编码器计算出的编码比特率上调，包括：将编码器计算出的编码比特率上调第一预置值；或者，确定所述亮度均值差所属的预置的数值区间；按照预置的与该数值区间对应的调整值，对编码器计算出的编码比特率进行上调。
- 根据权利要求2所述的方法，其特征在于，所述按照第二预置调整方式，将编码器计算出的编码比特率下调，包括：将编码器计算出的编码比特率下调第二预置值；或者，确定所述亮度均值差所属的预置的数值区间；按照预置的与该数值区间对应的调整值，对编码器计算出的编码比特率进行下调。
- 一种图像处理装置，其特征在于，针对发送给编码器的图像进行处理，该装置包括：计算单元，用于计算当前帧图像与上一帧图像的亮度均值差，以及相比于上一帧图像，当前帧图像中产生亮度变化的像素点占当前帧图像中所有像素点的比例；第一判断单元，用于判断所述亮度均值差是否大于第一晃动阈值；第二判断单元，用于判断所述比例是否大于第二晃动阈值；比特率上调单元，用于在确定所述亮度均值差大于第一晃动阈值，且所述比例大于第二晃动阈值时，按照第一预置调整方式，将编码器计算出的编码比特率上调。
- 根据权利要求5所述的装置，其特征在于，还包括：第三判断单元，用于判断所述亮度均值差是否小于第一静止阈值；第四判断单元，用于判断所述比例是否小于第二静止阈值；比特率下调单元，用于在确定所述亮度均值差小于第一静止阈值，且所述比例小于第二静止阈值时，按照第二预置调整方式，将编码器计算出的编码比特率下调。
- 根据权利要求5所述的装置，其特征在于，所述比特率上调单元包括：第一比特率上调子单元，用于将编码器计算出的编码比特率上调第一预置值；第二比特率上调子单元，用于确定所述亮度均值差所属的预置的数值区间；按照预置的与该数值区间对应的调整值，对编码器计算出的编码比特率进行上调。
- 根据权利要求6所述的装置，其特征在于，所述比特率下调单元包括：第一比特率下调子单元，用于将编码器计算出的编码比特率下调第二预置值；第二比特率下调子单元，用于确定所述亮度均值差所属的预置的数值区间；按照预置的与该数值区间对应的调整值，对编码器计算出的编码比 特率进行下调。
Applications Claiming Priority (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201510144740.7A CN104702964B (zh) | 2015-03-30 | 2015-03-30 | 一种图像处理方法及装置 |
| CN201510144740.7 | 2015-03-30 |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| WO2016155125A1 true WO2016155125A1 (zh) | 2016-10-06 |
Family
ID=53349675
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| PCT/CN2015/081798 WO2016155125A1 (zh) | 2015-03-30 | 2015-06-18 | 一种图像处理方法及装置 |
Country Status (2)
| Country | Link |
|---|---|
| CN (1) | CN104702964B (zh) |
| WO (1) | WO2016155125A1 (zh) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US10291849B1 (en) * | 2015-10-16 | 2019-05-14 | Tribune Broadcasting Company, Llc | Methods and systems for determining that a video-capturing device is unsteady |
Families Citing this family (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104702964B (zh) * | 2015-03-30 | 2018-05-15 | 北京布卡互动科技有限公司 | 一种图像处理方法及装置 |
| CN106548763B (zh) * | 2015-09-22 | 2020-09-15 | 中兴通讯股份有限公司 | 一种图像显示方法和装置、以及终端 |
| CN107767712B (zh) * | 2017-11-07 | 2019-12-31 | Oppo广东移动通信有限公司 | 调整匹配度的方法、装置、存储介质及电子设备 |
| CN114339431B (zh) * | 2021-12-16 | 2023-09-01 | 杭州当虹科技股份有限公司 | 缩时编码压缩方法 |
| CN114245048B (zh) * | 2021-12-27 | 2023-07-25 | 上海集成电路装备材料产业创新中心有限公司 | 信号传输电路及图像传感器 |
| CN114786012B (zh) * | 2022-06-16 | 2022-09-02 | 深圳比特微电子科技有限公司 | 一种码率控制方法、装置和可读存储介质 |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102572381A (zh) * | 2010-12-29 | 2012-07-11 | 中国移动通信集团公司 | 视频监控场景判别方法及其监控图像编码方法、及装置 |
| CN102946505A (zh) * | 2012-11-22 | 2013-02-27 | 四川虹微技术有限公司 | 一种基于图像分块统计的自适应运动检测方法 |
| CN103945214A (zh) * | 2013-01-23 | 2014-07-23 | 中兴通讯股份有限公司 | 终端侧时间域视频质量评价方法及装置 |
| CN104702964A (zh) * | 2015-03-30 | 2015-06-10 | 北京三好互动教育科技有限公司 | 一种图像处理方法及装置 |
Family Cites Families (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US5751378A (en) * | 1996-09-27 | 1998-05-12 | General Instrument Corporation | Scene change detector for digital video |
| US7869503B2 (en) * | 2004-02-06 | 2011-01-11 | Apple Inc. | Rate and quality controller for H.264/AVC video coder and scene analyzer therefor |
| CN101389029B (zh) * | 2008-10-21 | 2012-01-11 | 北京中星微电子有限公司 | 一种视频图像编码和检索的方法及装置 |
| CN101511014B (zh) * | 2009-04-10 | 2011-12-07 | 杭州华三通信技术有限公司 | 一种视频编码的方法和设备 |
| CN103905708B (zh) * | 2012-12-24 | 2017-06-27 | 联想(北京)有限公司 | 一种拍摄帧率调整方法及电子设备 |
-
2015
- 2015-03-30 CN CN201510144740.7A patent/CN104702964B/zh not_active Expired - Fee Related
- 2015-06-18 WO PCT/CN2015/081798 patent/WO2016155125A1/zh active Application Filing
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102572381A (zh) * | 2010-12-29 | 2012-07-11 | 中国移动通信集团公司 | 视频监控场景判别方法及其监控图像编码方法、及装置 |
| CN102946505A (zh) * | 2012-11-22 | 2013-02-27 | 四川虹微技术有限公司 | 一种基于图像分块统计的自适应运动检测方法 |
| CN103945214A (zh) * | 2013-01-23 | 2014-07-23 | 中兴通讯股份有限公司 | 终端侧时间域视频质量评价方法及装置 |
| CN104702964A (zh) * | 2015-03-30 | 2015-06-10 | 北京三好互动教育科技有限公司 | 一种图像处理方法及装置 |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US10291849B1 (en) * | 2015-10-16 | 2019-05-14 | Tribune Broadcasting Company, Llc | Methods and systems for determining that a video-capturing device is unsteady |
| US10593365B2 (en) | 2015-10-16 | 2020-03-17 | Tribune Broadcasting Company, Llc | Methods and systems for determining that a video-capturing device is unsteady |
Also Published As
| Publication number | Publication date |
|---|---|
| CN104702964A (zh) | 2015-06-10 |
| CN104702964B (zh) | 2018-05-15 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| WO2016155125A1 (zh) | 一种图像处理方法及装置 | |
| US9270941B1 (en) | Smart video conferencing system | |
| US10003765B2 (en) | System and method for brightening video image regions to compensate for backlighting | |
| US10467734B2 (en) | Group management method, terminal, and storage medium | |
| WO2021114684A1 (zh) | 图像处理方法、装置、计算设备和存储介质 | |
| WO2013135067A1 (zh) | 一种使视频中目标物体图像稳定的方法及装置 | |
| WO2017032035A1 (zh) | 调节方法、调节装置和终端 | |
| WO2017049939A1 (zh) | 一种图像显示方法和装置、以及终端 | |
| JP2017028633A (ja) | 映像配信端末、プログラム、及び、映像配信方法 | |
| WO2020244328A1 (zh) | 图像处理方法及装置 | |
| JP7334470B2 (ja) | 映像処理装置、ビデオ会議システム、映像処理方法、およびプログラム | |
| CN106550227A (zh) | 一种图像饱和度调整方法及装置 | |
| CN103813103A (zh) | 一种移动终端的防抖动拍照方法及装置 | |
| JP2022036991A (ja) | 画像処理装置、画像処理方法およびプログラム | |
| CN104811631A (zh) | 一种图像处理方法及图像处理器 | |
| CN107438155A (zh) | 智能图像拍摄方法 | |
| CN110570441B (zh) | 一种超高清低延时视频控制方法及系统 | |
| CN104811802A (zh) | 图像播放方法及装置 | |
| WO2016155124A1 (zh) | 一种去除奇点噪声的方法及装置 | |
| KR101491963B1 (ko) | 모바일 단말의 아웃 포커싱 영상 통화 방법 및 장치 | |
| WO2018184408A1 (zh) | 视频拍摄方法及装置 | |
| TWI598655B (zh) | 攝影機自動對焦方法及裝置 | |
| CN105304058B (zh) | 自动调节显示参数的方法及装置 | |
| TW201032578A (en) | Method for enhancing image contrast | |
| WO2017107605A1 (zh) | 一种图像细节处理方法和装置、终端、存储介质 |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| 121 | Ep: the epo has been informed by wipo that ep was designated in this application |
Ref document number: 15887092 Country of ref document: EP Kind code of ref document: A1 |
|
| NENP | Non-entry into the national phase |
Ref country code: DE |
|
| 122 | Ep: pct application non-entry in european phase |
Ref document number: 15887092 Country of ref document: EP Kind code of ref document: A1 |