CN111901606A - Video coding method for improving caption coding quality - Google Patents
Video coding method for improving caption coding quality Download PDFInfo
- Publication number
- CN111901606A CN111901606A CN202010759205.3A CN202010759205A CN111901606A CN 111901606 A CN111901606 A CN 111901606A CN 202010759205 A CN202010759205 A CN 202010759205A CN 111901606 A CN111901606 A CN 111901606A
- Authority
- CN
- China
- Prior art keywords
- block
- coding
- subtitle
- frame
- caption
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 33
- 238000013139 quantization Methods 0.000 claims abstract description 18
- 239000013598 vector Substances 0.000 claims abstract description 11
- 238000004458 analytical method Methods 0.000 claims description 8
- 230000009286 beneficial effect Effects 0.000 abstract description 2
- 230000000694 effects Effects 0.000 description 5
- 238000000605 extraction Methods 0.000 description 5
- 238000001514 detection method Methods 0.000 description 4
- 238000010586 diagram Methods 0.000 description 4
- 238000004364 calculation method Methods 0.000 description 3
- 238000005516 engineering process Methods 0.000 description 3
- 230000006835 compression Effects 0.000 description 2
- 238000007906 compression Methods 0.000 description 2
- 230000007547 defect Effects 0.000 description 2
- 239000000284 extract Substances 0.000 description 2
- 238000004891 communication Methods 0.000 description 1
- 238000005457 optimization Methods 0.000 description 1
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
- H04N19/169—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
- H04N19/186—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being a colour or a chrominance component
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
- H04N19/102—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
- H04N19/124—Quantisation
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
- H04N19/134—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or criterion affecting or controlling the adaptive coding
- H04N19/136—Incoming video signal characteristics or properties
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
- H04N19/134—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or criterion affecting or controlling the adaptive coding
- H04N19/146—Data rate or code amount at the encoder output
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N5/00—Details of television systems
- H04N5/222—Studio circuitry; Studio devices; Studio equipment
- H04N5/262—Studio circuits, e.g. for mixing, switching-over, change of character of image, other special effects ; Cameras specially adapted for the electronic generation of special effects
- H04N5/278—Subtitling
Abstract
The invention discloses a video coding method for improving caption coding quality. The method specifically comprises the following steps: in the airspace, the variance of the brightness value of each coding block in a frame is obtained, and if the variance is greater than a set threshold, the coding block is judged as a subtitle candidate block; in time domain, calculating the reference times of each subtitle candidate block in one frame by subsequent frames, if the reference motion vector is zero and the reference times exceed a set threshold, further judging the subtitle candidate block as a subtitle block, otherwise, judging the subtitle candidate block as a non-subtitle block; when the code rate of the coding block layer is controlled, if a certain block belongs to a caption block, the coding quantization parameter QP is reduced and then coding is carried out. The invention has the beneficial effects that: the existing x265 coding frame is fully utilized, and the time domain and space domain characteristics of the caption are combined, so that the caption area is quickly extracted, the coding quality of the caption area is improved, and the coding quality of the caption area is improved under the condition that the coding performance is not basically reduced.
Description
Technical Field
The invention relates to the technical field related to video coding, in particular to a video coding method for improving subtitle coding quality.
Background
With the rapid development of communication technology and multimedia technology, video becomes an important way for information dissemination, and subtitles in video usually contain story content and key information, so that viewers can understand the content of video playing more conveniently, and therefore subtitle detection and subtitle enhancement of video also lead to intensive research of broad scholars.
How to extract video captions efficiently and accurately is a difficult point, and the difficulty of caption extraction is increased by the continuous change of the size and the alignment mode of caption characters and the change of the texture of a video background. The existing caption detection methods mainly include edge-based methods, stroke-based methods, texture-based methods, connected domain-based methods and the like, and these methods usually use pixels as units for calculation, so that the calculation amount is large, the real-time live broadcast of videos is not facilitated, and non-caption areas are easily mistakenly detected as caption areas.
x265 is an open-source HEVC (High Efficiency Video Coding) Video encoder, which adopts the technologies of scene pre-analysis, code rate control based on Lookahead-vbv, inter-frame parallel Coding, instruction set optimization, macroblock mode fast decision and the like, so that the HEVC encoder can better meet the requirements of High-Efficiency and real-time Coding. In the pre-analysis stage, the matching blocks and the corresponding motion vectors of the coding blocks in the reference frame in the subsequent N frames are analyzed, so that the motion complexity of each frame is judged, and a basis is provided for bit number distribution of the subsequent frames. In the code rate control stage, the control algorithm can be basically divided into two layers: frame layer rate control and coding block layer rate control. Dividing a target code rate into each frame by frame layer code rate control, and calculating a quantization parameter QP (quantization parameter) of each frame according to the complexity of each frame, the weight factor of each frame and the saturation of a buffer area; and the code rate control of the coding block layer calculates the QP of each coding block on the basis of the frame level QP according to the importance degree of each coding block, and if the coding block is the important coding block, a smaller QP is set to reduce the coding distortion of the coding block. If it is a trivial coding block, a larger QP is set, although the distortion increases, a certain number of bits can be saved in this area.
Disclosure of Invention
The present invention provides a video coding method for improving coding quality and improving subtitle coding quality to overcome the above-mentioned disadvantages in the prior art.
In order to achieve the purpose, the invention adopts the following technical scheme:
a video coding method for improving caption coding quality specifically comprises the following steps:
(1) in the airspace, the variance of the brightness value of each coding block in a frame is obtained, and if the variance is greater than a set threshold, the coding block is judged as a subtitle candidate block;
(2) in time domain, calculating the reference times of each subtitle candidate block in one frame by subsequent frames, if the reference motion vector is zero and the reference times exceed a set threshold, further judging the subtitle candidate block as a subtitle block, otherwise, judging the subtitle candidate block as a non-subtitle block;
(3) when the code rate of the coding block layer is controlled, if a certain block belongs to a caption block, the coding quantization parameter QP is reduced and then coding is carried out.
Aiming at the defects of the existing subtitle detection method, the invention provides a video coding method for improving the subtitle coding quality by taking an x265 video coder as a platform, which comprises the following steps: because the edge strength of the caption is high and a group of captions usually continuously appear in a plurality of frames, the method of the invention fully utilizes the existing x265 coding frame, combines the time domain and space domain characteristics of the caption, takes the coding block as a unit, quickly extracts the caption area, adjusts the coding quantization parameter aiming at the caption area and improves the coding quality of the caption area. Under the condition that the encoding performance is not reduced basically, the encoding quality of the subtitle area is improved.
Preferably, in the step (1), specifically: calculating by taking the coding block as a unit, and combining an x265 video encoder, solving the variance var of the brightness value of each coding block in the current frame in a pre-analysis stage by using a matching block and motion vector information provided by the existing pre-analysis module, recording the maximum value var _ max and the minimum value var _ min of var in the current frame, and calculating the texture complexity tex of each coding block in the current frame:
and if the tex is greater than a threshold TH1, judging the coding block as a subtitle candidate block, otherwise, judging the coding block as a non-subtitle block, wherein the threshold TH1 is in the range of [100, 255 ].
Preferably, in the step (2), specifically: and recording the time domain duration of each subtitle candidate block in the current frame as N, if the current subtitle candidate block is referred by a coding block in a certain subsequent frame and the motion vector is zero, recording N as N +1, after the subsequent N frames are analyzed, if the time domain duration of the subtitle candidate block exceeds a threshold TH2, further judging the current subtitle candidate block as a subtitle block, otherwise, judging the subtitle candidate block as a non-subtitle block, wherein the value range of N is between [10 and 50], and the value range of the threshold TH2 is between [3 and N ].
Preferably, in the step (3), specifically: in the coding block layer rate control, if a certain coding block belongs to a subtitle block, defining a coding quantization parameter QP _ mb of the current coding block as QP _ fm-dQp, where QP _ fm is a frame-level coding quantization parameter, dQp has a value range of [0, 10], and coding the coding block by using QP _ mb.
The invention has the beneficial effects that: the existing x265 coding frame is fully utilized, and the time domain and space domain characteristics of the caption are combined, so that the caption area is quickly extracted, the coding quality of the caption area is improved, and the coding quality of the caption area is improved under the condition that the coding performance is not basically reduced.
Drawings
FIG. 1 is a diagram of raw video;
FIG. 2 is a diagram of the effect of subtitle extraction according to the present invention;
fig. 3 is a diagram of subtitle coding effects for x265 coding;
fig. 4 is a subtitle encoding effect diagram of the present invention.
Detailed Description
The invention is further described with reference to the following figures and detailed description.
In the embodiment shown in fig. 1, a video encoding method for improving the quality of subtitle encoding specifically includes the following steps:
(1) in the airspace, the variance of the brightness value of each coding block in a frame is obtained, and if the variance is greater than a set threshold, the coding block is judged as a subtitle candidate block; the method specifically comprises the following steps: calculating by taking the coding block as a unit, and combining an x265 video encoder, solving the variance var of the brightness value of each coding block in the current frame in a pre-analysis stage by using a matching block and motion vector information provided by the existing pre-analysis module, recording the maximum value var _ max and the minimum value var _ min of var in the current frame, and calculating the texture complexity tex of each coding block in the current frame:
and if the tex is greater than a threshold TH1, judging the coding block as a subtitle candidate block, otherwise, judging the coding block as a non-subtitle block, wherein the threshold TH1 is in the range of [100, 255 ].
(2) In time domain, calculating the reference times of each subtitle candidate block in one frame by subsequent frames, if the reference motion vector is zero and the reference times exceed a set threshold, further judging the subtitle candidate block as a subtitle block, otherwise, judging the subtitle candidate block as a non-subtitle block; the method specifically comprises the following steps: and recording the time domain duration of each subtitle candidate block in the current frame as N, if the current subtitle candidate block is referred by a coding block in a certain subsequent frame and the motion vector is zero, recording N as N +1, after the subsequent N frames are analyzed, if the time domain duration of the subtitle candidate block exceeds a threshold TH2, further judging the current subtitle candidate block as a subtitle block, otherwise, judging the subtitle candidate block as a non-subtitle block, wherein the value range of N is between [10 and 50], and the value range of the threshold TH2 is between [3 and N ]. Wherein: the time domain duration of the subtitle candidate block refers to the number of times the subtitle candidate block appears in the subsequent frame.
(3) When the code rate of a coding block layer is controlled, if a certain block belongs to a caption block, coding is carried out after a coding quantization parameter QP is reduced; the method specifically comprises the following steps: in the coding block layer rate control, if a certain coding block belongs to a subtitle block, defining a coding quantization parameter QP _ mb of the current coding block as QP _ fm-dQp, where QP _ fm is a frame-level coding quantization parameter, dQp has a value range of [0, 10], and coding the coding block by using QP _ mb. Wherein: qp _ mb is a quantization value for performing quantization compression on the macroblock, and if Qp _ mb is larger, the larger the quantization amplitude is, the larger the image quality distortion encoded by the macroblock is; if Qp _ mb is smaller, it means that the quantization amplitude is smaller, and the picture quality distortion coded by the macroblock is smaller.
Aiming at the defects of the existing subtitle detection method, the invention provides a video coding method for improving the subtitle coding quality by taking an x265 video coder as a platform, which comprises the following steps: because the edge strength of the caption is large and a group of captions usually continuously appear in a plurality of frames, the method mainly comprises spatial domain extraction and time domain extraction when extracting the caption, wherein the variance of the brightness value of each coding block in one frame is solved in the spatial domain, and if the variance is greater than a certain threshold value, the coding block is judged as a caption candidate block; in time domain, calculating the reference times of each subtitle candidate block in one frame by the subsequent frame, if the reference time is zero and the reference times exceeds a certain threshold, further judging the subtitle candidate block as a subtitle block, otherwise, judging the subtitle candidate block as a non-subtitle block. When the code rate of the coding block layer is controlled, if a certain block belongs to a caption block, the coding quantization parameter QP is reduced, so that the coding quality of the caption area is improved. In addition, the method has low calculation complexity, and compared with the original x265 encoder, the encoding speed is not reduced basically.
The invention optimizes the quality of caption coding by a method for efficiently extracting the caption, and can be applied to video compression standards such as H.264, HEVC, AVS2, AVS3 and the like. Fig. 1 and fig. 2 show the subtitle extraction effect of the original video and the method of the present invention, respectively, and it can be seen that the method of the present invention can completely extract the subtitle region, and the other regions are not mistakenly detected as the subtitle region, which indicates that the accuracy of extracting the subtitle by the method of the present invention is high. Fig. 3 and fig. 4 respectively show the subtitle encoding effect of the original x265 encoding method and the method of the present invention, and it can be seen that the periphery of the subtitle of the original x265 encoding method is fuzzy, while the periphery of the subtitle of the method of the present invention is clear, which can better retain the information of the original video.
Claims (4)
1. A video coding method for improving caption coding quality is characterized by comprising the following steps:
(1) in the airspace, the variance of the brightness value of each coding block in a frame is obtained, and if the variance is greater than a set threshold, the coding block is judged as a subtitle candidate block;
(2) in time domain, calculating the reference times of each subtitle candidate block in one frame by subsequent frames, if the reference motion vector is zero and the reference times exceed a set threshold, further judging the subtitle candidate block as a subtitle block, otherwise, judging the subtitle candidate block as a non-subtitle block;
(3) when the code rate of the coding block layer is controlled, if a certain block belongs to a caption block, the coding quantization parameter QP is reduced and then coding is carried out.
2. The video coding method for improving the coding quality of subtitles according to claim 1, wherein in the step (1), the steps are specifically as follows: calculating by taking the coding block as a unit, and combining an x265 video encoder, solving the variance var of the brightness value of each coding block in the current frame in a pre-analysis stage by using a matching block and motion vector information provided by the existing pre-analysis module, recording the maximum value var _ max and the minimum value var _ min of var in the current frame, and calculating the texture complexity tex of each coding block in the current frame:
and if the tex is greater than a threshold TH1, judging the coding block as a subtitle candidate block, otherwise, judging the coding block as a non-subtitle block, wherein the threshold TH1 is in the range of [100, 255 ].
3. The video coding method for improving the coding quality of subtitles according to claim 1, wherein in the step (2), the steps are specifically as follows: and recording the time domain duration of each subtitle candidate block in the current frame as N, if the current subtitle candidate block is referred by a coding block in a certain subsequent frame and the motion vector is zero, recording N as N +1, after the subsequent N frames are analyzed, if the time domain duration of the subtitle candidate block exceeds a threshold TH2, further judging the current subtitle candidate block as a subtitle block, otherwise, judging the subtitle candidate block as a non-subtitle block, wherein the value range of N is between [10 and 50], and the value range of the threshold TH2 is between [3 and N ].
4. The video coding method for improving the coding quality of subtitles according to claim 1, wherein in the step (3), the steps are specifically as follows: in the coding block layer rate control, if a certain coding block belongs to a subtitle block, defining a coding quantization parameter QP _ mb of the current coding block as QP _ fm-dQp, where QP _ fm is a frame-level coding quantization parameter, dQp has a value range of [0, 10], and coding the coding block by using QP _ mb.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010759205.3A CN111901606A (en) | 2020-07-31 | 2020-07-31 | Video coding method for improving caption coding quality |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010759205.3A CN111901606A (en) | 2020-07-31 | 2020-07-31 | Video coding method for improving caption coding quality |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN111901606A true CN111901606A (en) | 2020-11-06 |
Family
ID=73182922
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010759205.3A Pending CN111901606A (en) | 2020-07-31 | 2020-07-31 | Video coding method for improving caption coding quality |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111901606A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113207003A (en) * | 2021-04-15 | 2021-08-03 | 上海顺久电子科技有限公司 | Motion estimation method of video image and electronic equipment |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1937779A (en) * | 2006-10-18 | 2007-03-28 | 深圳创维－Rgb电子有限公司 | Method for improving television terminal device digital caption data processing efficiency |
| CN102625029A (en) * | 2012-03-23 | 2012-08-01 | 无锡引速得信息科技有限公司 | Self-adaptive threshold caption detection method |
| CN103379335A (en) * | 2012-04-19 | 2013-10-30 | Vixs系统公司 | Detection of video feature based on variance metric |
| CN105282475A (en) * | 2014-06-27 | 2016-01-27 | 澜起科技（上海）有限公司 | Mobile subtitle detection and compensation method and system |
| US20170085892A1 (en) * | 2015-01-20 | 2017-03-23 | Beijing University Of Technology | Visual perception characteristics-combining hierarchical video coding method |
| CN107087163A (en) * | 2017-06-26 | 2017-08-22 | 杭州当虹科技有限公司 | A kind of coding method of lifting HDR Subjective video qualities |
-
2020
- 2020-07-31 CN CN202010759205.3A patent/CN111901606A/en active Pending
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1937779A (en) * | 2006-10-18 | 2007-03-28 | 深圳创维－Rgb电子有限公司 | Method for improving television terminal device digital caption data processing efficiency |
| CN102625029A (en) * | 2012-03-23 | 2012-08-01 | 无锡引速得信息科技有限公司 | Self-adaptive threshold caption detection method |
| CN103379335A (en) * | 2012-04-19 | 2013-10-30 | Vixs系统公司 | Detection of video feature based on variance metric |
| CN105282475A (en) * | 2014-06-27 | 2016-01-27 | 澜起科技（上海）有限公司 | Mobile subtitle detection and compensation method and system |
| US20170085892A1 (en) * | 2015-01-20 | 2017-03-23 | Beijing University Of Technology | Visual perception characteristics-combining hierarchical video coding method |
| CN107087163A (en) * | 2017-06-26 | 2017-08-22 | 杭州当虹科技有限公司 | A kind of coding method of lifting HDR Subjective video qualities |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113207003A (en) * | 2021-04-15 | 2021-08-03 | 上海顺久电子科技有限公司 | Motion estimation method of video image and electronic equipment |
| CN113207003B (en) * | 2021-04-15 | 2023-01-24 | 上海顺久电子科技有限公司 | Motion estimation method of video image and electronic equipment |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US20220312021A1 (en) | Analytics-modulated coding of surveillance video | |
| CN106231214B (en) | High-speed CMOS sensor image approximate lossless compression method based on adjustable macro block | |
| CN109688414B (en) | VVC intra-frame coding unit candidate prediction mode reduction and block division early termination method | |
| CN106682094B (en) | Face video retrieval method and system | |
| AU2012211249A1 (en) | Encoding of video stream based on scene type | |
| CN102665077A (en) | Rapid and efficient encoding-transcoding method based on macro block classification | |
| CN100574447C (en) | Fast intraframe predicting mode selecting method based on the AVS video coding | |
| CN102917225A (en) | Method for quickly selecting HEVC (high-efficiency video coding) inframe coding units | |
| CN107018412A (en) | A kind of DVC HEVC video transcoding methods based on key frame coding unit partition mode | |
| CN104601992A (en) | SKIP mode quickly selecting method based on Bayesian minimum hazard decision | |
| US11212518B2 (en) | Method for accelerating coding and decoding of an HEVC video sequence | |
| CN111901606A (en) | Video coding method for improving caption coding quality | |
| CN108881905B (en) | Probability-based intra-frame encoder optimization method | |
| CN106878754B (en) | A kind of 3D video depth image method for choosing frame inner forecast mode | |
| CN102158702A (en) | Self-adaption H.264 code rate control method | |
| CN102592130B (en) | Target identification system aimed at underwater microscopic video and video coding method thereof | |
| CN103905818B (en) | Method for rapidly determining inter-frame prediction mode in HEVC standard based on Hough conversion | |
| Zhao et al. | Fast CU Size Decision Method Based on Just Noticeable Distortion and Deep Learning | |
| CN113205010A (en) | Intelligent disaster-exploration on-site video frame efficient compression system and method based on target clustering | |
| Yuan et al. | Fast CU size and PU partition decision for AVS2 intra coding | |
| CN103533348A (en) | Multi-view rapid mode selecting method based on visual perception | |
| CN114501007B (en) | 422 video-based rapid coding method and coder | |
| CN112040246B (en) | Low-delay low-complexity fixed code rate control method | |
| Wu et al. | A novel RDO based on perceptual algorithm | |
| Tang et al. | Optimization of CU Partition Based on Texture Degree in H. 266/VVC |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20201106 |