CN109922281A - A kind of real-time video is stingy as system - Google Patents
A kind of real-time video is stingy as system Download PDFInfo
- Publication number
- CN109922281A CN109922281A CN201910059157.4A CN201910059157A CN109922281A CN 109922281 A CN109922281 A CN 109922281A CN 201910059157 A CN201910059157 A CN 201910059157A CN 109922281 A CN109922281 A CN 109922281A
- Authority
- CN
- China
- Prior art keywords
- blue
- background
- value
- processing
- video
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Landscapes
- Image Processing (AREA)
- Image Analysis (AREA)
Abstract
Scratch as system the present invention relates to a kind of real-time video, including under NTSC standard or PAL standard video matting flash processing and blue processing of overflowing.Beneficial effects of the present invention: the quality of video keying under blue, green background is improved, big data processing rear video broadcasting holding 30 frames/second is added, (speed of NTSC standard or 25 frames/second (PAL standard) are constant, it ensure that foreground image marginal steady simultaneously, reduce blue excessive phenomenon.This is scratched as algorithm is to run algorithm on GPU, and this system is write with opencl and two kinds of GPU language of Cuda (do not write GPU implementation method herein respectively.Because these are the specification of opencl or Cuda).
Description
Technical field
The present invention relates to blue, green background videos to scratch as field, it particularly relates to which a kind of real-time video is scratched as system.
Background technique
One classical imaging problem-is scratched as problem-is by non-rectangle foreground image and (usual) rectangular background image point
From-for example, performer is extracted from background scene to allow to replace different backgrounds in moving-picture frame.By required foreground image from
The special circumstances separated in constant or nearly constant background colour background.This background colour is often blue, Suo Youwen
Topic and its solution, always referred to as blue screen image cutting.However, other background colours, such as green, also have been used, so we
It is often summarised as constant color and scratches picture.Although it is stingy as being scratched naturally as artificial to be known as nature for we for any background picture
Intelligent depth digging technology is all applied, but corresponds to mathematical model, is still intangibility and is obtained.Although blue screen image cutting is easy to do
It arrives, but high quality is scratched as being also a problem.
Blue screen image cutting has three big technological difficulties, and first is video speed, it is understood that video must reach 30 frame per second
(NTSC standard) or 25 frames (PAL standard).After big data processing is added, it is a difficult point that video playout speed, which remains unchanged,；
Second is foreground image marginal steady problem, we can be by adjusting α value, to reach between foreground image edge and background
Gentle transition, term are exactly that the passivation to the problem of sharpening solves.It but because is to the stingy as processing, continuous broadcasting of each frame figure
It will appear new problem, be exactly edge flare (or sudden strain of a muscle is made to tremble).Especially in the continuous motion process of foreground people, this problem
It is more prominent.Third is blue overflow problem, due to lan settings are reflective etc., blue light can be projected the upper of foreground object
(green background is exactly green light, but is due up and is commonly called as blue overflow).The fundamental formular for scratching picture is:
I=αF+(1-α)B
Here I is mixed image, and F is prospect, and B is background.It scratches in present blue curtain as all quoting F.van den substantially in system
The formula mentioned in the article of Bergh, V. Lalioti [1]: 2B-R-G (green case is exactly 2G-R-B), this method are scratched as speed
Degree is fast, simple and practical, but foreground edge sharpens seriously.The prospect of deducting has a kind of unstable bounce.
Alvy Ray Smith and James F. Blinn [2] demonstrate one it is unique, general solution is true
Real storage exists, but their method requires reference object to face two different backgrounds, and when going on the stage participates in, this is infeasible
Method.
Vlahos [3] proposes formula:a o = 1–a 1(B f – a 2 G f )
The a1 in the formula, a2 are that tuning reconciles constant.This conciliation gives user to complete.The method that vlahos is proposed is logical
Normal hardware proposes that very high request, debugging difficulty is very high to user's professional standards to complete.
Under the conditions of blue curtain scratches picture, in the article of F.van den Bergh, V. Lalioti [1] when B > R and B > g
Calculate d=2B-R-G;It is less than threshold value a by d, to judge absolute prospect, d is greater than threshold value b, to judge absolute background.A value
0.3 Bd, b value 0.8Bd, BdFor the blue color difference of lan settings.Under handling in this way, there is no problem for single frames picture, once video
Continuous broadcasting prospect boundary sharpening is serious, and the boundary of prospect is rapidly dodged and trembled.
One phenomenon of video keying said before when being exactly that 30 frame speed per second plays, can be generated to dodge and be trembled in foreground edge,
This problem was discussed without paper substantially.Be because currently all article all concentrates on the clear clean of a frame image button,
Also the method that Gauss more than one scratches figure naturally is provided in opencv, but unfortunately these are all that single frames scratches figure, heard in Lee etc.
[4] it in paper, proposing a label key frame approach, trembles and improve speed to reduce sudden strain of a muscle, practice is infeasible, because this
Sample can generate motion blur to sport foreground, while blue dizzy (term is adulterated in motion blur.It is exactly blue line from level to level).Strictly
Ground says that foreground edge sudden strain of a muscle is trembled and can not be eliminated, but can reduce, and is reduced to human eye acceptable degree.
The method operand consumption that Yao Guilin [5] doctor proposes is too big, not only to calculate feature vector and characteristic value, also
The big sparse matrix of Laplce is calculated, GPU crashes.
Fran ois Desch ê nes [6] method is more suitable for handling overflow problem under advanced blue case, and there are many light sources, and
And reflectivity can be measured, it is that a profession is scratched as platform.Building a speciality platform is one than no small spending, in normal blue
In case system, because shining normal light, it can not provide and accurately reconcile metric parameter, so can not apply.
For the problems in the relevant technologies, currently no effective solution has been proposed.
Summary of the invention
For above-mentioned technical problem in the related technology, the present invention proposes that a kind of real-time video is scratched as system, can be improved
The quality of video keying under blue, green background.
To realize the above-mentioned technical purpose, the technical scheme of the present invention is realized as follows: real-time video is scratched as system first
It is the algorithm run on GPU, this algorithm content includes dodging to tremble processing and blue processing of overflowing；
Processing is trembled in sudden strain of a muscle
S1, d=2B-R-G is calculated first；
If S2, one parameter a1 B < a1 of setting are considered prospect, α=1；When B > a1, second parameter a2, d*255/ are reset
A2>245 are background, and α=0, d*255/a2<0 is prospect, α=1；0 < d*255/a2 < 245 are transitional region, reset third
A parameter a3, alpha value: α=(255- a3+d*255/a2)/255；
S3, for transitional region, search the immediate outlook and nearest background, then seek gradient:
div||Ωp-Ωf||<KC
div||Ωp-Ωb||<KC
Wherein: p is transitional region, and f is prospect, and b is background, and KC is threshold value；
Lan Yi is handled
S4, blue light coefficient is set as β, obtain formula: I=β * (α1*F+(1-α1)*B)+(1-β) BL；
Wherein: BL is to overflow bias light, α1To require the actual value after removal blue light；
S5, assume BL=B, can obtain and actually scratch figure formula: I=β * (α1*F+(1-α1*β)*B)；
S6, it is set as αc=α1* β, for all 0 < αc< 1 transitional region point, takes any to seek the mean value:
Dc=2*Bmean-Gmean-Rmean
K=dc/Bd
Wherein: Bmean, Gmean, Rmean are that each point mean value RGB, Bd are the d values that step S1 is found out, this k is α1With α value
Ratio；
S7, α is calculatedk=αc*k
Wherein: αkFor actual value under monochrome；
F(b,g,r)=I(b,g,r)-(1-αk)B(b,g,r)
αz(b,g,r)=((240-B(b,g,r))/(F(b,g,r)-B(b,g,r))
α z=(α zb+αzb+αzr)/3
It obtains close to actual value β, β=αk/αz；
Wherein background colour is B, and the color of certain point is I, and Background color to be replaced is Bz；
S8、α1=αc/β
Calculate F, F=(I- (1- α1)B)/α1
Wherein: unknown here is foreground F, obtains approximate F, further seeks α2:
α2=(α2b+α2g+α2r)/3；
Further, to α2It is filtered, Filtering Formula is as follows:
A=α2* λ is average weight to λ/(L- λ) here, and taking 0.1, L is that Laplacian Matrix Filtering Formula is as follows:
Wherein: ΣkFor covariance matrix, μkFor window WkThe color mean vector of interior 3X1, I3For 3X3 unit matrix, δijFor Crow
Interior gram of symbol, ε are that error coefficient takes 0.001.
Beneficial effects of the present invention: improving the quality of video keying under blue, green background, handles backsight big data is added
Frequency, which plays, keeps 30 frames/second speed constant, while ensure that foreground image edge is stablized, and reduces blue excessive phenomenon.
Detailed description of the invention
It in order to more clearly explain the embodiment of the invention or the technical proposal in the existing technology, below will be to institute in embodiment
Attached drawing to be used is needed to be briefly described, it should be apparent that, the accompanying drawings in the following description is only some implementations of the invention
Example, for those of ordinary skill in the art, without creative efforts, can also obtain according to these attached drawings
Obtain other attached drawings.
Fig. 1 is that endless form is searched in a kind of sampling of the video keying system described according to embodiments of the present invention.
Specific embodiment
Following will be combined with the drawings in the embodiments of the present invention, and technical solution in the embodiment of the present invention carries out clear, complete
Site preparation description, it is clear that described embodiments are only a part of the embodiments of the present invention, instead of all the embodiments.It is based on
Embodiment in the present invention, those of ordinary skill in the art's every other embodiment obtained belong to what the present invention protected
Range.
A kind of real-time video is scratched as system according to embodiments of the present invention, is a kind of GPU operating system, the system base
It scratches in holding real-time video as the rate of 30 frame (NTSC standard) per second is constant.
Processing is trembled in sudden strain of a muscle:
I combines method mentioned above to be improved to, and calculates first
d=2B-R-G
If setting a parameter a1 B < a1 is considered prospect, α=1；When B > a1, second parameter a2, d*255/a2 > 245 are reset
For background, α=0, d*255/a2 < 0 is prospect, α=1.0 < d*255/a2 < 245 be non-determined prospect and non-determined background area, then
Third parameter a3 is set, alpha value is as follows:
α=(255-a3+d*255/a2)/255
It does so compared with V.Lalioti method, alpha parameter will be stablized, and the sudden strain of a muscle of preliminary reduction edge is trembled.
It is mentioned above scratch as formula change to reduce dodge tremble it is beneficial, but this or it is inadequate, also to take and further arrange
It applies.
For the non-determined prospect generated in first time FIG pull handle and non-determined background area, name is defined herein
For transitional region, the immediate outlook and nearest background are searched, gradient is then sought.When gradient value is less than threshold value, before absolutely
Scape or absolute background.Using Fig. 1 circulation searching mode, all travelling points are found as far as possible.
div||Ωp-Ωf||<KC
div||Ωp-Ωb||<KC
Here p is transitional region, and f is prospect, and b is background, and KC is threshold value.
It does so and just reduces missed point, video foreground sudden strain of a muscle is trembled can human eye, it is not easy to perceive.
Lan Yi processing
Lan Yi is handled good, can also reduce shake, next, carrying out blue processing of overflowing.
β is set as blue light coefficient, α1To require the practical alpha value after removal blue light.Obtain formula:
I=β * (α1*F+(1-α1) * B)+(1- β) BL
BL is to overflow bias light to assume that BL=B is obtained:
I=β*α1*F+(1-α1* β) * B)
This formula is practical stingy figure formula.
If αcIt handles to obtain alpha value for upper section, we are set as αc=α1* β, it can be seen that this αcValue is less than
It is practical, therefore background colour is enlarged, so there is blue light spilling.
For all 0 < αc< 1 transitional region point, takes any to seek the mean value,
dc=2*Bmean-Gmean-Rmean
k=dc/Bd
Here Bmean, Gmean, Rmean are that each point mean value RGB, Bd are the d values found out in d=2B-R-G, this k is α1With α value
Ratio.
αk=αc*k(10)
This αkFor actual value under monochrome
F(b,g,r)=I(b,g,r)-(1-αk)B(b,g,r)
αz(b,g,r)=((240-B(b,g,r))/(F(b,g,r)-B(b,g,r))
αz=(αzb+αzb+αzr)/3
β=αk/αz
β is found out in this way.
It is only a close to actual value from this β that can be seen that of front solution procedure, blue excessive, institute can not be also eliminated at all
Also to further calculate.
First it is known that lan settings color is B, the color of certain point is I, our the Background colors to be replaced are Bz,
Alpha value α1=αc/β
Here unknown is foreground F
F=(I-(1-α1)B)/α1
We have found out an approximate F in this way, further seek alpha value α2
α2b=
α2g=
α2r=
α2=(α2b+α2g+α2r)/3
Thus further find out alpha value α2。
The alpha value α finally found out2, it is also necessary to it is filtered, Filtering Formula is as follows:
A=α2*λ/(L-λ)
Here λ is average weight, and taking 0.1, L is that Laplacian Matrix Filtering Formula is as follows
Here ΣkFor covariance matrix, μkFor window WkThe color mean vector of interior 3X1, I3For 3X3 unit matrix, δijFor Crow
Interior gram of symbol, ε are that error coefficient takes 0.001.
The foregoing is merely illustrative of the preferred embodiments of the present invention, is not intended to limit the invention, all in essence of the invention
Within mind and principle, any modification, equivalent replacement, improvement and so on be should all be included in the protection scope of the present invention.
Claims (2)
1. a kind of real-time video is scratched as system, which is characterized in that tremble processing and blue processing of overflowing including dodging；
Processing is trembled in sudden strain of a muscle
S1, d=2B-R-G is calculated first;
If S2, one parameter a1 B < a1 of setting are considered prospect, α=1；When B > a1, second parameter a2 is reset, d*255/a2 >
245 be background, and α=0, d*255/a2 < 0 is prospect, α=1；0 < d*255/a2 < 245 are transitional region, reset third parameter
A3, alpha value: α=(255-a3+d*255/a2)/255；
S3, for transitional region, search the immediate outlook and nearest background, then seek gradient:
div||Ωp-Ωf||<KC
div||Ωp-Ωb||<KC
Wherein: p is transitional region, and f is prospect, and b is background, and KC is threshold value；
Lan Yi is handled
S4, blue excessive coefficient is set as β, obtain formula: I=β * (α1*F+(1-α1) * B)+(1- β) BL；
Wherein: BL is to overflow bias light, α1To require the actual value after removal blue light；
S5, assume BL=B, can obtain and actually scratch figure formula: I=β * (α1*F+(1-α1* β) * B)；
S6, it is set as αc=α1* β, for all 0 < αc< 1 transitional region point, takes any to seek the mean value:
dc=2*Bmean-Gmean-Rmean
k=dc/Bd
Wherein: Bmean, Gmean, Rmean are that each point mean value RGB, Bd are the d values that step S1 is found out, this k is α1With the ratio of α value
Rate；
S7, α is calculatedk=αc*k
Wherein: αkFor actual value under monochrome
F(b,g,r)=I(b,g,r)-(1-αk)B(b,g,r)
αz(b,g,r)=((240-B(b,g,r))/(F(b,g,r)-B(b,g,r))
αz=(αzb+αzb+αzr)/3
It obtains close to actual value β, β=αk/αz
Wherein background colour is B, and the color of certain point is I, and Background color to be replaced is Bz；
S8、α1=αc/β
Calculate F, F=(I- (1- α1)B)/α1
Wherein: unknown here is foreground F, obtains approximate F, further seeks α2:
α2b=
α2g=
α2r=
α2=(α2b+α2g+α2r)/3。
2. a kind of video keying system according to claim 1, which is characterized in that the α in the step S82It also needs to carry out
Filtering processing, Filtering Formula are as follows:
A=α2* λ is average weight to λ/(L- λ) here, and taking 0.1, L is that Laplacian Matrix Filtering Formula is as follows:
Wherein: ΣkFor covariance matrix, μkFor window WkThe color mean vector of interior 3X1, I3For 3X3 unit matrix, δijFor Crow
Interior gram of symbol, ε are that error coefficient takes 0.001.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910059157.4A CN109922281B (en) | 2019-01-22 | 2019-01-22 | Real-time video keying system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910059157.4A CN109922281B (en) | 2019-01-22 | 2019-01-22 | Real-time video keying system |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN109922281A true CN109922281A (en) | 2019-06-21 |
| CN109922281B CN109922281B (en) | 2021-11-09 |
Family
ID=66960579
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910059157.4A Expired - Fee Related CN109922281B (en) | 2019-01-22 | 2019-01-22 | Real-time video keying system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109922281B (en) |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103581571A (en) * | 2013-11-22 | 2014-02-12 | 北京中科大洋科技发展股份有限公司 | Video image matting method based on three elements of color |
| CN104200470A (en) * | 2014-08-29 | 2014-12-10 | 电子科技大学 | Blue screen image-matting method |
| CN105681686A (en) * | 2015-12-31 | 2016-06-15 | 北京奇艺世纪科技有限公司 | Image processing method and system |
| US9786055B1 (en) * | 2016-03-29 | 2017-10-10 | Adobe Systems Incorporated | Method and apparatus for real-time matting using local color estimation and propagation |
| CN108965647A (en) * | 2017-05-18 | 2018-12-07 | 北京金山云网络技术有限公司 | A kind of foreground image preparation method and device |
-
2019
- 2019-01-22 CN CN201910059157.4A patent/CN109922281B/en not_active Expired - Fee Related
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103581571A (en) * | 2013-11-22 | 2014-02-12 | 北京中科大洋科技发展股份有限公司 | Video image matting method based on three elements of color |
| CN104200470A (en) * | 2014-08-29 | 2014-12-10 | 电子科技大学 | Blue screen image-matting method |
| CN105681686A (en) * | 2015-12-31 | 2016-06-15 | 北京奇艺世纪科技有限公司 | Image processing method and system |
| US9786055B1 (en) * | 2016-03-29 | 2017-10-10 | Adobe Systems Incorporated | Method and apparatus for real-time matting using local color estimation and propagation |
| CN108965647A (en) * | 2017-05-18 | 2018-12-07 | 北京金山云网络技术有限公司 | A kind of foreground image preparation method and device |
Non-Patent Citations (1)
| Title |
|---|
| 姚桂林: "数字图像抠图关键技术研究", 《中国博士学位论文全文数据库》 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN109922281B (en) | 2021-11-09 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN105654436B (en) | A kind of backlight image enhancing denoising method based on prospect background separation | |
| CN106204491B (en) | A kind of adapting to image defogging method based on dark channel prior | |
| CN106846263B (en) | Based on the image defogging method for merging channel and sky being immunized | |
| Yu et al. | Fast single image fog removal using edge-preserving smoothing | |
| US7310443B1 (en) | Automatic red eye detection and correction in digital images | |
| US8520089B2 (en) | Eye beautification | |
| CN107424198A (en) | Image processing method, device, mobile terminal and computer-readable recording medium | |
| US20200396434A1 (en) | Image White Balance Processing System and Method | |
| CN109191390A (en) | A kind of algorithm for image enhancement based on the more algorithm fusions in different colours space | |
| CN104504745B (en) | A kind of certificate photo generation method split based on image and scratch figure | |
| CN105959510B (en) | A kind of video rapid defogging method | |
| CN106709893A (en) | All-time haze image sharpness recovery method | |
| KR20150142038A (en) | Reference image selection for motion ghost filtering | |
| JP2002524003A (en) | Comprehensive method for removing backgrounds surrounding selected subjects from images | |
| JPH08510875A (en) | Screen filtering boundary detection for image synthesis | |
| CN103581571A (en) | Video image matting method based on three elements of color | |
| CN104318535B (en) | The method, device and mobile terminal of image defogging | |
| US7522314B2 (en) | Image sharpening | |
| CN111917994A (en) | Method and apparatus for combining image frames captured using different exposure settings into a blended image | |
| CN106651811B (en) | A kind of simple lens imaging ambiguity removal method of luminance channel guiding | |
| CN104680518B (en) | A kind of blue screen image cutting method based on colourity Overflow handling | |
| CN106530309A (en) | Video matting method and system based on mobile platform | |
| CN113034509A (en) | Image processing method and device | |
| He et al. | Single image dehazing with white balance correction and image decomposition | |
| Liba et al. | Sky optimization: Semantically aware image processing of skies in low-light photography |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| CF01 | Termination of patent right due to non-payment of annual fee | ||
| CF01 | Termination of patent right due to non-payment of annual fee |
Granted publication date: 20211109 Termination date: 20220122 |