CN108566521A - A kind of image synthesizing system for scratching picture based on natural image - Google Patents
A kind of image synthesizing system for scratching picture based on natural image Download PDFInfo
- Publication number
- CN108566521A CN108566521A CN201810665631.3A CN201810665631A CN108566521A CN 108566521 A CN108566521 A CN 108566521A CN 201810665631 A CN201810665631 A CN 201810665631A CN 108566521 A CN108566521 A CN 108566521A
- Authority
- CN
- China
- Prior art keywords
- video
- image
- audio
- unit
- data
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 230000002194 synthesizing effect Effects 0.000 title claims abstract description 18
- 238000006748 scratching Methods 0.000 title claims abstract description 12
- 230000002393 scratching effect Effects 0.000 title claims abstract description 12
- 238000012545 processing Methods 0.000 claims abstract description 16
- 230000003993 interaction Effects 0.000 claims abstract description 13
- 238000003708 edge detection Methods 0.000 claims abstract description 11
- 238000009499 grossing Methods 0.000 claims abstract description 11
- 238000005538 encapsulation Methods 0.000 claims abstract description 7
- 230000005540 biological transmission Effects 0.000 claims abstract description 4
- 230000002452 interceptive effect Effects 0.000 claims abstract description 4
- 238000012805 post-processing Methods 0.000 claims abstract description 4
- 230000015572 biosynthetic process Effects 0.000 description 7
- 238000003786 synthesis reaction Methods 0.000 description 7
- 230000006872 improvement Effects 0.000 description 5
- 238000000034 method Methods 0.000 description 5
- 230000009471 action Effects 0.000 description 2
- 238000010586 diagram Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000003780 insertion Methods 0.000 description 1
- 230000037431 insertion Effects 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000008092 positive effect Effects 0.000 description 1
- 238000002360 preparation method Methods 0.000 description 1
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N5/00—Details of television systems
- H04N5/222—Studio circuitry; Studio devices; Studio equipment
- H04N5/262—Studio circuits, e.g. for mixing, switching-over, change of character of image, other special effects ; Cameras specially adapted for the electronic generation of special effects
- H04N5/265—Mixing
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T3/00—Geometric image transformation in the plane of the image
- G06T3/40—Scaling the whole image or part thereof
- G06T3/4038—Scaling the whole image or part thereof for image mosaicing, i.e. plane images composed of plane sub-images
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/50—Image enhancement or restoration by the use of more than one image, e.g. averaging, subtraction
-
- G06T5/70—
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/12—Edge-based segmentation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/194—Segmentation; Edge detection involving foreground-background segmentation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10016—Video; Image sequence
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10024—Color image
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20172—Image enhancement details
- G06T2207/20192—Edge enhancement; Edge preservation
Abstract
The invention discloses a kind of image synthesizing systems for scratching picture based on natural image, including photographic device, microphone and resolution unit, obtain the data of photographic device in real time, obtain the not background video of foreground image and comprising the first video of foreground image；Pretreatment unit, each frame image of the first video subtract each frame RGB image of background video and obtain the second video frame；Combining unit merges foreground image and virtual scene, obtains synthetic video；Post-processing unit carries out edge detection and smoothing processing；The audio data of sound microphone and synthetic video are packaged into audio, video data by encapsulation unit；The audio, video data is uploaded to third-party server by transmission unit by http protocol；Third-party server generates the internet link of the audio, video data；Interactive unit receives the interaction instruction that client is sent, and according to interaction instruction update or switching virtual scene.
Description
Technical field
The present invention relates to technical field of video processing more particularly to a kind of Video Composition systems for scratching picture based on natural image
System.
Background technology
In technical field of video processing, video keying technology can be usually applied to.So-called video keying technology, refer to by
Personage in one video or other foregrounds are scratched by intelligence and are crawled out as technology, and it is embedded be synthesized to it is another virtual
Or in other actual scene backgrounds.Main effect is after so that the user of viewing video is felt that the personage in video is in
In the scene of phase synthesis, and personage and scene background can further be made to have interaction.It is scratched with traditional as the difference of technology is
Traditional scratches as technology needs the background of pure color for example：The background colour of green or blue could complete to scratch figure and the synthesis of video is dynamic
Make, greatly limits the convenience of the range and user that use, and the present invention is in addition to that can use green or blue background
Come synthesize outside can also using natural environment when background come complete video stingy figure act, can not be limited by environment, greatly
The use scope of each environment of big opening and industry, for example, sing k in the parlor of ktv or family, during singing K, pass through by
Singer's real-time image scratching and insertion be synthesized in free video image background, thus allow chanteur on the spot in person in video
In sing, like concert scene.
Video keying technology main problem to be solved in practice is carried out in fact to the personage that video camera is shot
When scratch picture, and synthesized with the video background of offer, finally by edge detection and smoothing technique to scratching as being caused with synthesis
Character contour edge sawtooth be smoothed.
Invention content
Purpose of the present invention is to solve the above problems, a kind of image synthesizing system for scratching picture based on natural image is designed, is overcome
Tradition scratches the limitation that figure could be completed with the background of pure color that must above arrange in pairs or groups, and can be completed when background using natural environment
The stingy figure of video acts, and greatly opens the use scope of each environment and industry.
To achieve the goals above, the technical scheme is that：
A kind of image synthesizing system for scratching picture based on natural image, including：
Photographic device, microphone and
Resolution unit, in real time obtain photographic device data, obtain the not background video of foreground image and comprising
The first video of foreground image；
Each frame RGB image of first video is subtracted each frame RGB image of background video, institute by pretreatment unit
The difference of acquirement obtains each frame RGB image of the second video；
Combining unit merges foreground image and virtual scene, obtains synthetic video；
Post-processing unit carries out edge detection and smoothing processing to synthetic video；
Encapsulation unit while obtaining photographic device data, obtains the audio data of microphone, by sound microphone in real time
Audio data is packaged into audio, video data with synthetic video；
The audio, video data is uploaded to third-party server by transmission unit by http protocol；Third-party server
Generate the internet link of the audio, video data；
Interactive unit receives the interaction instruction that client is sent, and according to interaction instruction update or switching virtual scene.
The resolution unit includes Video Decoder, and combining unit includes Video Composition module, and encapsulation unit includes hardware
Coding module and Video Record Processor.
The image synthesizing system that picture is scratched based on natural image of the present invention, using the Video Composition for scratching picture based on natural image
Method completes the synthesis of video, and the image synthesizing method that should be scratched based on natural image picture is included the following steps：
The video for not including foreground image under step 1, shooting natural environment, as background video；
The video for including foreground image under step 2, shooting natural environment, as the first video；
Each frame RGB image of step 3, first video subtracts each frame RGB image of background video, acquired
Difference obtains each frame RGB image of the second video；And be compared the difference with preset value, to judge that second video frame is
Foreground image or background image；
Second video frame, rgb value are more than preset value, judge the pixel for foreground pixel, transparence value 1 is no
It is then 0；Transparence value=(255- (blue channel value-(red color channel value+green channel value)/2))/255；
Step 4, according to the second video frame, the transparency of the first video frame is set, foreground image is obtained；
Step 5 merges foreground image and virtual scene, obtains synthetic video；
Step 6 carries out edge detection to the synthetic video；
Step 7 is smoothed the edge of the synthetic video.
As the improvement to the technical program, the edge detection is then judged transparency calculating with edge detection algorithm
The pixel is edge pixel.
As the improvement to the technical program, the smoothing processing, centered on the edge pixel point, m*m sizes
The color-values of pixel in square range, which add up, makees arithmetic average, the color-values new as the edge pixel.
Further, further include after the synthetic video after smoothing processing：
While step 8, acquisition photographic device data, the audio data of microphone is obtained in real time, by the sound of smoothing processing
Frequency is packaged into audio, video data according to synthetic video；
Step 9, by http protocol, the audio, video data is uploaded into third-party server；Third-party server is given birth to
At the internet link of the audio, video data.
Further, further include the interaction instruction for receiving client and sending, and updated or switched empty according to interaction instruction
Quasi- scene.
Further, the virtual scene is still image or dynamic video scene.
Compared with prior art, the invention has the advantages and positive effects that：
The image synthesizing system that picture is scratched based on natural image of the present invention is scratched with traditional as the difference of technology is tradition
Scratch the synthesis action for needing the background of the pure color such as background colour of green or blue that could complete to scratch figure and video as technology, greatly
The big convenience for limiting the range and user that use, and the present invention is synthesized in addition to that can use the background of green or blue
It can also be acted outside when background to complete the stingy figure of video using natural environment, can not be limited by environment, greatly beat
Open the use scope of each environment and industry.The present invention can be to each pixel of the frame image that camera captures in GPU (figures
Processor) parallel processing is carried out, processing speed is very fast.
Description of the drawings
In order to more clearly explain the embodiment of the invention or the technical proposal in the existing technology, to embodiment or will show below
There is attached drawing needed in technology description to be briefly described, it should be apparent that, the accompanying drawings in the following description is only this
Some embodiments of invention without having to pay creative labor, may be used also for those of ordinary skill in the art
With obtain other attached drawings according to these attached drawings.
Fig. 1 is the module map of image synthesizing system；
Fig. 2 is image synthesizing system schematic diagram.
Specific implementation mode
Following will be combined with the drawings in the embodiments of the present invention, and technical solution in the embodiment of the present invention carries out clear, complete
Site preparation describes, it is clear that described embodiments are only a part of the embodiments of the present invention, instead of all the embodiments.It is based on
Embodiment in the present invention, it is obtained by those of ordinary skill in the art without making creative efforts every other
Embodiment, any modification, equivalent replacement, improvement and so on should all be included in the protection scope of the present invention.
As shown in Fig. 1 to 2, a kind of image synthesizing system for scratching picture based on natural image of the invention, including：
Photographic device 601, microphone 605 and
Resolution unit 10, the data for obtaining photographic device in real time, obtains the background video and packet of not foreground image
Containing the first video of foreground image；
Pretreatment unit 20, each frame RGB image of first video subtract each frame RGB image of background video, institute
The difference of acquirement obtains each frame RGB image of the second video；
Combining unit 30 merges foreground image and virtual scene, obtains synthetic video；
Post-processing unit 40 carries out edge detection and smoothing processing to synthetic video；
Encapsulation unit 50 while obtaining photographic device data, obtains the audio data of microphone, by sound microphone in real time
Audio data and synthetic video be packaged into audio, video data；
The audio, video data is uploaded to third-party server 606 by transmission unit 60 by http protocol；Third party
Server 606 generates the internet link of the audio, video data；
Interactive unit receives the interaction instruction that client is sent, and according to interaction instruction update or switching virtual scene.
The resolution unit 10 includes Video Decoder 602, and combining unit 30 includes Video Composition module 603, and encapsulation is single
Member 50 includes hardware encoding block and Video Record Processor 604.
The image synthesizing system that picture is scratched based on natural image of the present invention, using the Video Composition for scratching picture based on natural image
Method completes the synthesis of video, and the image synthesizing method that should be scratched based on natural image picture is included the following steps：
The video for not including foreground image under step 1, shooting natural environment, as background video；
The video for including foreground image under step 2, shooting natural environment, as the first video；
Each frame RGB image of step 3, first video subtracts each frame RGB image of background video, acquired
Difference obtains each frame RGB image of the second video；And be compared the difference with preset value, to judge that second video frame is
Foreground image or background image；
Second video frame, rgb value are more than preset value, judge the pixel for foreground pixel, transparence value 1 is no
It is then 0；Transparence value=(255- (blue channel value-(red color channel value+green channel value)/2))/255；
Step 4, according to the second video frame, the transparency of the first video frame is set, foreground image is obtained；
Step 5 merges foreground image and virtual scene, obtains synthetic video；
Step 6 carries out edge detection to the synthetic video；
Step 7 is smoothed the edge of the synthetic video.
As the improvement to the technical program, the edge detection is then judged transparency calculating with edge detection algorithm
The pixel is edge pixel.
As the improvement to the technical program, the smoothing processing, centered on the edge pixel point, m*m sizes
The color-values of pixel in square range, which add up, makees arithmetic average, the color-values new as the edge pixel.
Further, further include after the synthetic video after smoothing processing：
While step 8, acquisition photographic device data, the audio data of microphone is obtained in real time, by the sound of smoothing processing
Frequency is packaged into audio, video data according to synthetic video；
Step 9, by http protocol, the audio, video data is uploaded into third-party server；Third-party server is given birth to
At the internet link of the audio, video data.
Further, further include the interaction instruction for receiving client and sending, and updated or switched empty according to interaction instruction
Quasi- scene.
Further, the virtual scene is still image or dynamic video scene.
The image synthesizing system that picture is scratched based on natural image of the present invention is scratched with traditional as the difference of technology is tradition
Scratch the synthesis action for needing the background of the pure color such as background colour of green or blue that could complete to scratch figure and video as technology, greatly
The big convenience for limiting the range and user that use, and the present invention is synthesized in addition to that can use the background of green or blue
It can also be acted outside when background to complete the stingy figure of video using natural environment, can not be limited by environment, greatly beat
Open the use scope of each environment and industry.The present invention can be to each pixel of the frame image that camera captures in GPU (figures
Processor) parallel processing is carried out, processing speed is very fast.
Claims (4)
1. a kind of image synthesizing system for scratching picture based on natural image, it is characterised in that：Including：
Photographic device, microphone and
Resolution unit, the data for obtaining photographic device in real time, obtains the not background video of foreground image and comprising foreground
The first video of image；
Each frame RGB image of first video is subtracted each frame RGB image of background video by pretreatment unit, acquired
Difference obtain each frame RGB image of the second video；
Combining unit merges foreground image and virtual scene, obtains synthetic video；
Post-processing unit carries out edge detection and smoothing processing to synthetic video；
Encapsulation unit while obtaining photographic device data, obtains the audio data of microphone, by the audio of sound microphone in real time
Data are packaged into audio, video data with synthetic video；
The audio, video data is uploaded to third-party server by transmission unit by http protocol；Third-party server generates
The internet link of the audio, video data；
Interactive unit receives the interaction instruction that client is sent, and according to interaction instruction update or switching virtual scene.
2. image synthesizing system as described in claim 1, it is characterised in that：The resolution unit includes Video Decoder.
3. image synthesizing system as described in claim 1, it is characterised in that：The combining unit includes Video Composition module.
4. image synthesizing system as described in claim 1, it is characterised in that：The encapsulation unit include hardware encoding block and
Video Record Processor.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810665631.3A CN108566521A (en) | 2018-06-26 | 2018-06-26 | A kind of image synthesizing system for scratching picture based on natural image |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810665631.3A CN108566521A (en) | 2018-06-26 | 2018-06-26 | A kind of image synthesizing system for scratching picture based on natural image |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN108566521A true CN108566521A (en) | 2018-09-21 |
Family
ID=63554829
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201810665631.3A Pending CN108566521A (en) | 2018-06-26 | 2018-06-26 | A kind of image synthesizing system for scratching picture based on natural image |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN108566521A (en) |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109286760A (en) * | 2018-09-28 | 2019-01-29 | 上海连尚网络科技有限公司 | A kind of entertainment video production method and its terminal |
| CN111083312A (en) * | 2019-12-30 | 2020-04-28 | 北京文香信息技术有限公司 | Studio system and program video production method and device |
| CN112887583A (en) * | 2019-11-30 | 2021-06-01 | 华为技术有限公司 | Shooting method and electronic equipment |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103475826A (en) * | 2013-09-27 | 2013-12-25 | 深圳市中视典数字科技有限公司 | Video matting and synthesis method |
| CN105827998A (en) * | 2016-04-14 | 2016-08-03 | 广州市英途信息技术有限公司 | Image matting system and image matting method |
| CN106204567A (en) * | 2016-07-05 | 2016-12-07 | 华南理工大学 | A kind of natural background video matting method |
| CN106657719A (en) * | 2017-01-04 | 2017-05-10 | 海南大学 | Intelligent virtual studio system |
| CN106792246A (en) * | 2016-12-09 | 2017-05-31 | 福建星网视易信息系统有限公司 | A kind of interactive method and system of fusion type virtual scene |
| CN107920213A (en) * | 2017-11-20 | 2018-04-17 | 深圳市堇茹互动娱乐有限公司 | Image synthesizing method, terminal and computer-readable recording medium |
-
2018
- 2018-06-26 CN CN201810665631.3A patent/CN108566521A/en active Pending
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103475826A (en) * | 2013-09-27 | 2013-12-25 | 深圳市中视典数字科技有限公司 | Video matting and synthesis method |
| CN105827998A (en) * | 2016-04-14 | 2016-08-03 | 广州市英途信息技术有限公司 | Image matting system and image matting method |
| CN106204567A (en) * | 2016-07-05 | 2016-12-07 | 华南理工大学 | A kind of natural background video matting method |
| CN106792246A (en) * | 2016-12-09 | 2017-05-31 | 福建星网视易信息系统有限公司 | A kind of interactive method and system of fusion type virtual scene |
| CN106657719A (en) * | 2017-01-04 | 2017-05-10 | 海南大学 | Intelligent virtual studio system |
| CN107920213A (en) * | 2017-11-20 | 2018-04-17 | 深圳市堇茹互动娱乐有限公司 | Image synthesizing method, terminal and computer-readable recording medium |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109286760A (en) * | 2018-09-28 | 2019-01-29 | 上海连尚网络科技有限公司 | A kind of entertainment video production method and its terminal |
| CN112887583A (en) * | 2019-11-30 | 2021-06-01 | 华为技术有限公司 | Shooting method and electronic equipment |
| CN112887583B (en) * | 2019-11-30 | 2022-07-22 | 华为技术有限公司 | Shooting method and electronic equipment |
| CN111083312A (en) * | 2019-12-30 | 2020-04-28 | 北京文香信息技术有限公司 | Studio system and program video production method and device |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108848325A (en) | A kind of image synthesizing method for scratching picture based on natural image | |
| CN106789991B (en) | Multi-person interactive network live broadcast method and system based on virtual scene | |
| JP4698831B2 (en) | Image conversion and coding technology | |
| WO2018045927A1 (en) | Three-dimensional virtual technology based internet real-time interactive live broadcasting method and device | |
| CN110493630A (en) | The treating method and apparatus of virtual present special efficacy, live broadcast system | |
| CN107087123B (en) | Real-time high-definition image matting method based on cloud processing | |
| CN110475150A (en) | The rendering method and device of virtual present special efficacy, live broadcast system | |
| US10572984B2 (en) | Method for inverse tone mapping of an image with visual effects | |
| CN108566521A (en) | A kind of image synthesizing system for scratching picture based on natural image | |
| CN106303289A (en) | A kind of real object and virtual scene are merged the method for display, Apparatus and system | |
| CN109040576A (en) | The method and system of camera control and image procossing with the window based on multiframe counted for image data | |
| US20040189677A1 (en) | Remote graphical user interface support using a graphics processing unit | |
| WO2022048097A1 (en) | Single-frame picture real-time rendering method based on multiple graphics cards | |
| CN108347647B (en) | Video picture displaying method, device, television set and storage medium | |
| JP2019041389A (en) | Process and device for capturing and rendering stream of panoramic or stereoscopic image | |
| CN110248242B (en) | Image processing and live broadcasting method, device, equipment and storage medium | |
| CN106210525A (en) | For realizing camera and the method for net cast | |
| EP4036841A1 (en) | Image processing method and apparatus | |
| CN112448962B (en) | Video anti-aliasing display method and device, computer equipment and readable storage medium | |
| WO2023160234A1 (en) | Transition animation generation method, electronic device, and storage medium | |
| CN107454434A (en) | Virtual reality net cast method and video playing terminal | |
| CN112533075A (en) | Video processing method, device and system | |
| CN113297937A (en) | Image processing method, device, equipment and medium | |
| CN109658488A (en) | Accelerate the method for decoding camera shooting head video flowing in a kind of virtual reality fusion system by Programmable GPU | |
| CN107454318A (en) | Image processing method, device, mobile terminal and computer-readable recording medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20180921 |
|
| RJ01 | Rejection of invention patent application after publication |