CN106530312B - One kind being based on low-power-consumption embedded system real-time image scratching method - Google Patents
One kind being based on low-power-consumption embedded system real-time image scratching method Download PDFInfo
- Publication number
- CN106530312B CN106530312B CN201610978143.9A CN201610978143A CN106530312B CN 106530312 B CN106530312 B CN 106530312B CN 201610978143 A CN201610978143 A CN 201610978143A CN 106530312 B CN106530312 B CN 106530312B
- Authority
- CN
- China
- Prior art keywords
- foreground
- data
- point
- video
- mask
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T1/00—General purpose image data processing
- G06T1/20—Processor architectures; Processor configuration, e.g. pipelining
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10016—Video; Image sequence
Abstract
The invention discloses one kind to be based on low-power-consumption embedded system real-time image scratching method, belongs to technical field of image processing.Purpose is to provide a kind of method of blue, green curtain real-time image scratching, using ARM etc. with the system of the SoC processor+FPGA parallel processing of serial process, to solve the problems, such as to be scratched in current embedded system as performance deficiency.Method is that image pre-processing unit pre-processes foreground video source, foreground mask extraction unit by image pre-processing unit generate rgb format foreground video signal, according to the algorithm compared point by point distinguish whether be effective foreground part mask information；Video format weight encapsulation unit receives the differentiation of foreground mask extraction unit as a result, format encapsulates again；Image pre-processing unit, foreground mask extraction unit and video format weight encapsulation unit are realized that the vision signal that video format weight encapsulation unit transmits is decoded by SoC processing unit by FPGA, finally export the vision signal for completing to scratch picture.
Description
Technical field
It is especially a kind of to be based on low-power-consumption embedded system real-time image scratching the present invention relates to a kind of method that blue, green curtain scratches picture
Method belongs to technical field of image processing.
Background technique
In modern society, develop with video technique to digitlization, multimedia direction, demand of the people to visual sensory
It is continuously improved, before having shown wide application as the Video Object Extraction technology of representative using virtual studio, Campus TV Station
Scape.Virtual studio is that the character activities image of camera live shooting is carried out digitized real-time synthesis, makes personage and void
Quasi- background can synchronize variation, so that the flawless fusion of the two is realized, to obtain perfect synthesis picture.It scratches as technology
It is widely used in news hookup, the occasions such as film trick.With application system to function, reliability etc. has the embedded of dedicated custom
It completes to scratch in system and completes to scratch as having a share in integration technology with general purpose computer with existing as function.
Traditionally, real-time image scratching function uses the high-end processors in x86 to realize.Real-time image scratching system in the market,
It is all this mode without exception.And in low-power-consumption embedded system, there are no the precedents of Implementation of Embedded System.And it is real-time
It scratches as FPGA is used alone and does not use conventional processors, then there is a problem of inflexible.For example, during prospect, background show
It both may be the real-time video of local input, the video or picture of local hard drive real-time decoding from different sources, it is also possible to
The far-end video of network real-time decoding.Corresponding decoded video, the various SoC of embedded ARM, which pass through accelerator, easily to cope with,
It can the SoC of solution H.264/H.265 be in real time on the market very common；And realize that H.264/H.265 decoding is not using FPGA
Reality --- it also can not find the FPGA of built-in H.264/H.265 stone on the market now.
Complete real-time video to scratch as function, either traditional CPU of serial process or the FPGA of parallel processing, all without
Perfect result is provided separately in method.
It is typical to scratch as algorithm background: every frame foreground and background data, to be respectively put into Memory Buffer；Then sentence
Whether disconnected foreground pixel data are transparent --- and judgment principle is to judge whether R, G, B belong to " background colour " respectively, if it is right
Answer pixel that background color is taken to export, if not then foreground color being gone to export.Seem very simple, but modern processors lack hardly possible
To be competent at " simple operation " of such flood tide.Due to using pipeline system, the multiple condition judgement of every pixel enables flowing water
Line optimization is entirely ineffective；And read repeatedly prospect buffer, background buffer, write output buffer, and enable DDR2, DDR3,
DDR4 one kind memory is not available the optimal ways such as burst is read, burst is write again, therefore whole system efficiency is very low.SMPTE is fixed
The full HD video of justice, resolution ratio is 1920*1080 pixel, even if not using asm and multimedia special instruction on x86,
It cannot achieve the so big HD video of real-time processing data amount and scratch picture.
Common processor, either x86, ARM or PowerPC are all work in series modes.High-performance x86 processor
Although being capable of handling real-time image scratching, its power consumption is very big, is correspondingly embedded in the common ARM of formula system, Power PC Processor then
Substantially it cannot achieve.
Summary of the invention
Therefore, for the above-mentioned deficiency of the prior art, the present invention is all providing one kind based on low-power-consumption embedded system reality
When scratch picture method, using the system of a kind of serial general processor+FPGA parallel processing of ARM, to solve embedded system at present
The problem as performance deficiency is scratched in system.
It is provided by the invention to be based on low-power-consumption embedded system real-time image scratching method, the method are as follows:
Image pre-processing unit pre-processes foreground video source, and foreground video source is decomposed by image pre-processing unit
Parallel uncorrected data progress YCbCr is turned RGB color conversion by the parallel uncorrected data of the embedded method of synchronization；
Image pre-processing unit is generated the foreground video signal of rgb format by foreground mask extraction unit, according to point-by-point right
The algorithm of ratio distinguish whether be effective foreground part mask information；
Point-by-point comparison algorithm is specifically, SoC processing unit transmits the threshold value of a stingy picture, in blue curtain video source situation
Under, using G channel data plus threshold value as G component, using R channel data plus threshold value as R component, when channel B data are less than G
When component or channel B data are less than R component, then the pixel is as effective foreground point, and otherwise the pixel is mask prospect
Point；
Video format weight encapsulation unit receives the differentiation of foreground mask extraction unit as a result, recording the of each effective row
One there is effective foreground point and the coordinate of effective foreground point occurs with the last one, and this coordinate and mask information and prospect are regarded
Again format encapsulates frequency signal；
Image pre-processing unit, foreground mask extraction unit and video format weight encapsulation unit are realized by FPGA processor；
The vision signal that transmits of video format weight encapsulation unit is decoded by SoC processing unit, and to extracting
Each effective row front and back mark coordinate section according to mask information, distinguish each pixel be foreground data to be stored also
Background data, and to section outside effective pixel points all storage background datas, finally by complete scratch picture vision signal it is defeated
Out.
Further, in video format weight encapsulation unit by coordinate points and mask information and foreground video signal again
It is effective to be embedded into foreground video signal specifically, diminution script horizontal blanking section length by format encapsulation for coordinate points and mask information
The front end of picture point and the rear end for starting header, the row data format after encapsulation, which is followed successively by, to be terminated header (EAV), blanking zone, opens
Beginning header (SAV), valid data front and rear coordinate, mask information area, effective image.
The beneficial effects of the present invention are: the present invention provides a kind of side based on low-power-consumption embedded system real-time image scratching
Method is completed the abstraction function of foreground mask at the end FPGA, is judged at the end ARM according to mask information using the framework of FPGA+CPU
Foreground video data or background video data are stored, the point-by-point comparison algorithm used when extracting to mask is utilized well
FPGA parallel ability, and ARM only needs to handle the coordinate section of mask, largely reduces performance consumption, can be with
Accomplish real-time image scratching.Compared with prior art, method provided by the invention breaks insertion of the tradition based on serial processing mode
Formula processing chip can not be completed the limitation of real-time image scratching, and context of methods compensates in existing embedded system, by as ARM,
The SoC processor based on serial processing mode such as PowerPC works together with concurrent operation FPGA, is finally completed low-power consumption
The technological gap of embedded system real-time image scratching keeps the design of system more flexible changeable.
Detailed description of the invention
Fig. 1 is that the present invention is based on the flow diagrams of the method for low-power-consumption embedded system real-time image scratching；
Fig. 2 is row data form schematic diagram video format of the present invention encapsulates again after.
Specific embodiment
Description of specific embodiments of the present invention with reference to the accompanying drawing:
As shown in Figure 1, the present invention provides one kind based on low-power-consumption embedded system real-time image scratching method, the method are as follows:
Image pre-processing unit 2 pre-processes foreground video source 1, and image pre-processing unit 2 divides foreground video source 1
Solution is the parallel uncorrected data of the embedded method of synchronization, and parallel uncorrected data progress YCbCr is turned RGB color conversion；
Image pre-processing unit 2 is generated the foreground video signal of rgb format according to point-by-point right by foreground mask extraction unit 3
The algorithm of ratio distinguish whether be effective foreground part mask information；
Whether it is effective foreground part that video format weight encapsulation unit 4 will be distinguished good from foreground mask extraction unit 3
Mask information, first for recording each effective row effective foreground point occurs and the last one the seat of effective foreground point occurs
Mark, and this coordinate and mask information and foreground video signal again format are encapsulated；
The vision signal that video format weight encapsulation unit 4 transmits is decoded by SoC processing unit 5, and to extraction
According to mask information, distinguishing each pixel is foreground data to be stored in the front and back mark coordinate section of each effective row arrived
Or background data, and to section outside effective pixel points all storage background data, finally will complete scratch picture vision signal
Output 7.
Comparing algorithm in foreground mask extraction unit 3 point by point is the threshold value that SoC processing unit transmits a stingy picture,
In the case of blue curtain video source, G channel data is regard as G component plus threshold value, R channel data is added into threshold value as R component, when
When channel B data are less than G component or channel B data and are less than R component, then the pixel is as effective foreground point, otherwise the picture
Vegetarian refreshments is mask foreground point.
As preferred embodiment, by coordinate points and mask information and foreground video in video format weight encapsulation unit 4
Again format encapsulates signal, reduces script horizontal blanking section length, coordinate points and mask information, which are embedded into foreground video signal, to be had
It imitates the front end of picture point and starts the rear end of header, the row data format after encapsulation, which is followed successively by, terminates header (EAV) 8, blanking zone
9, start header (SAV) 10, coordinate/mask information area 11, effective image 12.
Image pre-processing unit 2, foreground mask extraction unit 3, video format weight encapsulation unit 4 realized by FPGA, due to
It can accomplish not having to prestore video frame, and not have to plug-in RAM, resource consumption is also few, can select with high performance-price ratio logic
The suitable FPGA of resource, SoC processing unit 5 are realized by ARM, are added to coordinate section and are dropped significantly to ARM operand to be processed
It is low, and can be inputted from local video, far-end network transmission, internal stationary picture etc. obtains required background video source, accomplishes
Flexible real-time image scratching.
As shown in Fig. 2, according to the format of embedded synchronous video signal, by coordinate points and being covered in each effective row format
Code information is embedded into the front end of foreground video signal effectively figure picture point and starts the rear end of header, and accordingly reduces script horizontal blanking
Section length keeps script row general tempo number constant.In the YCbCr data format that finite data bit wide is 16, the channel Y is arrived
The value of highest order is that a time high position negates, and it is that a time high position negates that similarly C-channel, which is also by the value of highest order, and doing so can be to avoid out
There is the case where header in the flag information now transmitted, and so originally can transmit 14 in one clock node of blanking zone has criterion
Will data improve the demand of arithmetic speed in order to facilitate the processing of rear end SoC, and Y is fixed as 1010 with C-channel high 4, low
The effective marker data of total transmission 8 are by four transmission effective marker data, such a clock node in video resolution
In the case where SMPTE standard 1080P, secured transmission of payload data front and rear coordinate needs two pixels, transmits the mask information of a line
242 pixels are needed, since totally 272 timeticks points are available for script blanking zone, Gu blanking zone is reduced into 30 clocks
Beat, extra 242 timeticks are used to transmit to carry flag information in a line.
Point-by-point comparison algorithm used in foreground mask extraction unit 3 mainly applies in three primary colors blue in RGB
Blue channel numerical value is big in color standard, and green channel and red channel data very little, forms biggish numerical difference in this way, and human body
The red channel numerical value of the colour of skin is big, and the numerical value in blue channel and green channel is small, and personage has very big contrast in this way under the background of blue curtain, holds
It is easily detained from being set as 20 to 50 section as threshold value when scratching in actual application, scratch as the effect come out is all well and good.Phase
Data to tri- channels RGB are all compared, the better effect that this algorithm is realized, is not needing to reach as effect to scratching
In the case where ultimate attainment, using point-by-point comparison algorithm deduct come image not only can satisfy most Man's Demands, but also can be with
Reduce the cost of product.
Since the transmission of data is compound BT1120 standard, so only needing the number using standard in SoC processing unit
The collecting work of foreground data can be completed plus the mask operation of acquisition blanking zone according to acquisition mode.
And scratch picture operation be to be carried out inside SoC processing unit, SoC processing unit can from acquisition in addition all the way
Unrestricted choice in the video file of video input signals, fixed video file or playback, picture group have been woven with bigger
Flexibility.
Picture composition is only left in work in SoC processing unit, and the work of picture composition is for each pixel list
Be for member it is reciprocity, steps are as follows:
(1) mask of 8 bit is extended into 8 bytes；
(2) foreground data and mask are carried out and are operated；
(3) inverted value of background data and mask is carried out and is operated；
(4) by the results added of upper two operations, the image value scratched is obtained.
Since operation length above is 8 byte-aligneds, and all it is the logical operation value of standard, is very suitable for SIMD
Class instruction optimizes, such as the NEON instruction of ARM.
Without any branch in the treatment process of entire algorithm, can be obtained in the instruction pipeline of ARM enough
Handle the performance of 1080P30 real-time operation.Meanwhile that the ARM CPU of multicore can also be divided parallel is more for the symmetry in structure
Road carries out piecemeal processing to the data of same frame image simultaneously.
The above is a preferred embodiment of the present invention, it is noted that for those skilled in the art
For, without departing from the principles of the present invention, it can also make several improvements and retouch, these improvements and modifications
It should be regarded as protection scope of the present invention.
Claims (4)
1. one kind is based on low-power-consumption embedded system real-time image scratching method, which is characterized in that the method is,
Image pre-processing unit pre-processes foreground video source, and foreground video source is decomposed into embedded by image pre-processing unit
Parallel uncorrected data progress YCbCr is turned RGB color conversion by the parallel uncorrected data of the method for synchronization；
Image pre-processing unit is generated the foreground video signal of rgb format by foreground mask extraction unit, according to what is compared point by point
Algorithm distinguish whether be effective foreground part mask information；
Point-by-point comparison algorithm is specifically, SoC processing unit transmits the threshold value of a stingy picture, in blue curtain video source,
Using G channel data plus threshold value as G component, using R channel data plus threshold value as R component, when channel B data are less than G points
Amount or channel B data be less than R component when, then this scratch picture the corresponding pixel of threshold value as effective foreground point, otherwise this
The corresponding pixel of threshold value of a stingy picture is mask foreground point；
Video format weight encapsulation unit receives the differentiation of foreground mask extraction unit as a result, recording first of each effective row
There is effective foreground point and the last one the coordinate of effective foreground point occurs, and this coordinate and mask information and foreground video are believed
Number again format encapsulates；
Image pre-processing unit, foreground mask extraction unit and video format weight encapsulation unit are realized by FPGA processor；
The vision signal that video format weight encapsulation unit transmits is decoded by SoC processing unit, and every to what is extracted
According to mask information, distinguishing each pixel is foreground data to be stored or back in the front and back mark coordinate section of one effective row
Scape data, and to section outside effective pixel points all storage background data, finally will complete scratch picture vision signal output.
2. being based on low-power-consumption embedded system real-time image scratching method as described in claim 1, which is characterized in that the video lattice
Coordinate points and mask information are encapsulated with foreground video signal again format in formula weight encapsulation unit and are disappeared specifically, reducing row originally
Hidden section length, after being embedded into the front end of foreground video signal effectively figure picture point by coordinate points and mask information and start header
End, the row data format after encapsulation, which is followed successively by, to be terminated header EAV, blanking zone, starts header SAV, valid data front and rear coordinate, covers
The code information area, effective image.
3. being based on low-power-consumption embedded system real-time image scratching method as described in claim 1, which is characterized in that at the SoC
It manages unit to complete to scratch as in, the processing step of the work of picture composition for each pixel unit are as follows:
The mask of 8 bi t is extended to 8 bytes by step 1；
Step 2 foreground data and mask are carried out and are operated；
The inverted value of step 3 background data and mask carries out and operation；
The results added of step 2 and step 3 operation is obtained the image value scratched by step 4.
4. being based on low-power-consumption embedded system real-time image scratching method as described in claim 1, which is characterized in that described point-by-point right
Than in algorithm, the threshold value value for the stingy picture that SoC processing unit transmits is 20 to 50.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201610978143.9A CN106530312B (en) | 2016-11-05 | 2016-11-05 | One kind being based on low-power-consumption embedded system real-time image scratching method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201610978143.9A CN106530312B (en) | 2016-11-05 | 2016-11-05 | One kind being based on low-power-consumption embedded system real-time image scratching method |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN106530312A CN106530312A (en) | 2017-03-22 |
| CN106530312B true CN106530312B (en) | 2019-03-12 |
Family
ID=58349671
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201610978143.9A Active CN106530312B (en) | 2016-11-05 | 2016-11-05 | One kind being based on low-power-consumption embedded system real-time image scratching method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN106530312B (en) |
Families Citing this family (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110390225A (en) * | 2018-04-16 | 2019-10-29 | 广州番禺职业技术学院 | Automobile video frequency inspection system |
| CN108616724A (en) * | 2018-05-08 | 2018-10-02 | 南京齿贝犀科技有限公司 | A kind of radio image collecting transmission platform based on FPGA and ARM structures |
| CN111563862B (en) * | 2020-07-16 | 2020-10-27 | 北京文香信息技术有限公司 | Method, device, storage medium and equipment for constructing virtual video based on FPGA |
| CN113766147B (en) * | 2020-09-22 | 2022-11-08 | 北京沃东天骏信息技术有限公司 | Method for embedding image in video, and method and device for acquiring plane prediction model |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN202261667U (en) * | 2011-08-29 | 2012-05-30 | 三缘汇科技(天津)有限公司 | Interactive entertainment set top box device for KTV and family |
| CN103137094A (en) * | 2012-12-04 | 2013-06-05 | 江苏清投视讯科技有限公司 | Split joint image processor used for image matting |
| CN104469089A (en) * | 2014-12-23 | 2015-03-25 | 山东建筑大学 | Multimedia interaction teaching system and teaching method |
| CN105184787A (en) * | 2015-08-31 | 2015-12-23 | 广州市幸福网络技术有限公司 | Identification camera capable of automatically carrying out portrait cutout and method thereof |
Family Cites Families (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US9305357B2 (en) * | 2011-11-07 | 2016-04-05 | General Electric Company | Automatic surveillance video matting using a shape prior |
| EP3046073A1 (en) * | 2015-01-14 | 2016-07-20 | Thomson Licensing | Method and apparatus for color correction in an alpha matting process |
-
2016
- 2016-11-05 CN CN201610978143.9A patent/CN106530312B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN202261667U (en) * | 2011-08-29 | 2012-05-30 | 三缘汇科技(天津)有限公司 | Interactive entertainment set top box device for KTV and family |
| CN103137094A (en) * | 2012-12-04 | 2013-06-05 | 江苏清投视讯科技有限公司 | Split joint image processor used for image matting |
| CN104469089A (en) * | 2014-12-23 | 2015-03-25 | 山东建筑大学 | Multimedia interaction teaching system and teaching method |
| CN105184787A (en) * | 2015-08-31 | 2015-12-23 | 广州市幸福网络技术有限公司 | Identification camera capable of automatically carrying out portrait cutout and method thereof |
Non-Patent Citations (2)
| Title |
|---|
| 交互式前景抠图技术综述;沈洋等;《计算机辅助设计与图形学学报》;20140430;第26卷(第4期);全文 * |
| 基于DSP抠像处理的KTV视频合成机顶盒设计与开发;韩柳等;《天津职业技术师范大学学报》;20111231;第21卷(第4期);全文 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN106530312A (en) | 2017-03-22 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN106530312B (en) | One kind being based on low-power-consumption embedded system real-time image scratching method | |
| KR101931466B1 (en) | Image tone adjustment using local tone curve computation | |
| CN108965740B (en) | Real-time video face changing method, device, equipment and storage medium | |
| US6226015B1 (en) | Method of automatically producing sketches and cartoon images from movies | |
| CN107087123B (en) | Real-time high-definition image matting method based on cloud processing | |
| CN103475826A (en) | Video matting and synthesis method | |
| US20110242113A1 (en) | Method And System For Processing Pixels Utilizing Scoreboarding | |
| CN104767956A (en) | Video processing with multiple graphical processing units | |
| CN106303289A (en) | A kind of real object and virtual scene are merged the method for display, Apparatus and system | |
| US20060181547A1 (en) | Method and system for image editing in a mobile multimedia processor | |
| CN109688382B (en) | Underwater image processing system of underwater robot | |
| US10861243B1 (en) | Context-sensitive augmented reality | |
| CN106506950A (en) | A kind of image processing method and device | |
| CN101867727A (en) | Method and device for processing video | |
| CN106899878A (en) | A kind of adjustable video and graph compound method and system of transparency based on OMAP chips | |
| CN105554416A (en) | FPGA (Field Programmable Gate Array)-based high-definition video fade-in and fade-out processing system and method | |
| CN109658488B (en) | Method for accelerating decoding of camera video stream through programmable GPU in virtual-real fusion system | |
| CN103413331B (en) | A kind of support edits the high resolution video image content sparse expression method propagated | |
| CN108307163B (en) | Image processing method and device, computer installation and readable storage medium storing program for executing | |
| CN108093175A (en) | A kind of adaptive defogging method of real-time high-definition video and device | |
| CN103516998A (en) | Station caption superposition method for analog video | |
| CN208424534U (en) | The cartridge assemblies and imager of imager | |
| CN105554485A (en) | Imaging method, imaging device and electronic device | |
| JP2008505556A (en) | Television visual artwork generation system and method | |
| CN105321153A (en) | Video monitor low-illumination image color restoration method and device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |