WO2015085637A1

WO2015085637A1 - Method for supplementarily drawing content at station logo region in video

Info

Publication number: WO2015085637A1
Application number: PCT/CN2013/090978
Authority: WO
Inventors: 张磊; 肖煜东; 索津莉; 张永兵; 戴琼海
Original assignee: 清华大学深圳研究生院
Priority date: 2013-12-09
Filing date: 2013-12-30
Publication date: 2015-06-18
Also published as: CN103618905A

Abstract

Disclosed is a method for supplementarily drawing content at a station logo region in a video. The method comprises the following steps: 1) detecting a station logo region in a video, and marking the station logo region; and 2) performing the following supplementary drawing operations on each image in the video: 21) estimating, according to front L images and rear K images, (L+K) global reference images of a current image by using a global motion estimation method; 22) marking a foreground and a background of the station logo region in the current image; 23) supplementarily drawing a station logo region of the current image; 24) determining whether the station logo region of the current image is completely drawn, if so, entering step 26), otherwise, entering step 25); and 25) supplementarily drawing the rest undrawn region in the station logo region by using spatial correlation of the current image; and 26) ending the supplementary drawing of the current image. The method for supplementarily drawing the content at a station logo region in a video in the present invention can effectively improve the accuracy of supplementarily drawing content at the station logo region in the video by fully using time information between adjacent images.

Description

Method for compiling the content of the logo area in the video

【Technical field 】

The present invention relates to the field of computer video processing, and in particular to a method for complementing a logo area in a video.

【 Background technique 】

As a statement of intellectual property ownership, almost all commercial TV channels have their own logo. When the new logo is overlaid with the original logo, the logo can also be used as a sign of authorized broadcast. At the same time, however, the presence of these logos both reduces the viewing comfort of the video and increases the difficulty of communication between the videos. Therefore, how to remove the logo and accurately retouch the logo area is a key issue to be solved.

In order to remove the logo in the video and accurately compensate the logo area, the current common method is to perform mosaic processing in the logo area or use the image processing tool to blur the logo area, but these methods have poor effect. The artificial processing traces have seriously affected the video quality and video viewing effects.

[Content of the Invention]

The technical problem to be solved by the present invention is: to make up for the deficiencies of the above prior art, and to propose a content complementing method for the logo area in the video, which effectively improves the complementing accuracy of the video logo area and has low computational complexity.

The technical problem of the present invention is solved by the following technical solution - a method for compiling a content of a logo area in a video, comprising the following steps: 1) detecting a logo area in a video and marking a logo area; 2) video The image of each frame in the frame is as follows: 21) According to the pre-L frame and the K-frame image, the global motion estimation method is used to estimate the L+K. global reference image of the current frame image; wherein, L and K are both An integer greater than or equal to 0, and L and K are not 0 at the same time. The specific value is set by the 3⁄4 household according to the retouching accuracy requirement; 22) marking the foreground and background of the logo area in the current frame image; 23) Drawing the logo area of the current frame image: for the background pixel, using the information of the corresponding pixel in the global reference image to complement the background pixel; for the foreground pixel, using the local estimation based on the front L frame and the back K frame image The method estimates L+K local reference images of the current frame image, and complements the foreground pixel points by using information of corresponding pixel points in the L÷K local reference images; 24) determining the logo area of the current frame image Whether the field is completely completed, if yes, proceed to step 26); if not, the agent proceeds to step 25) _; 25) using the spatial correlation of the current frame image to complement the remaining unpainted areas in the logo area; 26) Ends the interpolation of the current frame image.

The beneficial effects of the present invention compared to the prior art are: The content complementing method of the logo area in the video of the present invention uses the global motion estimation method to estimate the current frame image from the L+K frame domain image before and after the time, and uses the pixel point information in the domain image to view the current frame image. The front spot of the logo area is different from the background point. After using the time correlation re-painting, if there is an unpainted area, continue to use spatial correlation to supplement the re-painting. In the process of composing, the front spot and the background point are complementarily painted, and a large amount of redundant information existing in the video image frame sequence is fully utilized, thereby effectively improving the complementing accuracy of the video logo area. For each frame of image, because it mainly performs motion tree segmentation on the video image sequence, the known image information in the neighborhood image is used to draw the logo area in the current frame, so the number of iterations in the process of composing Less, ensuring less computational complexity.

I 酎 diagram description 1

1 is a flowchart of a method for compiling a content of a logo area in a video according to an embodiment of the present invention; FIG. 2 is a diagram illustrating a method for constructing a domain block by using Κ=3 as an example in the complementary drawing method according to an embodiment of the present invention; Process schematic.

【 detailed description 】

The present invention will be further described in detail below in conjunction with the specific embodiments and with reference to the accompanying drawings.

The idea of the present invention is: researching the temporal correlation between video image sequences by global motion estimation and local motion estimation, and preferentially using the temporal correlation of the video image sequence to complement the logo area in the current image frame, when utilizing time When there is still an area that has not been redrawn after the correlation is added, the spatial correlation of the current ^ image is used for the port painting. When using time information, firstly, global motion estimation is performed on adjacent 3⁄4 L+K frame images, and the L+ frame neighborhood image is aligned with the current frame image to obtain a L+K frame global reference image. Then, the 'empty neighborhood data of each pixel to be complemented in the logo area is analyzed, and the front point and the background point in the current image frame logo area are marked. Finally, for the unknown background pixel, the information in the aligned image frame of the global motion compensation is used to complement the image; for the unknown foreground pixel, the local motion is compensated and then combined with the information in the neighborhood image frame. Make a repaint.

As shown in the figure, it is a flowchart of a method for compiling the content of the logo area in the video in the specific embodiment. The input video image processed by the complement drawing method is visible in the logo area, and the video data format is not limited. The supplementary drawing method includes the following steps -

U1): Detects the logo area in the video and marks the logo area.

In this step, the detection method may select a plurality of methods for detecting the video logo area, for example, may include algorithm detection and user manual tick identification, and mark the logo area. The method of automatic detection by algorithm can be used but not It is limited to the frame difference based station detection and extraction algorithm proposed by Kartrin Meisinger et al. By manually checking the user's method, the input parameters indicating the logo area can be prepared in advance or the user operation interface can be provided for the user to mark the logo area. At this time, when detecting, the station logo area of the user check mark is directly received to complete the detection. After the mark is detected, the station mark area can be identified. The easiest way is to mark the pixels in the logo area as 0, which is black. Or, the uniform identifier is 255, which is rendered white. Other identification methods are also available, and are not limited to the above two.

U2) Perform the following supplementary operations on each frame image in the video, specifically as step U21) - U26 in Figure 1.

U21) According to the first L frame and the last K frame image, the global motion estimation if method is used to estimate the L+K global reference images of the current frame image. Among them, K is a positive integer, which is set by 3⁄4 households according to the accuracy requirements.

Global motion refers to motion in a sequence of video images resulting from the motion of the camera. The global motion estimation is to estimate the spatial displacement between the pixel points in the current frame and the corresponding pixel points in the front and back domain frames in the current sequence, usually defined by the motion parameter model. In the global motion estimation method, after the motion parameter model is defined, the current frame image may be estimated from the domain frame image to obtain an estimated image of the current frame; or the adjacent frame image may be estimated by the current frame image to obtain an adjacent frame. The image of the image is estimated. For example, let X ¹ denote the coordinates of the pixel points in the video image frame at time i, ^ denote the coordinates of the pixel points in the video image frame at time j, and represent the motion parameter model required when aligning the image frame at time j with the i-time image frame. Then, the operation of estimating the image of the image frame at time i from the image frame at time j is χ', that is, aligning the image frame at time j to the image at time i.

In this step, after the user sets the L and K values, for example, both L and K are 3, that is, the current frame image is estimated from the three frames of images before and after, and six global reference images are obtained. If you find that the offset accuracy is not satisfactory, you can return to reset the values of L and K to a larger value, thus improving the precision of the fill. Of course, L and K can be taken as other values, and L can be taken as 0, and only the information of the K frame image is used; or K is 0, and only the information of the previous L frame image is used, and the specific value combination is not limited, as long as there is a phase The information of the adjacent frame can be. In this step, the L window is selected before and after the L-frame, and the alignment of the video image frame in the time window can be prepared for the use of the inter-correlation of the video sequence in the subsequent steps.

U22) Marks the foreground and background of the logo area in the current frame image.

In this step, the normal foreground background marking method can be used to process the logo area of the current frame image. Preferably, the information of the L+K global reference images obtained in step U21) can be used to perform the foreground background marking, so that the reference image of the response time correlation information is obtained by global motion estimation, and the foreground background can be improved. The accuracy of the markup, which in turn improves the accuracy of subsequent fills. Specifically, the method includes the step U221) - U223) _: U221) constructing a domain block set 像素 of the pixel points for each pixel point I (x, y) in the logo area in the current frame image, the domain block set ψ including the current frame image and L+K respectively L+K+1 sub-blocks ψο, sub-block 取出 taken out from the global reference image. = { ϊ ( i, j ) I i≡ (x- W, x+W), j fc≡ (y W, y+W) }, where W is the spatial window size, which is based on the accuracy of the ffi The calculation of the amount of calculation is required.

In this step, as shown in Fig. 2, the case where the L and K are both 3 is taken as an example to illustrate the process of constructing the domain block set ψ. That is, six sub-blocks are taken out from the six global reference pictures R1 to R6, and one sub-block is taken out from the actual current frame image C, and a total of seven sub-blocks constitute a set 像素. Each sub-block is a square having a length and a width of 2 W which are centered at the pixel point I x, y).

U222) Calculate the variance of the pixel points in the field block set of each pixel.

As described above, the set ψ is a set of pixels of L+K+1 sub-blocks, so this step calculates the variance of the pixel values of the pixels in the set.

U223) setting a threshold, comparing the threshold to the variance corresponding to each pixel point calculated in step U222), if the variance is less than the threshold, marking the pixel as the background; if the variance is greater than the threshold, marking the pixel as the foreground .

Since the result of the reaction between the global motion estimation in step U21) is used, the motion displacement of the pixel belonging to the background area of the video is small, so the variance of the pixel value of the pixel in the domain block set is relatively Smaller; conversely, for a pixel belonging to the foreground area of the video, since the foreground has local motion, the motion displacement is large, so the variance of the pixel values of the pixel points in the domain block set is relatively large. In this step, a threshold is set, and the threshold is set by the user, and can be set as an empirical value according to the compensation accuracy requirement. By comparing the threshold value with the variance of the domain block set _Ψ corresponding to each pixel point, it is possible to distinguish whether each pixel point belongs to the foreground or the background, and complete the foreground-background mark of the pixel area of the station frame of the current frame video image.

U23) Complement the logo area of the current frame image. Specifically, for the background pixel, the background pixel is complemented by the information of the corresponding pixel in the L+K global reference images; and for the foreground pixel, the local estimation method is used according to the pre-L frame and the K-frame image. The L+K. local reference image of the current frame image is complemented by the information of the corresponding pixel in the L+K local reference images.

In this step, for the pixel points of the logo area, the background and the foreground are respectively complemented. For the background pixel of the logo area in the current frame image, the information in the already-aligned neighborhood image, that is, the information of the reference image is directly used for the complement. For the foreground pixel of the logo area in the current frame image, the local motion compensation is introduced to compensate the foreground pixel according to the information of the local reference image obtained after the local motion estimation. Since the first L frame and the last K frame are taken as the time window, the L÷K in the time window can be referred to the fishing image. When L÷K global reference images are used to complement background pixels or L+K local reference images to complement foreground pixels, there are various re-painting methods. Here are two ways to complement:

One is to use the average value of the pixel values of the corresponding pixel points in the L+K reference images as the pixel value of the pixel to be complemented, for example, to complement the pixel point I in the current frame image (X , y), take the corresponding pixel point I ( X, ν) in the L + K reference images, and use the average value of the pixel values of the L + K pixel points as the current intra-frame pixel point I ( X, y) Pixel value

The other is to first calculate the error between the IK reference image and the current frame image, and use the pixel value of the corresponding pixel in the reference image with the smallest error as the pixel value of the pixel to be painted. When calculating the error, there are a variety of ways, one way, i.e., in the current image is to be taken inpainting pixels I (X, y) pixel points around Ν, denoted as _{I k (x, y),} k = l, 2, ..., Ν; corresponding to the pixel corresponding to the pixel in the reference image, denoted as ' _k (x, y), k == l, 2, .. , N; Then calculate the square root of the difference of the pixel values of the corresponding pixel, and ί| / _Α (χ, _ν) / 'xj)) ² as the error value of the reference map 。. Of course, other ways of defining the error value may also be selected. The above examples are merely illustrative. The error value of the L÷K reference image is calculated in turn, and the error value is compared to obtain a reference image with the smallest error value, and the information in the neighborhood image is minimized by using the information in the neighborhood image with the smallest error value.

U24) It is judged whether the logo area of the current frame image is completely filled, if yes, the process proceeds to step U26); if not, the process proceeds to step U25).

U25) Fill in the remaining unpainted areas in the logo area with the spatial correlation of the current frame image. U26) Ends the interpolation of the current frame image. If it is judged by the step U24) that the contents of the logo area in the video have been complemented, the processing proceeds to step U26) to end the complement of the current frame image. Enter the process of drawing the next frame of image. However, the re-painting process in step U23) mainly utilizes the temporal correlation between video frames. If the motion in the video is very slow, some of the pixels in the logo area may not be able to find the corresponding pixel information from the adjacent video image frame. This step U24) judges that there is a pixel that is not recompressed, so That is, proceed to step U25) to perform supplementary re-painting, and mainly use the spatial correlation of the current frame image to make a supplementary drawing.

When using spatial correlation for re-painting, it is commonly used to use the static image restoration algorithm for re-painting, including digital repair algorithm based on partial differential equations and block-based matching algorithm based on the sample. Block matching algorithm based on sample feeding When performing spatial correlation compensation, specifically include the T step -

251) Determine the remaining unpainted areas.

252) Extract the boundary of the area to be complemented.

253) Calculate the block matching priority for each pixel on the boundary, and select the pixel point with the largest priority value as the center point of the current to-be-drawn block.

254) detecting a matching block that is closest to the current to-be-painted block in the re-mapped area.

255) using the information of each pixel in the matching block to complement each pixel in the current to-be-painted block. In the current block, the pixel point II is filled with the information of the corresponding pixel point II' in the matching block, and the pixel point 12 in the current block is complemented by the information of the corresponding pixel point 12' in the matching block, and then the analogy is correspondingly plotted.

256) Return to step 251) and end until there is no unpainted area.

After the above steps U21) - U26), the complement of the logo area in the current frame image is completed. Repeat step U21) - U26) to complete the reconstruction of each frame image in the video, thus completing the task of removing the logo after the logo is removed.

In this embodiment, the content and the foreground of the content of the logo area in the video are respectively to be drawn, and the time information carried in the image of the domain frame is fully utilized, so that a large amount of redundant information is fully used in the process of composing Therefore, the precision of the complement of the video logo area is effectively improved, and the logo area is reconstructed with high precision. When the time information cannot be completely rewritten, the supplement is spatially correlated and drawn to ensure that the entire logo area is complemented. In addition, for each frame of image, because it mainly uses the known image information of the domain image inversion to perform motion analysis on the video image sequence to complement the logo area in the current frame, the number of iterations in the process of composing Less, make sure you don't add more computational complexity.

Preferably, the re-painting method of the embodiment further includes a step U3) (not illustrated in FIG. 1): integrating all the inaccurate images of all the re-paintings in time sequence to remove the logo area and extracting Video. That is, through step U3), each image frame is integrated into a video, that is, the 3⁄4 video is obtained after the station logo is removed and the station logo area is reconstructed and reconstructed.

The above is a further detail of the present invention in combination with the specific preferred embodiments. It is not to be understood that the specific embodiments of the present invention are limited to the description. It will be apparent to those skilled in the art that the present invention may be practiced without departing from the spirit and scope of the invention.

Claims

L - a method for compiling the content of the logo area in the video, characterized in that: the following steps are included -

1) Detect the logo area in the video and mark the logo area;

2) Perform the following supplementary image operations on each frame image in the video - 21) According to the front L frame and the back K frame image, the global motion estimation method is used to estimate the L+K global reference images of the current frame image; wherein, L and K is an integer greater than or equal to 0, and L and K are not 0 at the same time, and the specific value is set by the user according to the requirements of the supplementary drawing accuracy -

22) Mark the foreground and background of the logo area in the current frame image -

23) Complement the station label area of the current frame image; for the background pixel point, use the information of the corresponding pixel point in the L÷K global reference images to complement the background pixel point; for the foreground pixel point, according to the front L frame, the back K The frame image is estimated by using a local estimation method to obtain L÷K local reference images of the current frame image, and the foreground pixel points are complemented by information of corresponding pixels in the L+K local reference images;

24) judging whether the logo area of the current frame image is full, if yes, proceed to step 26); if not, proceed to step 25);

25) using the spatial correlation of the current frame image to complement the remaining unpainted areas in the logo area;

26) End the redraw of the current frame image.

The method for compiling the content of the logo area in the video according to claim 1, wherein: in step 22), the information of the L+K global reference images is used to mark the logo area in the current frame image. Prospect and back

The content complementing method of the logo area in the video according to claim 2, wherein: the step 22) is specifically; 221) each pixel point in the logo area in the current frame image I (x, y), a domain block set 构建 constructing a pixel point, the domain block set ψ including L+KH sub-blocks ψ ₀ , sub-block ψ ₀ {from the current frame image and the K global reference images respectively ΐ (ΐ, j ) ii≡ (xW _? x+W), je (yW, y+W) }, where W is the size of the spatial window, which is set by the user according to the requirements of drawing precision and calculation amount; 222) Calculating a variance of the pixel points in the domain block set 各 of each pixel point; 223) setting a threshold value, comparing the threshold value to the variance corresponding to each pixel point obtained in step 222), if the variance is less than the threshold value, then the pixel point Mark as background; if the variance is greater than the threshold, mark the pixel as foreground.

The method for compiling the content of the logo area in the video according to claim 1, wherein: in the step 23), the corresponding one of the L+K global reference images or the L+K local reference images is used. When the pixel point information is used to fill the pixel point, the average value of the pixel value of the corresponding pixel point in the L+K reference image is used as the pixel value of the pixel to be complemented.

The method for compiling a content of a logo area in a video according to claim 1, wherein: in the step 23), the L+K global reference images or the L+K local reference images are included in the method. When the corresponding pixel point information is used to complement the pixel point, the error of the L+K reference image and the current frame image is first calculated, and the pixel value of the corresponding pixel point in the reference image with the smallest error is taken as the pixel value of the current pixel to be complemented. Do the painting.

The method for compiling a content of a logo area in a video according to claim 1, wherein: the step 25) is performed by using a still image repair algorithm.

The method for composing a logo area in a video according to claim 1, wherein: the static image repair algorithm comprises a digital repair algorithm based on a partial differential equation, and a block matching algorithm based on a sample.

The method for composing the content of the logo area in the video according to claim 7, wherein: when the step 25) adopts the block matching algorithm based on the sample, the method specifically includes the following steps: 251) determining remaining 252) extracting the boundary of the area to be complemented; 253) calculating the block matching priority for each pixel on the boundary, and selecting the pixel with the highest priority value as the center point of the current to-be-composed block 254) detecting a matching block that is closest to the current to-be-drawn block in the already-painted area; 255) using the information of each pixel in the matching block to perform a simple drawing on each pixel in the current to-be-composed block; 256) Returning to step 251), it ends when there is no unpainted area.

The method for compiling the content of the logo area in the video according to claim 1, further comprising the step 3): integrating all the frames of the completion of the re-painting in time sequence, and outputting the logo removed. The video obtained after the area is complemented.