KR20010025404A - System and Method for Virtual Advertisement Insertion Using Camera Motion Analysis - Google Patents

Abstract

PURPOSE: An apparatus and a method for inserting a virtual advertisement using an analysis of a camera operation are provided to enhance effects of an advertisement by reading an image screen by a frame unit and securing an advertisement inserting position and inserting the advertisement into the position. CONSTITUTION: A moving picture shown in a sports ground(110) is photographed in real time and an image signal transmitted from a digital camera(10) is received and an area for advertising the image is searched(50). A movement of the camera(10) is chased(60). An inserting module(65) is divided into an object image transformation mode(70), an obstacle detecting module(90) and an inserting mode(80). If a video frame is inserted, the area for advertising the image is searched(50). If the area is selected, a movement of the camera is forecasted by comparing to the prior frame(60). The virtual advertisement image to be inserted is selected from the pre-set advertisement image database(100) and the image is transformed(70). An obstacle positioned at the inserted portion is detected and an object occlusion mask is prepared(90). The virtual advertisement is inserted in the selected position by using a mask as a key(80).

Description

System and Method for Virtual Advertisement Insertion Using Camera Motion Analysis}

The present invention relates to an apparatus and a method for inserting a virtual advertisement into a broadcast relay video screen, and reads the video image screen of the broadcast broadcast in frame units to secure the insertion position of the advertisement by a template method and then artificially When the insertion position of the advertisement is first found, the insertion position of the advertisement is analyzed by analyzing camera motions between successive video frames from the next input frame. The present invention relates to an apparatus for inserting an advertisement in a broadcast relay video screen and a control method thereof, which can easily insert an advertisement signal at a desired position on a dynamically developed video image screen.

In general, a broadcast receiving medium such as a television, such as a television, receives a real-time information as a broadcast signal to the viewers to solve the needs of the viewers to meet a variety of information, in particular, such broadcast media By providing the same video to viewers in real time, they provide viewers with live sports information.

By the way, the sporting events such as the above is focused on people, so in order to utilize such a product as an advertisement of the advertisement board is installed on the bulletin board of the stadium or the wall around the stadium. Recently, a system that replaces an advertisement displayed directly around a stadium with an arbitrary advertisement by manipulating a portion thereof during sports broadcasting is widely used.

One such invention is disclosed in US Pat. Nos. 5,892,554 (190) and 5,436,672 (260), each of which relates to advertisement insertion and advertisement replacement. In addition, Korean Patent No. 2000-29844 discloses a method of automatically finding and inserting an insertion position of a virtual advertisement using a template structure.

Then, the advertisement insertion and replacement system of the conventional broadcast media will be described with reference to FIGS. 2A and 2B.

First, referring to the existing advertisement insertion system 190, the video signal of the camera 10 transmitted as a digital signal by taking a moving picture in real time in the stadium 110 is input and blurred using a Gaussian filter (not shown). A filter / decimation module 130 for tracking the advertisement insertion area by dividing the video signal into small images, and a landmark on the input video screen of the filter / decimation module 130. A search module unit 140 for finding a reference array by finding points or lines that may be characterized around the insertion of an advertisement, and a reference array of landmarks of an image input from the search module unit 140. And the landmark tracking module unit 150 to find a portion to insert an advertisement by comparing the reference array with respect to the reference image, and a portion to insert the advertisement of the landmark tracking module unit 150, for example. The overlap determination module unit 160 which grasps the position of an obstacle such as a ball or a ball, and the insertion module unit 170 which inserts an advertisement image at an advertisement insertion position by referring to the determination information of the overlap determination module 160. Include.

Meanwhile, referring to the operation of the conventional broadcast media advertisement insertion system as described above, first, the camera 10 installed in the stadium 110 converts a digital signal by taking a real-time moving picture, for example, a soccer relay screen, and then advertises. When inputted to the filter / decimation module unit 130 of the insertion system 190, the filter / decimation module unit 130 blurs the input digital image using a Gaussian filter. Then, the filter / decimation module unit 130 produces four blurred image signals as images each having 1/2 resolution. This is so that it takes very little time to find an area to insert the advertisement in the low resolution image. Therefore, the filter / decimation module unit 130 finds an area to insert an advertisement in the image having a small resolution, and then further finds an area to be inserted in the original image having the highest resolution, and then the search module unit 140 inputs the same. A reference array is created by finding points or lines that can be characterized around the insertion of advertisements called landmarks in the image, and four reference arrays are created according to the resolution. In addition, when the reference signal is generated for the landmark of the input image, the landmark tracking module unit 150 may generate a reference signal for the reference image. The comparison will find the part where the ad will be inserted. In this case, the overlap determination module 160 detects the position of the obstacle on the current video screen and inputs it to the insertion module unit 170 because the advertisement insertion part is covered if there is an obstacle such as a person or a ball in the portion to insert the advertisement. Then, the insertion module unit 170 finds a part to insert an advertisement, and then, if there are no overlapping obstacles by referring to the obstacle information on the part to be inserted, inputted from the overlap determination module unit 170, the preset advertisement image is displayed. Insert it.

Therefore, after inserting an advertisement into the digital video screen of the moving image transmitted through the camera 10 through the above process, the image is converted back to the NTSC format video signal 180 and transmitted to the broadcasting station. The broadcasting station then transmits the moving picture information, for example, the soccer game screen, into which the new inserted advertising signal is inserted, to each television system.

The advertisement replacement system 260 is composed of four components: a finder 220, a global motion tracker 270, an updater 230, a replacer 280, and the like. The finder module 220 finds a position of an advertisement to be replaced, and uses point pattern matching. Before replacing the advertisement, it analyzes the corner point of the advertisement image to be replaced, stores it in the system, and when the video input comes in, finds the corner point and finds the affine part among them. This method can be found even if a part of the advertisement is not visible and can be found at any angle, but it is expensive to calculate and relies on corner points. The Global Motion Tracker module 270 is a part that predicts the movement of the camera. There is an advertisement video to be replaced but no prediction is made every frame. The updater module 230 communicates video memories. When the scene change or advertisement image is found, the updater module 230 informs the position of the previous video frame. This information is exchanged because the billboard to be replaced must be replaced from when it slowly appears at the edge of the screen. Finally, the replacer module 280 replaces the advertisement. Object overlap is detected and replaced using the difference image between the insertion region and the original image to be replaced.

In this way, the system 260 is modular, and replaces the advertisement video in parallel processing. In addition, it is divided into an online module and an offline module. In the offline module, the information of the advertisement image to be replaced and the advertisement image to be replaced must be processed in advance, and information on the point pattern has to be collected, which takes a lot of time.

In Korean Patent No. 2000-29844, a method of automatically finding an insertion position of a virtual advertisement has been introduced. The insertion region is found using a template structure. As a more improved method, the present invention has invented an improved method for finding insertion positions in various stadiums.

In particular, the present invention finds the motion information of the camera in the image without receiving the camera's motion information directly from the camera with the memory head, that is, the conventional technology for analyzing the motion of the camera included in the video data. A method of generating and analyzing a spatiotemporal image from input video data, a method of using a histogram of optical flow calculated from adjacent images of a moving image, and a transformation model by calculating a correlation between adjacent images. There is a method of analyzing the movement of the camera included in the video by substituting for. The method of using the space-time image and the method of analyzing the camera movement using the histogram of the light flow can obtain only the direction information of the camera movement, and the motion size information cannot be known in the direction of the movement. In the method of using the correspondence information and the deformation model between adjacent images, the direction information and the size information of the camera movement can be obtained, but are directly affected by the few noises included in the input data. In order to reduce the influence of such noise, a method using a plurality of feature points is used. The method of using these multiple feature points has a problem of requiring high computational cost.

However, since the advertisement insertion system 190 and the advertisement replacement system 260 of the conventional broadcast medium, such as the video screen transmitted in real time through the camera 10 is dynamically developed, the desired advertising signal at the desired position on the video screen Could not be easily inserted. In addition, the conventional advertisement insertion and replacement system as described above has a hardware board for each module of the system, so the system is very complicated accordingly, and the signal processing operation for advertisement insertion / replacement has to be executed through several steps. There was a problem that the screen processing is not easy. In addition, in the case of the advertisement insertion system 190, the reference image and the landmark are used, but when the overall contrast of the background is changed when the reference image is used, there is a great difficulty in detecting the object. In order to solve this problem, the contrast value of the background is updated by using a landmark, which also does not have a great effect unless a certain number of landmarks are detected.

Accordingly, the present invention has been invented to solve the above problems, and has invented a method of effectively separating objects from an image and a method of accurately extracting camera movement from an image without using a camera with a memory head. . By reading the video screen of the broadcast broadcast frame-by-frame, by using the template structure to secure the insertion position of the advertisement, and artificially inserting the preset advertisement in the position, the desired position on the video screen that the video screen is dynamically developed It is an object of the present invention to provide an apparatus and method for inserting an advertisement on a broadcast relay video screen which can easily insert an advertisement signal into the broadcast relay video screen.

Still another object of the present invention is to insert an advertisement into a broadcast video screen that can maximize the effect of advertising through the broadcast media because the advertising signal is inserted into the appropriate position of the dynamic video screen to always watch the advertisement. And a method.

The present invention for achieving the above object is a portion 50 for determining the insertion position of the virtual advertisement using the template search window structure function on the video signal input from the camera photographing from the stadium in real time, and the search module ( An apparatus for inserting an advertisement on a broadcast relay video screen includes a portion 60 for tracking the insertion position set in each frame 50 and an insertion portion 65 for inserting a pre-selected virtual advertisement at the insertion position.

Another feature of the present invention is to advertise on the broadcast relay video screen including the step of effectively inserting only in the background avoiding obstacles such as players or balls in the insertion area and the camera motion tracking step of determining the camera movement between two consecutive images It provides a control method of the device to insert.

1 is a block diagram of a system of the present invention.

Figure 2a is a block diagram illustrating a conventional advertising insertion system.

Figure 2b is a block diagram illustrating a conventional advertising replacement system.

3A is a diagram showing a feature point distribution around a center circle of a soccer field and a template structure used therein;

3B is a diagram showing a feature point distribution around a tennis court and a template structure used therein.

4 is a flowchart illustrating a method of finding a center circle of a soccer field.

5A illustrates a method for obtaining perspective deformation parameters of an image, and illustrates a method when the insertion position is in the center of the screen.

5B illustrates a method for obtaining perspective deformation parameters of an image, illustrating a method when the insertion position is on the right side of the screen.

5A illustrates a method for obtaining perspective deformation parameters of an image, and illustrates a method when the insertion position is on the left side of the screen.

6 is a flowchart illustrating a method of detecting an obstacle.

7 is a block diagram illustrating an elliptic object tracking process according to the present invention.

8A illustrates an input image including an ellipse object.

8B illustrates preprocessing area selection in an input image including an ellipse object.

8C is a diagram illustrating a result of detecting a moving object region from a preprocessed image.

8D is a view illustrating an image in which a moving object region is removed from a preprocessed image.

9 is a block diagram illustrating a non-elliptic object tracking process according to the present invention.

10 shows an optical flow model database.

FIG. 11 illustrates an optical flow model generated using feature point information extracted from an input image. FIG.

12 is a diagram illustrating decomposition of an input optical flow using an optical flow model.

FIG. 13 is a diagram illustrating an input parameter expressed as a weighted sum of each model generation parameter. FIG.

Hereinafter, one example of the present invention will be described in detail with reference to the accompanying drawings.

As shown in FIG. 1, the apparatus of the present invention receives a video signal of a camera 10 which takes a moving picture taken in real time in a stadium 110 and transmits it as a digital signal, and finds an insertion position of a virtual advertisement. The module 60 is divided into a module 60 and an insertion module 65 for tracking movement of the object. The insertion module 65 is subdivided into three parts: an object image transformation 70, an obstacle detection 90, and an insertion 80.

The flow of the system is as follows. When the video frame is input, the area for inserting the virtual advertisement in the first frame is searched (50). It can be found automatically in standardized stadiums such as soccer fields and tennis courts, but in places where the shape of the stadium is not constant, such as golf courses, the user must manually set the insertion position. When the area in which the virtual advertisement is to be inserted is selected, the camera movement is predicted from the next frame from the next frame (60). The virtual advertisement image to be inserted in accordance with the region to be inserted is selected and modified from the pre-built advertisement image database 100 (70). Next, an obstacle is detected in the region to be inserted to form an object overlap mask (90), and then the insertion is performed using the mask as a key (80).

Next, a control method of an exemplary embodiment of the present invention having the above configuration will be described.

One embodiment of the present invention is to insert a virtual advertisement in an arbitrary position on a real-time moving image such as a video signal screen of a live sports broadcast, so that television viewers feel that the inserted advertisement is actually present in the sports stadium, watching the advertisement without reluctance Have them watch. For example, if you are broadcasting sports in the United States or Europe in Korea, you will insert the advertisements in Korea, while inserting the advertisements in that region when you broadcast sports events in Korea to other countries.

The present invention is divided into three parts, which will be described in turn.

end. Find insertion area

The position of insertion of a virtual object, such as an advertisement, on the floor of a sports arena varies. It can be inserted near the center circle or goal line of a soccer field, penalty area, the floor of a tennis court, on the grass of a baseball field, on the grass of a golf course, a free throw area of a basketball court, etc. Advertising effect is great. Since standardized stadiums such as soccer fields, tennis courts, basketball courts, and baseball fields have generalized feature points, the locations can be automatically found by using the relationship between the feature points of each area. However, if there are few feature points such as golf turf, or it is not a standardized stadium, the user must set the position manually.

3A and 3B show the distribution of the feature points of the center circle of the soccer field and the rectangular area of the tennis court and the templates used for each. First, referring to the football stadium 300A of FIG. 3A, there are four characteristic points of the center circle (310A-D). Two points of the top (310A) and the bottom (310B) of the "cross" shape, and the left (310C) and the right (310D) of the two points (',' ')'. The templates are connected in a diamond shape as 320 of FIG. 3A. In addition, the tennis court screen 300B of FIG. 3B shows feature points of the tennis court, and a vertex portion of the rectangular area may be a feature point (310A-D). Therefore, corresponding templates are connected in a trapezoidal shape as shown in 320 of FIG. 3B.

Here, an algorithm for finding an elliptic insertion position such as the center circle 330A of the soccer field 300A will be described. The center circle of the soccer field is composed of white ellipses and straight lines on the green grass as shown in FIG. The center circle has four feature points' top (310A) ',' bottom (310B) ',' left (310C) 'and' right (310D) ', and' top 'and' bottom 'feature points are' 十 (310A- B) 'shaped', 'left' is '((310C)', 'right' is ') (310D)' shape. Therefore, template matching may be attempted using the template 320 for such a feature point.

First, find the position of 'top 310A,' 'bottom 310B' of the center circle by using the 'double' template (310A-B), then 'top 310A,' 'bottom 310B'. 'Using the distance between the feature points, place the search window at the position where the' 310C, '' 310D 'feature points may be located, and then use the template matching to select' left 310C 'and' 310D 'points. You can find it. The entire algorithm is shown in FIG.

Extracting the grass area 400 and attempting template matching on the grass part may reduce unnecessary search time. After obtaining an edge of an image using a Laplacian filter 410, the result of closing the image 430 of the pixels 420 whose pixel value is greater than or equal to a predetermined value Template matching 440 is performed. In this case, if the athlete covers the 'twice' feature of the center circle, it will fail to locate the feature. Therefore, attempts should be made when the competitor does not cover the part. After finding two feature points of 'upper 310A' and 'lower 310B' (450), the distance between the two feature points is obtained. The center circle is circular, but is captured in the form of an ellipse deformed to the camera, so if you know the distance between the upper (310A) and the lower (310B), the features of the left (310C) and the right (310D) An approximate location of the can be seen (460). The search window is placed at an approximate location, and template matching is performed therein (470). In this case, of course, if the player hides the feature point, it cannot be found, but even if only one of the left (310C) and 'right (310D)' feature points appears, the other position can be found. This is because it is symmetrical with a straight line passing through the upper (310A) and the lower (310B) of the shape of the ellipse.

In this way, the insertion area 330B of the tennis court 300B can also be found. Like a soccer field, a tennis court has a white line on a solid floor, so you can find a specific area. In FIG. 3B, four feature point templates are needed to find a rectangle in front of the tennis court, and templates having the shapes 'ㅏ 310A', 'ㅓ' 310B, and 'ㅗ (310C-D)' may be used.

Therefore, since each insertion region has a different feature point, a corresponding algorithm must be applied to each template for each insertion region.

I. Camera movement tracking

7 is a flowchart illustrating an object tracking process according to the present invention when the object to be tracked is an ellipse object. By selecting and processing only the tracking object region in the image from the input video data, the preprocessing region selection step is performed to efficiently process the tracking operation. FIG. 8B is a diagram illustrating preprocessing of a binary image by setting a preprocessing area using information of a region where a tracking object is located with respect to an input image (FIG. 8A). The area 721 of the moving object other than the ellipse object must be removed for accurate tracking of the ellipse object. The preprocessed input image (FIG. 8B) removes the object area other than the ellipse object through the moving object removal step 720 so that the feature point is extracted only from the ellipse object. FIG. 8C is an image obtained by extracting a vertical edge, which is a characteristic of a moving object, to remove a moving object from a preprocessed input image (FIG. 8B), and removes the extracted vertical edge region from the preprocessed image. In order to remove noise included in the image (FIG. 8D) from which the moving object region is removed, a blob 731 is generated 730 through a labeling method on the image. When the number of pixels constituting the blob is less than or equal to a predetermined threshold, the blob area is removed from the preprocessed image by considering it as a noise blob (750). The moving object and the image from which the noise is removed (FIG. 8D) from the preprocessed image (FIG. 8B) are subjected to the feature point extraction step 760. Since the input image (FIG. 8D) of the feature point extraction step includes only the elliptic object region 761, all pixels present in the input image become the feature points for elliptic object tracking. The location information of the extracted feature point is transmitted to the input of the elliptic equation derivation step 770. In the elliptic equation derivation step, an elliptic equation consisting of extracted feature points is derived. The derived elliptic equation extracts information for elliptic object tracking through the parameter calculation step 780. In the parameter calculation step, the derived elliptic equation can calculate the center point coordinates of the ellipse and the size information of the long and short axes. The motion of the camera included in the video can be analyzed using the calculated center coordinates of the ellipse and the size information of the long and short axes. The movement information of the camera can be known through the change value of the center point coordinates of the extracted ellipse, and the enlargement / reduction information of the camera can be extracted by changing the size of the long and short axes of the ellipse.

9 is a flowchart illustrating an object tracking process according to the present invention when the object to be tracked is a non-elliptic object. The image of the input video undergoes an intersection search step 810 for extracting feature points. 3B is a diagram illustrating a template set to extract only feature points having characteristics corresponding to intersection points from an input image. The templates for extracting the feature points are 'TT' templates, '┗' templates and '+' templates. The location information of the extracted feature point is transferred to the optical flow calculation step 820 at the intersection to calculate the light flow at the extracted feature point location. An optical flow model generation step 830 is performed to analyze the optical flow on the calculated input video data. In the optical flow model generation step, the optical flow model (FIG. 11B) is generated by extracting only the optical flow component corresponding to the same position as the position of the feature point extracted from the image of the input moving image data from the optical flow model database. 10 is a view showing a light flow model database. FIG. 11B is a diagram illustrating an optical flow model generated by extracting only an optical flow component corresponding to a feature point location extracted from the optical flow model database. In the input optical flow analysis step 840 using the optical flow model, the input optical flow calculated using the optical flow model is decomposed. That is, the input optical flow is represented by a plurality of optical flow models. In this decomposition process, the weight of each model is calculated according to the similarity with the input optical flow. 12 is a diagram illustrating that the input optical flow is decomposed into a sum of weights of the optical flow model. The weight for each model calculated in the input optical flow analysis step using the optical flow model is transmitted to the parameter calculation step 850 to analyze camera movements included in the input video data. In the parameter calculation step, the input parameter is calculated by using a weighted sum applied to the parameters used for generating each model by the weights calculated for each model. FIG. 13 is a diagram illustrating that input parameters are calculated by combining and applying weights of each model to each model generation parameter. Through the calculated parameters, the movement of the camera included in the input video data may be analyzed, and thus the tracking object region may be tracked.

All. insertion

The insertion module 65 is roughly divided into three parts: a part 70 for deforming an advertisement image to be inserted, a part 90 for detecting an obstacle, and a part 80 for inserting an advertisement image. Let's take a closer look at each.

When the position of the region to be inserted is set, a virtual object must be inserted at the position. In order to naturally insert the object image, it must be modified to fit the region to be inserted. Since the image entered into the camera has a sense of perspective, it must be inserted into the image by changing the perspective to the same size as the perspective of the image. However, perspective cannot be obtained for every input image (video frame). The method invented here is to use the coordinates of the feature points of the part to be inserted. Since the coordinates of the feature point are on the input image, the position information of the feature point has perspective information of the input image.

In the case of a tennis court as shown in FIG. 3B or a penalty area of another soccer field, the position of the feature point may be used as it is, but in the case of a center circle of a soccer field as shown in FIG. 3A or a free throw area of a basketball court as shown in FIG. You can't get it right away. Therefore, in the case of a center circle, the perspective deformation parameter can be generated by the following method.

First, a rectangle 510 is created using the outermost coordinates of a feature point. A rectangle 510 having a horizontal a (530) and a vertical b (540) size is generated. Perspective parameters can be obtained using the lengths 530 and 540 of the two sides. The vertical length 540 is the same, and only the horizontal length 530 is changed. The length 560 of the horizontal segment in the lower portion is a + b × (1.0-c), and the length 550 of the horizontal segment in the upper portion is a-b × c (in FIG. 5, c is the vertical of the image). In the coordinate ratio (range is 0.0 to 1.0), d is the horizontal coordinate ratio of the image (d> 0 for the right side of the screen, d <0 for the left side). In this way, perspective deformation can be done automatically, but only when the insertion position is in the center of the image. Also Only when this is possible, otherwise the length 550 of the upper segment is a- (a-b) × c, and the length 560 of the lower segment is a + (a-b) × (1.0-c).

When the insertion position is in a position other than the center of the screen, for example, the right side (500B) or the left side (500C) of the screen, only the positions of the line segments are changed while maintaining the length of the two horizontal segments (520B-). C). FIG. 5B is a case where the insertion position (the position of the center circle) is on the right side of the screen, and FIG. 5C is when the left side of the screen. The position of the two horizontal segments varies depending on how far the horizontal segments are from the center of the screen. If the insertion position is far away from the center of the screen, the moved position of the two sides will also be large.

If there is an obstacle such as a person or a ball in the insertion area, the object should be inserted only on the floor of the playing field, avoiding the part. Therefore, before inserting an object, always check the insertion area for obstacles. The method for detecting an obstacle is as follows.

In general, the problem of separating the foreground such as the player or the ball and the background such as the playground has been dealt with a lot, and there are various methods. Usually, when separating an object, a moving object is regarded as an object and a difference image between adjacent frames is used, or a difference image with a reference image is used. This method uses low resolution images because objects are easily detected, while computation is expensive. And when the background color changes, it becomes difficult to detect the object. To overcome this drawback, sensor points were used. This method attaches markers (sensor points) around the actual stadium, finds the insertion position by using the camera first, finds the insertion position, compares the color with the sensor points in the previous frame, and updates the changed background color. However, this method has a disadvantage in that object detection fails if a certain number of sensor points are not detected. Thus, the present invention attempts to separate objects using color.

Most sports venues are of a single color. For example, a soccer field is green grass, usually composed of one or two colors, a tennis court is one color, and a golf or baseball field is one or two. Thus, a single background color can be used to separate using ultimattes, similar to chroma keys. This method does not require much computational cost and the algorithm is simple, but simple color filtering 600 does not accurately identify obstacles. For example, in a soccer game, if a player's uniform color is green, similar to the color of the playground, a part of the player may be recognized as the background, so that a virtual object is inserted above the player, making the player invisible. So we use edge image to compensate for this.

Instead of generating an existing binary key, a gray key is used. An occlusion mask 640 having a brightness is generated in the following manner.

In the previous expression Is the nth input frame, Is the nth input color filtered frame, Where n is the n th occlusion mask, G is a Gaussian function, θ is a Laplacian function, and K is a closing kernel.

The object detection algorithm is illustrated in FIG. When the video image is input, the boundary image is obtained simultaneously with the color filtering. Color filtering separates objects using the ultimatte method (linear method) using V and H × S in the HSV color model. The boundary image is blurred by using a Gaussian filter 610, and then a boundary image is generated by a Laplacian filter 620. The reason for finding the edge after blurring is to remove background noise. Therefore, only the boundary corresponding to the obstacle to be detected such as the player or the ball remains. However, this boundary is complemented by closing (630) because the foreground and the background color is weak when the color is similar. As a result, a gray image shows a relatively large value in the part with an obstacle and a small value in the part without an obstacle. Therefore, when the weight 650 of the gray image is given to the color filtered image, an obclusion mask 640 having a brightness is generated. Therefore, when the mask 640 is used, a portion where the color of the foreground and the background is similar can be detected. Therefore, the mask 640 is used as a key for alpha blending between the original image and the image to be inserted.

Therefore, according to the method of the present invention as described above, it is possible to naturally insert the advertisement of the stadium 110 in the broadcast relay to make viewers feel as if the advertisement is actually placed on the stadium, so that the effective advertising effect You can get it.

According to the present invention, it is possible to maximize the effect of the advertisement by inserting the animated advertisement in real time in the difficult part, for example, the center circle of the soccer field or the part that can insert only the picture advertisement during the television broadcast of sports events. have.

As described above, according to the present invention, the image screen is dynamically developed by reading an image of a broadcast broadcast on a frame-by-frame basis to secure an advertisement insertion position by a template method and then inserting an artificially preset advertisement at the position. Since the advertisement signal can be easily inserted at a desired position on the video screen, the advertisement display characteristics can be significantly improved.

According to the present invention, since an advertisement signal is inserted at an appropriate position of a dynamic video screen, the viewer always watches the advertisement, thereby maximizing the advertisement effect through a broadcasting medium.

Claims (12)

An advertisement insertion system that executes an advertisement insertion function on a video signal relayed in real time, comprising: a search module for determining a virtual advertisement insertion position using a template structure on a video signal input from the camera, and a template structure of the search module. And a tracking module for inserting an advertisement at a virtual advertisement insertion position set by the controller and a tracking module for predicting camera movement. The system of claim 1, further comprising analyzing the movement of the camera by analyzing the motion of the camera in the image without using the camera to which the memory head is attached. In the step of finding the insertion region of the advertisement insertion system, performing template matching on the feature points of the insertion position of the virtual advertisement using the template structure to find the insertion position of the virtual advertisement. Control method of the system to insert the advertisement. 4. The template structure of claim 3, wherein the template structure is composed of four templates, and each template is connected to each other in a circle so as to complement each other's positions among the templates, thereby efficiently finding an insertion position. Control method of the system to insert the advertisement on the broadcast relay video screen. In the step of inserting a virtual advertisement of the advertisement insertion system, the broadcast relay video screen comprising the step of modifying the object image to be inserted, detecting the obstacles appearing in the insertion region and inserting the modified virtual advertisement Control method of the system to insert the advertisement into. 6. The method of claim 5, wherein the step of transforming the object image comprises the step of predicting the perspective deformation degree of the entire image using the coordinates of the position to insert the virtual advertisement. How to control the system. The method of claim 5, wherein in the detecting of the obstacle, the background and the object are separated using V and H × S in the HSV color model, and the occlusion mask generated in the edge image and the previous frame is detected. And separating the object accurately by referring to the control method of the system for inserting an advertisement on a broadcast video screen. The method of claim 5, further comprising generating a gray valued mask in detecting the obstacle. 7. The method of claim 5, wherein the step of inserting a virtual advertisement by synthesizing a virtual advertisement image and an image input by a camera using the mask as a key when inserting the mask using a brightness mask in the inserting step. Control method of a system for inserting an advertisement on a broadcast relay video screen comprising a. During the moving object region removal step to extract only the elliptic object region from the input image for tracking the ellipse object, the moving object removal method using the characteristic of the moving object using the vertical edge component is included. Control method of inserting system. Camera motion information calculated by calculating elliptical equations using the extracted feature points after elliptic object tracking and calculating the motion information of the camera included in the video using the coordinates of the ellipse's center point and the length of the ellipse. And a method for tracking an ellipse object using the control system. In the non-elliptic object tracking method, an input optical flow analysis step using an optical flow model to analyze camera movements included in an input video, and a camera motion parameter for an input by applying weights for each model to each model generation parameter And a method for tracking a non-elliptic object using the information by calculating a.