KR20030003506A - image tracking and insertion system using camera sensors - Google Patents

Abstract

PURPOSE: An image tracing and inserting system using a camera sensor is provided to trace a marker position and insert an image in a precise position by using the value of a camera sensor when inserting the image in broadcasting images. CONSTITUTION: An image tracing and inserting system using a camera sensor includes a camera(10), photographing broadcasting images; a frame grabber(30), transmitting the image of the camera to an image analysis module(40) as the unit of a frame; a camera sensor, abstracting a camera variable; a camera sensor module(20), receiving the value of the camera sensor and transmitting the camera variable value to the image analysis module; a user interface(50), transmitting a user's demand to the image analysis module; an image analysis module, receiving the image frame, the camera sensor, and the demand, searching a marker position, and tracking a position for inserting the image, an image insertion module(70), inserting the image in broadcasting images and then delivering, and data base(80), storing the information of the marker, the information of the image to be inserted, and information of the camera. Accordingly, inserting an image into a precise position is possible by using the camera sensor value.

Description

Image tracking and insertion system using camera sensors}

The present invention relates to an image tracking and insertion system using a camera sensor, and relates to obtaining a camera variable using a camera sensor and using the same to track the position of a marker and inserting an image at a desired position of a user.

In the conventional broadcast image insertion method, the image is inserted into the screen without analyzing the broadcast image, and thus it is fixedly expressed in a certain area of the screen regardless of the movement of the camera such as pan, tilt and zoom. In other words, it could not express the phenomenon which is covered by the above object.

Recently, inventions related to changing the attributes of an embedded image in response to changes in broadcast video have been introduced beyond these limitations, and US Patent 5,264,933, US Patent 5,903,317, US Patent 5,353,392, US Patent 5,491,517, and the like are related thereto. . Such patents have a problem in that it is not easy to insert an image for a screen transmitted through a fast moving camera. That is, there is a problem in that an image cannot be inserted in a camera environment that moves rapidly along a batter or player during a baseball game relay. In addition, there is a problem in that real-time adaptation is difficult when the overall environment changes. For example, this may be the case when the day shifts to a night game or when the weather suddenly becomes cloudy.

Therefore, a technical problem of the present invention is to insert an image at a precise position by tracking a marker position by using a camera sensor value in order to insert an image desired by a user in a broadcast image.

In detail, camera variables are obtained by using pan, tilt, zoom, focus, position, and rotation information obtained from a camera sensor, and based on the analysis, image information is used to quickly and accurately search the position of the marker region and the insertion region, By inserting the, to configure an image insertion system that is not limited by the movement of the camera. In addition, by configuring a camera sensor consisting of an encoder and an inertial navigation device to obtain camera parameters not only fixed camera but also handheld mobile camera environment.

In addition, the present invention is to solve the problem caused by the change in brightness by tracking the position of the marker by applying a template matching based on the camera motion information and the brightness information of the current frame.

1 is a diagram illustrating an example of operating an image tracking and insertion system using a camera sensor.

2 is an overall structural diagram of an image tracking and insertion system using a camera sensor according to an embodiment of the present invention.

3 is a diagram showing camera parameters used in the present invention.

4 is a structural diagram illustrating an application of the camera sensor module shown in FIG. 2 to a fixed camera.

FIG. 5 is a structural diagram illustrating an application of the camera sensor module shown in FIG. 2 to a mobile camera.

6 is a structural diagram of a pre-performing operation for operating an image insertion system.

7 is a diagram illustrating a marker region and an insertion region according to an embodiment of the present invention.

8 is a flowchart illustrating an image tracking and insertion method using a camera sensor according to an exemplary embodiment of the present invention.

FIG. 9 is a flowchart illustrating the searching of a marker region shown in FIG. 8.

10 is a diagram illustrating an example of setting a location of a marker obtained using camera variable information and a search region setting of the marker based on the location of the marker.

11 is a diagram showing processing for an image to be inserted.

12 illustrates an example of inserting an image into a broadcast video according to an embodiment of the present invention.

An image tracking and insertion system using a camera sensor according to an aspect of the present invention for achieving the technical problem, a camera for photographing a broadcast image, a frame grabber for transferring the image from the camera to the image analysis module frame by frame, camera A camera sensor attached to the camera to extract camera variables, a camera sensor module that receives camera sensor values as inputs, and passes the camera variable values to the video analysis module, a user interface that delivers user requests to the video analysis module, and an image frame Image analysis module that searches the position of the marker based on the input, camera sensor, user's request, and tracks the position of the image to be inserted, adjusts the size of the image to be inserted, perspective, and separates the background. Image insertion module and marker to insert and send images in broadcast video Storing the management information, information of the image to be inserted, the camera information such as a data base.

Here, the frame grabber is a hardware device that delivers the image taken from the camera to the computer in frame units for processing in a computer system. The frame grabber should ensure that the image is acquired and delivered at a rate of 30 frames per second for real time operation.

The camera sensor is composed of a combination of an inertial navigation device and an encoder, and is attached to a camera to generate information related to camera parameters during real-time broadcast shooting. Among the camera sensors, the inertial navigation device generates roll, pitch, yaw, x-axis movement, y-axis movement, and z-axis movement information of the camera, and the encoder part tracks them by mechanical rotational movement. Information about pan, tilt, zoom, and focus, which are possible elements, is generated.

The camera sensor module converts the values received from the camera sensors into actual camera variables that can be used in the image insertion system. The camera parameters required to insert an image into the actual image require a combination of different values depending on whether the fixed camera or the mobile camera is used. In the case of a fixed camera operated with a tripod fixed, the camera parameters required for image tracking and insertion during real time operation consist of four elements: pan, tilt, zoom, and focus. Therefore, only the value input through the encoder can be interpreted to obtain the required camera variables. However, in the case of a mobile camera which is not fixed by the trivet and there is a positional movement or a movement and rotation by a camera photographer, there are camera variables that are difficult to obtain only by applying an encoder such as the position or roll of the camera itself. In this case, the inertial navigation device and the encoder are used simultaneously, or only the inertial navigation device is used, and the camera sensor module receives this value and generates a camera variable.

In the user interface, the user's request such as determining whether to insert an image, determining an image to be inserted, and determining a position to be inserted is received in real time and plays a role of delivering the information to the system. This allows the user to select in real time which location and which image to insert.

The image analysis module predicts the position of a reliable marker to some extent in the current frame by using the camera variable information, and reconfirms the marker region within the predicted region through a template matching method to increase accuracy. When no matching is made in the area, that is, when the marker is covered by another object, information such as the position of the matching marker, the position of the predicted marker, and the correlation between the positions of the markers is used. Confirm the position. Correlation information of positions between the respective markers is obtained before performing real time through the pre-work. The insertion region is determined by applying a correlation between the marker region and the insertion region obtained from the marker region search. The correlation between the marker region and the insertion region used in this process is also obtained before the real-time execution through the pre-work. The generated insertion region is verified by comparing with the insertion region predicted by the camera variable.

The image insertion module performs operations such as size and perspective adjustment so that the image exists in the insertion region calculated by the image analysis module, and separates the entire background inside the image to be inserted based on the background information of the database. As a result, after inserting the image, the frame in which the image is finally inserted is transmitted to the broadcast video.

Since the motion in the broadcast video is performed in precise units compared to the resolution of the video itself, in order to insert the image as if it is actually present in the broadcast video, position and insertion operations for the image to be inserted are performed at a higher precision than the resolution of the video. Should be. Therefore, in the image analysis module and the image insertion module of the present invention, all operations are performed in a sub-pixel unit in which one pixel of the actual broadcast image is divided into smaller units.

The pre-execution task refers to a task performed before the real-time broadcast, and includes marker region setting, template information extraction, insertion region setting, background information extraction, target region modeling, and camera modeling.

The marker region setting is to select a marker to be tracked in the image analysis module in the broadcasting target region and to establish a relationship between markers. Marker areas should have properties that are easy to navigate through template matching, including those that have an asymmetric structure around the origin and should not consist of the same color.

When the marker region is set, the template information extraction process of extracting the internal image information of the marker region as the basic template information is performed. Templates are divided into default templates that are set through pre-work and execution templates that are obtained during the matching process during execution. The base template is kept in the database as the base image of the marker and serves as an image representing the marker. The execution template stores the image of the marker region when the matching is performed in the actual matching process as a template. When the matching occurs, the execution template is replaced with the image of the matched marker region. Using two kinds of templates, if the marker area itself changes due to camera movement or overall environment change during real-time operation, it actively adapts to increase the accuracy of matching.

The insertion region setting actually selects candidate regions into which an image is to be inserted, thereby obtaining background information as well as position information of the region to be inserted. Where the image is inserted during the broadcast is selected by the user in real time among these candidate areas. If the area in which the image is to be inserted is obscured by an object or a person during broadcasting, the image should not be inserted in the hidden area. To do this, you must have background information in advance. Background information refers to background information of an area where an image is to be inserted, and is used as information for extracting a foreground area in an entire background separation operation.

When the marker region and the insertion region are set in this way, the target region modeling process processes the positional relationship between them and stores the result in the database. This information is used as basic information for real-time image insertion in the image insertion system.

Camera calibration is performed based on the set marker area and the insertion area, through which information on camera characteristics can be grasped. Camera modeling is performed based on the generated information. That is, it defines the correlation between the information input from the camera sensor and the actual camera variable. In the real-time operating system, the camera modeling information can be used to convert the value input from the camera sensor into a camera variable and extract the position of the marker and the insertion region.

In accordance with an aspect of the present invention, an image tracking and insertion method using a camera sensor includes: extracting a camera variable converting a value input from a camera sensor into a camera variable, extracting an insertion region based on the camera variable, and the region is a frame Checking whether there is an inside, searching the input area based on the camera variable to find the location of the marker, searching the marker area, and using the correlation between the marker area and the insertion area based on the found marker area Insertion area determination step to find out, insertion area verification step of checking whether the insertion area calculated based on the camera variable and the insertion area determination step match, image to be inserted using the location information of the insertion area and broadcasting image information To adjust the size of the image, the perspective, and compare the background information of the insertion area with the image of the broadcasting area. And determining whether there is another object or person, that is, a foreground element, on the image to be inserted, and finally synthesizing and sending the image.

DETAILED DESCRIPTION Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the accompanying drawings.

1 is a view showing an example of the actual operation of the image tracking and insertion system using a camera sensor.

When shooting a scene such as baseball relay with the camera 10 with the camera sensor 100 attached, the original broadcast image 2 and the camera parameters obtained from the camera sensor 100 are input to the image insertion system 1. The image insertion system 1 analyzes an image based on input information such as a camera variable and an original broadcast image 2 and inserts an image to output the broadcast image 3 into which the image is inserted. The viewer, while watching the broadcast image 3 inserted with the image, feels as if the image exists in the shooting site itself.

2 is an overall structural diagram of an image tracking and insertion system using a camera sensor according to an embodiment of the present invention.

The broadcast screen photographed from the camera 10 to which the sensor is attached is stored in the frame buffer by the frame grabber 30, which becomes a background into which an image is inserted. Camera information generated by the sensor attached to the camera 10 is converted into a camera variable in the camera sensor module 20 based on the camera modeling information stored in the database 80 and input to the image analysis module 40.

The operator 60 may control whether a virtual image is inserted, an insertion position, an image to be inserted, etc. in real time through the user interface 50, and this information also serves as an input of the image analysis module 40.

The image analysis module 40 receives an image from the frame grabber 30 and performs a role of searching for a marker region, determining an insertion region, verifying an insertion region, size of an insertion image, and applying perspective elements. In this process, the marker region is predicted based on camera variables, and template matching is applied to obtain a reliable marker region. Then, the position of the image to be inserted in the current frame is determined using the positional relationship of the found markers. The positional relationship between the marker region and the insertion region is automatically determined by the system as a mathematical relationship on affine coordinates in the target region modeling step before the real-time execution. The insertion region thus determined is compared with the insertion region calculated on the basis of the camera variable, and is validated when the error does not exceed the allowable range. In determining the insertion area, the insertion image is converted by considering factors such as size and perspective.

The image insertion module 70 performs an overlap determination operation for separating the foreground and the background of the region to be inserted based on the background information stored in the database 80, inserts the image based on the result, and synthesizes the broadcast. Output an image (90). The color keying technique is generally used to separate the foreground and background. If the background appears in several colors instead of one color, it is handled using a multi-color key technique within a certain limit (usually two colors). However, if it is composed of more colors, processing by chroma key technique is limited. In this case, the foreground and the background are separated by applying a pattern recognition technique in the image to be inserted. Create a background mask as a result of separating the foreground and background, and then insert the image into the background area.

3 is a diagram showing camera parameters used in the present invention.

Camera variables include pan, tilt, roll, zoom, focus, position information (x, y, z), rotation information about three axes, and the like. Common image insertion systems use only pan, tilt, zoom, and focus among camera parameters. A hardware device that provides only this information is called a memory head. This is only considering a fixed camera, there is a problem that the image insertion is impossible in the case of a hand-held camera that the operator changes the camera position or the camera moves. The present invention develops and uses a camera sensor of a type in which an encoder and an inertial navigation device are combined to be used in a moving camera environment while solving this problem. That is, the image insertion system operates in both the stationary camera and the mobile camera.

In the case of the fixed camera fixed to the tricycle, the linear movement of the angular axis (x, y, z) does not occur in FIG. 3, and only the movement of the pan, which is the left and right rotation information, and the tilt, which is the vertical rotation information. This happens. In addition, camera's unique characteristics such as zoom, focus, etc. are included in camera variables to be recognized by the system as values that change depending on the situation.

In the case of a mobile camera that can move freely and shoot, the movement of a roll corresponding to the z-axis rotation information and the straight line on each axis (x, y, z) in addition to the camera parameters of the fixed camera, pan, tilt, zoom, and focus The shift value is included in the camera variable.

4 is a diagram illustrating the structure of the camera sensor module 20 shown in FIG. 2 in the fixed camera. The camera 10 requires only the encoder 110 as a camera sensor to obtain camera parameters required by the fixed camera. The encoder 110 is an arithmetic apparatus that expresses a mechanical rotation value, and expresses a rotation value related to a pan, tilt, zoom, and focus, which are camera variables required by a fixed camera. The encoder analysis module 210 converts the output result of the encoder 110 numerically and delivers it to the camera variable extraction module 230. The camera variable extraction module 230 calculates a camera variable (pan, tilt, zoom, focus) applicable to the image analysis module 40 based on the camera modeling result stored in the database 80 as a pre-work.

5 is a diagram illustrating the structure of the camera sensor module 20 shown in FIG. 2 in the mobile camera. The camera 10 uses an inertial navigation device 120 that is advantageous for extracting information of the mobile camera as well as the encoder 110 as a camera sensor. The encoder analysis module 110 performs numerical transformation on zoom and focus, and the inertial navigation device (INS) analysis module 220 converts acceleration information and angular velocity information about an axis generated by the inertial navigation device 120. After generating the linear movement information and the rotation angle information for each axis and passes the result to the camera variable extraction module 230. The camera variable extraction module 230 calculates a camera variable (pan, tilt, roll, location information, zoom, focus) required by the image analysis module 40 based on the camera modeling result stored in the database 80.

As a preparatory step for tracking and inserting images using camera sensor information in real time, it is necessary to perform marker area setting, template information extraction, insertion area setting, background information extraction, target area modeling, and camera modeling before real-time execution. do. Fig. 6 shows the contents of such pre-work.

First, the process of setting the marker region 300 is performed. The marker area is required to find the insertion area during the real-time system operation and is represented by location information about the center of the marker. Based on the marker position information obtained in the marker region setting 300, the template information extraction 310 stores an image in a predetermined region from the center of the marker in the database 80 as basic template information. The basic template should have the characteristics that can be selected by the pattern matching algorithm as the image information for the marker.

Next, the insertion area setting step 320 is performed. The insertion area is a rectangular shape of a position where an image or a video is to be inserted. The insertion area is expressed as position information of each vertex of the rectangle. Background information is image information of the inside of the insertion area, which is used for the entire background separation in real time. The background information is stored in the database 80 through the background information extraction step 330.

When the marker region and the insertion region are selected as described above, the target region modeling 340 is performed. The relationship between the marker regions and the relationship between the marker and the insertion region is defined in this process. The insertion region 321 is redefined as a position relative to the position of the marker region 301 on the affine coordinates, and the result is stored in the database 80. In addition, the image information about the insertion region itself is stored as the background information and used as the basis information for determining the occurrence of the overlap problem.

Next, the camera modeling 350 is performed using the results of the marker region setting 300 and the insertion region setting 320. The camera calibration process and the camera variable information which generate information for converting the camera raw data input from the camera sensor 100 into camera variables based on the designated marker region 301 and the insertion region 321 information The relationship between the positions of the markers and the relationship between the camera variable information and the position of the insertion region are generated in a table form and stored in the database 80.

8 is a flowchart illustrating an image tracking and insertion method using a camera sensor according to an exemplary embodiment of the present invention.

First, a process of extracting a camera variable 400 is performed. In the camera variable extraction process 400, the camera variables are calculated by integrating the information input from the camera sensor and used as basic information for calculating the position information in the image insertion system. In the case of the fixed camera, the camera variable is extracted by the method shown in FIG. 4 and the method shown in FIG. 5 in the case of the mobile camera.

The insertion region extraction 410 is performed based on the camera variable value obtained in the camera variable extraction 400 and the camera modeling 340 information calculated in the pre-work. Insertion region extraction information is used for two purposes. First, it is used in the process of determining whether the insertion region is inside the current frame (420). If it is found that the insertion region does not exist in the frame, the following process is omitted and the current frame received is transmitted as it is. On the contrary, when the insertion region exists in the frame, the following process including the marker region search 430 is performed. Second, it is used to verify whether the insertion region determined based on the marker region in the insertion region verification step 450 is correct.

If there is an insertion region in the frame, the marker region search 430 is performed on the broadcast screen based on the camera variable value. 9 is a flowchart illustrating a process of searching for a marker region 430. First, the marker region in the current frame is predicted 431 based on the input camera variable value and the result of the camera modeling 340 configured as the pre-work. Then, as a preparation step for starting the search, the brightness adjustment 432 is performed. The change in brightness in the current frame is defined as the difference by comparing the average value of the overall brightness of the predicted marker area with the average value of the brightness of the base template. Based on this value, the brightness of the execution template is adjusted to solve the matching problem caused by the change in brightness.

After predicting the marker region 431 and adjusting the brightness 432, the precise marker search is performed using a template matching technique. In FIG. 10, the black point 302 is the position of the marker predicted based on the camera variable, and the circle 303 around the black point is actually the search region of the marker that will use the matching technique in the process of searching the marker region 430. to be. That is, camera variable values are used to identify the basic area for searching for markers, and precise marker positions are obtained through a template matching technique. Template matching proceeds as follows. First, if the search for all markers is finished by determining whether there is a marker region that has not been searched (433), and if the number of matched markers is obtained by the search result, four or more matching results are obtained, Go to decision 440, otherwise use the correlation between markers found in the pre-work based on the searched marker position and the predicted marker position to obtain the position of the unmatched marker and determine the insertion region. Go to step 440). That is, in the insertion region determination 440, the operation is performed with information about the positions of four or more markers. If there is a marker that has not been searched by determining whether or not there is a marker area that has not been searched, first, a matching 434 is performed in the area of the expected marker in the current frame based on the execution template. If a match is made, the execution template is updated to an image of the matched marker region (436), and then the process moves to a process of determining (433) whether there is a marker region not searched. If no match is made, the matching marker area is performed based on the default template 435. If a match is made, the execution template is updated to an image of the matched marker region (436), and then the process proceeds to a process of determining (202) whether there is a marker region not searched for. If no match is found here, the execution template for the current marker region is maintained (437), and it is concluded that no match is made. This happens when the marker region is covered by another object or person, and a completely different image exists in the marker region.

When the location of the marker is obtained as a result of the search (430) of the marker region, the process of determining the insertion region (440) is performed. In order to perform this process, the target region modeling 320 stored in the database 80 is used through a pre-work. That is, the position of the insertion region is determined by using a mathematical correlation on the affine coordinate between the position of the marker defined before the execution and the four vertex coordinates of the insertion region.

The position of the insertion region thus obtained is compared with the insertion region prediction information obtained in the insertion region extraction process 410 to verify the accuracy (450). By going through the verification process 450 for the insertion region, it is possible to ensure more accurate and reliable image insertion and to prevent the occurrence of errors. Since most image insertion systems for broadcast video must be operated in real time on a live broadcast, the performance of this verification process plays an important role in increasing the reliability of the system.

When the position of the insertion region is determined, the size and perspective of the image to be inserted are adjusted according to the insertion region (460). By matching the perspective of the image to be inserted with the perspective attribute of the original broadcast image to be inserted, the viewer is visually provided with a natural image. In addition, by changing the position and size according to the ever-changing broadcast video, the viewer feels that the inserted image was in the screen of the original broadcast video.

After processing the attributes of the image to be inserted, a background mask is generated through the overlap determination process 470 comparing the background information stored in the database 80 with the image of the image to be inserted, and based on the background mask, Isolate the background and insert an image.

11A-C illustrate processing for an image to be inserted, such as perspective adjustment, foreground separation, and the like. 11A is an image to be inserted, and no processing is performed. FIG. 11B is an image modified through a process of adjusting size and perspective 460 to fit perspective properties in a broadcast image. Finally, 11c is an overlap determination process. The broadcast image is obtained by inserting an image by separating the foreground and the background through 470.

Since the movement in the actual broadcast image is performed in a precise unit compared to the resolution of the image itself, in order to insert the image as if it is actually present in the broadcast image, image analysis and insertion must be performed at a higher precision than the resolution. Therefore, in the present invention, the sub-pixel unit analysis and insertion for dividing one pixel of the actual broadcast image into several small pixel units is performed.

Finally, the broadcast image in which the image is inserted is transmitted through the image synthesizing process 480. As a result, the viewer views the broadcast while feeling as if the image is actually present in the actual broadcast screen.

12A-C illustrate an example of inserting a video into a center of a crossroad according to an exemplary embodiment of the present invention. FIG. 12A is an image inserted into an image taken with the center of gravity at the center, and FIG. 12B is an image inserted with the camera panned to the right and tilted upward. 12C shows an image inserted in a camera zoom-in state.

Although the present invention has been described with reference to the most practical and preferred embodiments, the present invention is not limited to the above disclosed embodiments, but also includes various modifications and equivalents within the scope of the following claims.

According to the exemplary embodiment described above, the broadcast relay screen may be changed by inserting an image, a video, or a 3D object desired by the user as if the user is actually present in the space at the desired position.

According to the present invention, a camera sensor can be used to insert an image at a desired position in a broadcast image even under free and fast camera movement. In addition to image tracking, verification is performed by using camera variables in the insertion process, which ensures accurate and reliable image insertion and broadcast video transmission. The viewer can provide the effect of feeling that the inserted image is present in the original video screen because it is inserted through the exploration of a dynamically changing camera and the space photographed through the image rather than the existing insertion of the image at a fixed position.

The present invention can be applied to applications such as inserting and replacing a virtual advertisement on a broadcast relay screen, inserting a 3D virtual object, and providing information using an inserted image.

Claims (9)

A camera variable extraction step of converting a value input from a camera sensor into a camera variable; Extracting an insertion region based on a camera parameter; Checking whether the area exists inside the frame; A marker region search step of finding a position of a marker by searching an input image based on a camera variable; An insertion region determination step of finding an insertion region using a correlation between the found marker region and the insertion region; An insertion region verification step of checking whether the insertion region extracted based on the camera variable matches the insertion region obtained in the insertion region determination step; Adjusting the size and perspective of an image to be inserted using location information of the insertion region and image frame information; A background separation step of determining whether there is another object or person, that is, a foreground element, on the image to be inserted by comparing the background information of the insertion area with the image of the current image frame; Finally synthesizing and sending the image Image tracking and insertion system using a camera sensor comprising a. The method of claim 1, In the camera variable extraction step, The camera sensor is composed of a combination of an inertial navigation device and an encoder, and attached to the camera to generate information related to camera parameters during real-time broadcast shooting. The method of claim 1, The camera variable extraction step In case of fixed camera, only encoder is used as camera sensor, encoder analysis module that converts the output result of encoder numerically, based on camera modeling result stored in database, zoom, focus, pan and a camera variable extraction module that calculates tilt. In the case of a mobile camera, a combination of an inertial navigation device and an encoder is used as a camera sensor, and a camera variable is based on an encoder analysis module, an inertial navigation device analysis module, and a camera modeling result stored in a database. Camera variable extraction module that calculates in-zoom, focus, pan, tilt, roll, and linear movement information for each axis (x, y, z) Image tracking and insertion system using a camera sensor, characterized in that the configuration. The method of claim 3, The inertial navigation device calculates linear acceleration information and rotational angular velocity information for three axes, and the inertial navigation device analysis module converts it to generate linear movement information and rotation angle information for each axis. Image tracking and insertion system. The method of claim 1, An image tracking and insertion system using a camera sensor further comprising a user interface capable of reflecting the user's needs, such as whether to insert, select an inserted image, and determine an insertion position. The method of claim 1, Includes a database for storing and managing information of markers, correlations between markers, basic template information, execution template information, insertion region location information, insertion region background information, correlation between markers and insertion regions, camera modeling information, and the like. Image tracking and insertion system using camera sensor. The method of claim 1, Marker area search step, Using the camera variable information to predict the position of the marker, In order to search the marker region even in the environment where the brightness changes, the brightness information of the predicted marker region is analyzed and the brightness of the template is corrected according to the result. Matching is performed using the default template stored before execution and the execution template, which stores the matching results. Image tracking using a camera sensor, characterized in that the position of the marker predicted by the camera variable and the position of the marker extracted by matching, and the untraced marker position are obtained by the correlation between the markers stored in the database. Insertion system. The method of claim 1, Insertion area determination step, Based on four or more tracked markers, image tracking and insertion using a camera sensor, characterized in that the insertion region is determined by using the correlation between the marker stored in the database and the insertion region before performing on affine coordinates. system. A marker region setting step of selecting a position of a marker to be tracked in a broadcast video region and setting a relationship between markers; A template information extraction step of extracting basic template information from internal image information of the marker region; An insertion region setting step of selecting candidate regions into which an image is to be inserted; A background information extraction step of extracting image information in an insertion area to be inserted; A target area modeling step of setting a correlation between the position of the marker and the insertion area; A camera modeling step of identifying information on camera characteristics based on the set marker region and insertion region and defining a correlation between information input from a camera sensor and actual camera variables. Pre-Performance Task.