KR101319667B1 - System and method for providing augmented reality and method for tracking region of interest used in the same - Google Patents

Abstract

PURPOSE: A system and method, and a method and terminal for tracking an interested area used in the same can quickly and precisely track an interested area in a terminal and allow a user to be quickly and precisely provided with augmented reality video. CONSTITUTION: A video frame acquisition unit (101) obtains a video frame. A situation adaptation engine (103) calculates a previously stored template. The situation adaptation engine compares a feature point extracted from the video frame with a feature point extracted from the template to determine whether to perform feature point-based tracking based on the accuracy. The situation adaptation engine compares the template with the video frame to determine whether to perform template matching-based tracking. The situation adaptation engine tracks an interested area of the video frame according to the determination result. [Reference numerals] (101) Video frame acquisition unit; (103) Situation adaptation engine; (105) Communication unit; (107) User interface unit; (109) Storage unit; (201) Feature point extraction unit; (203) Search unit; (205) ROI recognition unit

Description

System and method for providing augmented reality and method for tracking Region of Interest used in the same}

The present invention relates to a system for providing augmented reality that can provide augmented reality images quickly and accurately, and a method thereof, and a method and terminal for tracking a region of interest used therein.

Augmented reality technology is a technology that matches the additional content to the real-world image, the utilization of the terminal is increasing as the terminal is developed.

In order to match additional content in augmented reality technology, a region of interest must be tracked. As a related art, a method of tracking a region of interest using a marker and a method of tracking a region of interest without a marker are known.

Meanwhile, in order to receive an augmented reality image without a marker outdoors, it is necessary to find a region of interest of a user by analyzing a feature point of an image acquired through a terminal in real time, and also to track a region of interest in a moving image in real time. Due to the limitation of resources, there is a problem that the terminal cannot effectively use the augmented reality service.

According to an exemplary embodiment of the present invention, an augmented reality providing system that can provide augmented reality quickly and accurately, and a method and a method of tracking a region of interest used therein can be provided.

According to another exemplary embodiment of the present invention, a terminal and a recording medium capable of providing augmented reality quickly and accurately may be provided.

According to an embodiment of the present invention, a method for tracking a region of interest (ROI) in an augmented reality providing system, the method comprising: obtaining an image frame; Calculating an accuracy of a previously stored template; And based on the accuracy, determine whether to perform feature point based tracking by comparing the feature point extracted from the image frame and the feature point extracted from the template, or perform template matching based tracking by comparing the template and the image frame. Tracking the ROI in the image frame according to the determination result; A region of interest tracking method in an augmented reality providing system may be provided.

According to another embodiment of the present invention, a computer-readable recording medium having recorded thereon a program for executing a method according to an embodiment of the present invention may be provided.

According to another embodiment of the present invention, a markerless based augmented reality providing system comprising: a terminal for acquiring image frames and matching and displaying additional content with the obtained image frames; The terminal may calculate the accuracy of a pre-stored template and perform feature-based tracking by comparing the feature point extracted from the image frame and the feature point extracted from the template based on the calculated accuracy. A markerless based augmented reality providing system may be provided in which a template matching based tracking is performed by comparing the image frame with the image frame, and tracking the region of interest in the image frame according to a determination result.

According to another embodiment of the present invention, a terminal for providing augmented reality, comprising: an image frame acquisition unit for acquiring an image frame; And calculating the accuracy of a pre-stored template and performing feature point based tracking by comparing the feature point extracted from the image frame and the feature point extracted from the template based on the accuracy, or comparing the template with the image frame. A context adaptation engine unit configured to determine whether to perform matching based tracking, and to track a region of interest in the image frame according to the determination result; A terminal including a may be provided.

According to one or more embodiments of the present invention, the terminal can quickly and accurately track the region of interest, thereby providing augmented reality images quickly and accurately.

1 is a view for explaining an augmented reality providing system according to an exemplary embodiment of the present invention,
2 is a flowchart illustrating a method for tracking a region of interest in an augmented reality providing system according to an exemplary embodiment of the present invention;
3 is a flowchart illustrating a method for tracking a region of interest in an augmented reality providing system according to another exemplary embodiment of the present invention;
4 is a flowchart illustrating a method for tracking a region of interest in an augmented reality providing system according to another exemplary embodiment of the present invention;
5 is a diagram for describing template matching based tracking according to an embodiment of the present invention.
6 is a view provided to explain the selection of a region of interest used in an embodiment of the present invention.
FIG. 7 is a view provided to explain recognition of a region of interest used in an embodiment of the present invention; and
8 is a view provided to explain the creation of a new template used in an embodiment of the present invention.

BRIEF DESCRIPTION OF THE DRAWINGS The above and other objects, features, and advantages of the present invention will become more readily apparent from the following description of preferred embodiments with reference to the accompanying drawings. However, the present invention is not limited to the embodiments described herein but may be embodied in other forms. Rather, the embodiments disclosed herein are provided so that the disclosure can be thorough and complete, and will fully convey the scope of the invention to those skilled in the art.

Further, when it is mentioned that an element (or component) is operated or executed on another element (or component) ON, the element (or component) Or may be operated or executed in an environment, or through interaction with other elements (or components) either directly or indirectly.

It is to be understood that when an element, component, apparatus, or system is referred to as comprising a program or a component made up of software, it is not explicitly stated that the element, component, (E.g., memory, CPU, etc.) or other programs or software (e.g., drivers necessary to drive an operating system or hardware, etc.)

It is also to be understood that the elements (or components) may be implemented in software, hardware, or any form of software and hardware, unless the context clearly dictates otherwise.

The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. In the present specification, the singular form includes plural forms unless otherwise specified in the specification. The terms "comprises" and / or "comprising" used in the specification do not exclude the presence or addition of one or more other elements.

Hereinafter, the present invention will be described in detail with reference to the drawings. In describing the specific embodiments below, various specific details have been set forth in order to explain the invention in greater detail and to assist in understanding it. However, those skilled in the art can understand that the present invention can be used without these various specific details. In some cases, it is mentioned in advance that parts of the invention which are commonly known in the description of the invention and which are not highly related to the invention are not described in order to prevent confusion in explaining the invention without cause.

1 is a view for explaining an augmented reality providing system according to an exemplary embodiment of the present invention.

Referring to FIG. 1, an augmented reality providing system may include a terminal 100 and a server 200.

The terminal 100 sequentially acquires image frames, tracks a region of interest (ROI) of the acquired image frame, and adds additional content related to the region of interest (ROI) to the region of interest (ROI). The additional content may be matched and displayed. Here, the terminal may be a mobile device such as, for example, a smartphone, a PDA, or a notebook, but the present invention is not limited thereto.

The server 200 provides additional content related to the ROI to the terminal 100 or extracts a feature point of an image frame according to an embodiment, recognizes the ROI, and then recognizes the ROI. 100).

For example, since the corner point has the most information about the shape of the object, it may be used as an important feature point for object representation, interpretation, and recognition, but the scope of the present invention is not limited to the corner point. Meanwhile, when the feature point is extracted, a descriptor of the feature vector for the extracted feature point may also be generated.

When the terminal 100 obtains an image frame, the terminal 100 checks whether a stored template exists. If the template does not exist, the terminal 100 generates a template from the image frame. Here, the pre-stored template may be generated based on the image frame previously acquired by the terminal 100, but is not limited thereto.

When the pre-stored template exists, the terminal 100 calculates the accuracy of the pre-stored template and performs template matching based tracking or feature point based tracking according to the accuracy of the template.

The accuracy of the template is, for example,

는 템플릿의 i 번째 픽셀의 그레이 레벨 값이고,

는 템플릿을 구성하는 픽셀들의 평균 그레이 레벨 값이고,

는 트래킹할 영상 프레임 섹션(템플릿과 같은 크기의 영상 프레임 부분)의 i번째 픽셀의 그레이 레벨 값이고,

는 트래킹할 영상 프레임 섹션(템플릿과 같은 크기의 영상 프레임 부분)을 구성하는 픽셀들의 평균 그레이 레벨 값이며, N은 템플릿의 크기(columns × rows)로서 정의된다.It may be a correlation coefficient calculated by. here,

Is the gray level value of the i th pixel of the template,

Is the average gray level value of the pixels constituting the template,

Is the gray level value of the i th pixel of the image frame section (the portion of the image frame of the same size as the template) to be tracked,

Is the average gray level value of the pixels constituting the image frame section to be tracked (the portion of the image frame of the same size as the template), and N is defined as the size of the template (columns x rows).

The correlation coefficient may be a measure of the degree of similarity or similarity between the template and the current image frame (hereinafter, referred to as a 'target image frame') to track a region of interest (ROI). The greater the likelihood of including the same or similar parts as, and the lower the correlation coefficient, the lower the likelihood that the target image frame includes the same or similar parts as the template.

In the present embodiment, the accuracy of the template is taken as an example, but other indexes may be used in addition to the correlation coefficient as long as the accuracy of the template may be reflected. In the following embodiments, it is assumed that the accuracy of the template is a correlation coefficient for the purpose of describing the present invention.

The terminal 100 sets a reference value and stores the reference value in advance. If the correlation coefficient is larger than the reference value, the terminal 100 may perform template matching-based tracking. If the correlation coefficient is less than or equal to the reference value, the terminal 100 may perform feature point-based tracking. In the present embodiment, the correlation coefficient may have a value of -1 to +1, and the reference value may be set to '0', but since these values are exemplary, the scope of the present invention is not limited only to those values. .

In the present specification, template matching based tracking is to track a region of interest by matching a target image frame and a template with each other, and feature point based tracking is to track a region of interest by comparing a feature point extracted from a target image frame and a feature point extracted from a template. It can mean doing.

According to an embodiment of the present invention, template matching-based tracking may be performed by comparing a template and a target image frame in a slide window manner. FIG. 5 exemplarily shows a comparison between the template T and the target video frame S in a slide window method. Referring to FIG. 5, the template T is compared with each of the sections a, b, and c having the same or similar size as the template T in the target image frame S, and this comparison operation is performed by the target image frame. It may be performed for all sections constituting (S).

Feature-based tracking can be implemented in any of the methods well known in the art or in any of the methods to be developed in the future.

For example, Korean Patent Publication No. 10-2011-0130309 (name of the invention: a real-time markerless three-dimensional object tracking system for augmented reality and its method) (hereinafter referred to as '0130309') tracking using a feature point Exemplary methods are disclosed. Korean Patent Laid-Open Publication No. 0130309 describes a method of extracting feature points and contents of comparing feature points, which are combined as part of the present specification.

When performing the feature point-based tracking, the terminal 100 may track the ROI by comparing the feature point extracted from the target image frame with the feature point extracted from the template. Here, the operation of extracting the feature points from the target image frame or comparing the feature points with each other may be performed by the terminal 100 or the server 200.

If the feature point-based tracking result shows that the feature point of the image frame and the feature point of the template are the same or similar, the terminal 100 may generate a new template from the feature point of the image frame and update and store the previously stored template as the new template.

After determining whether the terminal 100 has hardware (H / W) and / or software (S / W) resources capable of extracting feature points, the terminal 100 according to an embodiment of the present invention. When the 100 has H / W and / or S / W resources, the feature point of the target image frame is extracted, and the feature point of the extracted target image frame is compared with the feature point of the template to track the ROI. . If the terminal 100 does not have H / W and / or S / W resources for extracting the feature point from the target video frame, the server 200 extracts the feature point from the target video frame, The ROI is tracked by comparing the feature point and the feature point of the template, and the terminal 100 receives and displays the result tracked by the server 200.

Now, the operation of generating the template from the image frame by the terminal 100 will be described.

When the template does not exist, the terminal 100 according to an embodiment of the present invention may perform an operation of generating a template from the target image frame.

In detail, when the template does not exist, the terminal 100 determines whether the ROI is selected in the target image frame, and when the ROI is selected, the terminal 100 extracts a feature point from the selected ROI. The region of interest (ROI) may be recognized by searching the DB using the extracted feature points, and the recognized region of interest (ROI) may be generated as a template. The generated template may be stored in the terminal 100. The DB may be configured to store feature points extracted from various images and local structures and meta information necessary for searching for a corresponding area or object (the DB configured as such is called a 'feature point based DB'). Alternatively, the DB may not be configured to store feature points but may be configured to store images and meta information (the DB configured as such is called an 'image-based DB'), and in this case, the terminal 100 may store an image stored in the image-based DB. In addition, an operation of extracting feature points is performed.

Alternatively, if the template does not exist, the terminal 100 determines whether the region of interest is selected in the image frame, extracts feature points from the selected region of interest, and recognizes the region of interest. The operation may be implemented to be performed by the server 200. Thereafter, the terminal 100 may store a template from the ROI recognized by the server 200.

The selection of the ROI may be performed by the user of the terminal 100 or in a manner in which a central portion of the image frame is automatically selected. The selection method is exemplary and the scope of the present invention is not limited to the above-described selection methods. FIG. 6 is a view provided to explain the selection of a region of interest used in an embodiment of the present invention. The portion shown in FIG. 6 is a portion selected by a user or selected as a region of interest as a center region of an image frame. .

On the other hand, the recognition of the region of interest is to determine what the region of interest means. FIG. 7 is a view provided to explain recognition of a region of interest (ROI) used in an embodiment of the present invention. The portion shown in FIG. 7 is a portion recognized as the region of interest (ROI), and an object to be tracked is It is a part. For example, the part shown in FIG. 7 may be extracted and stored as a template.

Referring back to FIG. 1, the terminal 100 is configured to include an image frame acquisition unit 101, a context adaptation engine unit 103, a communication unit 105, a user interface unit 107, and a storage unit 109. Can be.

The image frame acquisition unit 101 may sequentially acquire image frames.

The context adaptation engine 103 calculates the accuracy (eg, correlation coefficient) between the template and the image frame as described above, and performs the template matching-based tracking operation when the accuracy is greater than the reference value, and the accuracy is less than or equal to the reference value. In this case, the feature-based tracking operation is performed.

The context adaptation engine 103 may also determine whether a pre-stored template exists and, if the template does not exist, generate a template from an image frame and store the template in the storage 109.

When generating the template because the template does not exist, the context adaptation engine unit 103 determines whether the ROI is selected in the image frame and then selects the ROI when the ROI is selected. A feature point of the area ROI is extracted, the ROI is recognized from the extracted feature point, and a template is generated from the recognized ROI.

The storage unit 109 may change and store a previously stored template into the new template.

The communication unit 105 supports communication with the server 200, and the user interface unit 107 may provide an interface with a user, and may be, for example, a device such as a monitor, a keyboard, a mouse, and the like.

The storage unit 109 may be implemented as a storage medium such as a memory, a hard disk (HDD), or a semiconductor disk (SSD), and may store a template. The storage unit 1090 updates and stores new templates when they are created, and may store feature point-based DB if necessary.

In the embodiment of FIG. 1, the server 200 may be configured to include a feature point extractor 201, a searcher 203, a ROI recognizer 205, and a DB 207.

The feature point extractor 201, upon request from the terminal 100, i) extracts a feature point from a region selected as the ROI, or ii) extracts a feature point from a target image frame for feature-based tracking. Perform the action.

The search unit 203 searches the DB 207 using the feature points extracted from the i), and the ROI recognition unit 205 recognizes the ROI based on the search result of the search unit 203. Perform the action.

The DB 207 may be a feature point based DB or an image based DB.

2 is a flowchart illustrating a method for tracking a region of interest in an augmented reality providing system according to an exemplary embodiment of the present invention.

Assuming that the embodiment of FIG. 2 is applied to the system of FIG. 1, it will be described with reference to the drawings of FIGS. 1 and 2.

The image frame acquisition unit 101 may sequentially acquire at least one or more image frames (S201).

The context adaptation engine 103 determines whether the template exists (stores) in the storage unit 109 when the image frame acquisition unit 101 acquires an image frame (S203).

As a result of the determination in step S203, the context adaptation engine unit 103 calculates a correlation coefficient between the template stored in the storage unit 109 and the image frame (target image frame) acquired by the image frame acquisition unit (S205). .

The situation adaptation engine unit 103 determines whether or not the correlation coefficient calculated in step S205 is larger than the reference value Th (S207).

As a result of the determination in step S207, the situation adaptation engine unit 103 performs template-based tracking using the template stored in the storage unit 109 when the correlation coefficient is larger than the reference value Th (S207: Y) (S209). do.

As a result of the determination in step S207, the situation adaptation engine unit 103 extracts the feature point from the target video frame when the correlation coefficient is equal to or less than the reference value Th (S207: N) (S210).

The context adaptation engine 103 tracks the region of interest (ROI) in the target image frame by comparing the feature points extracted in the step S210 and the feature points extracted from the template (S212).

The user interface 107 displays an image frame by reflecting the tracking result of step S209 or S212 (S211).

On the other hand, the situation adaptation engine unit 103 generates a new template using the execution result of step S212 (S214), and updates a template previously stored in the storage unit 109 with the new template (S216). Here, the new template is exemplarily illustrated in FIG. 8, and the small picture on the right may be the new template.

In operation S212, for example, the context adaptation engine unit 103 generates a new template based on the feature points extracted from the image frame when the feature points extracted from the image frame and the feature points extracted from the template are the same or similar to each other.

If the acquisition of the image frame is terminated (S213: Y), the tracking of the ROI is terminated. If the acquisition of the image frame is not terminated (S213: N), the operation below S203 is performed again.

3 is a flowchart illustrating a method for tracking a region of interest in an augmented reality providing system according to another exemplary embodiment of the present invention.

Assuming that the embodiment of FIG. 3 is applied to the system of FIG. 1, it will be described with reference to the drawings of FIGS. 1 and 3.

The image frame acquisition unit 101 may sequentially acquire at least one or more image frames (S301).

The context adaptation engine unit 103 determines whether the template exists in the storage unit 109 when the image frame acquisition unit 101 acquires the image frame (S303).

As a result of the determination in step S203, the context adaptation engine unit 103 calculates a correlation coefficient between the template stored in the storage unit 109 and the image frame (target image frame) acquired by the image frame acquisition unit (S305). .

The situation adaptation engine unit 103 determines whether or not the correlation coefficient calculated in step S305 is larger than the reference value Th (S307).

As a result of the determination in step S307, the situation adaptation engine unit 103 performs template-based tracking using the template stored in the storage unit 109 when the correlation coefficient is larger than the reference value Th (S307: Y) (S309). do.

As a result of the determination in step S307, the situation adaptation engine unit 103 extracts the feature point from the target video frame when the correlation coefficient is equal to or less than the reference value Th (S307: N) (S313).

The context adaptation engine unit 103 tracks the region of interest ROI in the target image frame by using the feature point extracted in operation S313 (S315).

The user interface unit 107 displays an image frame by reflecting the tracking result of step S309 or step S315 (S311).

On the other hand, the situation adaptation engine unit 103 generates a new template using the execution result of step S313 (S312), and updates a template previously stored in the storage unit 109 with the new template (S329).

The situation adaptation engine unit 103 ends the operation when the acquisition of the image frame is terminated (S333: Y), and the operation below S301 is performed when the acquisition of the image frame is not terminated (S333: N). .

As a result of the determination in step S303, if the template does not exist, the context adaptation engine 103 determines whether the ROI is selected in the target image frame (S319).

When the ROI is selected, the context adaptation engine unit 103 extracts a feature point from the selected ROI (S321), and uses the extracted feature point to store the feature point based DB stored in the storage unit 109. 109) (S323).

The context adaptation engine unit 103 recognizes the ROI by referring to the result of the operation S323 (S325) and generates the recognized ROI as a template (S327). The template generated in step S327 is stored in the storage unit 109.

4 is a flowchart illustrating a method for tracking a region of interest in an augmented reality providing system according to another exemplary embodiment of the present invention.

A case where the embodiment of FIG. 4 is applied to the system of FIG. 1 will be described with reference to the drawings of FIGS. 1 and 4.

The terminal 100 may sequentially acquire at least one or more image frames (S301).

When the terminal 100 obtains the image frame, it determines whether the template exists (stored) in the 100 (S303).

As a result of the determination in step S303, the terminal 100 calculates a correlation coefficient between the template and the target image frame (S305).

The terminal 100 determines whether the correlation coefficient calculated in step S305 is larger than the reference value Th (S307).

As a result of the determination in step S307, when the correlation coefficient is larger than the reference value Th (S307: Y), the terminal 100 performs template-based tracking using the template (S309).

If the correlation coefficient is less than or equal to the reference value Th (S307: N), the terminal 100 determines whether there is sufficient H / W and / or S / W resources from which the feature 100 extracts the feature point. Judge.

If it is determined that the resources of the 100 are sufficient, the terminal 100 extracts the feature points from the target image frame (S313). The terminal 100 tracks the region of interest (ROI) in the target image frame by using the feature point extracted in step S313 (S315).

The terminal 100 reflects the tracking result and displays an image frame (S311).

On the other hand, the terminal 100 generates a new template using the execution result of step S315 (S312), and updates and stores the previously stored template with the new template.

If it is determined that the resources of the 100 are not sufficient, the terminal 100 requests the server 200 to extract the feature points from the target image frame. Here, the terminal 100 extracts the feature points, for example. While transmitting the image frame to the server 200, the template may be transmitted together. Alternatively, the server 200 may transmit the feature point itself extracted from the template to the server 200 together with the image frame.

 The server 200 extracts a feature point from the target image frame by such a request, performs tracking based on the feature point, and transmits the tracking performance result to the terminal 100.

The terminal 100 displays the tracking performance result received from the server 200 (S311), generates a new template using the tracking performance result (S312), and stores the previously stored template as a new template. do. In operation S312, when a feature point extracted from an image frame and a feature point extracted from a template are the same or similar to each other, a new template may be generated based on the feature point extracted from the image frame.

On the other hand, if it is determined in step S303 that the template does not exist, the terminal 100 determines whether the ROI is selected in the target image frame (S319), and if the ROI is selected, In operation S319: Y, the server 200 may request recognition of the ROI while transmitting the target image frame.

The server 200 extracts a feature point from a region of interest (ROI) selected from a target image frame received from the terminal 100 (S321), and searches a feature-based DB stored by the own 200 using the extracted feature point. In step S323, the ROI is recognized (S325). Thereafter, the server 200 transmits the recognition result to the terminal 100, and the terminal 100 generates and stores a template based on the recognition result (S312).

The above-described methods may be implemented in the form of a program executable by a computer and stored in a recording medium. For example, the terminal may be configured to include a computer processor and a memory. In this configuration, the context adaptation engine may be configured by program instructions to be loaded and operated in a memory. Here, the computer processor performs an operation of loading and executing program instructions into a memory.

It will be apparent to those skilled in the art that various modifications and variations can be made in the present invention without departing from the spirit and scope of the invention. Accordingly, such modifications or variations are intended to fall within the scope of the appended claims.

100: terminal
101; Image frame acquisition unit 103: situation adaptation engine unit
105: communication unit 107: user interface unit
109:
200: Server
201: feature point extraction unit 203: search unit
205: ROI recognition unit 207: DB

Claims (25)

In the method of tracking a region of interest (ROI) in an augmented reality providing system,
Obtaining an image frame;
Calculating an accuracy of a previously stored template; And
Based on the accuracy, it is determined whether to perform feature point based tracking by comparing the feature point extracted from the image frame and the feature point extracted from the template, or to perform template matching based tracking by comparing the template and the image frame. Tracking the region of interest in the image frame according to a result; Method of interest tracking in the augmented reality providing system comprising a. The method of claim 1,
Determining whether a previously stored template exists; More,
If it is determined that a pre-stored template exists, calculating the accuracy and performing the tracking, wherein the region of interest tracking method in the augmented reality providing system. The method of claim 1,
Determining whether a previously stored template exists;
If it is determined that the pre-stored template does not exist, generating a template from the image frame; And
Storing the template generated in the step of generating the template; further comprising, area of interest tracking method in the augmented reality providing system. The method of claim 3,
Wherein the generating the template comprises:
Determining whether a region of interest is selected in the image frame;
Extracting feature points from the selected region of interest when the region of interest is selected;
Recognizing a region of interest by matching a feature point extracted from the selected region of interest with a DB; And
Generating the recognized region of interest as a template;
The DB includes a feature point of the images acquired in the service area to provide augmented reality, the area of interest tracking method in the augmented reality providing system. 5. The method of claim 4,
Searching for the additional content corresponding to the recognized region of interest in the DB;
The DB further includes additional content information corresponding to each feature point, the region of interest tracking method in the augmented reality providing system. The method of claim 1,
The augmented reality providing system includes a terminal and a server,
If the terminal has a resource for extracting the feature point from the image frame, the terminal performs an operation for extracting the feature point,
If the terminal does not have a resource for extracting a feature point from the image frame, the server performs an operation of extracting the feature point from the image frame. The method according to claim 6,
The server tracks the region of interest by comparing the feature point of the image frame extracted from the image frame and the feature point extracted from the template, and transmits the tracking result to the terminal, the region of interest in the augmented reality providing system. Tracking method. The method of claim 7, wherein
When the feature point of the image frame and the feature point of the template are the same, the terminal generates a new template from the feature point of the image frame, and updates and stores a previously stored template with the new template. How to track region of interest. The method of claim 1,
The step of calculating the accuracy,
Comprising the step of performing by comparing the image frame and the pre-stored template, region of interest tracking method in the augmented reality providing system. The method of claim 1,
The step of calculating the accuracy,
And a gray level value of the pixels constituting the template and a gray level value of the pixels constituting the image frame. The method of claim 10,
The step of calculating the accuracy,

To calculate the accuracy
here,

Is the gray level value of the i th pixel of the template,

Is the average gray level value of the pixels constituting the template,

Is the gray level value of the i th pixel of the image frame section (the portion of the image frame of the same size as the template) to be tracked,

Is the average gray level value of the pixels that make up the image frame section to be tracked (the portion of the image frame of the same size as the template), where N is defined as the size of the template (columns x rows). Zone tracking method. The method of claim 1,
The tracking of the ROI in the image frame may include performing template matching based tracking when the accuracy is greater than a reference value, and performing feature point based tracking when the accuracy is less than or equal to the reference value. Method of tracking region of interest in a reality providing system. A computer-readable recording medium having recorded thereon a program for executing a method according to any one of claims 1 to 12. In the markerless based augmented reality providing system,
A terminal for acquiring image frames and matching and displaying additional content with the obtained image frames; Including;
The terminal calculates the accuracy of a pre-stored template and compares the feature point extracted from the image frame with the feature point extracted from the template based on the calculated accuracy, or performs the feature-based tracking or the template and the image frame. Determining whether to perform a template matching-based tracking by comparing the, and tracking the region of interest in the image frame according to the determination result, markerless based augmented reality providing system. 15. The method of claim 14,
The terminal determines whether the pre-stored template exists, and calculates the accuracy when it is determined that the pre-stored template, markerless based augmented reality providing system. 15. The method of claim 14,
The terminal determines whether a pre-stored template exists, and if it is determined that the pre-stored template does not exist, to generate and store a template from the image frame, markerless based augmented reality providing system. 15. The method of claim 14,
If the feature point-based tracking results in the feature point of the image frame being identical to the feature point of the template, the terminal generates a new template from the feature point of the image frame, and updates and stores a previously stored template with the new template. Markerless based augmented reality providing system. 15. The method of claim 14,
The accuracy is
Markerless based augmented reality providing system that is calculated by comparing the image frame and the pre-stored template. 15. The method of claim 14,
The accuracy is
And a gray level value of the pixels constituting the template and a gray level value of the pixels constituting the image frame. 15. The method of claim 14,
The terminal performs the template matching based tracking when the accuracy is greater than the reference value, and performs the feature point based tracking when the accuracy is less than or equal to the reference value, markerless based augmented reality providing system. 15. The method of claim 14,
The server further includes;
If the terminal has a resource for extracting the feature point from the image frame, the terminal performs the operation of extracting the feature point from the image frame,
If the terminal does not have a resource to extract a feature point from the image frame, the server extracts the feature point from the image frame, markerless based augmented reality providing system. The method of claim 21,
The server tracks the ROI by comparing the feature points extracted from the image frame and the feature points extracted from the template, and transmits a tracking result to the terminal. In the terminal providing augmented reality,
An image frame obtaining unit obtaining an image frame; And
Compute the accuracy of the pre-stored template, and based on the accuracy, whether to perform a feature-based tracking by comparing the feature point extracted from the image frame and the feature point extracted from the template or template matching by comparing the template and the image frame A situation adaptation engine unit configured to determine whether to perform based tracking and to track a region of interest in the image frame according to the determination result; Lt; / RTI > 24. The method of claim 23,
And a storage unit,
The situation adaptation engine unit determines whether a template is stored in the storage unit,
And if it is determined that the template is stored in the storage unit, calculating the accuracy and tracking the region of interest in the image frame. 25. The method of claim 24,
The situation adaptation engine unit performs tracking based on the template matching when the accuracy is greater than a reference value, and performs the feature point based tracking when the accuracy is less than or equal to the reference value.

Images