KR100957189B1 - Augmented reality system using simple frame marker, and method therefor, and the recording media storing the program performing the said method - Google Patents

Augmented reality system using simple frame marker, and method therefor, and the recording media storing the program performing the said method Download PDF

Info

Publication number
KR100957189B1
KR100957189B1 KR1020080012852A KR20080012852A KR100957189B1 KR 100957189 B1 KR100957189 B1 KR 100957189B1 KR 1020080012852 A KR1020080012852 A KR 1020080012852A KR 20080012852 A KR20080012852 A KR 20080012852A KR 100957189 B1 KR100957189 B1 KR 100957189B1
Authority
KR
South Korea
Prior art keywords
marker
augmented reality
frame
information
image
Prior art date
Application number
KR1020080012852A
Other languages
Korean (ko)
Other versions
KR20090087545A (en
Inventor
김혜진
우운택
Original Assignee
광주과학기술원
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 광주과학기술원 filed Critical 광주과학기술원
Priority to KR1020080012852A priority Critical patent/KR100957189B1/en
Publication of KR20090087545A publication Critical patent/KR20090087545A/en
Application granted granted Critical
Publication of KR100957189B1 publication Critical patent/KR100957189B1/en

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/14—Digital output to display device ; Cooperation and interconnection of the display device with other functional units

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • General Engineering & Computer Science (AREA)
  • Human Computer Interaction (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Processing Or Creating Images (AREA)
  • User Interface Of Digital Computer (AREA)

Abstract

The present invention relates to an augmented reality system using a simple frame marker and a method thereof, and a recording medium on which a program for implementing the method is recorded. In particular, the marker can be easily found through a marker frame indicating the position of the marker. The purpose of the present invention is to efficiently improve both the speed of natural view recognition and marker frame tracking when augmented reality is realized using natural video images by using location information and orientation information set on a marker frame. . The present invention includes a simple frame marker comprising a marker representing a virtual object (natural view) of the augmented reality environment, and a marker frame surrounding the marker to display position information and orientation information of the marker. Provide a reality system.

Augmented reality, virtual reality, markers, marker frames, simple frame markers, natural view recognition, marker frame tracking, natural image

Description

Augmented reality system using simple frame marker, and method therefor, and the recording media storing the program performing the said method}

The present invention relates to an augmented reality system using a simple frame marker, a method thereof, and a recording medium having recorded thereon a program for implementing the method. In more detail, the marker frame can be easily found through a marker frame indicating the position of the marker, and the recognition process when the augmented reality is realized through the natural image image using the position information and the orientation information set in the marker frame (natural view) An augmented reality system using a simple frame marker that efficiently improves both the speed of recognition and marker frame tracking, and a method thereof, and a recording medium on which a program for implementing the method is recorded.

In general, Augmented Reality (AR) is a modified form of Virtual Reality (VR), and refers to a user interface technology that matches a computer-generated virtual image to a live image viewed by a user. Vision-based augmented reality may be implemented through optical-based systems or image-based systems. In the case of optical-based systems, augmented reality is realized by projecting virtual objects into a real environment using a translucent screen, HMD (Head Mounted Display). On the other hand, in the image-based system, augmented reality is implemented by synthesizing virtual information with an image acquired through a camera. Such augmented reality has been applied in various fields in medicine, industry, entertainment, military, etc. in order to provide a user with improved reality and awareness.

It is very important to accurately estimate the motion of the camera or tracking object to implement augmented reality. The conventional general augmented reality implementation method is as follows. First, augmented reality is realized using feature points collected from objects in the real world. This method recognizes feature points by performing a full search of the image every frame and performs a tracking function accordingly. Therefore, when the provided image or the texture is very complicated, the recognition speed or the tracking speed may be drastically degraded, resulting in a problem that a lot of time is required to implement augmented reality. In addition, when the collected feature points are low or environmental conditions such as lighting are unstable, a problem of greatly deteriorating performance also occurs.

Second, augmented reality is implemented using an AR marker. AR markers are artificially inserted into the real world in order to accurately estimate the position and orientation of the tracking subject. Such conventional AR markers required a unique ID that was arbitrarily assigned to enable recognition and tracking. Accordingly, in the related art, it is possible to find and enhance the position of the marker only by searching the image to extract the unique ID and interpreting the unique ID.

On the other hand, conventionally it was also possible to implement augmented reality using a 3D model. However, in this case, as shown in FIG. 1, when the 2D image 110 is matched with the 3D image 100, posture estimation according to the positional change is required, thereby reducing the speed of the recognition and tracking processes. Even if the AR marker is introduced to solve this problem, the slowdown in the recognition process and the tracking process is not eliminated due to the above-mentioned problems.

The present invention has been made to solve the above problems, augmented reality system having a simple frame marker (simple frame marker) consisting of a natural video image and a marker frame formed in a shape surrounding the natural video image and its method An object of the present invention is to provide a recording medium on which a program for implementing the method is recorded.

The present invention has been made to achieve the above object, in a system for implementing augmented reality, a marker representing a virtual object (natural view) of the augmented reality environment, surrounding the marker and the position information of the marker and It provides an augmented reality system comprising a simple frame marker having a marker frame for displaying azimuth information.

Preferably, the augmented reality system includes a marker recognizing unit for detecting the marker after detecting the marker frame in advance; A simple frame marker tracker to track or posture the marker using the marker frame; An image processor configured to augment a virtual image at a position of the marker in association with the marker recognizer and the simple frame marker tracker; And an image display unit displaying an image generated by the image processor through augmentation.

In addition, the present invention provides an augmented reality implementation method of a system for implementing augmented reality, (a) using a photographing or sensing surrounding the marker representing the virtual object in the augmented reality environment and the position information and orientation information of the marker Classifying a marker frame to be displayed; (b) recognizing the marker using location information of the classified marker frame; (c) tracking or posture estimation of the marker using information of the marker frame derived in step (a); (d) augmenting a virtual image at the position of the marker using the results of steps (a) to (c); And (e) provides an augmented reality implementation method comprising the step of displaying the image generated by the augmentation.

The present invention produces the following effects in accordance with the configuration and method described below. First, feature markers can be easily extracted by indicating the position of natural video images, and feature points can be collected only inside the marker frame. You can run Accordingly, it is possible to efficiently improve the speed in natural view recognition and marker frame tracking. Second, depth information for augmentation can be easily obtained by deriving position information and orientation information of the natural video image using a marker frame that is easier to recognize than the natural video image. In addition, it is possible to easily determine how much the natural video image is relatively inclined from the front, and thus the process of comparing the texture image or the 3D model by posture and recognizing it may be omitted. Third, it is suitable for augmented reality using mobile devices.

Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the accompanying drawings. First of all, in adding reference numerals to the components of each drawing, it should be noted that the same reference numerals are used as much as possible even if displayed on different drawings. In addition, in describing the present invention, when it is determined that the detailed description of the related known configuration or function may obscure the gist of the present invention, the detailed description thereof will be omitted. In addition, the following will describe a preferred embodiment of the present invention, but the technical idea of the present invention is not limited thereto and may be variously modified and modified by those skilled in the art.

2 is an internal block diagram of an augmented reality system according to a preferred embodiment of the present invention. As shown in FIG. 2, the augmented reality system 200 according to a preferred embodiment of the present invention includes a simple frame marker 210, a marker recognizer 220, and a simple frame marker tracker 230. , An image processor 240 and an image display unit 250.

The augmented reality system 200 according to the present invention includes a marker 211 provided in a real environment and a marker frame 212 surrounding the marker 211 to correspond to a virtual object to be augmented in augmented reality environment. Characterized by placing a simple frame marker 210 made. In the above description, the marker 211 may be represented by a logo, a picture or a picture frame, a symbol, or a character that displays a virtual object in an augmented reality environment. Then, the marker recognition unit 220 can easily identify the virtual object to be augmented using a visual sensor or the like. The marker 211 may be a natural view image in an embodiment of the present invention.

The simple frame marker 210 is composed of a marker frame 212 having a frame shape having a predetermined thickness at the edge of the marker 211 and the marker 211 in the embodiment of the present invention. Such a simple frame marker 210 may be represented as visible information or invisible information at the position of the virtual object in the augmented reality environment in the embodiment of the present invention. Hereinafter, the marker 211 and the marker frame 212 will be described in detail.

The marker frame 212 is implemented to have the same information as the position information and the orientation information of the marker 211 in the embodiment of the present invention. Accordingly, the marker recognition unit 220 and the simple frame marker tracking unit 230 may have an effect of easier function performance than before.

The marker 211 generally carries location information. Accordingly, the marker frame 212 indicates the position of the marker 211 as four vertex information in the embodiment of the present invention. As shown in FIG. 3, when the marker 211 is formed at the place where the virtual object is located in the AR environment, the marker frame 212 uses the four vertex information 311 to 314 to position the marker 211. Will be displayed. Then, not only can the position of the marker 211 be found more easily than before, but also the fast tracking can be realized.

On the other hand, the marker frame 212 may represent the position of the marker 211 as two or three vertex information in the embodiment of the present invention. In particular, when the marker frame 212 indicates the position of the virtual object in the augmented reality environment as two vertex information, it is noted that the two vertices are located in a diagonal direction. Referring to FIG. 3, two vertices in this case may correspond to 311 and 313 or 312 and 314. The vertex information may be represented by planar coordinate value information (x, y) or stereoscopic coordinate value information (x, y, z).

The marker frame 212 may augment a virtual image by using azimuth information considering a tilt in an embodiment of the present invention. Referring to FIG. 4, the 3D model 400 of the printer is displayed as shown in (b) according to the orientation information (that is, coordinate values for the x-axis, the y-axis, and the z-axis) of the marker frame 212 shown in (a). Augmentation can be implemented.

The marker recognition unit 220 performs a function of photographing and recognizing the simple frame marker 210 in the embodiment of the present invention. Specifically, the marker recognition unit 220 detects the marker frame 212 of the simple frame marker 210 first, and then recognizes the marker 211 through the marker frame 212.

The marker recognizer 220 may be implemented by including an RGB camera (or an optical sensor or a visual sensor) when the simple frame marker 210 is represented as visible information. On the other hand, the marker recognition unit 220 may consider an infrared camera (or IR sensor) in addition to the RGB camera when considering that the simple frame marker 210 is represented by invisible information using IR reflective coating materials. It may be implemented to include more.

Meanwhile, in the embodiment of the present invention, when the size of the simple frame marker 210 (or the marker frame 212) is very small, a recognition distance problem may arise. To this end, the marker recognition unit 220 may further include an optical zoom camera, and may insert a parameter to be referred to a calculation for easy tracking.

The simple frame marker tracking unit 230 performs a function of tracking the marker 211 and estimating a posture using the marker frame 212 in the embodiment of the present invention. For this purpose, the simple frame marker tracking unit 230 preferably secures vertex information of all marker frames 212 at all times.

The image processor 240 augments a previously prepared virtual image at a position where the simple frame marker 210 (specifically, the marker 211) is displayed in the embodiment of the present invention to generate a new image. In the above, the augmentation preferably comprises rendering. Rendering refers to giving colors or effects through three-dimensional computer graphics (CG) to each side of a real object drawn on the screen, thereby improving the three-dimensional and realism of the real object on the screen.

The image processor 240 additionally performs the following function to perform the function. That is, the image processor 240 monitors the position and posture of the marker 211 in real time using the marker frame 212 in cooperation with the simple frame marker tracker 230. Through this, the image processor 240 may execute virtual object augmentation at any time by implementing augmented reality.

Meanwhile, in the embodiment of the present invention, when the marker 211 represents a specific logo (eg, HP), it is necessary to discern which virtual object is to be augmented at the specific logo position. On the other hand, the image processor 240 may solve this problem by configuring the spatial map as a sensor and combining the context information with respect to the spatial map in the embodiment of the present invention.

The image display unit 250 performs a function of outputting an image from the image processor 240 to the screen in an embodiment of the present invention. Examples of the image display unit 250 include stereoscopic glasses such as HMD (Head Mounted Display), Crystal Eyes (Crystal Eyes), glasses type optical see-through HMD (HMD), etc. It may also be a liquid crystal display device (eg, a mobile phone, a PDA, etc.).

Next, the augmented reality implementation method of the augmented reality system 200 according to a preferred embodiment of the present invention. 5 is a flowchart illustrating a method of implementing augmented reality of an augmented reality system according to a preferred embodiment of the present invention.

Referring to FIG. 5, first, the marker recognition unit 220 searches for the marker frame 212 through photographing (or sensing) (S500). Thereafter, the marker recognition unit 220 determines the position of the marker frame 212 based on the data obtained in the result of operation S500 (S505). For example, the vertex information derivation of the marker frame 212 may correspond to this.

Thereafter, the marker recognizer 220 recognizes the marker 211 based on the result of step S505 (S510). The marker 211 is located inside the marker frame 212, so that the recognition process can be performed more easily than in the related art.

Next, the simple frame marker tracking unit 230 tracks the marker 211 using the marker frame 212 and estimates the attitude (S515). At this time, the simple frame marker tracker 230 utilizes vertex information (and parameters) of the marker frame 212. Thereafter, the image processor 240 augments the virtual image (eg, 3D CG content) at the position of the marker 211 based on the three additional functions described above (S520). In operation S520, a new image reflecting the virtual image is generated, which is displayed through the image display unit 250 (S525).

Meanwhile, the above-described embodiments of the present invention can be written as a program that can be executed in a computer, and can be implemented in a general-purpose digital computer that operates the program using a computer-readable recording medium. The computer-readable recording medium may include a magnetic storage medium (for example, a ROM, a floppy disk, a hard disk, etc.), an optical reading medium (for example, a CD-ROM, a DVD, etc.) and a carrier wave (for example, Storage media such as transmission over the Internet).

The above description is merely illustrative of the technical spirit of the present invention, and those skilled in the art to which the present invention pertains various modifications, changes, and substitutions without departing from the essential characteristics of the present invention. will be. Accordingly, the embodiments disclosed in the present invention and the accompanying drawings are not intended to limit the technical spirit of the present invention but to describe the present invention, and the scope of the technical idea of the present invention is not limited by the embodiments and the accompanying drawings. . The scope of protection of the present invention should be interpreted by the following claims, and all technical ideas within the scope equivalent thereto should be construed as being included in the scope of the present invention.

The augmented reality system according to the present invention efficiently improves both the speed of recognition and tracking when implementing augmented reality based on a natural video image through a simple frame marker composed of a natural video image and a marker frame surrounding the natural video image. The height can be effective. Considering that augmented reality can be applied to various fields, such as arranging home appliances using virtual images in a model house or providing information on scores or teams (or players) on the floor of a stadium during sports broadcasting, The industrial applicability of the augmented reality system according to the invention is expected to be very high. Moreover, these augmented reality systems are expected to contribute to speeding up the ubiquitous computing era.

1 is a conceptual diagram illustrating a problem of a conventional augmented reality system,

2 is an internal block diagram of an augmented reality system according to a preferred embodiment of the present invention;

3 and 4 is a conceptual diagram for explaining the characteristics of the augmented reality system according to a preferred embodiment of the present invention,

5 is a flowchart illustrating a method of implementing augmented reality of an augmented reality system according to a preferred embodiment of the present invention.

<Description of Symbols for Main Parts of Drawings>

200: augmented reality system according to the present invention 210: simple frame marker

211: marker 212: marker frame

220: marker recognition unit 230: simple frame marker tracking unit

240: image processing unit 250: image display unit

Claims (10)

In the system for implementing augmented reality, A simple frame marker comprising a marker representing a virtual object of the augmented reality environment, and a marker frame surrounding the marker to display position information and orientation information of the marker and invisibly. Augmented reality system comprising a. The method of claim 1, The marker is augmented reality system, characterized in that implemented as any one of a picture, a character and a sign indicating the virtual object. The method of claim 1, The marker frame has vertex information displayed using coordinate values as position information of the marker as one of two to four, and when the vertex information is two, the two vertex information are located in a diagonal direction. Augmented reality system, characterized in that derived from two vertices. The method of claim 1, A marker recognizing unit which detects the marker frame in advance and subsequently recognizes the marker; A simple frame marker tracker to track or posture the marker using the marker frame; An image processor configured to augment a virtual image at a position of the marker in association with the marker recognizer and the simple frame marker tracker; And Image display unit for displaying the image generated by the image processing unit augmentation Augmented reality system characterized in that it further comprises. delete The method of claim 1, The marker is an augmented reality system, characterized in that the image for a natural view (natural view). In the augmented reality implementation method of the system for implementing augmented reality, (a) classifying a marker frame that is displayed invisibly by enclosing a marker representing a virtual object of the augmented reality environment by using photographing or sensing, displaying position information and orientation information of the marker; (b) recognizing the marker using location information of the classified marker frame; (c) tracking or posture estimation of the marker using information of the marker frame derived in step (a); (d) augmenting a virtual image at the position of the marker using the results of steps (a) to (c); And (e) displaying the image generated by the augmentation Augmented reality implementation method comprising the. The method of claim 7, wherein The marker in the step (b) is augmented reality implementation method characterized in that it is implemented by any one of a picture, a character and a sign indicating the virtual object. The method of claim 7, wherein The marker frame in step (a) has at least one of two to four pieces of vertex information displayed using coordinate values as position information of the marker, and the two vertices when the vertex information is two. Augmented reality implementation method characterized in that the information is derived from two vertices located in the diagonal direction. In a computer-readable recording medium, A record carrier on which a program for implementing the method according to any one of claims 7 to 9 is recorded.
KR1020080012852A 2008-02-13 2008-02-13 Augmented reality system using simple frame marker, and method therefor, and the recording media storing the program performing the said method KR100957189B1 (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
KR1020080012852A KR100957189B1 (en) 2008-02-13 2008-02-13 Augmented reality system using simple frame marker, and method therefor, and the recording media storing the program performing the said method

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
KR1020080012852A KR100957189B1 (en) 2008-02-13 2008-02-13 Augmented reality system using simple frame marker, and method therefor, and the recording media storing the program performing the said method

Publications (2)

Publication Number Publication Date
KR20090087545A KR20090087545A (en) 2009-08-18
KR100957189B1 true KR100957189B1 (en) 2010-05-11

Family

ID=41206509

Family Applications (1)

Application Number Title Priority Date Filing Date
KR1020080012852A KR100957189B1 (en) 2008-02-13 2008-02-13 Augmented reality system using simple frame marker, and method therefor, and the recording media storing the program performing the said method

Country Status (1)

Country Link
KR (1) KR100957189B1 (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR101295862B1 (en) * 2010-11-25 2013-08-12 에스케이플래닛 주식회사 Karaoke apparatus and method thereof for providing augmented reality images

Families Citing this family (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR101325757B1 (en) * 2010-07-09 2013-11-08 주식회사 팬택 Apparatus and Method for providing augmented reality using generation of virtual marker
KR101062929B1 (en) * 2011-01-04 2011-09-07 (주)올라웍스 Method, terminal, and computer-readable recording medium for supporting collection of object included in the image which is taken
KR101056418B1 (en) * 2011-03-31 2011-08-11 주식회사 맥스트 Apparatus and method for tracking augmented reality contents using mobile sensors
KR101257889B1 (en) * 2011-04-26 2013-04-23 한국과학기술원 Apparatus and method of user input interface for portable device
KR101242551B1 (en) * 2011-05-30 2013-03-19 (주)포스트미디어 Stereo images display apparatus with stereo digital information display and stereo digital information display method in stereo images
KR102431712B1 (en) * 2017-09-04 2022-08-12 삼성전자 주식회사 Electronic apparatus, method for controlling thereof and computer program product thereof

Citations (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20070242886A1 (en) * 2004-04-26 2007-10-18 Ben St John Method for Determining the Position of a Marker in an Augmented Reality System

Patent Citations (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20070242886A1 (en) * 2004-04-26 2007-10-18 Ben St John Method for Determining the Position of a Marker in an Augmented Reality System

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR101295862B1 (en) * 2010-11-25 2013-08-12 에스케이플래닛 주식회사 Karaoke apparatus and method thereof for providing augmented reality images

Also Published As

Publication number Publication date
KR20090087545A (en) 2009-08-18

Similar Documents

Publication Publication Date Title
CN106875431B (en) Image tracking method with movement prediction and augmented reality implementation method
KR100957189B1 (en) Augmented reality system using simple frame marker, and method therefor, and the recording media storing the program performing the said method
KR100932634B1 (en) 3D web map service method and system using augmented reality
US8933966B2 (en) Image processing device, image processing method and program
US8983184B2 (en) Vision image information storage system and method thereof, and recording medium having recorded program for implementing method
KR101227255B1 (en) Marker size based interaction method and augmented reality system for realizing the same
US20140282220A1 (en) Presenting object models in augmented reality images
KR20140082610A (en) Method and apaaratus for augmented exhibition contents in portable terminal
KR101410273B1 (en) Method and apparatus for environment modeling for ar
CN107646109B (en) Managing feature data for environment mapping on an electronic device
US9361731B2 (en) Method and apparatus for displaying video on 3D map
Pucihar et al. Exploring the evolution of mobile augmented reality for future entertainment systems
CN111833458B (en) Image display method and device, equipment and computer readable storage medium
EP3906527B1 (en) Image bounding shape using 3d environment representation
JP2015001875A (en) Image processing apparatus, image processing method, program, print medium, and print-media set
CN113359986B (en) Augmented reality data display method and device, electronic equipment and storage medium
CN106780757B (en) Method for enhancing reality
CN116057577A (en) Map for augmented reality
CN111569421A (en) Virtual scene change synchronization method and system, VR playing equipment and storage medium
KR101522842B1 (en) Augmented reality system having simple frame marker for recognizing image and character, and apparatus thereof, and method of implementing augmented reality using the said system or the said apparatus
JP2012146305A (en) Device and method for providing window-like augmented reality
CN111918114A (en) Image display method, image display device, display equipment and computer readable storage medium
WO2023124691A1 (en) Display of augmented reality scene
CN110827411B (en) Method, device, equipment and storage medium for displaying augmented reality model of self-adaptive environment
Treskunov et al. Range Camera for Simple behind Display Interaction.

Legal Events

Date Code Title Description
A201 Request for examination
E902 Notification of reason for refusal
E701 Decision to grant or registration of patent right
GRNT Written decision to grant
FPAY Annual fee payment

Payment date: 20130313

Year of fee payment: 4

FPAY Annual fee payment

Payment date: 20140421

Year of fee payment: 5

LAPS Lapse due to unpaid annual fee