KR20120135967A - Method of using infrared marker based on augmented reality for providing marker information with real-time changing and producing multiple markers - Google Patents

Abstract

PURPOSE: A method for using infrared marker based on augmented reality for providing marker information with real-time changing and producing multiple markers is provided to prevent a maker from covering a desired object by not exposing a maker for a tracked object. CONSTITUTION: A marker information receiving unit receives information about an IR (Infrared Ray) marker from outside. An IR LED light emitting unit includes IR LEDs of an IR camera. An IR marker display unit transmits IR of the infrared LED light emitting unit to outside. A control unit controls the IR LED light emitting unit by using received IR ray markers information. The arrangement and radiation of the infrared LEDs include information about an AR(Augmented Reality) image.

Description

Method of using Infrared marker based on augmented reality for providing Marker information with real-time changing and producing multiple markers}

The present invention relates to an augmented reality-based infrared marker for real-time change of marker information and the production of a plurality of markers and a method of using the same. That is, the present invention relates to a marker for implementing augmented reality and a method for implementing augmented reality using the same, and more particularly, to a marker using an infrared LED and a method thereof.

Augmented Reality (AG) is a technology that superimposes a virtual object on the real world that the user sees with the eyes.It shows the virtual world with additional information in real time and displays it as a single image. Also called MR). Augmented reality, a concept that complements the real world with the virtual world, uses a virtual environment made of computer graphics, but the main role is the real environment. Computer graphics serve to provide additional information needed for the real world. This means that the user can blur the distinction between the real environment and the virtual screen by overlapping the 3D virtual image with the live image. Augmented reality technology, which is a mixture of real environment and virtual objects, provides the user with a better sense of reality and additional information.

For example, when the camera is illuminated with a smartphone camera, information such as the location of a nearby store, a phone number, etc. is displayed in a 3D image so that the user may be provided with virtual content.

There are GPS and LBS-based methods for this purpose, there is a vision, that is, a vision-based method, the present invention relates to a vision-based augmented reality system.

Vision-based augmented reality system is a marking technology applied to a building or a store in advance, and read through a camera, such marking technology is a QR (Quick Response) code, which is a kind of two-dimensional bar code, or a structured letter code (ePosition) code.

However, the conventional vision-based augmented reality system is a system that shoots a target object (TO) that is visible to the visible marker (Visible Marker, VM) and outputs a screen that renders a virtual image on the captured image In this conventional system, the visually viable markers are used to implement augmented reality accurately and quickly. However, since these markers are additives that must be artificially added to the traced object, they can be used to cover the actual targets. There is a problem that bothers you.

In addition, since the virtual image corresponding to each marker is specified because it is a visible marker, the respective markers must be separately provided for each object, and thus the application range is narrow.

In addition, the visually visible markers have a limitation that the virtual image may be acquired only when they are daytime and close because they are not recognized by the camera at a distance from the object or at night.

An object of the present invention is to provide an infrared marker and a method for implementing augmented reality using the same, which can be variously applied without the appearance of a marker applied to an object.

Infrared marker according to the present invention, the marker information receiving unit for receiving information from the outside, an infrared LED light emitting unit including a plurality of infrared LEDs recognized by the infrared camera, the infrared ray transmitting the infrared rays from the infrared LED light emitting unit to the outside And a marker display unit and a controller for controlling the infrared LED light emitting unit according to the information on the infrared marker received from the marker information receiving unit, and includes information about an augmented reality image by arranging and emitting light of the plurality of infrared LEDs. Characterized in that,

The infrared LED light emitting unit includes a location area indicating information on a relative position and attitude of the infrared camera and a type area indicating information on a type of augmented reality image to be implemented.

The location area is arranged so that four infrared LEDs correspond to the corners of the rectangle, and further includes two infrared LEDs positioned on two line segments connecting two infrared LEDs adjacent to one infrared LED corresponding to the corner of the rectangle. It is characterized by the specific features.

In addition, the kind region is characterized in that a plurality of infrared LEDs are arranged in a matrix form.

The method for implementing augmented reality according to the present invention includes a location area indicating information on a relative position and attitude of an infrared camera by a plurality of infrared LEDs, and a type representing information on the type of augmented reality image to be implemented by the plurality of LEDs. An image acquisition step of acquiring an image of an infrared marker composed of an area, and a posture of analyzing a relative position and attitude of an infrared marker with respect to an infrared camera through an infrared LED arranged in the location area of the infrared marker acquired in the image acquisition step. A type analysis step of analyzing a kind of augmented reality image through an infrared LED arranged in the type region of the infrared marker acquired in the image obtaining step, the posture analysis step and the type analysis step; Extract the augmented reality image corresponding to the posture and type of the marker Characterized in that it comprises a step.

In addition, the location area is arranged so that four infrared LEDs correspond to the corners of the rectangle, and two infrared LEDs are positioned on two line segments connecting two infrared LEDs adjacent to one infrared LED corresponding to the corner of the rectangle. The kind region is characterized in that a plurality of infrared LEDs are arranged in a matrix form,

In the posture analysis step, the LED position detection step for detecting the respective positions of the infrared LED by extracting a value corresponding to the pixel value range of the pre-stored infrared LED from the image obtained in the image acquisition step, extracted in the position detection step An outer point extraction step of extracting an outer point using a Convex Hull algorithm based on the center point of the infrared LED, an intermediate point extraction step of extracting an intermediate point from the outer points extracted in the outer point extraction step,

(Px is the X coordinate of each point, Py is the Y coordinate of each point, and n is the center point number of the extracted infrared LED.)

It is another specific feature that includes an upper left corner point extracting step of extracting the upper left corner point through the following equation by using the outer point and the middle point extracted in the outer point extraction step and the middle point extraction step.

(Pn is the coordinate of the outer point, not the midpoint, and C1 and C2 represent the midpoint.)

The type analyzing step may further include analyzing a type of an augmented reality image by lighting each of a plurality of infrared LEDs arranged in the type region of the infrared marker obtained in the image obtaining step.

According to the present invention,

Since the markers applied to the tracer are not visible, they do not obscure or obstruct the actual target.

In addition, since a plurality of images corresponding to one marker may be implemented, it is not necessary to have a separate marker for each object, thereby enabling various applications.

In addition, since the infrared marker is photographed by the infrared camera, the virtual image can be output even if the distance from the object is far, and the virtual image can be output even at night.

1 shows an augmented reality system according to the present invention.
2 is a schematic diagram of a portion of an augmented reality system according to the present invention.
3 is a block diagram of an infrared marker applied to the present invention.
Figure 4 shows the outer points extracted by using the Convex Hull algorithm from the infrared marker.
5 is a configuration diagram of detecting the upper left corner point of the infrared marker.
6 is a block diagram for extracting the marker type of the infrared marker.
7 illustrates a virtual image implemented by an infrared marker according to the present invention.

Preferred embodiments of the present invention will be described in more detail with reference to the accompanying drawings, and descriptions of well-known techniques will be omitted.

Figure 1 shows an augmented reality system according to the present invention, Figure 2 is a schematic of a portion of the augmented reality system according to the present invention.

1 to 2, the augmented reality system 100 according to the present invention includes an object 110, an augmented reality apparatus 120, a marker information transmitter 130, and an output device 140. It is composed.

The object 110 is an object having an infrared marker for augmented reality, the object to which the user wants to view virtual content, and the marker information transmitter 130 provides information on the infrared marker IM to be implemented in the object 110. Means for transmitting.

The infrared marker IM of the present invention applied to the object 110 includes a marker information receiver, an infrared LED light emitter, an infrared marker display, and a controller.

The marker information receiver is configured to receive information on the infrared marker IM transmitted from the marker information transmitter 130, and the infrared LED light emitting unit is arranged in a plurality of infrared LEDs so that each of the plurality of infrared LEDs emits light by power control. It is a configuration. The infrared marker display unit allows light emitted from the infrared LED to be transmitted to the outside of the object 110 and corresponds to a kind of cover.

That is, the marker information transmitter 130 transmits a control signal for implementing information on the corresponding object 110 in the form of an infrared marker IM on the object 110, and marker information of the object 110. The receiver receives the control signal transmitted from the marker information transmitter 130. The control signal received by the marker information receiving unit is transmitted to the control unit, and the control unit controls the power of each infrared LED of the infrared LED light emitting unit according to the control signal. The infrared LED emitted by the power control of the controller is transmitted to the outside through the infrared marker display unit.

Here, the marker information transmitter 130 and the marker information receiver described above may transmit and receive information by wired or wireless communication networks such as wired, Zigbee, and Bluetooth.

Next, the augmented reality implementing apparatus 120 captures an actual image of the object 110 and an infrared marker IM of the object 110, and renders a virtual image read from the infrared marker IM on the captured real image. As a device for generating a new image, the apparatus includes a visible light camera unit 121, an infrared camera unit 122, an image processing unit 123, a virtual image DB 124, and a communication unit 125.

The visible light camera unit 121 of the apparatus 100 for augmented reality is a means for capturing an actual image of the object 110 and receives light in a visible wavelength range.

Augmented reality implementation apparatus 120 in the present invention is configured to further include an infrared camera unit 122 in addition to the visible light camera unit 121.

The infrared camera unit 122 is a means for photographing the infrared marker IM of the object 110 and receives light in an infrared wavelength region longer than the visible wavelength region.

Images captured by the two camera units 121 and 122 are acquired by the image processor 123. The image processor 123 converts the analog image signals received from the two camera units 121 and 122 into digital image signals and processes the image signals digitally by the digital signal processing processor.

That is, the image processing unit 123 tracks and analyzes the position and posture of the infrared marker with respect to the infrared camera unit 122 in real time from the image signal obtained by photographing the infrared marker IM, thereby analyzing the position and posture of the object 110. Obtains the information and analyzes the position and type of the infrared marker IM based on the acquired position and posture information of the object 110, and then the virtual image DB 124 corresponding to the position and type of the infrared marker IM. The saved virtual image is received.

Here, the attitude means the angle and the direction, and means information about the angle and the direction of the infrared marker IM with respect to the infrared camera unit 122 by the imaging of the infrared camera unit 122.

The analysis of the position and type of the infrared marker IM will be described later.

Thus, an augmented reality image overlapping the virtual image with the image obtained by photographing the actual image is generated.

As shown, the virtual image DB 124 is a means for storing a virtual image corresponding to the infrared marker IM, and the communication unit 125 may be configured in addition or exclusively to the virtual image DB 124.

The communication unit 125 is a means capable of transmitting information to and receiving information from an external network through a communication network. The control unit 125 controls the image processing unit 123 to generate infrared rays from an image signal obtained by photographing an infrared marker IM. By analyzing the position and type of the marker IM, the virtual image corresponding to the infrared marker IM can be received from a server (not shown) existing in an external network via a communication network.

By the above configuration, the augmented reality image generated by the augmented reality implementation apparatus 120 by photographing the object 110 may be configured to be output by the output device 140 that is a display means for outputting it, which is a mobile phone It may be a three-dimensional glasses such as a display, a computer monitor, a head mounted display (HMD), crystal eyes (Optical See-Through HMD), or the like.

3 to 6 relate to the configuration of the infrared LED light emitting unit of the infrared marker (IM) applied to the present invention and a method of analyzing the position and type of the object therefrom.

Figure 3 shows the arrangement of the infrared LED light emitting unit which is one configuration of the infrared marker IM applied to the present invention.

Referring to FIG. 3, the infrared LED light emitting unit of the infrared marker IM includes a marker position region 220 and a marker type region 230.

The marker position area 220 is an area in which six infrared LEDs 310, 320, 330, 340, 350, and 360 for displaying position information of the infrared camera unit 122 of the infrared marker IM are arranged, and the marker type region 230 is an infrared marker ( It is an area arranged with four infrared LEDs (510, 520, 530, 540) for indicating information on the type of IM), that is, the type of virtual image to implement augmented reality.

The number of infrared LEDs in each region is the simplest example, and of course, more applications are possible depending on the arrangement of LEDs of a higher quantity.

Since the image processor 123 stores the pixel value range of the infrared LED in advance, first extracts a value corresponding to the pixel value range of the infrared LED obtained from the image received through the infrared camera 122. The center point of the LED position is extracted.

The center point refers to the center pixel of the LED, and the position of each LED is detected by calculating the center pixel of the LED.

That is, the center point is extracted by scanning the entire input image in pixel units to find the position of the LED in the input infrared image. If the LED is found and the pixel falls within the RGB color value range, the area is gradually searched for adjacent pixels by spreading the area up, down, left and right.

Next, after detecting the center point of each LED, the outer points are extracted by using a known Convex Hull algorithm based on the center point of the extracted infrared LED.

4 shows the outer points extracted by using the Convex Hull algorithm.

This outer point is for extracting posture information regarding which direction the infrared camera unit 122 is looking at the infrared marker IM. In other words, when one infrared LED is used, the position can be easily extracted, but information about which direction the camera is looking at cannot be understood. Thus, by arranging the LEDs at the four corners of the marker position region 220, the angle viewed by the camera can be known.

However, only the corners of the four corners show the angle the camera looks at, but not which direction it is looking at. In order to accurately grasp in which direction the camera is looking and to augment the virtual image accordingly, detection of the upper left corner point, which is a reference point of the marker position region 220, is required, and the infrared ray of the present invention is required to clarify the upper left corner point. In the marker, the concept of midpoints 320 and 360 is added.

Extraction of the midpoints 320 and 360 of the extracted outer points may be extracted through Equation 1.

(Px is the X coordinate of each point, Py is the Y coordinate of each point, and n is the center point number of the extracted infrared LED.)

5 is a configuration diagram of detecting the upper left corner point of the marker.

Midpoints 320, 360 for each corner point 310, 330, 340, 350 of the six outline points 310, 320, 330, 340, 350, 360 extracted to extract the upper left point of the marker ) Is to select the corner point extracted as the minimum value as the upper left corner point.

The upper left corner point may be extracted through Equation 2.

(Pn is the coordinate of the corner points 310, 330, 340, 350, and C1 and C2 represent intermediate points 320, 360, respectively. In the case of P1 310, P1 310 and C1 ( The distance 370 between 320 and the sum of the distances 380 between P1 310 and C2 360, and in the case of P2 330, the distance 390 between P2 330 and C1 320. ) And the distance 400 between the P2 330 and the C2 360, and in the case of the P3 340, the distance 410 between the P3 340 and the C1 320, and the P3 340. And the distance 420 between C2 360, and in the case of P4 350, the distance 430 between P4 350 and C1 320, and between P4 350 and C2 360. Sum of distances (440).)

As described above, since the camera is calculated in which direction the upper left corner point is detected, the augmented image is also generated according to the posture of the camera and overlapped with the image, which is illustrated in FIG. 7.

6 is a block diagram for extracting the marker type of the infrared marker.

Points 510, 520, 530, and 540 indicating the marker type indicate marker types (marker numbers) by turning on / off the infrared LED, and ON / OFF of the points 510, 520, 530, and 540 indicating the marker type. Marker number according to the following Table 1. Among the values of the points 510, 520, 530, and 540, 0 is an infrared LED turned off and 1 is an infrared led turned on.

Point 1 (510) Dot 2 (520) Dot3 (530) 4 points (540) Marker number 0 0 0 0 0 0 0 0 One One 0 0 One 0 2 0 0 One One 3 0 One 0 0 4 0 One 0 One 5 0 One One 0 6 0 One One One 7 One 0 0 0 8 One 0 0 One 9 One 0 One 0 10 One 0 One One 11 One One 0 0 12 One One 0 One 13 One One One 0 14 One One One One 15

Points representing the marker types will each have a value corresponding to the n power of 2. As shown in Table 1, point 1 (510) corresponds to the third power of 2, and point 2 (520) equals 2 to 2. Point 4 and point 3 530 corresponding to a win have a value of 1 corresponding to a power of 2 and point 4 540 corresponding to a power of 2.

As described above, the marker type region may be utilized in various ways, such as 2 × 2, 3 × 3, 3 × 4, 4 × 4, etc. according to the number of types of markers, and in the case of 3 × 3, the number of markers may be 512.

7 is an example of a state in which points 1 510 and 4 540 are turned on. As shown in the above table, this state corresponds to marker number 9, and an image corresponding to marker number 9 is implemented. Is an example in which the type of marker number 9 represents a three-dimensional truck image.

While the present invention has been particularly shown and described with reference to exemplary embodiments thereof, it is to be understood that the invention is not limited to the disclosed exemplary embodiments. And the scope of the present invention should be understood as the following claims and their equivalents.

IM: infrared marker
220: marker location area
230: marker type area

Claims (9)

Marker information receiving unit for receiving information from the outside,
Infrared LED light emitting unit including a plurality of infrared LEDs recognized by the infrared camera,
An infrared marker display unit for transmitting infrared rays from the infrared LED light emitting unit to the outside;
Including a control unit for controlling the infrared LED light emitting unit by the information on the infrared marker received from the marker information receiving unit,
Infrared markers, characterized in that containing the information on the augmented reality image by the array and the emission of the plurality of infrared LEDs. The method of claim 1,
The infrared LED light emitting unit,
A location area indicating information on a relative position and attitude of the infrared camera;
Infrared marker, characterized in that consisting of the type region indicating information on the type of augmented reality image to be implemented. The method of claim 2,
The location area,
Infrared marker, characterized in that four infrared LEDs are arranged to correspond to the corner of the rectangle. The method of claim 3,
And two infrared LEDs positioned on two line segments connecting two infrared LEDs adjacent to one infrared LED corresponding to a corner of the quadrangle. The method of claim 2,
The kind area is,
Infrared marker, characterized in that a plurality of infrared LED is arranged in a matrix form. Acquiring an image of an infrared marker composed of a location area indicating information on the relative position and attitude of the infrared camera by a plurality of infrared LEDs and a type area indicating information on the type of augmented reality image to be implemented by the plurality of LEDs. Image Acquisition Stage,
A posture analysis step of analyzing a relative position and posture of the infrared marker of the infrared marker with the infrared LED arranged in the position area of the infrared marker obtained in the image acquisition step;
A type analysis step of analyzing a type of an augmented reality image through an infrared LED arranged in the type region of the infrared marker obtained in the image acquisition step;
And extracting an augmented reality image corresponding to the posture and type of the infrared marker analyzed in the posture analysis step and the type analysis step. The method according to claim 6,
The location area is arranged so that four infrared LEDs correspond to the corners of the rectangle, and two infrared LEDs are positioned on two line segments connecting two infrared LEDs adjacent to one infrared LED corresponding to the corner of the rectangle.
The type region, augmented reality implementation method using an infrared marker, characterized in that a plurality of infrared LED is arranged in a matrix form. The method according to claim 6,
The posture analysis step,
LED position detection step of detecting the position of each of the infrared LED by extracting a value corresponding to the pixel value range of the previously stored infrared LED from the image obtained in the image acquisition step,
An outer point extraction step of extracting an outer point using a Convex Hull algorithm based on the center point of the infrared LED extracted in the position detecting step,
An intermediate point extraction step of extracting an intermediate point through the following equation among the outer points extracted in the outer point extraction step;

(Px is the X coordinate of each point, Py is the Y coordinate of each point, and n is the center point number of the extracted infrared LED.)
Augmented reality using an infrared marker, comprising extracting the upper left corner point by extracting the upper left corner point by using the outer point and the middle point extracted in the outer point extraction step and the middle point extraction step Implementation method.

(Pn is the coordinate of the outer point, not the midpoint, and C1 and C2 represent the midpoint.) The method according to claim 6,
The type analysis step,
And analyzing the type of augmented reality image by lighting each of a plurality of infrared LEDs arranged in the type region of the infrared marker obtained in the image acquisition step.

Images