KR20130089032A - Augmented reality system using the morse code - Google Patents

Abstract

PURPOSE: An augmented reality system based on Morse code does not expose a marker applied to a tracking object, thereby preventing the marker from blocking an actually desired object or irritating the sight. CONSTITUTION: An object produces an IR marker with an infrared ray (IR) light emitting diode (LED). A code provided by the IR marker is Morse code. A visible ray camera unit (121) shoots a real image of the object. An IR camera unit (122) shoots the IR marker of the object. An image processor (123) analyzes an image signal obtained by shooting the IR marker in real time, and interprets the information on an actual image and a virtual image to render, a location of the virtual image to render, and a posture of the virtual image to locate. [Reference numerals] (121) Visible ray camera unit; (122) Infrared ray camera unit; (123) Image processing unit; (124) Virtual image DB; (125) Communication unit

Description

Augmented Reality System based on Morse Code {AUGMENTED REALITY SYSTEM USING THE MORSE CODE}

The present invention relates to an augmented reality system, and more particularly to an augmented reality system based on Morse code.

Augmented Reality (AG) is a technology that superimposes a virtual object on the real world that the user sees with the eyes, and shows the virtual world with additional information in real time as a single image to show mixed images. Also called MR).

Augmented reality, a concept that complements the real world using the virtual world, uses a virtual environment made of computer graphics, but the main role is the real environment, and computer graphics plays a role of providing additional information needed in the real environment. This means that the user can blur the distinction between the real environment and the virtual environment by overlapping the 3D virtual image with the real image that the user sees. Augmented reality technology, which is a mixture of real environment and virtual objects, provides the user with a better sense of reality and additional information.

For example, when the camera is illuminated with a smartphone camera, information such as the location of a nearby store, a phone number, etc. is displayed in a 3D image so that the user may be provided with virtual content.

There is a method for GPS and LBS-based, there is a vision, that is, a vision-based method, the present invention relates to augmented reality system based on the vision, but using Morse code.

In the vision-based augmented reality system, markers applied to buildings or shops are read by a camera. Such marking technology includes a QR (Quick Response) code, which is a kind of two-dimensional bar code, or a structured letter code ( ePosition) code.

However, in the conventional vision-based augmented reality system, a screen shot of a target object (TO) in which a visible marker (VM) is visible to the naked eye is displayed and a virtual image is rendered on the actual image of the captured object. In the conventional system, since the marker is visible to the naked eye, the augmented reality can be accurately and quickly implemented. However, since the marker is an additive to be artificially added to the tracking object, an actual target object may be used. There is a problem that obscures or disturbs the naked eye.

In addition, the respective markers corresponding to the object must be provided separately, which limits the application area.

In addition, the visually visible markers have a limitation that the virtual image may be acquired only when they are daytime and close because they are not recognized by the camera at a distance from the object or at night.

An object of the present invention is to provide an augmented reality system that can provide a variety of information by using a Morse code that can be combined in various combinations without the appearance of a marker applied to the object in appearance. There is this.

Augmented reality system of the present invention, the object generating an infrared marker by the infrared LED; And an augmented reality implementation apparatus for capturing an actual image of the object and an infrared marker of the object to render a virtual image read from the infrared marker, wherein the code provided by the infrared marker may be a Morse code. .

The apparatus for implementing augmented reality may include: a visible light camera unit for photographing an actual image of the object; An infrared camera unit photographing a lighting state of the infrared LED; And an image processor which generates a virtual reality image by overlapping the real image and the virtual image.

The apparatus for implementing augmented reality may include: a position detection unit extracting information on a position where the virtual image is to be disposed in the virtual reality image from the lighting state of the photographed LED; And a type detector configured to extract information about contents of a virtual image to be displayed from the lighting state of the photographed LED.

The apparatus for implementing augmented reality may include: a virtual image DB for storing the virtual image; It may further include, and may further include a communication unit capable of transmitting and receiving information through a communication network.

Augmented reality system of the present invention is a marker information transmitter for transmitting information about the infrared marker to be implemented on the object; The object may further include a marker information receiver configured to receive information on an infrared marker from the marker information transmitter; An infrared marker display unit which transmits infrared rays from the infrared LED light emitting unit to the outside; And a controller configured to control the infrared LED light emitting unit based on the information on the infrared marker received from the marker information receiving unit. You may.

According to the present invention,

Since the markers applied to the tracer are not apparent in appearance, they do not obscure or distract the actual desired object.

In addition, there is no need to provide a separate marker for each object, and since a large amount of markers can be generated at a time, various applications are possible.

In addition, since the infrared marker is photographed by the infrared camera, the virtual image can be output even if the distance from the object is far, and the virtual image can be output even at night.

In addition, it is possible to reproduce a more perfect image by correcting the infrared LED that can be revealed in the real image.

Furthermore, since the information on the position or type of the object provided by the Morse code is utilized, the information on the object can be provided through a simple configuration capable of displaying the Morse code, thereby simplifying the configuration of the system.

1 shows an augmented reality system according to the present invention.
2 is a schematic diagram of a portion of an augmented reality system according to the present invention.
Figure 3 shows the application of the augmented reality system according to the present invention.
4 is a configuration diagram of an infrared marker applied to the present invention.
Figure 5 shows the outer points extracted by using the Convex Hull algorithm from the infrared marker.
6 is a block diagram of upper left corner point detection of a virtual image.
7 is a configuration diagram of the type region in the infrared marker.
Fig. 8 illustrates the configuration of the Morse code used in the present invention.

Preferred embodiments of the present invention will be described in more detail with reference to the accompanying drawings, and descriptions of well-known techniques will be omitted.

Figure 1 shows an augmented reality system according to the present invention, Figure 2 is a schematic of a portion of the augmented reality system according to the present invention.

1 to 2, the augmented reality system 100 according to the present invention includes an object 110, an augmented reality implementing device 120, a marker information transmitter 130, and an output device 140. It is composed.

The object 110 has an infrared marker implemented by an infrared LED, and is a target object for which a user wants to view virtual content, and the marker information transmitter 130 transmits information on an infrared marker to be implemented on the object 110. It is meant for.

The object 110 in the augmented reality system 100 of the present invention includes a marker information receiver (not shown), an infrared LED light emitting unit (not shown), an infrared marker display unit (IM), and a controller (not shown). .

The marker information receiving unit is configured to receive information on the infrared marker transmitted from the marker information transmitter 130, and the infrared LED light emitting unit is configured to emit each of the plurality of infrared LEDs by power control because a plurality of infrared LEDs are arranged. to be. The infrared marker display unit IM is configured to allow the light of the emitted infrared LED to be transmitted to the outside of the object 110.

That is, the marker information transmitter 130 transmits a control signal for implementing information on the corresponding object 110 in the form of an infrared marker on the object 110, and the marker information receiving unit of the object 110 is a marker. The control signal transmitted from the information transmitter 130 is received. The control signal received by the marker information receiving unit is transmitted to the control unit, and the control unit controls the power of each infrared LED of the infrared LED light emitting unit according to the control signal. The infrared LED emitted by the power control of the controller is transmitted to the outside through the infrared marker display unit IM.

Here, a blocking film (not shown) made of translucent colored acrylic or the like may be further configured so that the infrared light emitting part is not recognized from the outside.

The marker information transmitter 130 and the marker information receiver described above may transmit and receive information by a wired or wireless communication network such as a wired line, a Zigbee, or a Bluetooth.

Next, the augmented reality implementation apparatus 120 photographs the real image of the object 110 and the infrared marker of the object 110, and renders a virtual image read from the infrared marker on the real image to generate a new image. The apparatus includes a visible light camera unit 121, an infrared camera unit 122, an image processing unit 123, a virtual image DB 124, and a communication unit 125.

The visible light camera unit 121 of the apparatus 100 for augmented reality is a means for capturing an actual image of the object 110 and receives light in a visible wavelength range.

Augmented reality implementation apparatus 120 in the present invention is configured to further include an infrared camera unit 122 in addition to the visible light camera unit 121.

The infrared camera unit 122 is a means for photographing an infrared marker of the object 110 and receives light in an infrared wavelength region longer than the visible wavelength region.

Images captured by the two camera units 121 and 122 are acquired by the image processor 123. The image processor 123 converts analog image signals received from the two camera units 121 and 122 into digital image signals, and then digitally processes the image signals using a digital signal processing processor.

That is, the image processor 123 analyzes an image signal obtained by capturing an infrared marker of the object 110 in real time, and the virtual image to be rendered from the analyzed image signal should be disposed. Interpret the information about the position to be done and the posture of the virtual image to be arranged.

The virtual image to be displayed together with the actual image at the time of rendering is stored in the virtual image DB 124. The virtual image DB 124 may be additionally or exclusively provided by the communication unit 125.

The infrared marker may include a plurality of infrared LEDs, and the plurality of infrared LEDs may be arranged in a two-dimensional form. In this case, some of the infrared LEDs may provide information on the position and posture of the virtual image to be rendered, and other infrared LEDs may be configured to provide information on the content of the virtual image that needs to be displayed. Can be.

The infrared marker image captured by the infrared camera 122 is transmitted to the position detector 126 and the type detector 127. The position detector 126 is a component for extracting information about a position where the virtual image is to be arranged and a posture of the arranged virtual image from the photographed infrared marker image. The type detector 127 is a component for extracting information on the content of the marker for which display is required from the captured infrared marker image.

In this case, the infrared LED may be classified into an LED providing information and an LED providing information on the content according to the arranged position. When the use of the infrared LED is classified by the arrangement position as described above, the position detector 126 and the type detector 127 may classify the infrared LEDs for which they need to obtain information according to the arrangement position.

The image processor 123 uses the information received from the position detector 126 and the type detector 127 to determine the type of virtual image to be displayed and the position and posture to be displayed. The image processor 123 retrieves and calls the virtual image to be displayed in the virtual image DB 124.

The image processor 123 generates the augmented reality image by overlapping the actual image photographed by the visible light camera unit 121 with the called virtual image.

The generated augmented reality image is displayed by the output device 140 for. As the output device 140, a stereoscopic glasses such as a display of a mobile phone, a computer monitor, a head mounted display (HMD), crystal eyes, or the like, an optical see-through HMD (HMD) may be used.

In this case, the infrared LED may not be recognized by the naked eye, but may be recognized by the visible light camera unit 121, and as a result, the image obtained by the visible light camera unit 121 may include not only an actual image of the object but also infrared light. There may be an image of the LED, which acts as a kind of noise in the augmented reality image. Accordingly, in the present invention, the image processor 123 of the apparatus for implementing augmented reality 120 may obtain an actual image of an object whose noise is corrected.

That is, the image processing unit 123 extracts the position information of the infrared marker, that is, the infrared LED, from the image acquired from the infrared camera unit 122 through the Convex Hull algorithm, and uses the extracted infrared LED position information to display the visible light camera. The region where the infrared LED present in the real image acquired from the unit 121 is photographed is corrected.

As a correction method, an inpainting technique in which the light emitting area of the infrared LED is inferred and filled from the surrounding real image area may be used. As a result, a virtual reality image more fully realized may be obtained. The corrected virtual reality image is displayed through the output device 140 described above.

Hereinafter, with reference to Figures 4 to 6 will be described in more detail the configuration of the infrared marker 210 applied to the present invention and a method for interpreting the necessary information therefrom. In this case, the infrared marker 210 is a term used to collectively refer to an image obtained through the infrared camera unit 122 or a form in which the light emitting unit of the infrared LED is arranged in the acquired image.

4 is a block diagram of an infrared marker 210 applied to the present invention.

Referring to FIG. 4, the infrared marker 210 includes a location area 220 and a type area 230.

Since the image processing unit 123 stores the pixel value range of the infrared LED in advance, the image processing unit 123 extracts a value corresponding to the pixel value range of the infrared LED obtained from the image input through the infrared camera unit 122 to locate the infrared LED. Extract the center point to the starting point of, and extract the outer points using the known Convex Hull algorithm based on the center point of the extracted infrared LED.

5 shows the outer points extracted by using the Convex Hull algorithm.

Extraction of the midpoints 320 and 360 of the extracted outer points may be extracted through Equation 1.

(Px is the X coordinate of each point, Py is the Y coordinate of each point, and n is the center point number of the extracted infrared LED.)

FIG. 6 is a diagram illustrating a process of detecting a position where a virtual image is to be disposed in a process of overlapping with an actual image. The position where the virtual image is to be arranged is determined by the point where the upper left point of the virtual image is disposed.

An intermediate point 320 for each corner point 310, 330, 340, 350 of the six outer points 310, 320, 330, 340, 350, 360 extracted to extract the upper left point of the virtual image 360) and select the corner point extracted as the minimum value as the upper left corner point.

The upper left corner point may be extracted through Equation 2.

(Pn is the coordinate of the corner points 310, 330, 340, 350, and C1 and C2 represent intermediate points 320, 360, respectively. In the case of P1 310, P1 310 and C1 ( The distance 370 between 320 and the sum of the distances 380 between P1 310 and C2 360, and in the case of P2 330, the distance 390 between P2 330 and C1 320. ) And the distance 400 between the P2 330 and the C2 360, and in the case of the P3 340, the distance 410 between the P3 340 and the C1 320, and the P3 340. And the distance 420 between C2 360, and in the case of P4 350, the distance 430 between P4 350 and C1 320, and between P4 350 and C2 360. Sum of distances (440).)

FIG. 7 illustrates the type region 230 of the infrared marker 210. When the type region information is analyzed, the type of the virtual image to be used for the virtual reality image may be known.

Although FIG. 7 illustrates a case where four infrared LEDs are arranged in the form of a 2 × 2 matrix, the arrangement of the infrared LEDs in the type region 230 is not limited thereto. For example, it may be arranged linearly, such as 1XN form, an appropriate number of NXN form may be used according to the format of the information provided by the infrared LED. In particular, the infrared LED may be interpreted to mean '1' when it is turned on (ON state) and to mean '0' when it is turned off (OFF state). Therefore, only one infrared LED may be used if necessary. have.

In the present invention, the Morse code is applied as a method of providing information in the type region 230. 8 shows a form of providing information of a type region according to the present invention, and illustrates the configuration of a Morse code used in the present invention.

8 exemplifies a case where a Morse code is generated using 90 pieces of data. The configuration of the infrared LED for this may be a 9X10 arrangement, but may be configured as an infrared LED in the form of 1 * X, only one infrared LED may be used. In FIG. 8, '1' means a case where the infrared LED is turned on, and '0' means a case where the infrared LED is turned off.

On the other hand, since the Morse code is composed only of a combination of 1 and 0, it is possible to decipher the code only when a rule is provided to identify a start point and an end point of providing information. .

To do this, you can use an array of specific numbers as the termination code. For example, it is possible to use the case where the third occurrence of '1' is used as the termination code and to provide the information again after the termination course.

On the other hand, the area may be classified according to the number or position of the infrared LEDs, and may be implemented by a method of pre-allocating the information providing information data and the data gap (inter gap).

FIG. 8 illustrates a case in which the initial six infrared LEDs provide information data for each line, and the subsequent three LEDs are assigned to mean data intervals. Referring to FIG. 8, when the Morse code provided by the initial six data is analyzed, the type of virtual image required by the Morse code can be known, and the DB 124 searches for the virtual image. do. After the virtual image matching the Morse code is retrieved from the virtual image DB 124, the process of generating and displaying the virtual reality image by overlapping with the actual image is the same as described above, and thus, repeated description thereof will be omitted.

Although the detailed description of the present invention described above has been made by the embodiments with reference to the accompanying drawings, the above-described embodiment is only described with a preferred example of the present invention, the present invention is limited only to the above embodiments It should not be understood that the scope of the present invention is to be understood by the claims and equivalent concepts described below.

100: augmented reality system 110: the object
120: augmented reality realization device 121: infrared camera unit
122: visible light camera unit 123: image processing unit
124: virtual image DB 125: communication unit
126: position detector 127: type detector
130: marker information transmitter 140: output device
210: infrared marker 220: location area
230: Kind area

Claims (7)

An object generating an infrared marker by the infrared LED; And
And an augmented reality implementation apparatus for rendering a virtual image read from the infrared marker by capturing an actual image of the object and an infrared marker of the object,
Augmented reality system, characterized in that the code provided by the infrared marker is a Morse code.
The method of claim 1,
The augmented reality implementation device,
A visible light camera unit for photographing an actual image of the object;
An infrared camera unit photographing a lighting state of the infrared LED; And
An augmented reality system comprising an image processor for generating a virtual reality image by overlapping the real image and the virtual image.
The method of claim 2,
The augmented reality implementation device,
A position detector for extracting information on a position where the virtual image is to be disposed in the virtual reality image from the lit state of the photographed LED; And
And a type detecting unit extracting information on the content of the virtual image to be displayed from the lighting state of the photographed LED.
The method according to claim 2 or 3,
The augmented reality implementation device,
A virtual image DB for storing the virtual image; Augmented reality system characterized in that it further comprises.
The method according to claim 2 or 3,
The augmented reality implementation device,
A communication unit capable of transmitting and receiving information through a communication network; Augmented reality system characterized in that it further comprises.
The method of claim 1,
A marker information transmitter for transmitting information on an infrared marker to be implemented in the object; Further comprising:
The object,
A marker information receiver configured to receive information on an infrared marker from the marker information transmitter;
An infrared marker display unit which transmits infrared rays from the infrared LED light emitting unit to the outside;
A control unit controlling the infrared LED light emitting unit based on the information on the infrared marker received from the marker information receiving unit; Augmented reality system comprising a.
The method of claim 1,
An output device for outputting an augmented reality image generated by the augmented reality implementation device; Augmented reality system characterized in that it further comprises.

Images