KR101040253B1 - Method of producing and recognizing marker for providing augmented reality - Google Patents

Abstract

Disclosed are a method of manufacturing and recognizing a marker for providing augmented reality. The marker manufacturing method according to the present invention includes the steps of defining context information associated with a marker, generating a binary code by encoding the defined context information, and forming a binary pattern according to the generated binary code. Characterized in that. The marker recognition method according to the present invention includes a binary pattern in which a marker contains contextual information related to the marker, wherein the binary pattern is formed according to a binary code generated by encoding the contextual information. And capturing an image by capturing an image, extracting a binary code from the binary pattern from the obtained image, and obtaining the context information by decoding the extracted binary code. According to the present invention, various contextual information related to the marker can be contained in the marker.

Augmented Reality, Markers, Marker Making, Marker Recognition

Description

Method of producing and recognizing marker for providing augmented reality}

The present invention relates to augmented reality, and more particularly, to a marker for providing augmented reality, and a method of manufacturing and recognizing a marker.

Most systems using the existing augmented reality technology have been manufactured based on markers or tags. The conventional marker or tag simply contains ID information of the marker or tag. In this case, it is assumed that information such as the size or position of the marker is known in advance when the marker is recognized, and the ID is extracted from the marker by analyzing the marker or the tag, and the content corresponding to the ID is augmented in the coordinates obtained from the marker.

Considering mobile augmented reality application services in a ubiquitous computing environment where different markers are distributed in the surrounding environment, the markers generally have different sizes and positions. If the information extracted from the marker is only one ID as in the existing marker design, various problems may occur as follows.

Since the information of the marker is only an ID, other information related to the marker can be obtained only through a database provided at a remote or mobile device. However, it is virtually impossible to contain all the information about numerous markers on a mobile device. In case of storing the related information in the remote database, the search speed may be slowed down as the number of markers increases and the communication with the database should be possible at all times.

Since the markers are interpreted under the assumption that all markers have the same size, the scale of the 3D coordinate system obtained from the markers varies in proportion to or inversely proportional to the size of the markers. In this case, when 3D content of the same size is displayed based on markers of different sizes, the size of the displayed content varies according to the size of the marker. In general, given that the screen of the mobile device is not large, the displayed content may be larger than the screen, and the user may not be able to see the content properly.

The 3D coordinate system obtained from the marker is a local coordinate system based on the marker. Therefore, when there are two different markers, the relationship between the two markers A and B can be represented only by relative coordinate system transformation. Therefore, when interacting with multiple markers, if all the markers are not visible on the screen, the relationship between the two different markers A and B is unknown. In addition, coordinate system transformation should always be performed when the displayed contents for each of markers A and B interact with each other.

In addition, the conventional marker interpretation method is designed to operate only for one type of marker, and thus, if various kinds of markers exist, the remaining markers cannot be interpreted except for specific markers.

Accordingly, an aspect of the present invention is to provide a marker capable of containing various contextual information related to a marker for providing augmented reality, a method of manufacturing such a marker, and a method of recognizing the marker.

In order to solve the above technical problem, a method for manufacturing a marker for providing augmented reality according to the present invention includes: defining context information related to a marker; Generating a binary code by encoding the defined context information; And forming a binary pattern according to the generated binary code.

In order to solve the above technical problem, a marker for providing augmented reality according to the present invention includes a binary pattern containing context information associated with the marker, and the binary pattern is based on a binary code generated by encoding the context information. It is characterized by being formed.

In the marker recognition method for providing augmented reality according to the present invention to solve the technical problem, the marker includes a binary pattern containing the context information associated with the marker, the binary pattern is generated by encoding the context information It is formed according to the binary code, the method comprising the steps of: photographing the marker to obtain an image; Extracting a binary code from the binary pattern in the obtained image; And decoding the extracted binary code to obtain the contextual information.

In order to solve the above technical problem, there is provided a computer-readable recording medium having recorded thereon a program for executing a marker recognition method for providing augmented reality according to the present invention.

According to the above-described present invention, there is provided a marker capable of containing various contextual information related to a marker for providing augmented reality in a marker, a method of manufacturing such a marker, and a method of recognizing the marker.

Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the accompanying drawings. In the following description and the accompanying drawings, substantially the same components are denoted by the same reference numerals, and redundant description will be omitted. In addition, in the following description of the present invention, if it is determined that a detailed description of a related known function or configuration may unnecessarily obscure the subject matter of the present invention, the detailed description thereof will be omitted.

1 shows a configuration of a marker for providing augmented reality according to an embodiment of the present invention. Referring to FIG. 1, the marker 10 according to the present exemplary embodiment includes a binary pattern 11 containing a contextual information related to the marker 10, an image inserter 12 into which a logo, a picture, or a text image is inserted, and a marker. And an area defining portion 13 for defining the area of (10).

The binary pattern 11 is formed according to a binary code generated by encoding context information. The binary pattern 11 is composed of several gratings as shown, and represents 0 or 1 by filling each grating with white or black color. The context information contained in the binary pattern 11 includes a marker ID, a type of marker, a size of the marker, a position in an absolute coordinate system, presence or absence of character information, character information, rotation information, and the like.

The image inserter 12 may insert a logo, picture or text image, and the binary pattern 11 may contain information indicating whether a logo, picture or text image is inserted. 2 shows an example of a state in which various log, picture or text images are inserted into the image insertion unit 12 of the marker 10 according to the present embodiment. The log, picture or text image may be appropriately disposed in the space between the binary pattern 11 of the marker 10 and the area defining portion 13. On the other hand, as shown in the fourth of FIG. 2, the picture or text image may extend outside the boundary of the marker.

As shown, the user can grasp the contents indicated by the marker 10 through a logo, a picture, or a text image inserted into the image insertion unit 12 of the marker 10. The image insertion unit 12 may be expanded in size to the left or right or up and down within the range of not invading the binary pattern 11 and the area defining unit 13 according to the size of the image to be inserted.

The region defining portion 13 serves to specify the size of the marker and define its size. For example, the region defining portion 13 defines the region by inserting a specific pattern into the boundary portion of the marker. The area defining unit 13 can more accurately obtain a conversion matrix between the camera photographing the marker and the marker when the marker is recognized.

3 is a flowchart of a marker manufacturing method for providing augmented reality according to an embodiment of the present invention.

In step 310, context information related to the marker is defined. As already described, the context information includes a marker ID, a type of marker, a marker size, a position in an absolute coordinate system, presence or absence of character information, character information, rotation information, and the like.

In operation 320, the binary code is generated by encoding information representing the type of the marker in the context information. In operation 330, an appropriate encoding method is selected according to the type of the marker. do. Since the context information contained in the binary pattern of the marker varies according to the type of the marker, an appropriate encoding method (decoding method for marker recognition) should be selected accordingly.

In operation 340, the binary code is generated by encoding the remaining information except for the information indicating the type of the marker. The process of encoding contextual information related to the marker, including information indicating the type of marker, will be described in detail later.

In step 350, the binary code generated in steps 320 and 340 is re-encoded for error correction. In this case, for example, a BCH (Boss-Chaudhuri-Hocquenghen) encoding scheme may be used.

In operation 360, a binary pattern is formed at a position where a marker is to be provided according to the binary code generated through operations 320, 340, and 350. At this time, a binary pattern is formed by filling white or black in each of the grids as described with reference to FIG. 1.

In operation 370, the region defining part of the marker is formed according to the size information of the marker among the context information related to the marker, and if there is a logo, picture, or text image to be inserted into the marker, the corresponding image is inserted into the marker.

Hereinafter, a process of encoding context information related to a marker will be described in detail for each information.

을 만족하는 k의 최소값이 마커의 종류를 나타내는데 필요한 비트 수가 된다. 예를 들어 마커의 종류가 7개라면 마커의 종류를 나타내기 위해서는 3비트가 필요하다. 마커의 종류 정보를 마커에 담음으로 써 여러 종류의 마커를 동시에 인식할 수 있는 장점이 있다. Information indicating the type of marker will be described as follows. The type of marker is assigned a number according to the type of marker to be recognized and represented as a binary number. If there are n kinds of markers to display

The minimum value of k that satisfies is the number of bits necessary to indicate the type of marker. For example, if there are seven types of markers, three bits are required to indicate the types of markers. By putting the type information of the marker in the marker, there is an advantage that can recognize several kinds of markers at the same time.

과 같이 나타내고 n 과 m을 이진수로 변환한다. 길이의 경우 가로 길이에 대한 비율 k를 이진수로 변환한다. 이 때, 가로 또는 세로 중 어느 쪽이 더 긴 쪽인지를 나타내기 위해 이진수의 마지막에 역수(inverse) 여부를 나타내기 위해 1비트를 추가한다. 마커의 크기 정보를 마커에 담음으로써, 마커의 크기를 미리 정해 놓지 않더라도 마커를 촬영한 후 카메라와 마커 사이의 올바른 변환 행렬을 구할 수 있다.The size of the marker can be defined as the length of the marker. For example, the width

And convert n and m to binary. For length, convert the ratio k to the width to binary. At this time, one bit is added to indicate whether the inverse is at the end of the binary number to indicate whether the length is horizontal or vertical. By including the size information of the marker in the marker, it is possible to obtain a correct transformation matrix between the camera and the marker after shooting the marker even if the size of the marker is not predetermined.

의 형태로 표현 가능하다면 n과 m을 이진화함으로써 필요한 비트수를 줄일 수 있다. 종래의 마커 해석 방법에 의하면, 절대 좌표를 고려하지 않고 마커에 대한 지역 좌표계(local coordinate system)만을 계산하기 때문에 서로 다른 두 마커 사이의 관계는 상대적으로만 정의가 가능하다. 그러나 본 실시예에서와 같이 마커의 위치 정보를 마커에 담음으로써 서로 다른 여러 마커들 사이의 절대적인 위치 관계를 알 수 있다. Information indicating the position of the marker in the absolute coordinate system is encoded as follows. The location of the marker only needs to define the absolute coordinates (x, y) of the point corresponding to the upper left (or other vertex) of the four vertices of the marker when the width and length of the marker are known. And binarizes x and y. In this case, like the way to express the width of the marker

If you can express in the form of, you can reduce the number of bits needed by binarizing n and m. According to the conventional marker analysis method, since only a local coordinate system for a marker is calculated without considering absolute coordinates, the relationship between two different markers can be defined relatively. However, as in this embodiment, by including the position information of the marker in the marker, it is possible to know the absolute positional relationship between different markers.

The ID of a marker is usually defined as a positive integer, which is converted to binary. In 12-bit representation, up to 4096 marker IDs can be generated.

The presence or absence of the character information indicates whether the marker contains the character information and may be represented by 0 or 1. Therefore, it can be represented by 1 bit.

Character information can be contained in the marker in two ways. First, since characters can be represented with binary codes between 0 and 255, they are converted to binary codes and put in binary patterns. The second is to insert a character in the form of an image into the marker's image insert if there are not enough bits available for the binary pattern. The inserted character image may be identified using a character recognition algorithm such as optical character recognition (OCR) at the time of marker recognition.

When a character image is inserted into the image insertion unit of the marker, the position of the character string must be known in order to recognize the character by using an OCR or the like. Therefore, in this case, the location information of the string should be contained in the marker. The position information of the string may be defined as a distance (dx, dy) away from the upper left vertex of the marker, which is the position information of the marker, and the width and height (w, h) of the string. Where (dx, dy) is represented by the number of grids r and s away from the upper left corner of the marker with respect to the x- and y-axis directions, respectively, and the width and height of the string are represented by the number of grids u and v, respectively. . By converting each of these numbers into a binary code and arranging them, it is possible to binarize character position information.

The rotation information of the marker is information for knowing the direction in which the marker is recognized. The rotation information of the marker can be indicated by giving 0 or 1 to bits specific position of the binary pattern. For example, three of the left, right, top, and bottom four bits of the binary pattern are represented by giving 0 and the other one. Of course the opposite is also true. If the marker does not have rotation information, you should try decoding in all four directions when recognizing the marker. When the marker has the rotation information as in the present embodiment, the rotation information is extracted at the time of marker recognition, and accordingly, one time decoding is possible.

4 is a flowchart illustrating a marker recognition method for providing augmented reality according to an embodiment of the present invention.

In operation 410, a marker is photographed with a camera to obtain an image. In this embodiment, the marker includes a binary pattern containing contextual information related to the marker, as described above, and the binary pattern is formed according to the binary code generated by encoding the contextual information.

In operation 420, the binary pattern is detected from the image, and in operation 430, the binary code is extracted from the binary pattern. In operation 440, decoding for erotic detection and correction is performed on the extracted binary code.

In step 450, the type of the marker is identified by reading a bit representing the type of the marker among the extracted binary codes.

In operation 460, an appropriate decoding scheme is selected according to the identified marker type, and in operation 470, the binary code is decoded to obtain context information related to the marker.

5 is a flowchart illustrating in detail the step 470 of obtaining context information of a marker.

In step 510, the binary code is decoded to obtain context information related to the marker. In step 520, it is determined whether the marker includes a picture image. In such a case, the process proceeds to step 530 to recognize the picture. do.

In operation 540, it is determined whether the marker includes a character image. In such a case, the controller proceeds to operation 550 to recognize the character using a character recognition algorithm such as an OCR.

According to the embodiments of the present invention described above has the following advantages.

Conventional markers require a rectangular outline, so when a part of the outline is hidden, there is a limitation that the marker is not properly recognized, but the shape of the marker proposed in the present invention is applied by applying the boundary only to the binary pattern 11. It is possible to reduce the case where the marker is not recognized properly due to the outline obstruction.

In the case of the existing rectangular markers, since all patterns must be put inside the outline, the shape and size of the pattern is limited to the inside size of the outline. However, the marker according to the present invention can reduce the constraints due to the outline because the image inserting portion 12 can be extended.

When using existing markers to augment content on top of each other, if two different markers are used, the markers are recognized based on the size of a predefined marker. There is a different problem. On the other hand, according to the marker and the marker recognition method according to the present invention, since the size of the marker can be known at the marker recognition step even if the sizes of the markers are different from each other, the rendering can be performed while maintaining the size of the content. Therefore, when considering a mobile terminal device having a small screen size, the content size can be maintained consistently.

In addition, since the marker according to the present invention can include a binary pattern and a picture, a logo or a text image containing contextual information related to the marker, the marker related information can be extracted directly from the marker without preparing a database or the like in advance. Can be.

By providing a blank space inside the marker and inserting a logo, picture or text image, it can reduce the visual dissatisfaction compared to the existing marker, and the user can intuitively grasp what the marker means.

Since existing markers can only know relative relations with each other, it is necessary to always calculate relative transformation matrices between different markers. However, according to the present invention, by storing the position information of the marker in the binary pattern of the marker, it is not necessary to calculate the transformation matrix between two different markers, and it is possible to interpret the relationship between the two markers in the absolute coordinate system.

Meanwhile, the above-described embodiments of the present invention can be written as a program that can be executed in a computer, and can be implemented in a general-purpose digital computer that operates the program using a computer-readable recording medium. The computer-readable recording medium includes a storage medium such as a magnetic storage medium (e.g., ROM, floppy disk, hard disk, etc.), optical reading medium (e.g., CD ROM,

So far I looked at the center of the preferred embodiment for the present invention. Those skilled in the art will appreciate that the present invention can be implemented in a modified form without departing from the essential features of the present invention. Therefore, the disclosed embodiments should be considered in descriptive sense only and not for purposes of limitation. The scope of the present invention is shown in the claims rather than the foregoing description, and all differences within the scope will be construed as being included in the present invention.

1 shows a configuration of a marker for providing augmented reality according to an embodiment of the present invention.

2 illustrates an example in which various log, picture or text images are inserted into the image insertion unit 12 of the marker 10.

3 is a flowchart of a marker manufacturing method for providing augmented reality according to an embodiment of the present invention.

4 is a flowchart illustrating a marker recognition method for providing augmented reality according to an embodiment of the present invention.

5 is a flowchart illustrating step 470 of obtaining context information of a marker in more detail.

Claims (20)

In the marker manufacturing method for providing augmented reality, (a) defining contextual information associated with the marker; (b) encoding the context information defined above to generate a binary code; And (c) forming a binary pattern according to the generated binary code, In step (b), Selecting an encoding scheme according to the type of the marker; And generating a binary code by encoding the context information with the selected encoding scheme. The method of claim 1, The context information may include at least one of a kind, a size, a position, character information, and rotation information of the marker. The method of claim 1, And re-encoding the generated binary code in order to correct an error. delete The method of claim 2, Step (c) is a marker manufacturing method characterized in that to form a binary pattern by filling a white or black in accordance with the generated binary code in each of the grids forming the marker. The method of claim 2, And (b) generating a binary code corresponding to the type of the marker by encoding the number assigned according to the type of the marker in binary when encoding information about the type of the marker. The method of claim 2, When encoding the size of the marker in the step (b), the marker manufacturing method characterized in that to generate a binary code corresponding to the size of the marker by representing the length of the horizontal or vertical of the marker in binary. The method of claim 2, In case of encoding the position of the marker in the step (b), a binary code corresponding to the position of the marker is generated by representing the absolute coordinate of the point corresponding to any one of the four vertices of the marker in binary. How to make a marker. The method of claim 2, In the step (b), when the rotation information is encoded, the marker manufacturing method, characterized in that it is indicated by giving 0 or 1 to a specific position among the bits of the binary pattern. The method of claim 1, The method of claim 1, further comprising inserting a logo, picture or text image into the marker. delete delete delete delete delete In the marker recognition method for providing augmented reality, The marker includes a binary pattern containing contextual information related to the marker, wherein the binary pattern is formed according to a binary code generated by encoding the contextual information. (a) photographing the marker to obtain an image; (b) extracting a binary code from the binary pattern in the obtained image; And (c) decoding the extracted binary code to obtain the contextual information, In step (c), Reading the bit representing the type of the marker from the extracted binary code to identify the type of the marker; Selecting a decoding scheme according to the type of the identified marker; And And decoding the extracted binary code according to the selected decoding scheme to obtain the context information. The method of claim 16, The context information may include at least one of a type, a size, a position, character information, and rotation information of the marker. delete The method of claim 16, And recognizing a character using a character recognition algorithm when the marker includes a character image. A computer readable recording medium having recorded thereon a program for executing the marker recognition method for providing augmented reality according to any one of claims 16, 17 and 19.

Images