CN101339604A - A New Type of Marker Graphics Based on Visual Invariants and Its Recognition, Tracking and Positioning Algorithm - Google Patents

Abstract

The invention relates to a novel marked point pattern based on vision invariants and identification and tracking positioning algorithm thereof. The invention comprises a design of novel marked point pattern based on vision invariants and identification and tracking positioning algorithm based on the novel marked point. The novel marked point pattern based on vision invariants is a black pentagon with a round mark. The marked point can be a convex pentagon or a concave pentagon. The round mark is used for identifying a first vertex of the pentagon. The pattern is coded using the cross-ratio invariant of the pentagon and combining the concavity and convexity of the vertex. The marked point pattern based on vision invariants can conduct attitude calculation using self-characteristics and realize the coding of the marked point, thus saving the area needed by coding patterns, reducing the pattern size of the marked point and improving the registration precision in unit area. The tracking registration algorithm can be used for identifying the marked point, tracking the head posture of the user and calculating the accurate position and posture of virtual scenes in true environmental space.

Description

A New Type of Marker Graphics Based on Visual Invariants and Its Recognition, Tracking and Positioning Algorithm

technical field

The invention relates to a design of a mark point pattern based on visual invariance, in particular to an augmented reality tracking registration algorithm utilizing the special mark point, and belongs to the field of virtual reality.

Background technique

The tracking and registration technology based on artificial landmarks is the most mature and widely used registration technology in the current augmented reality system. The so-called "mark point" refers to the special graphic pattern printed by artificial design. Due to the particularity of its graphic pattern, it is difficult to be similar to the environmental background in various scenes, so it is easy to distinguish from the environment. After the identification and positioning algorithm of the landmark point, the corresponding point set of its three-dimensional coordinates in space and its projection image can be obtained, and the six-degree-of-freedom attitude information of the camera relative to the real scene can be obtained by calculating the attitude estimation algorithm.

等在2002年发表的TRIP系统。Many foreign laboratories have made important achievements in the tracking and registration technology based on artificial landmarks. For example, the HTL laboratory of the University of Washington in the United States has designed and developed the ARToolKit software package. The square landmarks used in this package are the most frequently adopted landmark style in the current AR system. Because ARToolKit uses binary images in the processing process, the precision is relatively low. At the same time, because the image correlation matching is used in the encoding matching, the matching efficiency and the number of encodings are restricted. In 1998, Japan's Sony Computer Laboratory published a landmark system similar to ARToolKit, and improved its coding algorithm to make it more convenient to design and code. X.Zhang and N.Navab from Siemens Research Institute have made important improvements to the ARToolKit-style landmark system, greatly improving its tracking accuracy. The algorithm of this marker point system has excellent properties and contributes a lot to the marker-based AR registration technology. And Charles B. et al. of Michigan State University in the United States, after analyzing the internal pattern coding method, provided a graphic design method that can theoretically obtain the maximum number of codes. Since then, M.Fiala published the ARTag landmark tracking registration system in 2005. The landmark system has made important contributions to the recognition of landmark registration, feature location and two-dimensional barcode encoding and decoding algorithms. In addition to the above square markers similar to the ARToolKit style, there are also some circular marker systems published publicly. Among them, the more representative ones are the circular landmark system published by Leonid Naimark et al. of InterSense in 2002 and Diego López de

et al. published the TRIP system in 2002.

However, in these landmark systems, the design of landmark patterns is often divided into two independent parts, one part is dedicated to the coding of logo graphics, and the other part is used for feature point extraction and pose calculation. The result of such separation is that the marking point pattern requires a large coding area, without reducing the size of the marking point pattern, and improving the accuracy that can be provided per unit area. At the same time, the code needs to be decoded during the tracking and registration process, which increases the computational burden of the computer.

Contents of the invention

In order to overcome the shortcomings of the existing augmented reality tracking and registration technology based on landmarks, the system of the present invention includes a new type of landmark graphics based on visual invariants and its identification, tracking and positioning algorithms, which can be applied to augmented reality tracking and registration systems middle. The beneficial effect of the present invention is that the new type of marker points based on visual invariants fully utilizes the visual invariant information of the marker point graphics itself, saves the area required for the coding pattern, and reduces the space size of the marker point graphics. Increased precision available per unit area. The enhanced tracking and registration system based on the visual invariant landmarks has high robustness to large viewing angle changes.

The invention proposes an augmented reality tracking and registration system using a black pentagon with a white circular mark as a marker point graphic. The vertices of the pentagon are used as matching features for attitude estimation; at the same time, the five sides that constitute the pentagon are selected to calculate its cross-ratio projective invariant and its concave-convexity, which are used to encode and identify marker points. In order to solve the problem of sequential matching of feature points, a circular mark is added to the pentagon graph to identify the first vertex of the pentagon. After identifying the first vertex of the pentagon, the sequence direction of its vertices is determined, so that the matching relationship of the feature points can be determined. Referring to Figure 1, the new type of marker point graphic based on visual invariance is a pentagon with a black or white circular mark, and the position of each vertex of the pentagon can be arbitrarily selected as long as any three vertices are not collinear That's it. And the new type of marking point does not limit the concave-convex characteristics of the pentagon, that is, the pentagon can be a convex pentagon or a concave pentagon. The circle flag is used to determine the first vertex of the pentagon. If the pentagon is a convex pentagon, the circle mark is white and located inside the pentagon, and if the pentagon is a concave pentagon, the circle mark is black and located outside the pentagon. After determining the order of vertices of the pentagon, the matching relationship of the feature points can be determined. The cross-ratio invariant (I ₁ , I ₂ ) is calculated by using the five sides of the pentagon, and the graph is coded by combining the concavity and convexity of the vertices.

${\mathrm{<span\; class="notranslate">\; I</span>}}_{\mathrm{<span\; class="notranslate">\; 1</span>}}<span\; class="notranslate">\; =</span>\frac{<span\; class="notranslate">\; |</span>{\mathrm{<span\; class="notranslate">\; m</span>}}_{\mathrm{<span\; class="notranslate">\; 431</span>}}<span\; class="notranslate">\; |</span><span\; class="notranslate">\; |</span>{\mathrm{<span\; class="notranslate">\; m</span>}}_{\mathrm{<span\; class="notranslate">\; 521</span>}}<span\; class="notranslate">\; |</span>}{<span\; class="notranslate">\; |</span>{\mathrm{<span\; class="notranslate">\; m</span>}}_{\mathrm{<span\; class="notranslate">\; 421</span>}}<span\; class="notranslate">\; |</span><span\; class="notranslate">\; |</span>{\mathrm{<span\; class="notranslate">\; m</span>}}_{\mathrm{<span\; class="notranslate">\; 531</span>}}<span\; class="notranslate">\; |</span>}<span\; class="notranslate">\; ,</span>$ ${\mathrm{<span\; class="notranslate">\; I</span>}}_{\mathrm{<span\; class="notranslate">\; 2</span>}}<span\; class="notranslate">\; =</span>\frac{<span\; class="notranslate">\; |</span>{\mathrm{<span\; class="notranslate">\; m</span>}}_{\mathrm{<span\; class="notranslate">\; 431</span>}}<span\; class="notranslate">\; |</span><span\; class="notranslate">\; |</span>{\mathrm{<span\; class="notranslate">\; m</span>}}_{\mathrm{<span\; class="notranslate">\; 532</span>}}<span\; class="notranslate">\; |</span>}{<span\; class="notranslate">\; |</span>{\mathrm{<span\; class="notranslate">\; m</span>}}_{\mathrm{<span\; class="notranslate">\; 432</span>}}<span\; class="notranslate">\; |</span><span\; class="notranslate">\; |</span>{\mathrm{<span\; class="notranslate">\; m</span>}}_{\mathrm{<span\; class="notranslate">\; 521</span>}}<span\; class="notranslate">\; |</span>}<span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; )</span>$

Where M _ijk = (l _i , l _j , l _k ), (l = (l ¹ , l ² , l ³ )l represents the equation of a straight line: l ¹ x+l ² y+l ³ =0), |M| is the determinant of the matrix.

Figure 2 is the design of the placement position of the circular marker in the marker point. To solve the problem of sequential matching of feature points, a circular marker is added to the pentagon to identify the first vertex of the pentagon. After identifying the first vertex of the pentagon, the order of the vertices on it is determined (the other vertices are selected clockwise), so that the matching relationship of the feature points can be determined. The position of the center of the circle mark is at (x ₀ , y ₀ ). in:

${\mathrm{<span\; class="notranslate">\; x</span>}}_{\mathrm{<span\; class="notranslate">\; 0</span>}}<span\; class="notranslate">\; =</span>\frac{{\mathrm{<span\; class="notranslate">\; x</span>}}_{\mathrm{<span\; class="notranslate">\; 1</span>}}<span\; class="notranslate">\; +</span>{\mathrm{<span\; class="notranslate">\; x</span>}}_{\mathrm{<span\; class="notranslate">\; c</span>}}}{\mathrm{<span\; class="notranslate">\; 2</span>}}<span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 2</span>}<span\; class="notranslate">\; )</span>$

${\mathrm{<span\; class="notranslate">\; the\; y</span>}}_{\mathrm{<span\; class="notranslate">\; 0</span>}}<span\; class="notranslate">\; =</span>\frac{{\mathrm{<span\; class="notranslate">\; the\; y</span>}}_{\mathrm{<span\; class="notranslate">\; 1</span>}}<span\; class="notranslate">\; +</span>{\mathrm{<span\; class="notranslate">\; the\; y</span>}}_{\mathrm{<span\; class="notranslate">\; c</span>}}}{\mathrm{<span\; class="notranslate">\; 2</span>}}$

(x ₁ , y ₁ ) is the coordinates of the first vertex of the pentagon, and (x _c , y _c ) is the coordinates of the center of gravity of a triangle formed by two vertices adjacent to the first vertex of the pentagon.

The identification, tracking and positioning algorithm flow of the novel marker point based on the visual invariant designed by the present invention is as follows:

(1) Extract all black areas in the collected image;

(2) Detect the connected regions of the extracted black regions;

(3) Pentagon detection;

(4) extract the pentagon edge;

(5) Edge thinning to obtain a single-pixel width edge;

(6) The straight line fitting algorithm obtains the straight line parameters;

(7) Detect the order arrangement of its vertices obtained by the circular marks in the marker point graphics;

(8) Perform error detection on all candidate pentagon regions. If the error detection result is less than the set threshold, it will be recognized as a correct marker point graphic; if not, it will exit;

(9) Detect the concave-convex characteristics of the pentagon;

(10) Calculate the cross-ratio invariant constituted by the sides of the pentagon;

(11) Utilize the concave-convex feature vector and the cross-ratio invariant to compare with the marker point feature information in the database to determine the encoding of the marker point;

(12) Repeat steps (9)-(11) until all marker points in the current image are detected;

(13) Using an augmented reality registration algorithm based on planar feature points for attitude calculation;

(14) Realize the integration of real and virtual scenes.

According to the above description of the present invention, the present invention can effectively solve the shortcomings of large area occupied by the mark point graphics and complex coding, and make full use of the characteristics of the figure itself to realize the augmented reality tracking registration based on the new mark point.

Description of drawings

Fig. 1 is the graphic design of the sign point based on the visual invariant in the present invention; Fig. 1a.-concave pentagon mark point, Fig. 1b.-convex pentagon mark point;

Figure 2 is the design of the circular mark in the marker point.

Detailed ways

The present invention will be described in further detail below in conjunction with specific implementation examples. Here, the exemplary embodiments and descriptions of the present invention are used to explain the present invention, but not to limit the present invention.

Place the pentagonal mark points designed by the present invention in the real scene, and use the camera to shoot the scene to obtain the image information of the scene. The specific implementation process of the identification, tracking and positioning algorithm is as follows:

(1) Extract all the black areas in the collected image, which does not limit the image binarization algorithm;

In this embodiment, the watershed adaptive threshold algorithm is used to extract black areas, so as to ensure the adaptability of the algorithm to illumination changes.

(2) Detect the connected regions of the extracted black regions, which does not limit the use of connected region detection algorithms;

In this embodiment, a label connected region detection algorithm is used.

(3) Utilize the pentagon detection algorithm to detect the pentagon, and extract the edge of the pentagon, and refine it to a single pixel width;

(4) Then adopt the straight line fitting algorithm to obtain the straight line parameters, and there is no limit to which straight line fitting algorithm to use;

In this embodiment, the least square method is used for straight line fitting.

(5) Use the white or black circle in the figure to determine the first vertex of the pentagon, and select other vertices on the pentagon in a clockwise direction;

(6) Perform homography error detection on all selected candidate regions. If the error detection result is less than the set threshold, it will be identified as the correct matching mark point, and if it is not detected, it will exit;

(7) Detect the concave-convex characteristics of each vertex of the marker point in turn, and form a concave-convex feature vector (a ₁ , a ₂ , a ₃ , a ₄ , a ₅ ). If it is a concave vertex, it is expressed as "1", and a convex vertex is expressed as "0". The concave-convex feature vector of the concave-convex pentagon mark point as shown in Fig. 1a is (1, 0, 0, 0, 0).

(8) Calculate the cross-ratio invariant (I ₁ , I ₂ ) formed by the sides of the pentagon; use the cross-ratio invariant and the concave-convex vector as the feature vector, and use the Mahalanobis distance to compare with the marker point information in the database Yes, if it is less than the set threshold, the system thinks that the marker point is correctly identified; if it is greater than the threshold, it will detect other candidate areas until all marker points in the current image are detected;

(9) Use the augmented reality registration algorithm based on plane feature points to calculate the attitude, and calculate the rotation matrix R and translation matrix T of the camera relative to the world coordinate system of the landmark points.

(10) Assign the rotation and translation matrix values to the virtual camera to make it move along the track to complete the fusion of virtual and real scenes.

The specific description above further elaborates the purpose, technical solution and beneficial effect of the invention. It should be understood that the above description is only a specific embodiment of the present invention and is not used to limit the protection of the present invention. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims (1)

1. A novel marker point graphic based on visual invariants and its recognition, tracking and positioning algorithm, characterized in that the novel marker point graphic based on visual invariants is a pentagon with a black or white circular sign, the The position of each vertex of the pentagon can be arbitrarily selected according to the needs, as long as any three vertices are not collinear; and this new type of marker point does not limit the concave-convex characteristics of the pentagon, that is, the pentagon can be a convex pentagon or It is a concave pentagon; the circle mark is used to determine the first vertex of the pentagon; if the pentagon is a convex pentagon, the circle mark is white and located inside the pentagon; if the pentagon is a concave pentagon For a polygon, the circle mark is black and is located outside the pentagon; the matching relationship of the feature points can be determined after determining the order of vertices of the pentagon; use the five sides of the pentagon to calculate its cross-ratio invariant (I ₁ , I ₂ ) Coding the graph with the concavo-convexity of the vertices; the recognition, tracking and positioning algorithm flow of the new marker point based on the visual invariant is as follows: (1) Extract all black areas in the collected image; (2) Detect the connected regions of the extracted black regions; (3) Pentagon detection; (4) extract the pentagon edge; (5) Edge thinning to obtain a single-pixel width edge; (6) The straight line fitting algorithm obtains the straight line parameters; (7) Detect the order arrangement of its vertices obtained by the circular marks in the marker point graphics; (8) Carry out error detection to all candidate pentagonal regions, if the error detection result is less than the threshold value of setting, will be identified as the correct mark point figure; If not then exit; (9) Detect the concave-convex characteristics of the pentagon; (10) Calculate the cross-ratio invariant constituted by the sides of the pentagon; (11) Utilize the concave-convex feature vector and the cross-ratio invariant to compare with the marker point feature information in the database to determine the encoding of the marker point; (12) Repeat steps (9)-(11) until all marker points in the current image are detected; (13) Using an augmented reality registration algorithm based on planar feature points for attitude calculation; (14) Realize the integration of real and virtual scenes.

Images