CN105894068B

CN105894068B - FPAR card design and rapid identification and positioning method

Info

Publication number: CN105894068B
Application number: CN201610137618.1A
Authority: CN
Inventors: 滕少华; 霍颖翔; 滕璐瑶; 张巍
Original assignee: Guangdong University of Technology
Current assignee: Guangdong University of Technology
Priority date: 2016-03-10
Filing date: 2016-03-10
Publication date: 2020-07-28
Anticipated expiration: 2036-03-10
Also published as: CN105894068A

Abstract

The invention discloses a design and rapid identification method of an FPAR (field programmable Gate array) card, which is characterized in that a plurality of groups of concentric circles/concentric polygons are arranged on the FPAR card, each group of concentric circles/concentric polygons is called as a positioning point, a plurality of groups of concentric circles/concentric polygons form identification patterns, and the total patterns of the plurality of groups of concentric circles/concentric polygons are centrosymmetric after being arranged. The quick identification positioning method comprises the following steps: s1, video input, namely decomposing the acquired video source file into a frame sequence; s2, RGB/YUV preprocessing; s3, performing binarization processing, namely performing binary separation on the image by adopting a fuzzy filter method; s4, performing image recognition processing by adopting a color filling method; s5, judging possible positioning points; s6, reversely deducing a normal vector of the FPAR card through an imaging picture to serve subsequent virtual reality rendering; s7, determining the position of the whole FPAR card in a three-dimensional space; and S8, rendering the virtual reality. The method has the advantages of zooming, dithering, blurred image recognition, partial shielding recognition, rotation, strong robustness and reliability, and also has the advantages of high static accuracy and high recognition speed.

Description

FPAR card design and rapid identification and positioning method

Technical Field

The invention relates to the field of image processing research, in particular to a design and rapid identification and positioning method for an FPAR card.

Background

Augmented Reality (AR) technology is a new technology for seamlessly integrating real world information and virtual world information, and is characterized in that entity information (visual information, sound, taste, touch and the like) which is difficult to experience in a certain time space range of the real world originally is overlapped after simulation through scientific technologies such as computers and the like, virtual information is applied to the real world and is perceived by human senses, and therefore sensory experience beyond Reality is achieved. The real environment and the virtual object are superimposed on the same picture or space in real time and exist simultaneously.

The augmented reality technology comprises new technologies and means such as multimedia, three-dimensional modeling, real-time video display and control, multi-sensor fusion, real-time tracking and registration, scene fusion and the like. Augmented reality provides information that is generally different from what human beings can perceive.

FPAR (fast Positioning acquired reality) is an AR card integrating fast Positioning and fast identification information.

Disclosure of Invention

The invention mainly aims to overcome the defects in the prior art and provide a FPAR card design and rapid identification and positioning method.

In order to achieve the purpose, the invention adopts the following technical scheme:

the invention provides a design method of an FPAR (field programmable Gate array) card, wherein a plurality of groups of concentric circles/concentric polygons are arranged on the FPAR card, each group of concentric circles/concentric polygons is called a positioning point, a plurality of groups of concentric circles/concentric polygons form an identification pattern, and the total pattern of the arranged plurality of groups of concentric circles/concentric polygons is not centrosymmetric.

As a preferred technical scheme, 5 groups of concentric circles/concentric polygons are arranged on the FPAR card, the shapes of the textures of each group of positioning points are the same, wherein 4 groups of concentric circles/concentric polygons are arranged at four corners of the rectangular card, and a fifth group of concentric circles/concentric polygons are positioned at the central position of the long edge of the card or at an asymmetric position where the FPAR card can be identified; the scaling ratio of two adjacent circles of edges of each group of concentric circles/concentric polygons is

The invention also provides a method for quickly identifying and positioning the FPAR card, which comprises the following steps:

s1, video input, namely decomposing the acquired video source file into a frame sequence;

s2, RGB/YUV preprocessing, wherein the extracted frame may be in RGB or YUV format, when the extracted frame is in RGB format, the quick identification method is to only take red or green channel and discard the rest channels, when the extracted frame is in YUV format, only Y channel is discarded U, V channel;

s3, performing binarization processing, and performing binary separation on the image;

s4, carrying out image recognition processing by adopting a fuzzy filter method; using a forward-backward order dual IIR filter, where the forward direction is P₀＝p₀Other P_n＝w₀*p_n+w₁*P_n-1+w₂*P_n-2Similarly, the inverse is of the form Q_len-1＝P_len-1And the rest of Q_n＝w₀*P_n+w₁*Q_n+1+w₂*Q_n+2The part of the subscript which is out of range is processed in a clamping mode, namely the nearest neighbor legal subscript is taken, and an IIR filter is used for simulating Gaussian blur in the mode;

s5, determining a possible positioning point,

s6, acquiring a three-dimensional normal vector of the FPAR card, and reversely pushing the normal vector of the FPAR card through an imaging picture to serve subsequent virtual reality rendering;

s7, determining the three-dimensional position of each positioning point through comparison and deviation correction, thereby determining the position of the whole FPAR card in a three-dimensional space;

and S8, rendering virtual reality, virtually pasting a plane picture on the three-dimensional FPAR card, and calculating the position of the color of each corresponding point on the imaging plane after the picture is pasted on the FPAR card.

As a preferred technical solution, in the step S3, the binarization processing is implemented by a blur filter, which specifically includes:

copying the image into two copies, namely copy 1 and copy 2;

if the video has more obvious noise points, copy 1 carries out extremely trace Gaussian blur, wherein sigma is less than 3 px;

copy 2 is gaussian blurred with a large radius, 1/30 to 1/10 of the image diagonal;

and establishing a blank dot matrix image, wherein the size of the blank dot matrix image is consistent with that of the first two copies and is marked as M, the position where the copy 1 at the corresponding position is brighter than the copy 2 is set as 1, and if not, the position is set as 0, and M is the result.

As a preferred technical solution, in the step S4, a specific method for performing image processing is as follows:

s41, determining three floating point values w0, w1 and w2 by an IIR filter simulating Gaussian blur, wherein P and P are both numerical values between 0 and 255, P is a floating point, and P is an integer; therefore, the product result of the part P and the P and the w is prestored to be made into a table for later use;

s42, listing a smaller precomputed multiplication table by adopting an approximate method;

the approximate calculation formula is that P W is approximately equal to P '. W is equal to W [ P ' ], and P W is equal to W ' [ P ], wherein W and W ' are the result of the product calculated in advance stored in the array, and P ' is the result of the single-precision floating point number P after the 16-bit mantissa is cut.

In a preferred embodiment, in step S4, integer addition is used instead of floating-point addition, and in the IIR filter positive order direction, since the calculation includes a fractional part, integer calculation is used to simulate fixed-point fractional calculation, and integer calculation is performed using a 16-bit short integer, where the integer and fractional parts each occupy 8 bits,

the calculation formula of the IIR filter in the positive sequence is P_n＝ω₀*(p_n<<8)+ω₁*P_n-1+ω₂*P_n-2，P_nThe calculation result is 256 times of the original formula, and during calculation, the omega table can be used to simplify multiplication, and the above formula is changed into P_n＝Ω'[p_n]+Ω[P_n-1]+Ω[P_n-2]；

The amplification problem of the positive sequence numerical value can be solved without processing, the reverse sequence can be repaired by directly using the right shift 8 bits after the calculation of the reverse sequence is completed, the table used by the reverse sequence and the positive sequence is consistent, only the calculation formula is slightly different and is Q_n＝ω₀*P_n+ω₁*Q_n+1+ω₂*Q_n+2Here, the multiplication can be solved by looking up the omega table, and the expression is Q_n＝Ω[P_n]+Ω[Q_n-1]+Ω[Q_n-2]Then R is_n＝Q_n>>8 is the result.

As a preferred technical solution, step S5 specifically includes:

s5.1, checking each pixel of the M, if the current pixel is 0 or 1, performing 4-communication or 8-communication filling by using a numerical value 2, and counting the maximum width and height of the filling and the area of the filling;

s5.2, limiting the filling area; if filling is started from the (x, y) point and the maximum filling area u is set, the left boundary is x-u, the right boundary is x + u, the upper boundary is y-u and the lower boundary is y + u; this limits the total fill area to no more than 4u²Thereby reducing the memory burden;

s5.3, selecting possible positioning point positions, wherein the conditions that the length-width ratio is legal when the length-width ratio is between 1:4 and 4:1 can be limited, and in addition, because the positioning point is designed into a fractal graph, the filling rate is consistent no matter which layer of pattern is, and the filling rate is near 0.436;

s5.4, eliminating error detection points, comparing and analyzing items in the table, and if some frames are independent or not concentric with any other frame, judging the frames to be error detection, combining the concentric frames in each group in the table L, recording the average center position and the score of the concentric frames, and storing the average center position and the score into a table H, wherein the score needs to consider the concentricity, the length-width ratio consistency, the frame overlapping number and the filling rate of each frame, and the score is higher and lower if the score meets the requirements;

and S5.5, identifying from the positioning points to the FPAR card pattern, enumerating all the positioning points by adopting a permutation and combination method, and substituting the enumerated positioning points into a calculation formula to test which corresponding mode is most reasonable.

As a preferred technical solution, in step S5.4, for anchor point identification, at least 2 layers are generally required to avoid false detection.

As a preferred technical solution, step S5 specifically includes: the determination of each positioning point specifically comprises the following steps:

(1) first judging convex polygon

The convex polygon judging method comprises the following steps:

b. eliminating the crossed option of the line segment, and judging the product with the same number;

considering a, b, c, d, and e as points in three-dimensional space, where the z-axis is 0, then the calculation is performed

Whether or not these values are the same sign as the axis, and may all be set to a positive sign;

(2) comprehensively judging;

because each positioning point is identified through images, whether the positioning points meet requirements or not is directly evaluated, and the degree of fit required by the rules is evaluated;

(3) optimizing decision;

when the calculation capacity permits, all the points in H can be selected from 5 points to carry out full arrangement, and the points are substituted into the above rule to carry out verification, if the candidate items in H are too many, the full arrangement is an astronomical number, and in order to reduce the calculation amount, the first n points with higher scores in H can be taken to carry out full arrangement substitution test.

As a preferred technical solution, step S7 specifically includes: using the drawttriangles method, the colors of all the points on the FPAR card are not completely calculated, but a sparse matrix grid is made, and then the grid is mapped to the positions of the holes of the grid on the imaging plane to be simulated by the transformation of triangular fragments.

Compared with the prior art, the invention has the following advantages and beneficial effects:

1. the invention has the advantages of zooming;

the pattern of the FPAR card is a fractal structure, and after removing the black circle of the outermost circle, the black circle of the inner circle is amplified to obtain an original image; reverse printing does not affect the recognition effect. Such a pattern has scaling advantages. When the resolution of the image captured by the camera is low or the distance of the FPAR card is long, the larger outer ring can still be identified; when the resolution of the lens is high or the distance between the FPAR cards is short, the area of the outer ring is too large and possibly exceeds the lens, the outer ring is not easy to identify, and only the inner ring with a small area is identified by neglecting the outer ring at the moment.

2. The method has the advantages of dithering and fuzzy image identification;

due to the shake of the lens and the scenery, the image shot by the camera cannot be always clear, and the obtained image often generates motion blur in many cases. The thicker and larger outer circle can help to identify the positioning point from the blurred image.

3. Static accuracy;

when a clear FPAR card is shot, not only the outer circle but also the small circle at the center of the positioning point can be analyzed clearly, which is beneficial to obtaining a recognition result with higher accuracy.

4. The advantage of rotation;

due to the concentric circle/concentric polygon design, the shape of the anchor point is the same regardless of rotation. Thus, the device is not influenced by rotation and angle.

5. Partial occlusion recognition advantage;

as long as the area visible for each anchor point is not less than two concentric circles/concentric polygons.

6. The robustness and the reliability are strong;

the card can be normally identified no matter whether the illumination is uneven, the picture is serious in noise, the card is inclined, the rotation is serious, the picture is bright, dark, fuzzy and the like.

7. The identification speed is high.

Compared with the similar technology, the method has the advantages of faster identification and positioning.

Drawings

FIG. 1 is a schematic diagram of an FPAR card of the present invention;

FIG. 2 is a schematic diagram of the scaling of concentric circles for the FPAR card of the present invention;

FIG. 3 shows the FPAR card recognition effect in an ideal environment;

FIG. 4 is a diagram of the FPAR card recognition effect in a semi-yin-yang environment;

FIG. 5 is a diagram of FPAR card recognition effect in an occluded environment;

FIG. 6 is a graph of the recognition effect of the conventional method in a low-contrast, noisy environment;

FIG. 7 is a diagram of the effects of the present invention on processing FPAR cards;

FIG. 8 is a FPAR card identification diagram in the fuzzy state;

fig. 9 is a binarization processing diagram of a fixed threshold value;

FIG. 10 is a schematic illustration of the color filling aspect of the present invention;

FIG. 11 is a block selection result of the invention after padding;

FIG. 12 is a schematic diagram in which the center position of the pattern is repeated;

FIG. 13 is a schematic diagram of a possible false detection;

FIG. 14 is a constructed locating point false detection pattern;

FIG. 15 is a schematic diagram of a mahjong bobbin pattern which may cause false detection;

FIG. 16 is a schematic diagram showing the steps of the region-limiting method;

FIG. 17 is a diagram showing the result of frame selection in the region-limiting method;

FIG. 18 is a diagram of a result of a box selection without using area restriction;

FIG. 19 is a schematic view of imaging;

FIG. 20 is a plan view of an FPAR card;

FIG. 21 is a diagram illustrating line segment crossing determination;

FIG. 22 is a schematic view of imaging for multiple solutions;

FIG. 23 is a diagram illustrating suspicious anchor points of a plurality of FPAR cards;

FIG. 24 is a modified FPAR print sample diagram;

FIG. 25 is a schematic view of multiple FPAR cards;

FIG. 26 is a schematic view of an FPAR card with openings;

FIG. 27 is a schematic view of multi-piece FPAR card identification with orientation;

FIG. 28 is a FPAR card multi-angle warp imaging view of a three-dimensional image;

FIG. 29 is a schematic view of three-dimensional image formation;

FIG. 30 is a real-time rendering graph with false positives;

FIG. 31 is a render screenshot;

fig. 32 is a rendering extension effect of the FPAR card.

Detailed Description

The present invention will be described in further detail with reference to examples and drawings, but the present invention is not limited thereto.

Examples

Introduction to AR technology

2. Introduction of card pattern

As shown in fig. 1, the design method of the FPAR card of the present embodiment performs video recognition on the pattern shown in fig. 1, where the pattern is composed of 5 sets of concentric circles/concentric polygons, each set of concentric circles/concentric polygons is referred to as a positioning point, and 5 sets of concentric circles/concentric polygons are referred to as recognition patterns. The grain shape of each group of positioning points is the same (actually, the grain shape can be different, but the grain shape is the same for design convenience and better recognition effect), wherein 4 groups of concentric circles/concentric polygons are arranged at four corners of the rectangular card, a fifth group of concentric circles/concentric polygons is positioned at the central position of the long edge of the card, and the circle center of the fifth group of concentric circles/concentric polygons is aligned with the circle centers of two adjacent grains. (in fact, the positions of all the concentric circles/concentric polygons are not necessarily arranged in this way, and can be arranged in other places, and the only requirement is that the total pattern after the arrangement of the concentric circles/concentric polygons cannot be similar to the centrosymmetric pattern.

2.1. Print/display request for pattern

The number of layers of the concentric circles/concentric polygons is infinite in design, but in practice, since the inner circle is small, the number of layers can be determined according to the printing and display accuracy of a printer or a display, and the inner circle stripes exceeding the range of the printing accuracy or the display capability can be omitted. At least two black stripes are generally required for accurate identification. The scaling of the edges of the turns is

As shown in fig. 2.

3. Image recognition method

3.1, video input

Videos collected from a camera or other video source are first broken down into framing sequences. According to the computing power of the machine, the identification can be carried out every frame or every few frames.

3.2 RGB/YUV preprocessing

The extracted frames may be in RGB or YUV format.

When in RGB format, the fast recognition method takes only the red or green channel and discards the remaining channels. The card is black and white, so the obtained channel can still restore the gray pattern of the card. The compression rate of the blue channel is adjusted to be high by the compression format of many videos, so that the definition of the blue channel is not high, and the blue channel is not used; the red and green weighted synthesis channel is recommended, the channel precision is high, and the calculation amount is moderate. The standard transition with high accuracy is 0.299R + 0.587G + 0.114B; the fast conversion may take the RGB average.

When extracting YUV format, only Y channel, discarding U, V channel.

By the processing of this step, the color image is changed into a grayscale image as a result.

3.3 binarization processing

The simplest way to quickly analyze a grayscale image is to change it to a black and white image. In general binarization processing, a threshold value is set, and the brightness is 1 when the brightness exceeds the threshold value and is 0 when the brightness is lower than the threshold value. In practical use, the threshold value is difficult to be determined in advance because the pattern is to be recognized under different lighting environments. Because of the uneven illumination, a proper threshold cannot be found to correctly separate the patterns. Therefore, the invention provides a dynamic and flexible method for carrying out binary separation on the patterns.

The method needs to be implemented with a blur filter (the blur filter will be discussed in 3.4).

The image is copied in two copies, denoted copy 1 and copy 2.

Replica 1 performs an extremely slight amount of gaussian blurring (σ < ═ 3px) to eliminate noise in the video. (this step may be omitted if efficiency is to be improved).

Copy 2 is gaussian blurred to a larger radius. The radii are 1/30 to 1/10 of the image diagonal (empirical values, calculated in terms of image aspect ratio 4: 3). The fuzzy radius is related to the capability of the lens, the using environment, even the printing material of the FPAR card, and the like, and can be set according to actual requirements.

Example 1: the example of fig. 3 is taken under a more ideal environment. The treatment was carried out using the method mentioned above. As can be seen from fig. 3, in the result of the binarization processing, the recognized pattern is very clear, and an ideal effect is achieved.

Example 2: FIG. 4 is an example of an FPAR card in a semi-yin-yang state, with one portion of the card under the sun and another portion under the shadow, processed using the above-mentioned method. From the above binarization results, even if the FPAR card of the original image is yin-yang, the binarization result is still ideal and is not affected by yin and yang. In fact, the parameters of each step are determined, and the effect is completely the same as that of example 1. The amount of Gaussian blur and the threshold value of the last step of binarization can be predetermined and cannot be changed due to the change of yin and yang environments.

Example 3: in order to show the advantages of the method, the original image as above is still used for binarization by using a traditional simple method. Namely, a threshold value is set, the brightness is changed into black when the brightness is lower than a certain threshold value, and is white when the brightness is not higher than the certain threshold value. In FIG. 5, the a diagram has only one complete anchor point, and other anchor points are all occluded or partially occluded; b, partially blocking the positioning point at the upper right corner of the graph; c, the positioning point of the lower right corner cannot be seen. During recognition, the threshold value is set to be higher in the graph a, the threshold value is set to be middle in the graph b, and the threshold value is set to be higher in the graph c. Experiments prove that no matter how many threshold values are set by using the traditional method, a satisfactory result is not easy to obtain. From the above discussion, the simple binarization method cannot accommodate these complex cases, and the threshold setting is extremely difficult. It follows that the algorithm presented herein clearly has advantages.

Example 4: low contrast and high noise. From FIG. 6, where the original white color is present, excessive black spots appear on the processed card; the original black place has white color on the processed card. And the threshold cannot be adjusted to achieve the desired effect anyway.

The results of the process applied to the algorithm are shown in fig. 7, where the leftmost image corresponds to copy 1 in 3.3, with slight gaussian blur noise reduction, and the other steps are exactly the same as in the first two examples. As can be seen from FIG. 7(d), the black circles in the recognized pattern are full, the white circles are clean, and there is almost no noise around the place where the recognized pattern is located, which is more beneficial to the recognition in the later period than the traditional threshold algorithm.

Example 5: a blurred picture.

The FPAR fringe-like design has the advantage of dithering. The picture of fig. 8 is taken from a dynamic video, where the FPAR card has a large jitter, thus generating a blurred picture. After binarization processing is carried out by the method, the black circle at the outermost circle can still be seen, so that the obtained pattern can still support identification.

If the pattern design of the FPAR card is identified by the traditional method and the method of the invention is not combined for binarization processing, good binarization results cannot be obtained. As shown in fig. 9, the threshold values are sequentially high to low from left to right. It is easy to see that five anchor points in none of the five figures can be analyzed correctly. From this, it is understood that the FPAR pattern design and the binarization processing of the present invention need to be used in combination. If the FPAR pattern design is not used, good identification effect cannot be obtained by only using the binarization processing method of the invention.

3.4 fuzzy filter

3.4.1 basic method

The following table lists the advantages and disadvantages of some commonly used image processing methods.

3.4.2 Rapid improvement for mobile device optimization

For the mainstream mobile devices at present, it is generally difficult for the computing power to support methods 1 and 2. The method 3 using simple coarseness, and the method 4 having better effects can be considered. Since the method 4 contains more floating point operations, which are not beneficial to the fast calculation of the mobile device, it can be improved to adapt to the mobile device.

According to the article "IIR Gaussian Filter approximation

As described in the advanced vector Extensions "and" Recurvedic Gaussian Derivative Filters ", forward and reverse order two-fold IIR Filters may be used, where the forward direction is P₀＝p₀Other P_n＝w₀*p_n+w₁*P_n-1+w₂*P_n-2Similarly, the inverse is of the form Q_len-1＝P_len-1And the rest of Q_n＝w₀*P_n+w₁*Q_n+1+w₂*Q_n+2. The part of the subscript which is out of bounds is processed in a clamping mode, namely the nearest neighbor legal subscript is taken. In this form, gaussian blur is simulated using an IIR filter. Where P, P, Q, and w are floating points. Therefore, at least 6 floating-point multiplications are required for each pixel calculation. Assuming that the resolution of the image obtained by the camera is 640 × 480, 25 frames per second are captured, and the floating point multiplication operation is performed 640 × 480 × 6 × 25 — 46080000 times per second. This amount of computation is clearly a significant overhead for the mobile device.

The invention provides a method, which uses a shaping table look-up to avoid floating point operation and multiplication operation in a space time conversion mode to obtain a result similar to the original floating point operation method.

First, w₀,w₁,w₂These three floating point values, determined by the radius of the gaussian blur, are pre-computed once the radius is determined. And P and P are both numerical values between 0 and 255, wherein P is a floating point and P is an integer. It is naturally contemplated that all the product results of P and P with w may be pre-stored and tabulated for future reference in order to avoid floating-point multiplication.

Because the number of the single-precision floating point is too many, all values need to be listed and stored in advance, and the memory space is extremely large. The present method uses an approximation approach to list a smaller precomputed multiplication table.

Current mainstream processors all use IEEE754 floating-point format, which features a one-bit symbol followed by several bit-level codes, and finally a mantissa. For a 32-bit single-precision floating-point numerical value, the truncation of the last 16 bits of the mantissa only affects the precision of the floating-point numerical value and has little influence on the numerical value. Therefore, the P 'is the numerical value of P after truncating 16 bits at the tail, the truncated data P' only has 16 bits, 65536 possible values are totally obtained, and the 65536 possible values and w are combined₀，w₁And w₂Respectively multiplying and storing the multiplication results, and the occupied memory space is small, which provides an improvement for the fast table look-up calculationProviding the possibility of using the method. The approximate calculation formula is P W ≈ P '═ W [ [ P'],p*w＝W’[p’]Where W and W' store the pre-computed product results for the array.

Considering that IIR calculations involve floating point addition operations, floating point addition is more time consuming than integer addition operations. The invention adopts integer addition to replace floating-point addition operation. Note that this step is an improvement over the previous paragraph, and the formulas will differ slightly from the previous paragraph. Taking the IIR positive sequence direction as an example, since the calculation includes a fractional part, integer calculation is used to simulate fixed point fractional calculation, and 16-bit short integer calculation is used for integer calculation, wherein the integer and fractional parts respectively occupy 8 bits.

The calculation formula of the IIR filter in the positive sequence is P_n＝ω₀*(p_n<<8)+ω₁*P_n-1+ω₂*P_n-2，P_nThe calculation result is 256 times of the original formula and is an integer. During calculation, the omega table can be used for simplifying multiplication operation, and the upper seed is changed into P_n＝Ω'[p_n]+Ω[P_n-1]+Ω[P_n-2]。

The amplification problem of the positive sequence numerical value can be solved by directly shifting 8 bits to the right after the reverse sequence calculation is finished. The tables used for the reverse and forward sequences are identical, except for a slight difference in the calculation formula. Is Q_n＝ω₀*P_n+ω₁*Q_n+1+ω₂*Q_n+2. The multiplication can be solved by looking up the omega table, and the expression is Q_n＝Ω[P_n]+Ω[Q_n-1]+Ω[Q_n-2]. Then R is_n＝Q_n>>8 is the result.

So far, all floating-point operations are replaced by fixed-point operations, and all multiplications become direct table lookup and fetching. The method has obvious improvement and improvement on the operation efficiency of the mobile equipment.

In addition, the following three operation steps can be reduced in a manner of reducing the resolution of the identification layer:

1) color conversion; 2) gaussian blur; 3) and (5) color filling.

Thereby significantly increasing the speed of the overall operation. For example, the original image is 640 × 480, but the step of converting the gray level has changed it to 320 × 240, and then the subsequent steps 1),2),3) are performed to reduce the operation amount to 1/4. When the results of generating the virtual reality are subjected to the mapping synthesis, the resolution of 640 x 480 is still used. This allows the output image sharpness to be maintained, except that it produces a map position error within 1 pixel, making it difficult for the user to visually see the difference in image quality.

3.5, judging possible positioning points

3.5.1 filling

Check each pixel of M and if the current pixel is 0 or 1, then with a value of 2, perform 4-pass or 8-pass fill. And counting the maximum width and height of the filling and the area of the filling, wherein the area is the number of pixels. If the aspect ratio is between 1:4 and 4:1 (this is an empirical value, and can be specifically adjusted), and the filling ratio is within

Nearby, the rectangular outline position and size of this filled range are recorded in the list L.

3.5.2 filling region limiting method

When the image is large, the filled area may be large, the filled pattern may be complex, and for the conventional filling algorithm, especially on the mobile device, a stack overflow or a large memory occupation may occur. In order to solve such a problem, the filling area may be limited, for example, by setting left, right, upper, and lower boundaries. That is, if the filling is started from the (x, y) point and the maximum filling area u is set, the left boundary is x-u, the right boundary is x + u, the upper boundary is y-u, and the lower boundary is y + u. This limits the total fill area to no more than 4u²Thereby reducing the memory burden. This method is called a fill region limiting method.

3.5.3 significance of filling operation

The contents of list L may be selected as possible anchor locations due to the following special features.

First, the positioning point of the FPAR card is a perfect circle, and each circle inside the positioning point is a circle, so that under the ideal distortion-free and rotation-free condition, the photographed FPAR clamping point is also a perfect circle, and the aspect ratio of the outer frame obtained by using the filling method is 1: 1.

However, since the FPAR card may be placed obliquely, the anchor point in the captured image is not necessarily a perfect circle, but may be distorted to appear like an ellipse, so the aspect ratio is not necessarily 1:1, and a certain distortion margin should be reserved. For example, it is possible to define that the aspect ratio is legal when it is between 1:4 and 4: 1.

In addition, because the design of the anchor point is a fractal graph, the filling rate is consistent no matter which layer of pattern is. Should be close to 0.436.

The above two are the necessary conditions for determining the anchor point, so all legal anchor points will appear in L, but there are some error detection points, which need to be eliminated by the following method.

3.5.4 example of filling box selection

An example of a fill box is given in fig. 10-11. The two diagonal regions represent substantially the same value, but are shown differently for clarity. The "\\\ slashed area indicates the area just filled in by the present step, and the"///"slashed area indicates the area once filled in by the previous step. Only a portion of the steps are listed in fig. 10.

3.5.5. Concentric frame

If a position in the frame is a true anchor point, the inner concentric circle/concentric polygon of the position must be selected by L repeatedly, see FIG. 12.

By utilizing the rule, items in L are compared and analyzed, if some frames are independent or not concentric with any other frame, the error detection can be considered, most error detection points can be eliminated, the coordinates of the center x and the y of each group of concentric frames are recorded and are recorded as a table H, if a recognizable complete pattern exists in a picture, H at least contains the information of 5 positioning points in the pattern, but sometimes due to error detection, the information of more than 5 suspicious positioning points possibly exists in H, and further judgment needs to be made at this moment to eliminate the error detection positioning points.

In addition, the set of frames forming each possible positioning point in H has the characteristic that when the positioning points exceed 5, each possible positioning point can be scored by comprehensively examining the concentricity, the length-width ratio similarity and the weight of the frame group, and the score is recorded in H for later use. And if the number of points in the H is less than 5, judging that no legal FPAR card identification pattern exists in the picture.

3.5.6. Possibly misdetected condition

In all the above-described identification processes, only sufficient conditions for identifying the positioning points are given, and the conditions are not necessary. There may actually be various false detected anchor point situations. As shown in fig. 13, more than 5 anchor points may be detected. But generally still can obtain correct results through the various screening and verification.

Fig. 13 lists some patterns that may lead to false detection. FIG. 14 lists three configurations of non-circular patterns that can cause false positives;

since there are many patterns similar to the anchor points, the false detection of the anchor points is inevitable in practical applications. Although the misjudgment situation of the positioning point can be improved by adding the judgment standard, the detection process is abnormally complicated, and another method is needed. In practical application, a plurality of error detection points may appear at the same time, but the probability that the arrangement mode of the error detection points is consistent with the arrangement mode of the positioning points in the identification pattern of the FPAR card is quite small. The positioning method provided by the invention can ensure that a satisfactory effect is achieved.

If deliberately sought, the pattern of the mahjong bobbin may cause false detection, as shown in fig. 15.

3.5.7. Identification from anchor points to FPAR (flash real estate array) card patterns

At 3.5.4, whether the number of H points exceeds 5 points or is exactly 5 points, there is a problem: how does each anchor point in H correspond to the FPAR card identify anchor points in the pattern? At this time, a permutation and combination method is adopted, and all correspondences are enumerated and substituted into a calculation formula (described in section 4 below) to test which corresponding manner is most reasonable.

3.5.8. Influence of filling region limiting method on recognition effect

When using the fill region limit method, the largest outer circle of some anchor points may be misidentified. But because the pattern has the scaling characteristic [2.2.1], when the patterns of the inner layers can be correctly recognized, a correct recognition result can be generated.

In fig. 16, a smaller fill boundary is used as the area constraint. The centroid of the fill boundary coincides with the location of the fill seed. After the region-limited method is used, it is seen from the result chart (see fig. 17) that the black region in the outermost circle of the pattern is colored 5 colors, i.e., divided into five boxes (shown in steps 1,2,3,4, 5), and the white region in the next outer circle is also divided into three colors, i.e., divided into three boxes (shown in steps 6,7, 8). The 8 selection boxes of the 5+3 selection boxes are invalid, but all the selection boxes can be correctly and completely selected from the second layer of black circle to the inner layers, so that the selection boxes are all valid. The inner layer black and white added up to a total of 5 layers is already sufficient for identification.

In fig. 18, the frame selection without region limitation is shown, and 7 layers of rings are used, i.e. a complete 7-layer pattern can be analyzed, and 7 layers can be used for identification.

For anchor point identification, at least 2 layers are generally required to avoid false detection. Although the result is more accurate as the number of layers is larger, the accuracy of the pixel level can be achieved after the number of layers reaches three or four layers, which is not meaningful in the above, so that the acceptable effect can be achieved as long as about three layers can be identified after the area is limited.

If the filling boundary of the region-limited method is set too narrow, the following effects may be produced:

1) identification failure caused by too few frame selection layers

2) The picture-shake resistance is impaired, and if the black concentric circles/concentric polygons of the outermost layer cannot be recognized due to the region limitation, as in the case of example 5 in fig. 3.3, the entire pattern recognition will be directly failed.

4. Two-dimensional to three-dimensional conversion method

4.1. Two-to-three dimensional conversion label definitions

The premise of the invention is that the camera lens is not distorted, if the camera lens has pincushion distortion or barrel distortion, the correction is firstly carried out, and then the following method is applied.

The FPAR card is placed in an actual three-dimensional space at any rotation angle, and its five positioning points are respectively marked as ABCDE, as shown in fig. 19. The FPAR card is shot by the camera, and the process of obtaining the two-dimensional image can be regarded as that each point on the FPAR card is connected with the O point and then is handed over to an imaging picture. And each positioning point of the FPAR card in the imaging picture is marked as abcde.

4.2. Determination of each anchor point

Since it is unknown which point corresponds to the point a, which point corresponds to the point b, which point corresponds to the points c, d, e, etc. on the FPAR card, and there may be mis-detection points included in H, see fig. 20. Therefore, to perform sorting and elimination, the position relationship among points a, b, c, d and e on the FPAR card has the following obvious characteristics:

(1) the polygon abcd should be a convex polygon;

(2) ∠ bec should be close to 180 °;

(3) the distances e to b and c should be relatively close.

The specific judging steps are as follows:

(1) first judging convex polygon

The convex polygon judging method comprises the following steps:

c. excluding line segments has the option of crossing, see fig. 21.

d. Judging whether the product has the same number

Whether or not these values are the same sign as the axis, and may all be positive.

(2) Comprehensive judgment

Since each anchor point is identified by an image, the evaluation criterion of the above 2, 3-point rule should be flexible. Therefore, in the operation, whether or not the required level is satisfied is not directly evaluated, but how much the degree of adhesion to the rule is. Therefore, the 3-point rule should be combined to determine a reasonable scoring criterion. The higher the score, the more satisfactory the representation.

(3) Optimization decision

When the calculation capability is allowed, all the points in H may be sorted into 5 points to be fully arranged, and the above rule is substituted for verification, and if there are too many candidate items in H, the fully arranged point is an astronomical number, and in order to reduce the calculation amount, the first n points with higher scores in H (n > -5) may be used for the substitution test.

Through the above calculation, it is possible to select the most desirable 5 points in H and obtain their corresponding relationship with the abcde points.

4.3. Obtaining three-dimensional normal vector of FPAR card

Now, the normal vector of the FPAR card is reversely deduced through the imaging picture

To serve subsequent virtual reality renderings. For this purpose, the O-point is crossed as an auxiliary line l₁//AD//BC；l₂// AB// DC; record OAB normal vector as

OCD normal vector of

Normal vector of OBC is

The normal vector of OAD is

The normal vector of the FPAR card is

l₁Has a direction vector of

l₂Has a direction vector of

Then there are:

to obtain finally

Where × denotes vector cross multiplication.

If it is

Very far from the spectrum, e.g. calculatedIf the FPAR card is almost vertical to the view plane, the result needs to be rejected, and 4.2 is returned to reselect other permutation and combination.

Z-axis multiple solution non-impact analysis

As shown in fig. 22, when the FPAR card is reversely derived from the two-dimensional imaging picture, there may be a plurality of solutions because the z-axis position is unknown. I.e., a small near FPAR card appears virtually indistinguishable from a large far FPAR card. The imaged picture is preceded and followed by only a difference in zoom. In fact, this distance cannot be determined if it is only monocular vision. However, these differences are only differences in scaling and do not affect the subsequent calculation. It is straightforward to simply set the distance from the viewpoint to the imaged picture to be n and the distance from the viewpoint to the center of the FPAR card to be m for subsequent calculations.

4.5. Verifying correctness

According to a distance assumption of 4.4, and 4.3

The three-dimensional position of the ABCDE point on the FPAR card in the three-dimensional space can be reversely deduced from the position of the ABCDE point in the imaging screen of the plane. The calculation method is as follows:

by dot-law, by use of

And the distance from O to the FPAR card, the equation of the plane sigma where the FPAR card is located is l_xx+l_yy+l_z(z-m)＝0；

The intersection of the rays Oa, Ob, Oc, Od, and Oe and Σ is the position of ABCDE in the three-dimensional space.

Take point A as an example, note t_a＝l_zm/(l_xa_x+l_ya_y+l_za_z) (ii) a The coordinate of A is (t)_a*a_x,t_a*a_y,t_a*a_z). The coordinates of each point of BCDE can be obtained in the same way.

As is apparent from the above formula, each point coordinate has a product m, so that m is not taken as a non-zero constant value, e.g. 2, and then a_zN is 1. So that all points sitThe target can calculate specific values to facilitate subsequent calculation.

If the numerical value is far away from the spectrum, for example, ∠ ABC is far away from the right angle of the space, or the E point position is far away from the middle point position of BC, or the aspect ratio of a rectangle formed by positioning points at four corners is far away from the design time of an FPAR card, the result is judged to be incorrect, the result needs to be rejected, and 4.2 is returned to reselect other possible permutation and combination.

Steps 4.5, 4.3, 4.2 are a set of mutually verified relationships that require repeated inter-inference to get the best results.

In addition, some auxiliary verification methods can be added. For example, if the length-width ratios of the frames in a suspected anchor point are different, it is considered that the anchor point may be a false detection point. Similarly, if the average aspect ratio of each ring in a set of anchor points in the identified pattern is not consistent, the identified pattern is considered to be possibly incorrect. In practice, the expected aspect ratio of the anchor point is calculated from the normal vector of the FPAR card calculated previously, and if the calculated value is far from the measured value, it can be determined that the recognition is incorrect.

The identification is carried out according to all the comprehensive methods, and when the image is clear and has no non-interference resistance (for example, two FPAR cards appear in the picture at the same time), the obtained identification result can be close to 100 percent correct. In the example given below, the background is a newspaper, which has a complex pattern and is noisy, but does not affect the recognition result.

If there are multiple FPAR cards in the same frame, many anchor points will be detected simultaneously, but it is difficult to distinguish which five anchor points belong to the same group. Since the computer finally perceives that there is not a picture but a set of suspicious anchor points, it is possible that the suspicious anchor points in a certain picture may be as shown in fig. 23.

Like the distribution of fig. 23, even the naked eye cannot distinguish which is the false detection point and which is the real point, and which positioning points form an FPAR card pattern. At this time, an improvement may be made to the pattern of the FPAR card, for example, changing concentric circles into a C-shape. One modification for reference is shown in fig. 24: then multiple FPAR cards may be put together with the effect of fig. 25.

The locating points of the patterns can also be identified by the algorithm of the invention, but the locating points have more information than the original locating points, namely the directional property of the FPAR card. Taking the leftmost card as an example, we can identify the opening direction of the C-shaped opening, as shown in fig. 26.

Besides the opening direction, the confidence level of the opening direction can be calculated by detecting the values of the definition, the angle and the like of the opening. The opening direction and the confidence level are two pieces of information, and the arrow direction can be used for visually representing the most possible opening direction, and the arrow size is used for representing the confidence level, which is shown in fig. 27.

The upper graph has direction information, and compared with the previous graph only having the positioning point position, it can be easily seen which are real positioning points and which positioning points can be combined into a group to form an FPAR pattern. The determination can be made as follows:

the multi-FPAR card identification method with the direction comprises the following steps:

1) the point with high reliability of the opening direction is preferentially considered as the positioning point.

2) If the directions of two anchor points are close and the direction of their connection line is close, they may belong to the same FPAR card and belong to points B and E respectively. Where point B points to point E.

3) If starting from a certain positioning point, the next point is searched through the pointed direction of each point, 5 positioning points can be found approximately continuously, and the direction reliability of the last positioning point is obviously lower than that of the first 4 points, the continuous 5 positioning points probably belong to the ABECD positioning points of the same FPAR card.

4) In the principle of proximity, if there are two other positioning points in the direction pointed by a certain positioning point, the closer one of the positioning points belongs to the same FPAR card.

In practical use, due to the distortion of the camera, the bending of the FPAR card, insufficient recognition accuracy, and the blur of the picture caused by the out-of-focus or jitter of the lens, the obtained vector result may have errors, so the direction pointing mentioned above is the determination of the approximation degree, rather than the necessity of just aligning with each other. So the measurement also needs to be done in certain permutation and combination to test which match is most reasonable. However, through the above 4 rules, we can prune the enumeration process to a large extent, greatly reducing the number of tests.

5. Rendering virtual reality

The actual virtual reality rendering may be adding a three-dimensional solid to the screen or simply pasting a map.

FIG. 28 is an example of an FPAR card multi-angle imaging implementation of a three-dimensional image.

The preset picture is now distorted and then filled into the picture. In order to increase practicality and reality, the filled-in photo needs to cover all positioning points, and no matter how the FPAR card is placed, the photo looks static relative to the FPAR card and looks like a photo actually stuck on the FPAR card. In order to do this, the key step is to calculate the position of each point in the imaging picture by the three-dimensional position of the FPAR card, see fig. 28. This calculation is an inverse pair of the method in 4.5.

Virtually pasting a plane picture on the three-dimensional FPAR card, and calculating the position of the color of each corresponding point to be displayed on the imaging plane after the picture is pasted on the FPAR card.

In actual operation, the position of a corresponding point on the FPAR card needs to be searched from a rendering imaging plane, the calculation is complex, the boundary is difficult to determine, and the calculation amount is large. Other compact methods need to be used instead. The simplest method is to use the drawTriangles method, not to compute the color of all the points on the FPAR card completely, but to make a sparse matrix grid and then map this grid onto the imaging plane. The positions of the holes of the mesh were simulated with a transformation of triangular patches. A triangular slicing scheme is shown in fig. 29.

The drawTriangeles or the synonymy equivalent method thereof are directly supported by various platforms, such as HTM L5, OpenG L, F L ASHActionScript3, cocos2D-x and the like, and are also supported by the quick operation of the video card, and the method is a general and conventional calculation method, and the following website address (explained by the version F L ASH ActionScript 3) http:// help. adobe. com/zh _ CN/as3/dev/WS84753F1C-5ABE-40b1-A2E4-07 D4973976Chtml.

In order to increase the sense of reality, when the FPAR card is under different light environments, the tone of the map should be adjusted to match the tone of the FPAR card. There are many empty positions that do not have the setpoint on the FPAR card, can take the colour of these positions to carry out white balance, adjustment such as luminance to the map to let its color and luster laminate reality environment.

Finally, supplement the following rendering example:

example 6: three stand photos are prepared, and one FPAR card is made according to the appearance of the stand photos, as shown in the picture on the desktop of FIG. 30.

The effect of real-time rendering is shown on the tablet computer on the right side of the picture 30, the FPAR card can be in a false and real state, and the picture is seen on the screen as if four photos are taken immediately. FIG. 31 is a rendering screenshot effect.

Fig. 32 shows the extended function of the FPAR card, in which not only the photograph is rendered in the screen, but also the handwritten "guangzhou 2016" is rendered, and the location of the "guangzhou 2016" has no corresponding recognition pattern. It can be seen that the rendered area is expandable. In addition, it is worth mentioning that in the real-time rendering example, the upper right corner of the photo is provided with a corner mark, and the position of the corner mark is even rendered outside the photo frame, which also reflects the expansibility of the rendering capability.

The above embodiments are preferred embodiments of the present invention, but the present invention is not limited to the above embodiments, and any other changes, modifications, substitutions, combinations, and simplifications which do not depart from the spirit and principle of the present invention should be construed as equivalents thereof, and all such changes, modifications, substitutions, combinations, and simplifications are intended to be included in the scope of the present invention.

Claims

1.A design method of a FPAR card is characterized in that the FPAR card integrates fast positioning and fast identification information, a plurality of groups of concentric circles/concentric polygon lines are arranged on the FPAR card, each group of concentric circles/concentric polygons is called a positioning point, the plurality of groups of concentric circles/concentric polygons form identification patterns, and the total pattern of the arranged plurality of groups of concentric circles/concentric polygons is not centrosymmetric.

2. The method of claim 1, wherein the FPAR card is provided with 5 sets of concentric circles/concentric polygons, each set of positioning points has the same texture shape, wherein 4 sets of concentric circles/concentric polygons are arranged at four corners of the rectangular card, and a fifth set of concentric circles/concentric polygons is positioned at a central position of a long side of the card or at an asymmetric position where the FPAR card can be identified; the scaling ratio of two adjacent circles of edges of each group of concentric circles/concentric polygons is

3. A FPAR card rapid identification and positioning method is characterized in that the FPAR card integrates rapid positioning and rapid identification information, and the rapid identification and positioning of the FPAR card comprises the following steps:

s4, carrying out image recognition processing by adopting a fuzzy filter method; using a forward-backward order dual IIR filter, where the forward direction is P₀＝p₀Other P_n＝w₀*p_n+w₁*P_n-1+w₂*P_n-2In the reverse form Q_len-1＝P_len-1And the rest of Q_n＝w₀*P_n+w₁*Q_n+1+w₂*Q_n+2Wherein the subscriptThe border-crossing part is processed in a clamping mode, namely, the nearest neighbor legal subscript is taken, and an IIR filter is used for simulating Gaussian blur in the mode; { p₀,...,p_len-1For IIR filter, one-dimensional ordered real number sequence is received, p₀、p_nAll receiving numerical values in a one-dimensional ordered real number sequence; { Q₀,...,Q_len-1Q is an ordered real number sequence output by the IIR filter_n、Q_n+1、Q_n+2Are all numerical values in the output ordered real number sequence; { P₀,...,P_len-1Is a temporary real number sequence, P, also used in the calculation process of the IIR filter_n-2、P_n-1、P_nAre all values in a temporary real number sequence; w0, w1 and w2 are all real weight parameters;

s5, judging possible positioning points;

4. The method for rapidly identifying and positioning the FPAR card according to claim 3, wherein in the step S3, the binarization processing is implemented by using a blur filter, which specifically comprises:

copying the image into two copies, namely copy 1 and copy 2;

if the video has more obvious noise points, copy 1 carries out extremely trace Gaussian blur, wherein the Gaussian radius sigma is less than 3 px;

5. The method for rapidly identifying and positioning the FPAR card according to claim 3, wherein in the step S4, the specific method for performing the image processing is as follows:

s41, determining three floating point values w0, w1 and w2 by an IIR filter simulating Gaussian blur, wherein P and P are both numerical values between 0 and 255, P is a floating point, and P is an integer; therefore, the product result of the parts P and P and the real number weight parameter w is prestored to be made into a table for future checking;

the calculation formula is that P ≈ W ≈ P ' ═ W ' ], P ≈ W ═ W ' [ P ], where W and W ' are arrays in which a pre-calculated product result is stored, and P ' is a result of a single-precision floating point number P truncated by a 16-bit mantissa.

6. The method for fast identifying and locating the FPAR card according to claim 3, wherein the step S5 specifically comprises:

s5.1, checking the dot matrix image, if the current pixel of each pixel of the dot matrix image is 0 or 1, performing 4-communication or 8-communication filling by using a numerical value 2, and counting the maximum width and height of the filling and the area of the filling;

s5.3, selecting possible positioning point positions, wherein the conditions that the length-width ratio is legal when the length-width ratio is between 1:4 and 4:1 can be limited, and in addition, because the positioning point is designed into a fractal graph, the filling rate is consistent no matter which layer of pattern is, and the filling rate is 0.436;

7. The FPAR card fast identification and positioning method of claim 6, wherein in step S5.4, for the positioning point identification, at least 2 layers are required to avoid false detection.

8. The method for fast identifying and locating the FPAR card according to claim 3, wherein the step S5 specifically comprises: the determination of each positioning point specifically comprises the following steps:

(1) first judging convex polygon

The convex polygon judging method comprises the following steps:

eliminating the crossed option of the line segment, and judging the product with the same number;

(2) comprehensively judging;

(3) optimizing decision;

when the calculation capacity permits, all the points in the table H can be selected to be 5 points for full arrangement, and the selected points are substituted into the above rule for verification, if the candidate items in the table H are too many, the full arrangement is an astronomical number, and in order to reduce the calculation amount, the top m points with higher scores in the table H can be selected for full arrangement and substitution test.

9. The method for fast identifying and locating the FPAR card according to claim 3, wherein the step S7 specifically comprises: the method of using the programmed library function drawttrianges does not fully compute the color of all the points on the FPAR card, but rather makes a sparse matrix grid, and then maps this grid to the positions of the holes of the grid on the imaging plane for simulation with the transformation of the triangular patches.