CN105740818A - Artificial mark detection method applied to augmented reality - Google Patents

Abstract

The present invention relates to an artificial sign detection method applied to augmented reality, and the specific steps include: S1: collecting a frame image, sampling the frame image roughly, using oblique grid scanning, and detecting edge pixels of the frame image; S2: based on the RANSAC algorithm , to detect the edge segments in the frame image; S3: fuse the edge segments; S4: extend and filter the edge segments; S5: detect the corner points of the quadrilateral, and construct a quadrilateral according to the corner points of the quadrilateral. The invention preprocesses frame images before operation, performs coarse grid sampling, and performs edge detection on each grid area, greatly reduces program operation time, improves detection speed, has good real-time performance, and meets real-time detection requirements. The invention adopts an edge-based detection method, firstly performs a line segment test, and then reconstructs a quadrilateral frame of a sign according to the line segment obtained by the line segment test, and has good robustness to illumination changes and occlusion conditions.

Description

An Artificial Landmark Detection Method Applied to Augmented Reality

technical field

The invention relates to an artificial mark detection method applied in augmented reality, and belongs to the technical field of augmented reality applications.

Background technique

Augmented reality technology is a new technology that "seamlessly" integrates real world information and virtual world information. , touch, etc.), through computer and other science and technology, simulate and then superimpose, apply virtual information to the real world, and be perceived by human senses, so as to achieve a sensory experience beyond reality. The real environment and virtual objects are superimposed on the same screen or space in real time.

The fiducial mark system is widely used in augmented reality, robot navigation, position tracking, image modeling and other general projects. It mainly uses image processing technology to detect and identify known 2D artificial signs placed in the environment, extract sign information, and calculate cameras and objects. Relative positional relationship. The most important parameters of the fiducial marking system are false detection rate, mixed code rate, minimum detection pixel and light immunity. The key technology of the fiducial marker system is the design of artificial markers and the corresponding identification and positioning methods. The existing artificial signs have a single design, which makes image recognition vulnerable to interference from light and complex objects. There is no internal storage information, so the data of virtual objects must be stored on the recognition device and cannot be changed flexibly.

The ARTag logo is a binary flat logo, and each logo has its own ID number, which is encoded inside the logo by a binary value. Compared with the previous ARToolkit logo, ARTag solves the problems of high error rate, false detection rate and mixed code rate. The performance of the benchmark marker system depends on the detection performance of 2D markers. In addition to considerations such as detection speed and error rate, the marker detection algorithm also needs to be robust to lighting conditions and occlusions.

Contents of the invention

Aiming at the deficiencies of the prior art, the present invention provides an artificial marker detection method applied to augmented reality;

The invention improves the recognition speed, anti-interference ability and accuracy of artificial signs.

Terminology Explanation

RANSAC algorithm, the abbreviation of RANdomSAmpleConsensus, is an algorithm that calculates the mathematical model parameters of the data based on a set of sample data sets containing abnormal data, and obtains effective sample data.

Technical scheme of the present invention is:

A method for detecting artificial signs applied to augmented reality, the specific steps comprising:

S1: Collect the frame image, roughly sample the frame image, use oblique grid scanning, and detect the edge pixels of the frame image;

S2: Based on the RANSAC algorithm, according to the edge pixels obtained in step S1, the edge line segment in the frame image is detected;

S3: Fusing the edge line segments detected in step S2;

S4: Extending and screening the edge line segments fused in step S3;

S5: Detect the corner points of the quadrilateral according to the extended and screened edge segments in step S4, and construct a quadrilateral according to the corner points of the quadrilateral.

Preferably according to the present invention, the step S1 specifically includes:

S11: divide the frame image into several regions, each region contains m×m pixels, m∈(20, 60); further preferably, m=40; divide the frame image into several regions, the detection speed is accelerated, Improved real-time performance.

S12: For each area obtained in step S11, use oblique x° and (180-x)° scanning lines to perform grid scanning, x∈(22.5,67.5), and the distance across each grid is y Pixel, y∈(3,9); further preferably, x=45, y=5;

S13: each scanning line in step S12 is convolved with the Gaussian first-order derivation, and the gradient intensity component of the pixel along the direction of each scanning line is calculated;

S14: According to the gradient strength component of the pixel point obtained in step S13 along the direction of each scanning line, calculate the gradient strength value of the pixel point, the pixel point corresponding to the local extremum in the gradient strength value is the local extremum point of the edge pixel gradient strength, The edge pixel is extracted, and the direction of the edge pixel is calculated according to the gradient intensity component.

Preferably according to the present invention, the step S3 specifically includes:

S31: Select an edge line segment obtained by step S2, and name it line segment a, randomly select another line segment from the remaining edge line segments, and call it line segment b, and perform fusion judgment on line segments a and b, that is, if line segments a and b satisfy : |θ _a -θ _b |∈Δ _θ , And Lab ∈ Δ _l , then merge the line segments a and _b to get a new line segment; otherwise, continue to select a line segment from the remaining edge line segments, and continue to fuse with line segment a until line segment a and all edge line segments except line segment a are completed Fusion judgment; θ _a and θ _b are the direction of line segments a and b, Δ _θ is the threshold value of the direction error of the line segment to be fused, θ _ab and Lab are the direction and length of line _ab connecting line segment a and b, and Δ _l is line segment a , The threshold of the allowable length of b connection ab;

S32: Step S31 is repeated until all edge line segments obtained in step S2 are fused.

Preferably according to the present invention, said step S4 specifically includes:

S41: According to the edge line segment obtained in step S3, arbitrarily select an edge line segment and select an end point thereof, and judge whether the direction along the edge line segment is consistent with the direction of the pixel adjacent to the end point, if they are consistent, then the pixel point Add it to the edge line segment, and continue to check the next pixel adjacent to the pixel point until the direction of a pixel point is inconsistent with the direction of the edge line segment, then the pixel point is updated as a new endpoint of the edge line segment;

S42: Execute step S41 on the two endpoints of all the edge segments obtained in step S3, to obtain the extended edge segments and their new endpoints;

S43: Preliminarily screen the extended edge line segment obtained in step S42, and delete the edge line segment whose length value is less than n pixels, n∈(15, 25), further preferably, n=20; the edge line segment whose length value is too small It cannot be the logo border, even if it is, it means that the logo border is too distorted or the scene scale is too large, it is no longer meaningful to detect, so it is deleted;

S44: Carry out a line segment test to the edge line segment after the preliminary screening of step S43, select a pixel point 2-4 pixels away from its end point along the direction of the edge line segment, detect the gray value of the pixel point, if the gray value of the pixel point If the value is in the range of 128-255, it is judged that the endpoint is qualified, and the edge line segment conforms to the feature of the frame of the sign; If unqualified, delete the edge segment;

S45: Step S44 is repeated until all edge line segments have been measured, and all edge line segments conforming to the characteristics of the sign frame are obtained.

Preferably according to the present invention, in step S5, according to the edge line segment after step S4 extension, screening, detect quadrilateral corner point, specifically include:

S51: From all the edge line segments that meet the characteristics of the frame of the sign obtained in step S45, arbitrarily select an edge line segment, set as line segment cd, as the first side of the quadrilateral, and select from the remaining edge line segments that meet the feature of the frame of the sign The line segment ef intersecting the line segment cd, the line segments cd and ef satisfy: θ _cd ! ≈θ _ef , min{ce, cf, de, df}≤Δ and the line segment cd and line segment ef satisfy the characteristics of the adjacent sides of the quadrilateral, and the intersection point of the line segment cd and ef is a corner point of the quadrilateral;

S52: Using the method described in step S51, obtain the corner point sequence of the quadrilateral;

S53: Go through all the edge line segments obtained in step S45 that meet the characteristics of the frame of the logo, and obtain all the corner point sequences of the quadrilaterals.

Preferably according to the present invention, according to the corner points of the quadrilateral, constructing the quadrilateral specifically includes:

S54: Construct a quadrilateral according to the number of corner points contained in all the corner point sequences obtained in step S53: if the number of corner points in the corner point sequence is 4, connect the 4 corner points to directly construct a quadrilateral;

If the number of corner points in the corner point sequence is 3, extend the 2 edge line segments that do not constitute the 4th corner point, and obtain the 4th corner point at the intersection point, connect the 4 corner points to construct a quadrilateral;

If the number of corner points in the corner point sequence is 2, extend the 2 edge line segments with only 1 corner point, if it intersects with the third edge line segment, the intersection point is the corner point, connecting 4 corner points , to construct a quadrilateral; otherwise, a quadrilateral cannot be constructed;

If the number of corner points in the corner point sequence is 1, a quadrilateral cannot be constructed;

S55: Traverse all corner point sequences to obtain all quadrilaterals, that is, detect all artificial signs.

The beneficial effects of the present invention are:

1. The present invention preprocesses the frame image before operation, performs coarse grid sampling, and performs edge detection for each grid area, which greatly reduces the program operation time, improves the detection speed, and has good real-time performance and meets the real-time detection requirements.

2. The present invention adopts an edge-based detection method. Firstly, the line segment test is performed, and then the quadrilateral frame of the sign is reconstructed according to the line segment obtained from the line segment test, which has good robustness to illumination changes and occlusion conditions.

Description of drawings

Fig. 1 is a flowchart of the present invention;

Figure 2 is a schematic diagram of the RANSAC algorithm;

The RANSAC algorithm is a commonly used line segment fitting method. In Fig. 2, the black and white points are the detected edge pixels, c1 and d1 are the edge pixels selected arbitrarily, as the end points of the hypothetical edge line segment, satisfying that c1, d1 and c1, d1 are connected in the same direction, Points that are close enough to the hypothetical edge line segment c1d1 and whose direction is consistent with the direction of the line segment c1d1 are regarded as support points of the line segment c1d1. In Figure 2, the line segment c1d1 has 12 support points, and the line segment e1f1 has 3 support points. Repeat the above steps to obtain the line segment with the most support points, that is, the line segment is identified as an existing line segment.

Fig. 3 is a schematic diagram of extending and screening the fused edge segments.

detailed description

The present invention will be further limited below in conjunction with the accompanying drawings and embodiments, but not limited thereto.

Example

A method for detecting artificial signs applied to augmented reality, the specific steps comprising:

S1: Collect the frame image, roughly sample the frame image, use oblique grid scanning, and detect the edge pixels of the frame image;

S2: Based on the RANSAC algorithm, according to the edge pixels obtained in step S1, the edge line segment in the frame image is detected;

S3: Fusing the edge line segments detected in step S2;

S4: Extending and screening the edge line segments fused in step S3;

S5: Detect the corner points of the quadrilateral according to the extended and screened edge segments in step S4, and construct a quadrilateral according to the corner points of the quadrilateral. As shown in Figure 1.

The step S1 specifically includes:

S11: Divide the frame image into several regions, each region contains 40×40 pixels, divide the frame image into several regions, speed up the detection, and improve the real-time performance.

S12: For each area obtained in step S11, use oblique 45° and 135° scanning lines to perform grid scanning, and the distance across the sides of each grid is 5 pixels;

The main reason for adopting oblique scanning lines for grid scanning is: generally, there are few pictures where signs are placed upright, and signs are generally placed obliquely in pictures, so oblique scanning lines are used for grid scanning.

S13: each scanning line in step S12 is convolved with the Gaussian first-order derivation, and the gradient intensity component of the pixel along the direction of each scanning line is calculated;

S14: According to the gradient strength component of the pixel point obtained in step S13 along the direction of each scanning line, calculate the gradient strength value of the pixel point, the pixel point corresponding to the local extremum in the gradient strength value is the local extremum point of the edge pixel gradient strength, Extract the edge pixel, the threshold of extracting the edge pixel is 30/256 pixel value, and calculate the direction θ=tan ^-1 (g _y /g _x ) of the edge pixel according to the gradient intensity component, g _y is the gradient intensity of the Y component , g _x is the gradient strength of the X component.

S2: Based on the RANSAC algorithm, according to the edge pixels obtained in step S1, the edge line segment in the frame image is detected;

The step S2 specifically includes:

S21: In all the regions of 40×40 pixels that have been divided in step S11, randomly select two edge pixels, and if the direction of the two edge pixels is consistent with the direction of their connection, it is assumed that there is an edge line segment between them;

S22: Calculate the number of edge pixels supporting the assumed edge line segment described in step S21, if the edge pixel satisfies: γ∈(0.1,0.25), and the direction θ1 of the edge pixel is consistent with the direction of the edge line segment, then it is considered that the edge pixel supports the Assumed edge line segment, Count plus 1; wherein, γ is the distance from the edge pixel to the assumed edge line segment, θ1 is the direction of the edge pixel, and Count is the number of edge pixels supporting the assumed edge line segment;

S23: The assumed edge line segments whose Count reaches 12 are considered to exist, and the edge pixels supporting them are removed from the image;

S24: Repeat steps S21 to S23 until most edge pixels in the image are removed and all edge segments are found.

In Figure 2, the black and white points are the detected edge pixels, and the c1 and d1 points are arbitrarily selected edge pixel points as the end points of the hypothetical edge line segment, satisfying that the directions of c1 and d1 points and the direction of the connecting line of c1 and d1 points are consistent , the point that is close enough to the hypothetical edge segment c1d1 and whose direction is consistent with the direction of the line connecting points c1 and d1 is regarded as the support point of the hypothetical edge segment c1d1. In Fig. 2, it is assumed that the support points of the edge line segment c1d1 are 12, and the support points of the edge line segment c1d1 are assumed to be 3, the edge line segment c1d1 is considered to exist, and the edge line segment c1d1 is assumed to be excluded.

The step S3 specifically includes:

Select an edge line segment obtained through step S2, call it line segment a, choose another line segment arbitrarily from the remaining edge line segments, call it line segment b, and make a fusion judgment on line segment a and b, that is: if line segment a and b satisfy: | θ _a −θ _b |∈Δ _θ , And Lab ∈ Δ _l , then merge the line segments a and _b to get a new line segment; otherwise, continue to select a line segment from the remaining edge line segments, and continue to fuse with line segment a until line segment a and all edge line segments except line segment a are completed Fusion judgment; θ _a and θ _b are the direction of line segments a and b, Δ _θ is the threshold value of the direction error of the line segment to be fused, θ _ab and Lab are the direction and length of line _ab connecting line segment a and b, and Δ _l is line segment a , The threshold of the allowable length of b connection ab;

S32: Step S31 is repeated until all edge line segments obtained in step S2 are fused.

The step S4 specifically includes:

S41: According to the edge line segment obtained in step S3, arbitrarily select an edge line segment and select an end point thereof, and judge whether the direction along the edge line segment is consistent with the direction of the pixel adjacent to the end point, if they are consistent, then the pixel point Add it to the edge line segment, and continue to check the next pixel adjacent to the pixel point until the direction of a pixel point is inconsistent with the direction of the edge line segment, then the pixel point is updated as a new endpoint of the edge line segment;

In Fig. 3, pixel point 1 is the pixel point of the selected edge line segment, pixel point 2 is the pixel point added to the edge line segment, and pixel point 3 is the new endpoint;

S42: Execute step S41 on the two endpoints of all the edge segments obtained in step S3, to obtain the extended edge segments and their new endpoints;

S43: Preliminarily screen the extended edge line segment obtained in step S42, and delete the edge line segment whose length value is less than 20 pixels; the edge line segment whose length value is too small cannot be the logo frame, even if it is, it means that the logo frame distortion is too large Or the scale of the scene is too large to be meaningful for detection, so it is deleted;

S44: Carry out a line segment test on the edge line segment after the preliminary screening in step S43, select a pixel point 3 pixels away from its end point along the direction of the edge line segment, and detect the gray value of the pixel point, if the gray value of the pixel point is in 250, it is judged that the endpoint is qualified, and the edge line segment conforms to the feature of the sign frame; otherwise, it is judged that the end point is unqualified, and the edge line segment does not meet the feature of the sign frame; Then delete the edge segment;

In Figure 3, pixel 4 is a test pixel, and it can be known that if the detected line segment conforms to the feature of the logo frame, pixel 4 should be a relatively bright point.

S45: Step S44 is repeated until all edge line segments have been measured, and all edge line segments conforming to the characteristics of the sign frame are obtained.

In step S5, according to the edge line segment after step S4 extension, screening, detect quadrilateral corner point, specifically include:

S51: From all the edge line segments that meet the characteristics of the frame of the sign obtained in step S45, arbitrarily select an edge line segment, set as line segment cd, as the first side of the quadrilateral, and select from the remaining edge line segments that meet the feature of the frame of the sign The line segment ef intersecting the line segment cd, the line segments cd and ef satisfy: θ _cd ! ≈θ _et , min{ce, cf, de, df}≤Δ and the line segment cd and line segment ef satisfy the characteristics of the adjacent sides of the quadrilateral, and the intersection point of the line segment cd and ef is a corner point of the quadrilateral;

S52: Using the method described in step S51, obtain the corner point sequence of the quadrilateral;

S53: Go through all the edge line segments obtained in step S45 that meet the characteristics of the frame of the logo, and obtain all the corner point sequences of the quadrilaterals.

Construct a quadrilateral according to the corner points of the quadrilateral, including:

S54: Construct a quadrilateral according to the number of corner points contained in all the corner point sequences obtained in step S53: if the number of corner points in the corner point sequence is 4, connect the 4 corner points to directly construct a quadrilateral;

If the number of corner points in the corner point sequence is 3, extend the 2 edge line segments that do not constitute the 4th corner point, and obtain the 4th corner point at the intersection point, connect the 4 corner points to construct a quadrilateral;

If the number of corner points in the corner point sequence is 2, extend the 2 edge line segments with only 1 corner point, if it intersects with the third edge line segment, the intersection point is the corner point, connecting 4 corner points , to construct a quadrilateral; otherwise, a quadrilateral cannot be constructed;

If the number of corner points in the corner point sequence is 1, a quadrilateral cannot be constructed;

S55: Traverse all corner point sequences to obtain all quadrilaterals, that is, detect all artificial signs.

Claims (8)

1. A kind of artificial mark detection method that is applied to augmented reality is characterized in that, concrete steps comprise: S1: Collect the frame image, roughly sample the frame image, use oblique grid scanning, and detect the edge pixels of the frame image; S2: Based on the RANSAC algorithm, according to the edge pixels obtained in step S1, the edge line segment in the frame image is detected; S3: Fusing the edge line segments detected in step S2; S4: Extending and screening the edge line segments fused in step S3; S5: Detect the corner points of the quadrilateral according to the extended and screened edge segments in step S4, and construct a quadrilateral according to the corner points of the quadrilateral. 2. A method for detecting artificial signs applied to augmented reality according to claim 1, wherein said step S1 specifically comprises: S11: divide the frame image into several regions, each region contains m×m pixels, m∈(20,60); S12: For each area obtained in step S11, use oblique x° and 180°-x° scanning lines to perform grid scanning, x∈(22.5,67.5), and the distance across each grid is y pixels , y∈(3,9); S13: each scanning line in step S12 is convolved with the Gaussian first-order derivation, and the gradient intensity component of the pixel along the direction of each scanning line is calculated; S14: According to the gradient strength component of the pixel point obtained in step S13 along the direction of each scanning line, calculate the gradient strength value of the pixel point, the pixel point corresponding to the local extremum in the gradient strength value is the local extremum point of the edge pixel gradient strength, The edge pixel is extracted, and the direction of the edge pixel is calculated according to the gradient intensity component. 3. A method for detecting artificial signs applied in augmented reality according to claim 2, characterized in that m=40; x=45, y=5. 4. A kind of artificial mark detection method applied to augmented reality according to claim 1, is characterized in that, described step S3, specifically comprises: S31: Select an edge line segment obtained by step S2, and name it line segment a, randomly select another line segment from the remaining edge line segments, and call it line segment b, and perform fusion judgment on line segments a and b, that is, if line segments a and b satisfy : |θ _a -θ _b |∈Δ _θ , And Lab ∈ Δ _l , then merge the line segments a and _b to get a new line segment; otherwise, continue to select a line segment from the remaining edge line segments, and continue to fuse with line segment a until line segment a and all edge line segments except line segment a are completed Fusion judgment; θ _a and θ _b are the direction of line segments a and b, Δ _θ is the threshold value of the direction error of the line segment to be fused, θ _ab and Lab are the direction and length of line _ab connecting line segment a and b, and Δ _l is line segment a , The threshold of the allowable length of b connection ab; S32: Step S31 is repeated until all edge line segments obtained in step S2 are fused. 5. A kind of artificial mark detection method applied to augmented reality according to claim 1, is characterized in that, described step S4, specifically comprises: S41: According to the edge line segment obtained in step S3, arbitrarily select an edge line segment and select an end point thereof, and judge whether the direction along the edge line segment is consistent with the direction of the pixel adjacent to the end point, if they are consistent, then the pixel point Add it to the edge line segment, and continue to check the next pixel adjacent to the pixel point until the direction of a pixel point is inconsistent with the direction of the edge line segment, then the pixel point is updated as a new endpoint of the edge line segment; S42: Execute step S41 on the two endpoints of all the edge segments obtained in step S3, to obtain the extended edge segments and their new endpoints; S43: Preliminarily screen the extended edge segments obtained in step S42, and delete the edge segments whose length is less than n pixels, n∈(15, 25); S44: Carry out a line segment test to the edge line segment after the preliminary screening of step S43, select a pixel point 2-4 pixels away from its end point along the direction of the edge line segment, detect the gray value of the pixel point, if the gray value of the pixel point If the value is in the range of 128-255, it is judged that the endpoint is qualified, and the edge line segment conforms to the feature of the frame of the sign; If unqualified, delete the edge segment; S45: Step S44 is repeated until all edge line segments have been measured, and all edge line segments conforming to the characteristics of the sign frame are obtained. 6. A method for detecting artificial signs applied in augmented reality according to claim 5, characterized in that n=20. 7. A kind of artificial mark detection method that is applied to augmented reality according to claim 5, it is characterized in that, in step S5, according to the edge segment after step S4 extension, screening, detect quadrilateral corner point, specifically comprise: S51: From all the edge line segments that meet the characteristics of the frame of the sign obtained in step S45, arbitrarily select an edge line segment, set as line segment cd, as the first side of the quadrilateral, and select from the remaining edge line segments that meet the feature of the frame of the sign The line segment ef intersecting the line segment cd, the line segments cd and ef satisfy: θ _cd ! ≈θ _ef , min{ce, cf, de, df}≤Δ and the line segment cd and line segment ef satisfy the characteristics of the adjacent sides of the quadrilateral, and the intersection point of the line segment cd and ef is a corner point of the quadrilateral; S52: Using the method described in step S51, obtain the corner point sequence of the quadrilateral; S53: Go through all the edge line segments obtained in step S45 that meet the characteristics of the frame of the logo, and obtain all the corner point sequences of the quadrilaterals. 8. A kind of artificial sign detection method that is applied to augmented reality according to claim 7, is characterized in that, according to quadrilateral corner point, constructs quadrilateral, specifically comprises: S54: Construct a quadrilateral according to the number of corner points contained in all the corner point sequences obtained in step S53: if the number of corner points in the corner point sequence is 4, connect the 4 corner points to directly construct a quadrilateral; If the number of corner points in the corner point sequence is 3, extend the 2 edge line segments that do not constitute the 4th corner point, and obtain the 4th corner point at the intersection point, connect the 4 corner points to construct a quadrilateral; If the number of corner points in the corner point sequence is 2, extend the 2 edge line segments with only 1 corner point, if it intersects with the third edge line segment, the intersection point is the corner point, connecting 4 corner points , to construct a quadrilateral; otherwise, a quadrilateral cannot be constructed; If the number of corner points in the corner point sequence is 1, a quadrilateral cannot be constructed; S55: Traverse all corner point sequences to obtain all quadrilaterals, that is, detect all artificial signs.