CN111179346B

CN111179346B - Feature extraction method and device of label image, positioning method and positioning equipment

Info

Publication number: CN111179346B
Application number: CN201911389534.7A
Authority: CN
Inventors: 宋堃; 肖勇; 张学敏; 范超; 施垒
Original assignee: Guangdong Starcart Technology Co ltd
Current assignee: Guangdong Starcart Technology Co ltd
Priority date: 2019-12-26
Filing date: 2019-12-26
Publication date: 2023-06-06
Anticipated expiration: 2039-12-26
Also published as: CN111179346A

Abstract

The invention relates to the field of visual positioning, and discloses a feature extraction method of a label image, a related method and equipment, wherein the feature extraction method mainly comprises the following steps: sub-pixel line segment extraction operation is carried out on the label image, and a second line segment is obtained; obtaining the position information of the sub-pixel intersection point; generating data of a linear vector corresponding to each second line segment and determining an adjustment range; adjusting the positions of the linear vectors until the relation between each linear vector and the pixel gradient vector in the corresponding adjusting range meets a second condition; and extracting the position information of the first endpoint at the moment as the characteristic of the label image. Some technical effects of the present disclosure are: the relationship between the linear vector and the pixel gradient vector and the prior information are utilized, so that the final position information of the intersection point of the first line segment is optimized in a global angle, the feature of the label image can be extracted more accurately, and the label image is ensured to have relatively more features for output.

Description

Feature extraction method and device of label image, positioning method and positioning equipment

Technical Field

The disclosure relates to the technical field of visual positioning, and in particular relates to a feature extraction method and device of a label image, and a related positioning method and positioning equipment.

Background

In the field of visual positioning, one of the mainstream positioning methods is: the method comprises the steps of placing a tag (or a visual tag, a visual mark and the like) in an indoor or outdoor scene, shooting the scene through a camera to obtain a scene image, processing the scene image (for example, performing feature recognition through a deep learning method), identifying or obtaining a tag image in the scene image, calculating the relative position of the camera to the corresponding tag according to the tag image, and obtaining the position information of the camera or a positioning target associated with the camera (namely, the positioning target has a known relative position relation with the camera) according to the actual position of the tag.

However, in the implementation, such a situation often occurs: the quality of the obtained label image is affected by interference (such as the label is partially damaged and the label is partially blocked in the shooting process), so that the feature completeness of the label image is low, that is, the label image cannot sufficiently reflect the features of the label, for example, a blocking area is arranged in the label image, intersecting points of some line segments of the label are blocked, for example, blank color blocks appear at corresponding positions in the label image due to the fact that the label is damaged, and the like. For this case, it is common practice to acquire the coordinate positions only for intersections that are visible in the tag image.

In fact, the line segments (called first line segments for distinguishing understanding) on the acquired label image include intersection points between the line segments and have a certain pixel width, and currently, the mainstream technology generally adopts a method of sub-pixel edge extraction (also can be understood as sub-pixel line segment extraction, sub-pixel edge detection, and the like, that is, a line segment of sub-pixel level is obtained by extraction processing from a line segment of pixel level, for example, an edge of pixel level is obtained by detection, and then a fitting method such as a least square method is used for processing to obtain a line segment of sub-pixel level), so as to process the label image, and obtain a sub-pixel line segment (called second line segment for distinguishing understanding) corresponding to each first line segment. Since the first line segments are of a certain pixel width, when a plurality of first line segments intersect at a point, it means that they have only one intersection point there. However, since the relative positions between the first line segments may be distorted due to the influence of shaking, reflection, etc. during the capturing process, and the second line segments are obtained by performing sub-pixel extraction operation on the first line segments, certain errors may occur in the extraction process, so that it is difficult to occur that a plurality of second line segments intersect at a point. As shown in fig. 2, fig. 2 (a) illustrates a case where three first line segments intersect, and fig. 2 (b) illustrates a case where three second line segments obtained after the subpixel segment extraction process is performed on fig. 2 (a). In this regard, in the conventional processing method, the positions of the intersections of the first line segments are generally indicated by directly calculating the center points of several intersections (referred to as sub-pixel intersections for the sake of distinction) intersecting the second line segments by directly calculating the center points.

The disadvantage of such a processing method is that, for the obtained image with distortion, the position of the second line segment obtained by performing sub-pixel line segment extraction has a large uncertainty, and the accuracy is low by reflecting the position of the intersection point of the first line segment obtained by directly performing the center point calculation based on the intersection point of the sub-pixels; in addition, when the feature completeness of the label image is low, there may be few visible intersections for feature extraction, which also affects the subsequent application of the feature information (such as location resolution).

Disclosure of Invention

To solve at least one of the foregoing technical problems, the present disclosure proposes, in a first aspect, a feature extraction method of a label image, where the label image includes intersecting first line segments, including the steps of: judging whether the first condition for judging the feature completeness is met or not according to the prior information; if yes, sub-pixel line segment extraction operation is carried out on the label image, and a second line segment used for representing the position of the first line segment is obtained; obtaining position information of sub-pixel intersection points according to the position information of the second line segment, wherein the sub-pixel intersection points comprise visible intersection points and non-display intersection points; generating data of a linear vector corresponding to each second line segment and determining an adjustment range according to the position information of the second line segment and the sub-pixel intersection point, wherein the linear vector is provided with at least one intersected first endpoint; adjusting the positions of the linear vectors in the adjustment range until the relation between each linear vector and the pixel gradient vector in the corresponding adjustment range meets a second condition; and extracting the position information of the first endpoint at the moment as the characteristic of the label image.

Preferably, the first condition includes: the number of the first line segment intersection points meets a third condition or the integrity of the first line segment meets a fourth condition.

Preferably, the second condition includes that the sum of dot products of each of the straight line vectors and each of the pixel gradient vectors in the corresponding adjustment range reaches a minimum value.

Preferably, the adjustment range includes a first range determined from the prior information and the positional information of the sub-pixel intersection; adjusting the position of the line vector includes: and adjusting the position of the first endpoint in the first range.

Preferably, the boundary of the first range includes a position where the sub-pixel intersection points are located.

Preferably, the adjustment range further includes a second range determined according to the position information of the second line segment; adjusting the position of the line vector further comprises: when one straight line vector has a second end point which is not intersected with other straight line vectors, the position of the second end point is adjusted in the second range.

In a second aspect, the present disclosure proposes a feature extraction device of a label image for extracting features of a label image having intersecting first line segments, comprising: (1) The feature completeness judging module is used for judging whether the first condition for judging the feature completeness is met or not according to the priori information; (2) The sub-pixel line segment extraction module is used for executing sub-pixel line segment extraction operation on the label image when the first condition is met, and obtaining a second line segment used for representing the position of the first line segment; the sub-pixel intersection point comprises a visible intersection point and a non-displayed intersection point; (3) The adjustment information generation module is used for generating data of a linear vector corresponding to each second line segment and determining an adjustment range according to the position information of the intersection point of the second line segments and the sub-pixels, and the linear vector is provided with at least one intersected first endpoint; (4) The position adjustment module is used for adjusting the positions of the linear vectors in the adjustment range until the relation between each linear vector and the pixel gradient vector in the corresponding adjustment range meets a second condition; (5) And the feature extraction module is used for extracting the position information of the first endpoint when the second condition is met as the feature of the tag image.

In a third aspect, a method for positioning a feature based on a label image is provided, including the following steps: acquiring a scene image shot by a camera; processing the scene image to obtain a label image; executing the feature extraction method to obtain the features of the tag image; and obtaining relative position information according to the characteristics of the label image.

In a fourth aspect, a positioning device is provided, including a camera and a processor, where the camera is configured to photograph a scene to obtain a scene image; the processor is used for processing the scene image to obtain a label image; the method is also used for executing the step of the feature extraction method to obtain the features of the tag image; and obtaining relative position information according to the characteristics of the label image.

In a fifth aspect, a computer-readable storage medium is proposed, on which a computer program is stored, which computer program, when being executed by a processor, implements the feature extraction method steps.

Some technical effects of the present disclosure are: the relationship between the linear vector and the pixel gradient vector and the prior information are utilized, so that the final position information of the intersection point of the first line segment is optimized in a global angle, the feature of the label image can be extracted more accurately, and the label image is ensured to have relatively more features for output.

Drawings

For a better understanding of the technical solutions of the present disclosure, reference may be made to the following drawings for aiding in the description of the prior art or embodiments. The drawings will selectively illustrate products or methods involved in the prior art or some embodiments of the present disclosure. The basic information of these figures is as follows:

FIG. 1 is a schematic illustration of a label before and after being obscured in one instance;

FIG. 2 is a schematic diagram of an original line segment in a label image and a subpixel segment obtained by processing the original line segment by a subpixel segment extraction method;

FIG. 3 is a schematic diagram of a second line segment and a corresponding straight line vector according to an embodiment;

FIG. 4 is a schematic diagram of a first range in one embodiment;

FIG. 5 is a schematic diagram of a second range in one embodiment;

FIG. 6 is a schematic illustration of a different first range in some embodiments;

FIG. 7 is a schematic diagram of a straight line vector and a pixel gradient vector in one embodiment.

In the above figures, the reference numerals and the corresponding technical features are as follows:

1-first range, 2-second range, 3-pixels of the second range, 4-straight line vectors, 5-pixel gradient vectors.

Detailed Description

Further technical means or technical effects to which the present disclosure relates will be described below, and it is apparent that examples (or embodiments) provided are only some embodiments, but not all, which are intended to be covered by the present disclosure. All other embodiments that can be made by those skilled in the art without the exercise of inventive faculty, based on the embodiments in this disclosure and the explicit or implicit presentation of the drawings, are intended to be within the scope of this disclosure.

The present disclosure proposes in a first aspect a feature extraction method of a label image, the label image including intersecting first line segments, comprising the steps of: judging whether the first condition for judging the feature completeness is met or not according to the prior information; if yes, sub-pixel line segment extraction operation is carried out on the label image, and a second line segment used for representing the position of the first line segment is obtained; obtaining position information of sub-pixel intersection points according to the position information of the second line segment, wherein the sub-pixel intersection points comprise visible intersection points and non-display intersection points; generating data of a linear vector corresponding to each second line segment and determining an adjustment range according to the position information of the second line segment and the sub-pixel intersection point, wherein the linear vector is provided with at least one intersected first endpoint; adjusting the positions of the linear vectors in the adjustment range until the relation between each linear vector and the pixel gradient vector in the corresponding adjustment range meets a second condition; and extracting the position information of the first endpoint at the moment as the characteristic of the label image.

It should be noted that the feature extraction method is applicable to a label image in which there are at least two intersecting first line segments. The label image refers to an image containing a part or a complete label pattern, and can be an image obtained by directly shooting a scene including a label on site or a new image obtained by processing the scene image obtained by shooting.

The first line segment refers to an original line segment in the label image, and each line segment has a certain pixel width, namely, the line width is a plurality of pixels. In general, there are a plurality of pixels between a line segment in an image and the background, and the gray scale of the pixels is graded.

Thus, the intersection of the first line segment can be understood as a collection of several pixels where the first line segment intersects, and obviously, if the position of the intersection of the first line segment is to be expressed more accurately, it is necessary to find a coordinate point of one subpixel and consider it as the position of the intersection of the first line segment.

In a real occasion, the acquired scene image can be preprocessed, and the image suitable for the determination method is screened out and then a specific processing mode is executed. Before the feature extraction method is executed, a certain process of filtering, denoising, gray level adjustment and the like can be performed on the label image, and the label image can be processed by referring to the prior art, so that detailed description is omitted herein according to specific requirements.

In addition, the label image may be either black and white or colored, which means that in some cases, there may be a different color difference between each first line segment. Even the first line segment may have a different width.

With respect to a priori information. The a priori information records characteristics of the tag, including, for example, one or more of the number of segments of the tag itself, the relative positions of the segments, the IDs of the segments (for distinguishing between the different segments, e.g., each segment of each tag has a different number than the other segments), the relative positions of the intersections of the segments, and the IDs of the tags. Most importantly, the prior information records the related information that there are intersections between the line segments. Thus, when feature recognition is performed on a tag image, it is possible to know to which tag the tag image corresponds, and whether the tag image reflects the full view of the tag. Identifying the first line segments in the label image, determining which line segments they correspond to, such a method is mature in the prior art, and is not developed here; similarly, finding the intersection point in the label image that corresponds to the a priori information may also employ well-established techniques.

With respect to feature completeness. Feature completeness is a term used to describe whether a label image reflects the original complete appearance of the label or how much of the original appearance is reflected. In general, feature completeness is relatively low when a tag is damaged or when a tag is occluded when a scene photograph is taken; in general, the larger the area of the blocked region in the label image, the lower the feature completeness; or the more the intersection of the line segments of the tag is occluded, the lower the feature completeness. The specific definition of the feature completeness is not particularly limited, and depends on the requirements of practical implementation. It should be noted that the method for determining the completeness of a feature is various as long as it does not depart from the foregoing principles. For example, the prior information may be combined, and whether the feature completeness meets the first condition may be determined according to a ratio of the area of the tag that is blocked to the area that is not blocked; for example, the prior information may be combined, and whether the first condition is met may be determined according to the number of line segment intersections where the tag is blocked; for example, the prior information may be combined, and whether the first condition is met may be determined according to the number of line segments that the tag is blocked; for example, the prior information can be combined, and whether the first condition is met or not is judged according to the total length of the line segment blocked by the label; for example, the prior information may be combined, and whether the first condition is met may be determined according to whether the position of the area where the tag is blocked is a critical position.

Accordingly, the first condition may be varied, and may be a number, an area, a position, a length, or the like. Features of the label image are often used for visual localization, for example, position information of an intersection Point of a first line segment in the label image may be extracted, and then a PnP (Perspective-n-Point) algorithm is used for visual localization. In general, the more features, the more assistance in improving the accuracy of positioning. Thus, in a relatively stringent application, when at least one intersection of the tag is occluded or at least one line segment of the tag is occluded, it may be considered "meeting the first condition for judging feature completeness".

In some embodiments, the first condition comprises: the number of the first line segment intersection points meets a third condition or the integrity of the first line segment meets a fourth condition. The third and fourth conditions depend on the number of features of the tag itself and the requirements of the implementation. For example, when the third condition is at most 3, the first condition may be: the number of the first line segment intersections is less than or equal to 3. For example, when the fourth condition is that the number of the first line segments is at most 5, the first condition may be: the number of the first line segments is less than or equal to 5. Of course, instead of number, the integrity can be evaluated by length-to-length ratio. For example, the fourth condition may be: the sum of the lengths of the first line segments is at most 40% of the sum of the total lengths of the line segments corresponding to the labels.

It should be noted that, in many cases, the feature completeness is low, or the first condition is met, which may not be caused by the shielding of the tag, for example, the tag is damaged, the shooting process cannot obtain a complete tag image due to reflection, or the like, but the principle of the feature extraction method mentioned in the document may be adopted to solve the related problem for any reason. In order to more concisely describe the scheme and effect related to the disclosure, the reason for the low feature completeness is described by uniformly adopting 'shielding' in a most probable way, but the feature extraction method is not indicated to be only applicable to the condition that the tag is shielded.

The second line segment refers to a concept of a first line segment, and refers to a sub-pixel line segment correspondingly obtained after sub-pixel line segment extraction processing is performed on an image. It is common practice to use a second line segment representing a first line segment, which in general can be understood as a more detailed representation of the position of the latter, since the accuracy of the coordinates of the former is sub-pixel level and the accuracy of the coordinates of the latter is pixel level. The characteristic information of the second line segment includes information indicating the position of the second line segment, and generally includes coordinates of an end point of the second line segment or an equation of the second line segment.

In general, techniques for sub-pixel line segment extraction (or sub-pixel edge detection, extraction, i.e., processing a line segment at a pixel level to obtain a line segment at a sub-pixel level) are well established, and many techniques are available to those skilled in the art, and may be implemented using a number of existing techniques (e.g., identifying an edge of a first line segment, then determining a centerline of the first line segment from the edge, and extracting the centerline to represent a position of the first line segment), which are not expanded herein.

Regarding the visual intersection point, the intersection point is not displayed. From the foregoing, it is understood that the first line segment is the line segment that points out the label image, and that the first line segment corresponds to the line segment of the objectively existing label. The visual intersection point is an intersection point of one or more sub-pixels corresponding to the intersection point of the first line segment appearing on the label image. The non-displayed intersection point is the intersection point of one or more sub-pixels corresponding to the intersection point where the index tag is blocked. For example, as shown in fig. 2, one intersection of the first line segment of fig. 2 (a) corresponds to the intersection of three sub-pixels in fig. 2 (b). The non-displayed intersection is not a real intersection, but a virtual intersection, which is located where the extensions of the second line segments meet. As shown in fig. 1, fig. 1 (a) shows the real state of the tag, and a plurality of line segments intersect to form five intersecting points A, B, C, D, E; fig. 1 (b) shows that only three intersections of points a, b, and e appear on the label image due to occlusion. The dashed line in fig. 1 (b) represents a first line segment that is not shown due to occlusion. At this time, it can be known from the prior information that there should be an intersection point near the point d and an intersection point near the point c, so that the second line segment corresponding to the first line segment can extend toward the direction of the point d and the point c, and a corresponding sub-pixel intersection point can be obtained, that is, the intersection point is not displayed; and for the visible points a, b and e, the intersection points of the corresponding second line segments appearing nearby are regarded as visible intersection points. For fig. 1 (b) and the like, due to occlusion, the visible feature points (generally referred to as intersecting points) are only three points a, b and e, and if the positions of the points d and c can be calculated, the number of feature points will be more. Since the point d and the point c are intersection points a priori, it is helpful to more accurately determine the angle of the second line segment to which the tag line segment AD, AC, BD, BC corresponds.

With respect to the straight line vector. In essence, the feature extraction method of the present disclosure is designed such that the position of the second line segment is desirably used to represent the position of the first line segment (the position of the center line where the second line segment is located), and the position of the straight line where the second line segment is located is then used to represent the position of the straight line vector. Generally, when determining the position of a subpixel of a first intersection around the intersection, a straight line vector corresponds to a second line segment. However, when there are a plurality of first intersections on a first line segment, this is equivalent to determining the positions of the intersections of the plurality of line segments, and in implementation, one or more straight line vectors corresponding to the first line segment may be generated using the principles described herein. In fact, the present disclosure contemplates that by matching the linear vectors to the gray level variations of the pixels surrounding the linear vectors, the final position of the linear vectors is determined to be as much as possible relatively perpendicular to the surrounding pixel gradient vectors, which relatively reasonably reflects the position of the first line segment, and more importantly, since the associated linear vectors intersect (meaning having common endpoints), the position of the associated linear vectors is adaptively adjusted to the global pixel gradient vector variations when determining the sub-pixel intersection position of a first line segment.

"related line vector" refers to a line vector used to solve for the location of a certain first line segment intersection. As shown in fig. 3 (a), a second line segment a ₁ A ₂ 、B ₁ B ₂ 、C ₁ C ₂ The a priori information is that the three first line segments intersect at a point corresponding to the three first line segments. After the sub-pixel line segments are extracted from the three first line segments, the three second line segments have three intersection points (point O ₁ 、O ₂ 、O ₃ ). This isThe time-dependent straight-line vectors are the arrowed, thick-line segment shapes in fig. 3 (b), each straight-line vector corresponding to a second line segment, the straight-line vectors having a common endpoint, which is determined by a priori information, indicating that the second line segment should also have only one intersection. The reason why the three second line segments shown in fig. 3 have three intersection points instead of one intersection point is that some errors occur mainly in the image processing process and the shooting process, so the position of the second line segment needs to be adjusted, and the method adopted in the present disclosure adjusts the position of the linear vector, and the position of the common endpoint of the adjusted linear vector can be regarded as the position of the final intersection point of the second line segment, and also can be regarded as the position of the (sub-pixel) intersection point of the first line segment.

Adjusting the line vector means that the position or even the direction of the line vector changes. In some embodiments, the positions of the two endpoints of the linear vector may be adjusted along the gradient direction of the pixel where the two endpoints of the linear vector are located. In some embodiments, a traversal method may also be used to adjust the positions of the two endpoints of the line vector. Of course, other methods (essentially moving the two end points over a range) may be used, without expanding, in any case, as long as the line vector is adjusted in some way within the adjustment range (in some embodiments, including the first range and the second range) so that the second condition is satisfied.

In some embodiments, to reduce the amount of computation, the related linear vectors may be normalized to be uniform in length. In other embodiments, the linear vectors may have different lengths without normalization.

With respect to pixel gradient vectors. A pixel gradient vector is used to represent the magnitude and direction of the change in gray scale around a pixel. Within the adjustment range, there are a plurality of pixels, and with each pixel as the center, there is a pixel gradient vector corresponding to this pixel. For the calculation of the pixel gradient vector, a classical image gradient algorithm can be used for reference, the classical image gradient algorithm considers the gray level change in a certain neighborhood of each pixel of an image, a gradient operator is arranged in a certain neighborhood of the pixel in an original image by utilizing a first-order or second-order derivative change rule of edge approach, and normally, a small-area template is used for convolution to calculate, and the algorithm comprises a Sobel operator, a Robinson operator, a Laplace operator and the like. Generally, in the prior art, one representation of a pixel gradient vector is shown in equation one:

the length of this gradient vector is shown in equation two:

The direction angle of this gradient vector is shown in formula three:

/>

in the first formula, the second formula and the third formula,

is a gradient vector of pixels (also called pixel gradient vector) having a length and a direction. G _x Is the gradient of the gradient vector in the x-axis direction, G _y Is the gradient of the gradient vector in the y-axis direction, the length of the gradient vector is +.>

The direction angle is +>

The matrix with the corner mark T represents the transposed matrix, ">

And +.>

First order partial derivatives for different directions; g (x, y) represents the image gradient, +.>

And->

Arctan represents the arctangent function for different second partial derivatives.

In some embodiments, the image blocks within the adjustment range may be filtered, and then the corresponding pixel gradient vectors are calculated; in other embodiments, the filtering process may not be performed, but rather the calculation may be performed directly.

In some embodiments, the adjustment range includes a first range determined from the prior information and the positional information of the sub-pixel intersection; adjusting the position of the line vector includes: and adjusting the position of the first endpoint in the first range.

In some embodiments, the boundary of the first range includes a location where the subpixel intersection is located.

In some embodiments, the adjustment range further includes a second range determined from the location information of the second line segment; adjusting the position of the line vector further comprises: when one straight line vector has a second end point which is not intersected with other straight line vectors, the position of the second end point is adjusted in the second range.

With respect to the first range. The main function of the method is to define a reasonable range for calculating/calculating the position of the sub-pixel level of the intersection point of the first line segment and the intersection point of the blocked label in the range, and meanwhile, the position is the common intersection point of more than two linear vectors. In some embodiments, when two first line segments intersect to obtain a first intersection point, two corresponding second line segments are included, where the "determining the adjustment range" includes determining the first range of intersection points (i.e. the intersection point of line segments) according to the position where the points where the second line segments intersect (including the non-displayed intersection point obtained by extending the second line segments) are located, including but not limited to the following manners:

(1) Taking the intersection point of the sub-pixels as a circle center, taking a set length (such as 3 to 7 pixel lengths, which can be specifically set according to actual requirements) as a radius, and taking a range covered by a circle formed by the circle center and the radius as a first range;

(2) Taking a range covered by a polygon formed by taking a sub-pixel intersection point as a center as a first range, wherein the side length of the polygon can be set according to actual requirements;

(3) The range covered by a regular or irregular polygon with a set relative position with the intersection point of the sub-pixels is taken as a first range, the side length of the polygon can be set according to actual requirements, and other closed shapes can be adopted to replace the polygon.

When the first line segment intersecting at one location is three or more, meaning that the corresponding second line segment is also three or more, and the sub-pixel intersection is also three or more, the first range may be designed according to the position of the sub-pixel intersection, which includes but is not limited to:

(1) Taking a certain sub-pixel intersection point as a reference point, and designing a corresponding closed interval (such as a circle, a polygon and the like) as a first range;

(2) With more than two sub-pixel intersection points as reference points, corresponding closed sections (such as circles, polygons and the like) are designed as a first range.

In order to make the reader more intuitively understand the first scope, some typical examples are shown in fig. 3 and fig. 4. FIG. 3 shows the case where three second line segments intersect, second line segment A ₁ A ₂ 、B ₁ B ₂ 、C ₁ C ₂ Intersecting to form a second intersection point of point O ₁ 、O ₂ 、O ₃ . As shown in fig. 4, the triangle O can be directly used ₁ O ₂ O ₃ As the first range 1. Of course, triangle O can also be used ₁ O ₂ O ₃ Centroid O of (2) ₀ The circle center is a coverage area formed by setting a radius as a circle, and the coverage area is a first range. In any case, such a first range of designs may be provided in one orA plurality of intersecting points (point O ₁ 、O ₂ 、O ₃ ) Formed as a basis.

As further shown in fig. 6, fig. 6 (a) shows a closed interval formed by using a plurality of sub-pixel intersections as vertices, and in some embodiments, the closed interval may be directly used as a search range; a circle may be formed outside the closed region, and the area covered by the circle may be used as the first range 1, and fig. 6 (b) shows the first range 1 with a larger range; fig. 6 (c) shows a first range 1 of a smaller range. The size of the first range may be subject to consideration of the amount of calculation or the accuracy required to perform the determination method.

With respect to the second range. The main function of the second range is to delineate a reasonable range for deriving/calculating the position of the line vector to be finally determined within this range. A second line segment corresponds to a second range. Generally, one end of a straight line vector will fall within a first range, while the other end will fall within a second range. It is not excluded that in some cases the second range overlaps the first range, even when viewed from the area, the first range may be a part of the second range.

The second range may be designed in a similar manner according to the design (or setting) principle of the first range described above. However, unlike the first range of designs which typically reference the sub-pixel intersection points, the second range of designs typically reference the location of the entire second line segment. As shown in FIG. 5, for the second line segment A ₁ A ₂ In other words, the boundary of the second range 2 is a rectangle, and the rectangle and the second line segment A ₁ A ₂ Depending on the particular requirements, the distance from the rectangle to the second line segment may generally be 3 to 10 pixels in length. In other embodiments, the boundaries of the second range 2 are not necessarily rectangular, but may be other polygonal or irregular closed curves.

In some embodiments, each pixel through which the second line segment passes may be centered on a set length as a radius, so that individual circles can be formed, and the area covered by these individual circles may be used as the second range.

With respect to the second condition. The second condition is used to determine the final position of each line vector, that is, the subpixel coordinates used to determine the intersection of the first line segments, and the subpixel coordinates where the occluded label intersection is reflected on the label image. In general, whether the position of the linear vector relatively accurately reflects the relative positional relationship between the first line segments can be determined by dot product relationship of the linear vector and the pixel gradient vector in the corresponding adjustment range. One embodiment is shown in fig. 7. Fig. 7 schematically, and generally, illustrates the positional relationship of a portion of the pixel gradient vector 5 to the line vector 4 in some embodiments, where the pixel gradient vector 5 points from a lower gray value pixel to a higher gray value pixel. Fig. 7 also shows, in part, pixels 3 within the second range, i.e. the individual tiles in the figure. The numbers within the tiles represent the gray values of the corresponding pixels. The linear vector 4 has an initial position, but its angle may be varied within a certain range to meet the set conditions.

When we assume that: the number of the related second line segment is i (i is more than 1 and less than or equal to L), L is the number of the second line segment, and the straight line vector corresponding to a certain second line segment i is

(its endpoint-also the common endpoint of the relevant straight line vector is P ₀ Starting at P _x -a point in the second range), the range to be analyzed corresponding to the second line segment i having m.n pixels,/for the second line segment i>

When the gradient vector is a gradient vector of a pixel with the number jk, the setting condition may be expressed in the following expression:

in mathematics, argmin represents the minimum value when a given expression operates within a given parameter range, when the value of the expression reaches the maximumHours, determine at this time

And +.>

In the above formula, when->

And +.>

When determined, corresponding P _x P ₀ And also determined. In addition, in the formula->

Representing the dot product of the two parameters.

In other embodiments, the transformation may be performed

The position of (i) is varied within a range of 1 or more and L or less, for example, the point P _x Varying within the range to be analysed, e.g. by making point P _O Varying within the search range; transformation->

The method of the position of (a) includes, but is not limited to, a steepest descent method along the pixel gradient direction, a gauss newton iteration method, etc., and also includes a method of dividing a pixel into a plurality of cells and moving the head and tail positions thereof in a plurality of cells, etc., and also includes a probability-based method such as a simulated annealing method, etc.), and once for each conversion, a value of the following formula is calculated, and when the value is equal to or smaller than a threshold value T, it can be regarded as satisfying a set condition:

The dot product relation expressed by the above formula is the sum of dot products, and the corresponding setting condition is that the sum of dot products is less than or equal to a threshold value T. The threshold T may generally take a number of 0 or more, the smaller the value, the more stringent the requirements.

In other embodiments, the dot product relationship may be a variation of the above formula or a relationship such as a difference between dot products, a product of dot products, or the like, and in any case the dot product relationship helps to determine the order

And->

As much as possible, the inner product (or dot product) is 0 (or close to 0).

In some embodiments, the associated plurality of linear vectors

Is P at the common end point ₀ The direction of these linear vectors is also the pointing point P ₀ A kind of electronic device.

In some embodiments, after determining the coordinates of the common endpoint, i.e., the first endpoint, of the related line vector, the coordinate information of the first endpoint may be extracted as a feature of the label image.

In some embodiments, the second condition includes that a sum of dot products of each of the straight line vectors and each of the pixel gradient vectors within the corresponding adjustment range reaches a minimum.

In some embodiments, "determining a search range for an intersection" includes the steps of: and determining the boundary of the search range by taking the second intersection point as a reference point according to the second intersection point formed by the second line segment.

In some embodiments, when the number of the sub-pixel intersections is 3 or more, a range corresponding to a closed polygon formed by using the second intersection as a vertex is taken as the first range.

In some embodiments, the determining of the second range comprises the steps of: and determining the boundary of the second range by taking each pixel position as a reference point according to the pixel position of the second line segment. That is, the boundary of the second range is determined with the pixel position through which the second line segment passes as a reference point.

In some embodiments, the determining method further comprises the steps of: filtering the image blocks in the second range (namely, the partial images in the second range); the "pixel gradient vector in the adjustment range" is a gradient vector of the pixel subjected to the filtering processing. The purpose of the filtering is to reduce noise, so that the result obtained by the calculation of the formula above is more reliable. This is also one of the focus of the present disclosure. The filtering mode can be used for referencing various prior technologies, such as Gaussian filtering, median filtering, linear filtering and the like, and can be selected according to own requirements when the scheme is implemented, so that excessive expansion is avoided.

In some embodiments, the label image has a plurality of the first line segments and a plurality of the first intersection points; the first range is provided with a plurality of first intersection points which are respectively in one-to-one correspondence with the plurality of first intersection points. Generally, there are a plurality of first line segments in a label image, and the intersection points of the line segments are often more than one, and when the determining method is executed, correlation calculation can be sequentially performed on the intersection points; of course, it is also possible to perform a one-time calculation, i.e. to determine a plurality of first ranges simultaneously in the label image, so as to determine the positions of the line segment intersections corresponding to these first ranges simultaneously. In a more specific case, if one of the first line segments has more than 3 first intersection points, the positions of the first end points in the first range corresponding to the first intersection points are adjusted so that the first end points are located on the same straight line at the same time.

The modules may be integrated together or may be separate. A module may also be made up of a plurality of smaller units. The feature extraction means may also perform further steps, such as the various steps mentioned in the previous embodiments.

In a third aspect, a method for positioning a feature based on a label image is provided, including the following steps: acquiring a scene image shot by a camera; processing the scene image to obtain a label image; executing the feature extraction method to obtain the features of the tag image; and according to the characteristics of the label image, obtaining relative position information by performing operation of visual positioning by using the prior art. The relative position information includes position information obtained by using the relative position of the camera and the tag as a basis. The feature extraction method of the label image is applied to positioning, and can improve the condition of poor positioning effect when the label is shielded.

In a fifth aspect, a computer-readable storage medium is presented, on which a computer program is stored, which computer program, when being executed by a processor, implements the steps of the feature extraction method.

It will be appreciated by those skilled in the art that all or part of the steps in the embodiments may be implemented by a computer program to instruct related hardware, and the program may be stored in a computer readable medium, and the readable medium may include various media that may store program codes, such as a flash disk, a removable hard disk, a read-only memory, a random access device, a magnetic disk, or an optical disk.

It is within the knowledge and ability of one skilled in the art to combine the various embodiments or features mentioned herein with one another as additional alternative embodiments without conflict, and such limited number of alternative embodiments, not listed one by one, formed by a limited number of combinations of features, still fall within the skill of the present disclosure, as would be understood or inferred by one skilled in the art in view of the drawings and the foregoing.

In addition, the description of most embodiments is based on different emphasis and, where not explicitly described, may be understood with reference to the prior art or other related description herein.

It is emphasized that the embodiments described above are merely exemplary and preferred embodiments of the present disclosure, and are merely used to describe and explain the technical solutions of the present disclosure for the convenience of the reader to understand and not to limit the scope or application of the present disclosure. Any modifications, equivalent substitutions, improvements, or the like, which are within the spirit and principles of the present disclosure, are intended to be encompassed within the scope of the present disclosure.

Claims

1. The feature extraction method of the label image, the label image includes the first line segment that intersects, characterized by that, include the following steps:

judging whether the first condition for judging the feature completeness is met or not according to the prior information; the first condition is when at least one intersection point of the tag is blocked or at least one line segment of the tag is blocked;

if yes, sub-pixel line segment extraction operation is carried out on the label image, and a second line segment used for representing the position of the first line segment is obtained;

obtaining position information of sub-pixel intersection points according to the position information of the second line segment, wherein the sub-pixel intersection points comprise visible intersection points and non-display intersection points;

generating data of a linear vector corresponding to each second line segment and determining an adjustment range according to the position information of the second line segment and the sub-pixel intersection point, wherein the linear vector is provided with at least one intersected first endpoint;

Adjusting the positions of the linear vectors in the adjustment range until the relation between each linear vector and the pixel gradient vector in the corresponding adjustment range meets a second condition; the second condition includes that the sum of dot products of each linear vector and each pixel gradient vector in the corresponding adjustment range reaches a minimum value;

and extracting the position information of the first endpoint at the moment as the characteristic of the label image.

2. The feature extraction method according to claim 1, characterized in that:

the adjustment range comprises a first range determined according to the prior information and the position information of the sub-pixel intersection point;

adjusting the position of the line vector includes: and adjusting the position of the first endpoint in the first range.

3. The feature extraction method according to claim 2, characterized in that:

the boundary of the first range includes a location where the sub-pixel intersection is located.

4. The feature extraction method according to claim 2, characterized in that:

the adjusting range further comprises a second range determined according to the position information of the second line segment;

adjusting the position of the line vector further comprises: when one straight line vector has a second end point which is not intersected with other straight line vectors, the position of the second end point is adjusted in the second range.

5. A feature extraction device of a label image for extracting features of the label image having intersecting first line segments, comprising:

the feature completeness judging module is used for judging whether the first condition for judging the feature completeness is met or not according to the priori information; the first condition is that when at least one intersection point of the tag is blocked or at least one line segment of the tag is blocked;

the sub-pixel line segment extraction module is used for executing sub-pixel line segment extraction operation on the label image when the first condition is met, and obtaining a second line segment used for representing the position of the first line segment; the sub-pixel intersection point comprises a visible intersection point and a non-displayed intersection point;

the adjustment information generation module is used for generating data of a linear vector corresponding to each second line segment and determining an adjustment range according to the position information of the intersection point of the second line segments and the sub-pixels, and the linear vector is provided with at least one intersected first endpoint;

the position adjustment module is used for adjusting the positions of the linear vectors in the adjustment range until the relation between each linear vector and the pixel gradient vector in the corresponding adjustment range meets a second condition; the second condition includes that the sum of dot products of each linear vector and each pixel gradient vector in the corresponding adjustment range reaches a minimum value;

And the feature extraction module is used for extracting the position information of the first endpoint when the second condition is met as the feature of the tag image.

6. The positioning method based on the characteristics of the label image is characterized by comprising the following steps of:

acquiring a scene image shot by a camera;

processing the scene image to obtain a label image;

performing the feature extraction method of any one of claims 1 to 4, obtaining features of the label image;

and obtaining relative position information according to the characteristics of the label image.

7. Positioning device, including camera and treater, its characterized in that:

the camera is used for shooting a scene to obtain a scene image;

the processor is used for processing the scene image to obtain a label image; further for performing the steps of the feature extraction method of any one of claims 1 to 4, obtaining features of the label image; and obtaining relative position information according to the characteristics of the label image.

8. A computer readable storage medium having stored thereon a computer program characterized by: the computer program, when executed by a processor, implements the feature extraction method steps of any one of claims 1 to 4.