CN111652825B - Edge tracking straight line segment rapid detection device and method based on gradient direction constraint - Google Patents
Edge tracking straight line segment rapid detection device and method based on gradient direction constraint Download PDFInfo
- Publication number
- CN111652825B CN111652825B CN202010780646.1A CN202010780646A CN111652825B CN 111652825 B CN111652825 B CN 111652825B CN 202010780646 A CN202010780646 A CN 202010780646A CN 111652825 B CN111652825 B CN 111652825B
- Authority
- CN
- China
- Prior art keywords
- straight line
- edge
- gradient
- line segment
- points
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 64
- 238000001514 detection method Methods 0.000 title description 35
- 230000008569 process Effects 0.000 claims description 25
- 230000011218 segmentation Effects 0.000 claims description 25
- 238000004364 calculation method Methods 0.000 claims description 12
- 238000012545 processing Methods 0.000 claims description 10
- 230000001629 suppression Effects 0.000 claims description 10
- 230000008030 elimination Effects 0.000 claims description 5
- 238000003379 elimination reaction Methods 0.000 claims description 5
- 239000011159 matrix material Substances 0.000 claims description 4
- 238000013139 quantization Methods 0.000 claims description 4
- 238000009499 grossing Methods 0.000 claims description 3
- 230000001186 cumulative effect Effects 0.000 claims description 2
- 238000001914 filtration Methods 0.000 claims description 2
- 230000008901 benefit Effects 0.000 abstract description 4
- 230000009471 action Effects 0.000 description 8
- 238000010586 diagram Methods 0.000 description 6
- 238000000605 extraction Methods 0.000 description 6
- 238000012795 verification Methods 0.000 description 6
- 230000006870 function Effects 0.000 description 4
- 230000008859 change Effects 0.000 description 2
- 230000003247 decreasing effect Effects 0.000 description 2
- 230000000694 effects Effects 0.000 description 2
- 230000000007 visual effect Effects 0.000 description 2
- 230000005540 biological transmission Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000005457 optimization Methods 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/13—Edge detection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/70—Denoising; Smoothing
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/12—Edge-based segmentation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/136—Segmentation; Edge detection involving thresholding
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/20—Analysis of motion
- G06T7/269—Analysis of motion using gradient-based methods
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Multimedia (AREA)
- Image Analysis (AREA)
Abstract
The invention discloses a device and a method for rapidly detecting edge tracking straight line segments based on gradient direction constraint, wherein the method comprises the following steps: obtaining edge points of a single pixel of an image; according to the constraint of the gradient, connecting the adjacent edge points when and only when the difference value between the gradient tangential direction of the searched adjacent edge points and the extension direction of the tracked straight line segment is less than a threshold value to obtain the straight line segment; for the obtained initial straight line segment, namely an edge segment, the ratio of the distance l between two end points of the initial straight line segment to the maximum distance d between each point on the edge segment and a straight line connecting the two end points is calculated, the ratio represents the significance degree of the edge segment as a straight line, and whether the edge segment is divided into a plurality of straight line segments as a curve segment is judged; and judging straight line segments and eliminating false straight line segments. The method has the advantages of high speed of detecting the straight line segments and strong adaptability to different image scales and contrast levels.
Description
Technical Field
The invention belongs to the technical field of visual detection, and relates to a device and a method for quickly detecting an edge tracking straight line segment based on gradient direction constraint.
Background
The straight line edge is one of the main geometrical features describing the shape and structure of objects in the image. The straight line segment detection is always a basic task and a research hotspot in computer vision and photogrammetry, and has wide application in many fields such as camera calibration, target identification and matching, industrial detection and the like.
The Line Segment Detector (LSD) algorithm is a representative high-performance straight Line Segment detection method at present due to the fact that the Line Segment Detector (LSD) algorithm is well balanced in speed and detection performance stability. The basic idea of the LSD algorithm is to adopt a general threshold decision theory called a background counterexample model proposed by Desolneux and the like to distinguish and eliminate false straight-line segments on the basis of an improved phase grouping algorithm. However, the LSD algorithm does not get rid of the disadvantage of the large number of operated pixels inherent in the phase grouping algorithm, and the speed advantage is mainly obtained by using a simplified gradient solver. In addition, in order to ensure the accuracy of the detection of the straight line segment, a series of parameter adjustment measures are designed to optimize the straight line segment before a straight line support region is judged as false detection, and the execution efficiency of the algorithm is influenced to a certain extent.
Disclosure of Invention
The invention aims to provide a device and a method for rapidly detecting an edge tracking straight line segment based on gradient direction constraint, which improve the operation efficiency on the premise of ensuring the detection performance.
According to a first aspect of the present invention, there is provided an edge tracking straight-line segment fast detection apparatus based on gradient direction constraint, including:
the single-pixel edge image acquisition module is used for acquiring edge points of single pixels of the image;
the straight line segment extraction module is used for connecting the adjacent edge points to obtain the straight line segment only when the difference value between the gradient tangential direction of the searched adjacent edge points and the extension direction of the tracked straight line segment is less than a threshold value according to the constraint of the gradient;
the curve segmentation module is used for judging whether the obtained initial straight-line segment is used as a curve segment and is segmented into a plurality of straight-line segments; and
and the verification module is used for judging the straight line segment after the curve segmentation module finishes segmentation and eliminating false straight line segments.
According to a second aspect of the present invention, there is provided a method for rapidly detecting an edge tracking straight-line segment based on gradient direction constraint, including:
(1) obtaining edge points of a single pixel of an image;
(2) according to the constraint of the gradient, connecting the adjacent edge points when and only when the difference value between the gradient tangential direction of the searched adjacent edge points and the extension direction of the tracked straight line segment is less than a threshold value to obtain the straight line segment; the method comprises the following steps: the extension direction angle of the line segment is estimated in the edge tracking process by using the following formula
In the formula, GiThe gradient amplitude at the i-th point, θiIs the gradient tangent angle of the ith point. If record Gi xAnd Gi yThe gradient values of the i point in the transverse direction and the longitudinal direction respectively have
When a new edge point is connected, the extending direction of the primary line segment is estimated again until no new edge point meeting the direction condition exists;
(3) for the obtained initial straight line segment, namely an edge segment, the ratio of the distance l between two end points of the initial straight line segment to the maximum distance d between each point on the edge segment and a straight line connecting the two end points is calculated, the ratio represents the significance degree of the edge segment as a straight line, and whether the edge segment is divided into a plurality of straight line segments as a curve segment is judged; performing the segmentation if any of the two edge segments that are obtained by breaking the edge segment furthest away from a line connecting the two end points is more pronounced than the primary edge; repeating the steps until a more obvious edge section cannot be obtained after segmentation, or the number of pixels contained in the two edge sections obtained after segmentation is less than a set threshold value TL; and
(4) and the verification module judges straight line segments and eliminates false straight line segments.
Optionally, for the edge tracking straight-line segment fast detection method based on the gradient direction constraint, the method further includes:
(1.1) smoothing and filtering the image by adopting a 7 multiplied by 7 Gaussian kernel, and then performing gradient calculation by adopting a 3-order Sobel operator;
(1.2) after obtaining the image gradient, carrying out threshold processing to obtain a pixel set only containing edge information in the image, eliminating small gradient amplitude points,
in the formula: rho is a gradient amplitude threshold value, q is a maximum gray level error possibly brought in 256-level gray level quantization, tau represents an allowed maximum error in a gradient direction, and all image points with gradient amplitudes smaller than the threshold value are ignored in the subsequent processing process;
(1.3) carrying out non-maximum suppression treatment on the obtained result along the gradient direction to obtain a refined edge, wherein only the pixel points which are local maximum values in the gradient direction are reserved through the non-maximum suppression, and the rest pixel points are removed to obtain the edge of a single pixel.
Optionally, for the edge tracking straight-line segment fast detection method based on the gradient direction constraint, the method further includes:
pseudo-ordering the gradient magnitude values to select seed points: and after the pseudo-sorting is finished, selecting the seed points from the grade with the highest gradient value, and sequentially descending until all grades are traversed.
Optionally, for the edge tracking straight-line segment fast detection method based on the gradient direction constraint, the method further includes:
counting the number of edge points of which the phase directions are approximately vertical to the fitted straight line direction on the candidate straight line segment, and converting the elimination of the false straight line segment into an exception for judging whether the false straight line segment is an unstructured background noise hypothesis; selecting a local background area, namely a linear support area, for each linear segment, wherein the area is selected as a rectangle, the center of the rectangle is selected as the weighted average of all points on the linear segment, the main direction is the direction of the eigenvector corresponding to the maximum eigenvalue of the covariance matrix of the weighted distribution of all points, wherein the weight of each point is the gradient amplitude of the point, the length and the width of the rectangle are respectively selected as values when the rectangle just can completely cover all the points on the corresponding linear segment, and in addition, the calculated center and the main direction of the rectangle are also used as the fitting result of the linear segment passing through the center of the rectangle along the main direction of the rectangle.
Optionally, for the edge tracking straight-line segment fast detection method based on the gradient direction constraint, the method further includes: assuming that the background model is a white noise model, that is, the phase direction of each pixel point in the image satisfies independent uniform random distribution within [0, 2 pi ], the region with smooth gray scale change on the image is considered to be in accordance with the assumption, the effective straight line segment is a region with consistent phase direction height, otherwise, the effective straight line segment is a false straight line segment, in the N × M image, the number of all possible straight lines is C (N × M, 2), and C (N × M, 2) is a combined number obtained by randomly taking two nonrepeating points from N × M points
The width of the linear bearing zone varies from 1 to N, with the number of all possible linear bearing zones being (N M)2.5(ii) a The false alarm number for generating the image point phase direction distribution in the specific straight line support region is
In the formula: n represents the total number of points in the linear support region, k represents the number of points in the linear support region that coincide with the linear direction, p is determined by an angle threshold, the local principal direction of an image point is set as its gradient tangent, NFLess than or equal to the straight line segment is accepted, NF>The straight line segments of (a) are eliminated as false straight line segments, which are empirically set thresholds, B (n, k, p) is a binomial distribution cumulative probability,
。
optionally, for said gradient direction constraint-basedThe method for rapidly detecting the edge tracking straight line segment further comprises the following steps: before one candidate straight line segment is eliminated, the width of the rectangle is gradually increased by taking 1 pixel as a step size, and N is performed againFUntil the determination result is true or the set maximum allowable width T is reachedWIf the decision result is always false, then N is reservedFThe width value with the minimum value is used for further judgment by adjusting the p value;
after the rectangle width adjustment is finished, taking five values of p/2, p/4, p/8, p/16 and p/32 in sequence except p for each candidate line segment to carry out NFAnd (4) if the result of the calculation of any one of the five values is true, accepting the straight line segment.
According to a third aspect of the present invention, there is provided a computer-readable storage medium having stored therein computer-executable instructions which, when executed by a processor, implement the steps in the method according to the second aspect.
Compared with the prior art, the edge tracking straight-line segment rapid detection device and method based on the gradient direction constraint provided by the invention have the advantages that firstly, on a refined edge image, the edge tracking algorithm based on the gradient direction constraint is utilized to carry out rapid scanning and extraction on the straight-line segment; and then, carrying out effectiveness judgment on the preliminarily obtained straight line segments based on a background counterexample model, and eliminating invalid detection results to obtain a final straight line segment set. The speed of the line segment detection of the common real scene picture is improved by nearly 1 time compared with the current classical high-performance algorithm, the adaptability to different image scales and contrast levels is stronger, and the requirements of various visual detection systems on the line segment detection performance and the real-time performance can be better met.
Drawings
FIG. 1 is a flowchart of a method for fast detecting an edge tracking straight-line segment based on a gradient direction constraint according to an embodiment of the present invention.
FIG. 2 is a diagram illustrating correct tracking directions according to an embodiment of the present invention.
FIG. 3 is a diagram illustrating the direction of the error tracking according to an embodiment of the present invention.
FIG. 4 is a schematic view of the gradient tangency in one embodiment of the present invention.
FIG. 5 is a diagram illustrating an angle difference between two line segments smaller than a threshold τ according to an embodiment of the present invention.
FIG. 6 is a diagram illustrating segment division according to an embodiment of the present invention.
Detailed Description
In the following description, numerous technical details are set forth in order to provide a better understanding of the present application. However, it will be understood by those skilled in the art that the technical solutions claimed in the present application can be implemented without these technical details and with various changes and modifications based on the following embodiments.
In order to make the objects, technical solutions and advantages of the present invention more apparent, embodiments of the present invention will be described in detail with reference to the accompanying drawings.
Example 1
The embodiment 1 of the invention provides an edge tracking straight-line segment rapid detection device based on gradient direction constraint. Implementation details of the present embodiment are specifically described below, and the following description is provided only for the sake of understanding and is not necessary for implementing the present embodiment. The present embodiment 1 includes:
the single-pixel edge image acquisition module is used for acquiring edge points of single pixels of the image;
the straight line segment extraction module is used for connecting at least adjacent edge points according to the constraint of the gradient to obtain a straight line segment;
the curve segmentation module is used for judging whether the obtained initial straight-line segment is used as a curve segment and is segmented into a plurality of straight-line segments; and
and the verification module is used for judging straight line segments and eliminating false straight line segments.
Therefore, the embodiment of the invention provides structural guarantee for the realization of the rapid detection of the edge tracking straight line segment based on the constraint of the gradient direction.
Example 2
The embodiment 2 of the invention provides a depth image portrait edge optimization processing device. Implementation details of the present embodiment are specifically described below, and the following description is provided only for the sake of understanding and is not necessary for implementing the present embodiment. In this embodiment, the method specifically includes:
the single-pixel edge image acquisition module at least has the following functions:
(1.1) the image can be smoothed with a 7 x 7 gaussian kernel and then gradient calculated with a 3 rd order Sobel operator.
(1.2) after the image gradient is obtained, the amplitude of the image gradient is thresholded to obtain a pixel set which only contains edge information in the image, namely an edge image. Small gradient magnitude points tend to appear in smooth regions, the presence of which tends to severely affect the computation of the gradient direction, resulting in a wrong gradient direction. For the pixel points, no matter the gradient amplitude information or the direction information is utilized, the edge information cannot be accurately detected from the pixel points, but false detection results can be caused, and therefore the pixel points are eliminated in the method. For this purpose, the single-pixel edge image acquisition module is implemented using the following gradient magnitude thresholds:
in the formula: ρ is a gradient magnitude threshold, q is a maximum gray level error possibly brought about in 256-level gray level quantization, τ represents an allowable maximum error in gradient direction, and all image points with gradient magnitudes smaller than the threshold are ignored in subsequent processing.
In a specific example, q =2 and τ =22.5 ° can be taken, which are two empirical values recommended in the present invention, and practice shows that relatively good results can be obtained. Of course, those skilled in the art can flexibly set other specific values according to actual situations, and the selection of the above empirical value is not particularly limited by the present invention.
(1.3) further, the edge tracking is carried out on the single-pixel edge to obtain a better result, so that after the gradient calculation and thresholding are carried out, the single-pixel edge image acquisition module can also carry out non-maximum suppression processing on the obtained result along the gradient direction to obtain a refined edge. Through non-maximum suppression, only those pixels which are local maximums in the gradient direction of the pixel are reserved, namely the pixel points with the gradient amplitude larger than that of the front and the rear pixel points are reserved, and the rest pixels are removed, so that the edge of a single pixel can be obtained.
The straight line segment extraction module can at least utilize gradient direction information to constrain the tracking process. Linking the searched adjacent boundary points if and only if the difference between the gradient tangents of the boundary points and the extension direction of the already tracked straight line segment is less than a certain threshold value tau. In one particular option, τ =22.5 °.
The definition of two adjacent pixels can adopt an 8-neighborhood mode, and the definition of the gradient tangential direction is as shown in figure 4, is vertical to the gradient direction and points to the right side.
Specifically, the straight line segment extraction module at least has the following functions:
(2.1) the extension direction angle of the line segment is estimated in the edge tracking process using the following formula:
in the formula, GiThe gradient amplitude at the i-th point, θiIs the gradient tangent angle of the ith point. If record Gi xAnd Gi yThe gradient values of the i point in the transverse direction and the longitudinal direction respectively have
The purpose of weighting using the gradient magnitude in equation (2) is to ensure that significant edge points have a greater influence on the line segment extension direction. And re-estimating the extending direction of the line segment once every time a new edge point is connected until no new edge point meeting the direction condition exists.
It is considered that the output result of the edge tracking algorithm has certain dependency on the initially selected seed point. For a significant straight edge segment, the same tracking result will be obtained regardless of the seed point selection. However, different seed point selections may yield different tracking results when a non-straight curve segment or a broken line segment is approximated by a straight line segment. A more easily understood example is that when a circle is included in an image, different seed points can decompose the circle into a plurality of line segments, but the starting point and the ending point of the line segment are not the same, and thus the line segment is not the same.
The inventor further studies and finds that the edge points with larger gradient values are more suitable as seed points. However, fully ordering the gradient values is a time consuming task, typically with a complexity above o (nlogn).
For this purpose, the functions of the straight line segment extraction module further include:
(2.2) pseudo-ordering the gradient magnitude values to select seed points: a number of levels are divided equally spaced between 0 and the maximum gradient magnitude, into which each pixel is classified according to its corresponding gradient magnitude. And after the pseudo sorting is finished, selecting the seed points from the grade with the highest gradient value, and descending sequentially until all grades are traversed.
In the pseudo sorting of the embodiment of the invention, the number of the selected levels is 1024, and for the gray image with 256 levels, the difference between the pseudo sorting result and the full sorting result of 1024 levels is not large, so that more levels are not needed to be used. Of course, it is also possible to select more levels.
For the curve segmentation module, it is considered that although the gradient direction constraint is added in the present invention, the algorithm still cannot distinguish between the special case in fig. 5 where the angle difference between the two line segments is smaller than the threshold τ, or a curve segment with a large curvature radius (as in fig. 6). For this reason, the output still needs to be segmented after the end of the connection process. The specific process of the curve segmentation module in the segmentation is as follows: performing the segmentation if any of the two edge segments that are obtained by breaking the edge segment furthest away from a line connecting the two end points is more pronounced than the primary edge; the above steps are repeated until no more significant edge segment can be obtained after the segmentation, or the number of pixels included in the two edge segments obtained after the segmentation is less than a certain set threshold TL, for example, 6 pixels are selected from TL in the embodiment of the present invention, and the threshold TL may also be other values, for example, 2 to 10 pixels, and the like.
As shown in fig. 6, the edge segment AB meeting the segmentation conditions will be broken at point C, resulting in new edge segments AC and CB.
And for the verification module, performing straight line segment judgment detection, and counting the number of edge points of which the phase directions on the candidate straight line segments are approximately vertical to the fitted straight line direction, so that the elimination problem of the false straight line segment can be converted into the exception of judging whether the false straight line segment is under the assumption of non-structural background noise.
Since the geometric structures (including line segments) in the actual image are displayed in a certain background area, a local background area, i.e., a linear support area, is selected for each line segment to achieve the above determination. The verification module in the embodiment of the invention generally selects the area as a rectangle, the center of the rectangle is selected as the weighted average of all points on the line segment, the main direction is the direction of the eigenvector corresponding to the maximum eigenvalue of the covariance matrix of the weighted distribution of all points, and the weight of each point is the gradient amplitude of the eigenvector. The length and width of the rectangle are each chosen to be the value at which the rectangle is just able to completely cover all points on the corresponding line segment. Furthermore, the calculated rectangle center and principal direction are also taken as the fitting result of a straight line segment passing through the center thereof in the rectangle principal direction.
Further, the verification module may assume that the background model is a white noise model, that is, the phase direction of each pixel point in the image is [0, 2 pi ]]The effective straight line segment is a region with consistent phase direction height, otherwise, the effective straight line segment is a false straight line segment, and in the N multiplied by M image, the number of all possible straight lines is C (N is a function of the formula)M, 2) strips, the width of the rectilinear support areas varying from 1 to N, the number of all possible rectilinear support areas being (N M)2.5(ii) a The false alarm number for generating the image point phase direction distribution in the specific straight line support region is
In the formula: n represents the total number of points in the linear support region, k represents the number of points in the linear support region that coincide with the linear direction, and p is determined by an angle threshold. In a specific example, the angle threshold is chosen to be 22.5 °, and then p = 22.5/180 = 1/8. The local principal direction of an image point is set as its gradient tangent. N is a radical ofFLess than or equal to the straight line segment is accepted, NF>The straight line segments of (1) are eliminated as false straight line segments, which are empirically set thresholds.
It should be noted that, in this step of calculation, the points removed in the non-maximum suppression process also participate in the number statistics, because the background counter-example model needs to determine by using the phase information of most of the pixel points except the pixel points determined as noise in the thresholding process.
In fact, the dependency of the detection result of the present invention on the threshold is very weak, so that it can satisfy the needs in most cases by directly taking a fixed value, and the value is, for example, 0.01 to 1, such as 0.05, 0.1, 0.5, etc.
For a potential straight line segment, the selection of the rectangle parameter and the p value for representing the straight line support region may influence NFTherefore, a more careful determination process is required to avoid the missing detection as much as possible. Considering the process of extracting potential straight line segments by the algorithm, adjusting the length of the rectangle may destroy the effect of curve segment segmentation, and therefore we choose to make further decision by adjusting the width of the rectangle. Before one candidate straight-line segment is eliminated, the width of the supporting rectangle is gradually increased by taking 1 pixel as a step length, and N times of operation are carried out againFUntil it is determinedAs a result, the maximum allowable width TW is true or reached, for example, TW is selected to be 4 pixels in the embodiment of the present invention, it is understood that other values are also possible, and the present invention is not limited to this. If the decision result is always false, then keeping NFThe width value with the smallest value is used for further determination by adjusting the p value. It should be noted that the straight line segment candidates are derived from single-pixel edges, and therefore, it does not make sense to reduce the width of the rectangle.
Since the magnitude of the p-value directly corresponds to the magnitude of the angle threshold, increasing the p-value will tend to make N largerFThe angle threshold is decreased, but increased at the same time, which corresponds to the relaxation of the constraints on the gradient direction. However, for a line segment with a good structure, a small p value can instead yield a smaller NFTherefore, in order to ensure that line segments with good structure are not mistakenly eliminated, after the width adjustment of the supporting rectangle is completed, the five values of p/2, p/4, p/8, p/16 and p/32 are sequentially taken for N of each candidate line segment except for pFAnd (4) accepting the candidate straight-line segment if the result of the calculation of any one of the five values makes the determination true.
Example 3
The embodiment 3 of the invention provides a method for rapidly detecting an edge tracking straight line segment based on gradient direction constraint. Implementation details of the present embodiment are specifically described below, and the following description is provided only for the sake of understanding and is not necessary for implementing the present embodiment. Fig. 1 is a schematic diagram of the present embodiment, which includes:
(1) obtaining edge points of a single pixel of an image;
(2) according to the constraint of the gradient, connecting the adjacent edge points when and only when the difference value between the gradient tangential direction of the searched adjacent edge points and the extension direction of the tracked straight line segment is less than a threshold value to obtain the straight line segment;
(3) for the obtained initial straight line segment, namely an edge segment, the ratio of the distance l between two end points of the initial straight line segment to the maximum distance d between each point on the edge segment and a straight line connecting the two end points is calculated, the ratio represents the significance degree of the edge segment as a straight line, and whether the edge segment is divided into a plurality of straight line segments as a curve segment is judged; and
(4) and (5) judging straight line segments, and removing false straight line segments.
On the premise of ensuring the detection performance, the method has obviously improved operation efficiency compared with the traditional algorithms such as LSD and the like.
Example 4
the LSD algorithm uses a simple template of 2 multiplied by 2 to carry out gradient calculation, can meet the requirements for the phase grouping algorithm, but is difficult to ensure the gradient calculation precision for the edge tracking algorithm. Furthermore, the gradient calculated using this template output is located at (x +0.5, y +0.5) in position, and this deviation must be compensated for in the following procedure.
Thus, in the embodiment of the present invention, for step (1), specifically, the method includes:
(1.1) smoothing the image with a 7 × 7 gaussian kernel and then gradient-computing it with a 3 rd order Sobel operator.
(1.2) after the image gradient is obtained, thresholding is carried out on the amplitude of the image gradient to obtain a pixel set which only contains edge information in the image, namely an edge image. Small gradient magnitude points tend to appear in smooth regions, the presence of which tends to severely affect the computation of the gradient direction, resulting in a wrong gradient direction. For the pixel points, no matter the gradient amplitude information or the direction information is utilized, the edge information cannot be accurately detected from the pixel points, but false detection results can be caused, and therefore the pixel points are eliminated in the method. To this end, the following gradient magnitude thresholds are used:
in the formula: ρ is a gradient magnitude threshold, q is a maximum gray level error possibly brought about in 256-level gray level quantization, τ represents an allowable maximum error in gradient direction, and all image points with gradient magnitudes smaller than the threshold are ignored in subsequent processing.
In a specific example, q =2 and τ =22.5 ° can be taken, which are two empirical values recommended in the present invention, and practice shows that relatively good results can be obtained. Of course, those skilled in the art can flexibly set other specific values according to actual situations, and the selection of the above empirical value is not particularly limited by the present invention.
(1.3) further, the edge tracking is performed on the single-pixel edge to obtain a better result, so that after the gradient calculation and thresholding are performed, the obtained result can be subjected to non-maximum suppression processing along the gradient direction to obtain a refined edge. Through non-maximum suppression, only those pixels which are local maximums in the gradient direction of the pixel are reserved, namely the pixel points with the gradient amplitude larger than that of the front and the rear pixel points are reserved, and the rest pixels are removed, so that the edge of a single pixel can be obtained.
In the embodiment of the present invention, for step (2), edge tracking is a method for gradually detecting an image edge by starting from an edge point in a gradient map, and sequentially searching and connecting adjacent edge points. But usually only the magnitude information of the gradient is utilized without utilizing its phase (direction) information, and also without utilizing the distribution information of the edge points that have been detected. Therefore, the stability of the tracking direction cannot be guaranteed during the edge tracking process, which may result in some unreasonable detection results. As shown in fig. 2, the ideal detection result for the situation in fig. 2 should be as indicated by the arrow in fig. 2, but the actual detection result may also be as indicated by the arrow in fig. 3, with the determining factor being the selected neighborhood traversal order, e.g., outputting the result in fig. 2 in a counter-clockwise direction and outputting the result in fig. 3 in a clockwise direction.
The gradient direction information is used in the embodiment of the invention to constrain the tracking process. Linking the searched adjacent boundary points if and only if the difference between the gradient tangents of the boundary points and the extension direction of the already tracked straight line segment is less than a certain threshold value tau. In one particular option, τ =22.5 °.
The definition of two adjacent pixels can adopt an 8-neighborhood mode, and the definition of the gradient tangential direction is as shown in figure 4, is vertical to the gradient direction and points to the right side.
Specifically, the step (2) comprises the following steps:
(2.1) the extension direction angle of the line segment is estimated in the edge tracking process using the following formula:
in the formula, GiThe gradient amplitude at the i-th point, θiIs the gradient tangent angle of the ith point. If record Gi xAnd Gi yThe gradient values of the i point in the transverse direction and the longitudinal direction respectively have
The purpose of weighting using the gradient magnitude in equation (2) is to ensure that significant edge points have a greater influence on the line segment extension direction. And re-estimating the extending direction of the line segment once every time a new edge point is connected until no new edge point meeting the direction condition exists.
The output result of the edge tracking algorithm has a certain dependency on the initially selected seed point. For a significant straight edge segment, the same tracking result will be obtained regardless of the seed point selection. However, different seed point selections may yield different tracking results when a non-straight curve segment or a broken line segment is approximated by a straight line segment. A more easily understood example is that when a circle is included in an image, different seed points can decompose the circle into a plurality of line segments, but the starting point and the ending point of the line segment are not the same, and thus the line segment is not the same.
The inventor further studies and finds that the edge points with larger gradient values are more suitable as seed points. However, fully ordering the gradient values is a time consuming task, typically with a complexity above o (nlogn).
For this purpose, the step (2) further comprises:
(2.2) pseudo-ordering the gradient magnitude values to select seed points: a number of levels are divided equally spaced between 0 and the maximum gradient magnitude, into which each pixel is classified according to its corresponding gradient magnitude. And after the pseudo sorting is finished, selecting the seed points from the grade with the highest gradient value, and descending sequentially until all grades are traversed.
In the pseudo sorting of the embodiment of the invention, the number of the selected levels is 1024, and for the gray image with 256 levels, the difference between the pseudo sorting result and the full sorting result of 1024 levels is not large, so that more levels are not needed to be used. Of course, it is also possible to select more levels.
For step (3), it is considered that although the gradient direction constraint is added in the present invention, the algorithm still cannot distinguish between the special case of fig. 5 where the angle difference between the two line segments is smaller than the threshold τ, or a curve segment with a large curvature radius (as in fig. 6). For this reason, the output still needs to be segmented after the end of the connection process. The specific process of segmentation is as follows: performing the segmentation if any of the two edge segments that are obtained by breaking the edge segment furthest away from a line connecting the two end points is more pronounced than the primary edge; the above steps are repeated until no more significant edge segment can be obtained after the segmentation, or the number of pixels included in the two edge segments obtained after the segmentation is less than a certain set threshold TL, for example, 6 pixels are selected from TL in the embodiment of the present invention, and the threshold TL may also be other values, for example, 2 to 10 pixels, and the like.
As shown in fig. 6, the edge segment AB meeting the segmentation conditions will be broken at point C, resulting in new edge segments AC and CB.
Through the above operations, we complete the basic straight line segment detection algorithm. Although measures such as direction constraint and curve segment segmentation are also adopted to ensure that the algorithm only outputs straight line segments, experimental results show that some false straight line segments are still detected in an image region with complex texture, and further judgment and elimination are needed.
In the embodiment of the present invention, for step (4), the straight line segment determination and detection are performed, and the number of edge points on the candidate straight line segment, where the phase direction is approximately perpendicular to the fitted straight line direction, may be counted, so that the elimination problem of the false straight line segment may be converted into an exception for determining whether it is under the assumption of the unstructured background noise.
Since the geometric structures (including line segments) in the actual image are displayed in a certain background area, a local background area, i.e., a linear support area, is selected for each line segment to achieve the above determination. The region is generally selected to be a rectangle, the center of the rectangle is selected to be the weighted average of all points on the line segment, the main direction is the direction of the eigenvector corresponding to the maximum eigenvalue of the covariance matrix of the weighted distribution of all points, and the weight of each point is the gradient amplitude of the point. The length and width of the rectangle are each chosen to be the value at which the rectangle is just able to completely cover all points on the corresponding line segment. Furthermore, the calculated rectangle center and principal direction are also taken as the fitting result of a straight line segment passing through the center thereof in the rectangle principal direction.
Further, assume the background model is a white noise model, i.e. the phase direction of each pixel point in the image is [0, 2 π]The effective straight line segment is a region with consistent phase direction height, otherwise, the effective straight line segment is a false straight line segment, in the N multiplied by M image, the number of all possible straight lines is C (N multiplied by M, 2), the width of the straight line support region is changed from 1 to N, and the number of all possible straight line support regions is (N multiplied by M)2.5(ii) a Generating image point phase direction distribution in specific straight line support areaThe false alarm number is
In the formula: n represents the total number of points in the linear support region, k represents the number of points in the linear support region that coincide with the linear direction, and p is determined by an angle threshold. In a specific example, the angle threshold is chosen to be 22.5 °, and then p = 22.5/180 = 1/8. The local principal direction of an image point is set as its gradient tangent. N is a radical ofFLess than or equal to the straight line segment is accepted, NF>The straight line segments of (1) are eliminated as false straight line segments, which are empirically set thresholds.
It should be noted that, in this step of calculation, the points removed in the non-maximum suppression process also participate in the number statistics, because the background counter-example model needs to determine by using the phase information of most of the pixel points except the pixel points determined as noise in the thresholding process.
In fact, the dependency of the detection result of the present invention on the threshold is very weak, so that it can satisfy the needs in most cases by directly taking a fixed value, and the value is, for example, 0.01 to 1, such as 0.05, 0.1, 0.5, etc.
For a potential straight line segment, the selection of the rectangle parameter and the p value for representing the straight line support region may influence NFTherefore, a more careful determination process is required to avoid the missing detection as much as possible. Considering the process of extracting potential straight line segments by the algorithm, adjusting the length of the rectangle may destroy the effect of curve segment segmentation, and therefore we choose to make further decision by adjusting the width of the rectangle. Before one candidate straight-line segment is eliminated, the width of the supporting rectangle is gradually increased by taking 1 pixel as a step length, and N times of operation are carried out againFUntil the determination result is true or the set maximum allowable width TW is reached, for example, TW is selected to be 4 pixels in the embodiment of the present invention, and it is understood that other values are also possible, and the present invention is not limited thereto. If the judgment result is alwaysIf false, then keep NFThe width value with the smallest value is used for further determination by adjusting the p value. It should be noted that the straight line segment candidates are derived from single-pixel edges, and therefore, it does not make sense to reduce the width of the rectangle.
Since the magnitude of the p-value directly corresponds to the magnitude of the angle threshold, increasing the p-value will tend to make N largerFThe angle threshold is decreased, but increased at the same time, which corresponds to the relaxation of the constraints on the gradient direction. However, for a line segment with a good structure, a small p value can instead yield a smaller NFTherefore, in order to ensure that line segments with good structure are not mistakenly eliminated, after the width adjustment of the supporting rectangle is completed, the five values of p/2, p/4, p/8, p/16 and p/32 are sequentially taken for N of each candidate line segment except for pFAnd (4) accepting the candidate straight-line segment if the result of the calculation of any one of the five values makes the determination true.
Accordingly, other embodiments of the present invention may also provide a computer-readable storage medium having stored therein computer-executable instructions that, when executed by a processor, implement the method embodiments of the present application. Computer-readable storage media, including both non-transitory and non-transitory, removable and non-removable media, may implement information storage by any method or technology. The information may be computer readable instructions, data structures, modules of a program, or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), Static Random Access Memory (SRAM), Dynamic Random Access Memory (DRAM), other types of Random Access Memory (RAM), Read Only Memory (ROM), Electrically Erasable Programmable Read Only Memory (EEPROM), flash memory or other memory technology, compact disc read only memory (CD-ROM), Digital Versatile Discs (DVD) or other optical storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transmission medium that can be used to store information that can be accessed by a computing device.
It is noted that, in the present patent application, relational terms such as first and second, and the like are used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Also, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, the use of the verb "comprise a" to define an element does not exclude the presence of another, same element in a process, method, article, or apparatus that comprises the element. In the present patent application, if it is mentioned that a certain action is executed according to a certain element, it means that the action is executed according to at least the element, and two cases are included: performing the action based only on the element, and performing the action based on the element and other elements. The expression of a plurality of, a plurality of and the like includes 2, 2 and more than 2, more than 2 and more than 2.
In some cases, the actions or steps recited in the claims may be performed in a different order than in the embodiments and still achieve desirable results. In addition, the processes depicted in the accompanying figures do not necessarily require the particular order shown, or sequential order, to achieve desirable results. In some embodiments, multitasking and parallel processing may also be possible or may be advantageous.
Claims (7)
1. A method for quickly detecting an edge tracking straight line segment based on gradient direction constraint is characterized by comprising the following steps:
(1) obtaining edge points of a single pixel of an image;
(2) according to the constraint of the gradient, connecting the adjacent edge points when and only when the difference value between the gradient tangential direction of the searched adjacent edge points and the extension direction of the tracked straight line segment is less than a threshold value to obtain the straight line segment; the method comprises the following steps: the extension direction angle of the line segment is estimated in the edge tracking process by using the following formula
In the formula, GiThe gradient amplitude at the i-th point, θiThe gradient tangent angle of the ith point, if denoted Gi xAnd Gi yThe gradient values of the i point in the transverse direction and the longitudinal direction respectively have
When a new edge point is connected, the extending direction of the primary line segment is estimated again until no new edge point meeting the direction condition exists;
(3) for the obtained initial straight line segment, namely an edge segment, the ratio of the distance l between two end points of the initial straight line segment to the maximum distance d between each point on the edge segment and a straight line connecting the two end points is calculated, the ratio represents the significance degree of the edge segment as a straight line, and whether the edge segment is divided into a plurality of straight line segments as a curve segment is judged; performing the segmentation if any of the two edge segments that are obtained by breaking the edge segment furthest away from a line connecting the two end points is more pronounced than the primary edge; repeating the steps until a more obvious edge section cannot be obtained after segmentation, or the number of pixels contained in the two edge sections obtained after segmentation is less than a set threshold value TL; and
(4) and (5) judging straight line segments, and removing false straight line segments.
2. The method for fast detecting edge tracking straight line segment based on gradient direction constraint as claimed in claim 1, wherein the (1) further comprises:
(1.1) smoothing and filtering the image by adopting a 7 multiplied by 7 Gaussian kernel, and then performing gradient calculation by adopting a 3-order Sobel operator;
(1.2) after obtaining the image gradient, carrying out threshold processing to obtain a pixel set only containing edge information in the image, eliminating small gradient amplitude points,
in the formula: rho is a gradient amplitude threshold value, q is a maximum gray level error possibly brought in 256-level gray level quantization, tau represents an allowed maximum error in a gradient direction, and all image points with gradient amplitudes smaller than the threshold value are ignored in the subsequent processing process;
(1.3) carrying out non-maximum suppression treatment on the obtained result along the gradient direction to obtain a refined edge, wherein only the pixel points which are local maximum values in the gradient direction are reserved through the non-maximum suppression, and the rest pixel points are removed to obtain the edge of a single pixel.
3. The method for fast detecting edge-tracking straight-line segment based on gradient direction constraint as claimed in claim 1, wherein the step (2) further comprises:
pseudo-ordering the gradient magnitude values to select seed points: and after the pseudo-sorting is finished, selecting the seed points from the grade with the highest gradient value, and sequentially descending until all grades are traversed.
4. The method for fast detecting edge tracking straight line segment based on gradient direction constraint as claimed in claim 1, wherein the step (4) further comprises:
counting the number of edge points of which the phase directions are approximately vertical to the fitted straight line direction on the candidate straight line segment, and converting the elimination of the false straight line segment into an exception for judging whether the false straight line segment is an unstructured background noise hypothesis; selecting a local background area, namely a linear support area, for each linear segment, wherein the area is selected as a rectangle, the center of the rectangle is selected as the weighted average of all points on the linear segment, the main direction is the direction of the eigenvector corresponding to the maximum eigenvalue of the covariance matrix of the weighted distribution of all points, wherein the weight of each point is the gradient amplitude of the point, the length and the width of the rectangle are respectively selected as values when the rectangle just can completely cover all the points on the corresponding linear segment, and in addition, the calculated center and the main direction of the rectangle are also used as the fitting result of the linear segment passing through the center of the rectangle along the main direction of the rectangle.
5. The method for fast detecting edge tracking straight line segment based on gradient direction constraint as claimed in claim 4, wherein the step (4) further comprises: the background model is assumed to be a white noise model, that is, the phase direction of each pixel point in the image is [0, 2 pi ]]The effective straight line segment is a region with consistent phase direction height, otherwise, the effective straight line segment is a false straight line segment, in the N multiplied by M image, the number of all possible straight lines is C (N multiplied by M, 2), the width of the straight line support region is changed from 1 to N, and the number of all possible straight line support regions is (N multiplied by M)2.5(ii) a The false alarm number for generating the image point phase direction distribution in the specific straight line support region is
In the formula: n represents the total number of points in the linear support region, k represents the number of points in the linear support region that coincide with the linear direction, p is determined by an angle threshold, the local principal direction of an image point is set as its gradient tangent, NFLess than or equal to the straight line segment is accepted, NF>The straight line segments of (a) are eliminated as false straight line segments, and are threshold values set according to experience, and B (n, k, p) is a binomial distribution cumulative probability.
6. The method for fast detecting edge tracking straight line segment based on gradient direction constraint as claimed in claim 5, wherein the step (4) further comprises:
before one candidate straight line segment is eliminated, the width of the rectangle is gradually increased by taking 1 pixel as a step size, and N is performed againFUntil the determination result is true or the set maximum allowable width T is reachedWIf the decision result is always false, then N is reservedFThe width value with the minimum value is used for further judgment by adjusting the p value;
after the rectangle width adjustment is finished, taking five values of p/2, p/4, p/8, p/16 and p/32 in sequence except p for each candidate line segment to carry out NFAnd (4) if the result of the calculation of any one of the five values is true, accepting the straight line segment.
7. A computer-readable storage medium having stored thereon computer-executable instructions which, when executed by a processor, implement the steps in the method of any one of claims 1 to 6.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010780646.1A CN111652825B (en) | 2020-08-06 | 2020-08-06 | Edge tracking straight line segment rapid detection device and method based on gradient direction constraint |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010780646.1A CN111652825B (en) | 2020-08-06 | 2020-08-06 | Edge tracking straight line segment rapid detection device and method based on gradient direction constraint |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111652825A CN111652825A (en) | 2020-09-11 |
| CN111652825B true CN111652825B (en) | 2020-11-13 |
Family
ID=72346131
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010780646.1A Active CN111652825B (en) | 2020-08-06 | 2020-08-06 | Edge tracking straight line segment rapid detection device and method based on gradient direction constraint |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111652825B (en) |
Families Citing this family (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112446894B (en) * | 2020-12-04 | 2024-03-26 | 沈阳工业大学 | Image segmentation method based on direction space |
| CN112257717B (en) * | 2020-12-22 | 2021-03-30 | 之江实验室 | Straight line, ellipse and intersection point identification and positioning method for uncorrected image |
| CN112950627B (en) * | 2021-04-01 | 2023-01-20 | 上海柏楚电子科技股份有限公司 | Detection and control method and system for laser cutting |
| CN116485904A (en) * | 2023-03-26 | 2023-07-25 | 重庆大学 | Improved mobile robot EDLines line segment detection method based on image gradient threshold calculation |
| CN117274288B (en) * | 2023-09-27 | 2024-05-03 | 河海大学 | Shaft part sub-pixel edge detection method based on improved LSD algorithm |
Family Cites Families (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20060098877A1 (en) * | 2004-11-09 | 2006-05-11 | Nick Barnes | Detecting shapes in image data |
| CN105261031B (en) * | 2015-11-26 | 2019-01-15 | 四川汇源光通信有限公司 | The line detection method and device calculated based on gradient |
| CN109978878A (en) * | 2019-04-04 | 2019-07-05 | 厦门商集网络科技有限责任公司 | Color image line segment detecting method and its system based on LSD |
| CN110288624A (en) * | 2019-06-28 | 2019-09-27 | 苏州大学 | Detection method, device and the relevant device of straightway in a kind of image |
| CN111445510A (en) * | 2020-03-24 | 2020-07-24 | 杭州东信北邮信息技术有限公司 | Method for detecting straight line in image |
-
2020
- 2020-08-06 CN CN202010780646.1A patent/CN111652825B/en active Active
Also Published As
| Publication number | Publication date |
|---|---|
| CN111652825A (en) | 2020-09-11 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111652825B (en) | Edge tracking straight line segment rapid detection device and method based on gradient direction constraint | |
| Hoang et al. | Metaheuristic optimized edge detection for recognition of concrete wall cracks: a comparative study on the performances of roberts, prewitt, canny, and sobel algorithms | |
| CN116168026A (en) | Water quality detection method and system based on computer vision | |
| Lestriandoko et al. | Circle detection based on hough transform and Mexican Hat filter | |
| Hwang et al. | A practical algorithm for the retrieval of floe size distribution of Arctic sea ice from high-resolution satellite Synthetic Aperture Radar imagery | |
| Pirzada et al. | Analysis of edge detection algorithms for feature extraction in satellite images | |
| EP3953691A1 (en) | Methods and systems for crack detection using a fully convolutional network | |
| AlAzawee et al. | Using morphological operations—Erosion based algorithm for edge detection | |
| CN115953421A (en) | Harris honeycomb vertex extraction method for detecting regularity of honeycomb structure | |
| CN117474918B (en) | Abnormality detection method and device, electronic device, and storage medium | |
| CN102509265B (en) | Digital image denoising method based on gray value difference and local energy | |
| CN114511575A (en) | Image segmentation positioning-assisted point cloud registration-based high-reflectivity object grabbing method | |
| CN117788472A (en) | Method for judging corrosion degree of rivet on surface of aircraft skin based on DBSCAN algorithm | |
| CN112950594A (en) | Method and device for detecting surface defects of product and storage medium | |
| Bode et al. | Bounded: Neural boundary and edge detection in 3d point clouds via local neighborhood statistics | |
| Scharfenberger et al. | Image saliency detection via multi-scale statistical non-redundancy modeling | |
| Shetty | Circle detection in images | |
| CN112686222B (en) | Method and system for detecting ship target by satellite-borne visible light detector | |
| Li et al. | Graph network refining for pavement crack detection based on multiscale curvilinear structure filter | |
| Hernández | Structural analysis of textures based on LAW´ s filters | |
| CN104156696A (en) | Bi-directional-image-based construction method for quick local changeless feature descriptor | |
| MAARIR et al. | Building detection from satellite images based on curvature scale space method | |
| Zhang et al. | A level set method with heterogeneity filter for side-scan sonar image segmentation | |
| Zhang | A critical overview of image segmentation techniques based on transition region | |
| Najgebauer et al. | Representation of edge detection results based on graph theory |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| CP03 | Change of name, title or address |
Address after: Room 411, 4th floor, main building, No. 835 and 937, Dangui Road, China (Shanghai) pilot Free Trade Zone, Pudong New Area, Shanghai, 200131 Patentee after: Shanghai hailichuang Technology Co.,Ltd. Address before: 201203 Room 411, 4th Floor, Main Building (1 Building) of Zhangjiang Guochuang Center, 899 Dangui Road, Pudong New Area, Shanghai Patentee before: SHANGHAI HYNITRON MICROELECTRONIC Co.,Ltd. |
|
| CP03 | Change of name, title or address |