CN112598665A - Method and device for detecting vanishing points and vanishing lines of Manhattan scene - Google Patents
Method and device for detecting vanishing points and vanishing lines of Manhattan scene Download PDFInfo
- Publication number
- CN112598665A CN112598665A CN202011632601.6A CN202011632601A CN112598665A CN 112598665 A CN112598665 A CN 112598665A CN 202011632601 A CN202011632601 A CN 202011632601A CN 112598665 A CN112598665 A CN 112598665A
- Authority
- CN
- China
- Prior art keywords
- edge
- line
- response
- straight line
- vanishing
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/0002—Inspection of images, e.g. flaw detection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F17/00—Digital computing or data processing equipment or methods, specially adapted for specific functions
- G06F17/10—Complex mathematical operations
- G06F17/18—Complex mathematical operations for evaluating statistical data, e.g. average values, frequency distributions, probability functions, regression analysis
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N99/00—Subject matter not provided for in other groups of this subclass
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/13—Edge detection
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Computational Mathematics (AREA)
- Mathematical Optimization (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Mathematical Analysis (AREA)
- General Engineering & Computer Science (AREA)
- Pure & Applied Mathematics (AREA)
- Bioinformatics & Computational Biology (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Evolutionary Biology (AREA)
- Life Sciences & Earth Sciences (AREA)
- Quality & Reliability (AREA)
- Operations Research (AREA)
- Probability & Statistics with Applications (AREA)
- Computing Systems (AREA)
- Algebra (AREA)
- Databases & Information Systems (AREA)
- Image Analysis (AREA)
Abstract
The application discloses a method and a device for detecting a disappearing point and a disappearing line of a Manhattan scene, wherein the method comprises the steps of calculating the difference of line integrals of two adjacent straight lines in the same direction and the same length as the straight line aiming at each straight line, calculating the edge response of the straight line according to the difference, obtaining the disappearing line according to the edge response of each straight line and a non-maximum value inhibition method, establishing a Bayesian probability model of Manhattan scene parameters of an image to be detected, searching the position of the disappearing point in the vertical direction in the image to be detected, determining the position of the disappearing point in the horizontal direction in a Hough transformation mode, initializing the Manhattan scene parameters according to the position of the disappearing point in the horizontal direction, and optimizing the initialized Manhattan scene parameters based on the Bayesian probability model and a simulated annealing algorithm to. According to the technical scheme, the straight line reflecting the structural information in the image to be detected can be robustly detected, and the globally optimal vanishing point can be detected to avoid falling into local optimization.
Description
Technical Field
The application relates to the technical field of perspective analysis, in particular to a method and a device for detecting a vanishing point and a vanishing line of a Manhattan scene.
Background
The vanishing point and the vanishing line are important features in many scenes, and therefore, the vanishing point and the vanishing line need to be detected.
In the stage of line detection and extraction, the existing method is usually based on a Canny edge detection mode, and the extracted lines in a complex scene with low signal-to-noise ratio or a large amount of interference contain a large amount of noise, so that the lines containing scene structure information are difficult to extract.
In the vanishing point estimation modeling stage, the grouping of the detected straight lines and the vanishing point calculation are generally realized by a heuristic rule in the existing method, so that the local optimization is easy to fall into.
Disclosure of Invention
The application provides a method and a device for detecting vanishing points and vanishing lines of a Manhattan scene, which can robustly detect straight lines reflecting structural information in an image to be detected, can detect globally optimal vanishing points and avoid falling into local optimization.
In a first aspect, the present application provides a method for detecting a vanishing point and a vanishing line of a manhattan scene, including:
acquiring an image to be detected;
calculating line integrals of all straight lines in the image to be detected, calculating the difference value of the line integrals of two adjacent straight lines in the same direction and the same length as the straight line for each straight line, calculating the edge response of the straight line according to the difference value, and obtaining a vanishing line according to the edge response of each straight line and a non-maximum suppression method;
establishing a Bayesian probability model of Manhattan scene parameters of the image to be detected;
searching the position of a vertical vanishing point in the image to be detected, determining the position of a horizontal vanishing point in a Hough transform mode, and initializing the Manhattan scene parameters according to the position of the horizontal vanishing point;
and optimizing the initialized Manhattan scene parameters based on the Bayesian probability model and the simulated annealing algorithm to obtain the positions of vanishing points.
Optionally, the step of calculating the line integrals of all the straight lines in the image to be detected includes:
for the two-dimensional data g (x, y) in the image to be detected, the method uses
Define a length L and a center point of
Normalized line integral with direction θ:
wherein x is the abscissa of the center point of the straight line, y is the ordinate of the center point of the straight line,
the linear edge response method is characterized in that the linear edge response method is a method for obtaining linear edge response, and comprises the following steps of obtaining linear integral, obtaining sampling positions along a straight line, obtaining two-dimensional image data, obtaining linear integral, obtaining linear edge response, and obtaining linear edge response.
Optionally, the step of calculating, for each straight line, a difference between line integrals of two adjacent straight lines having the same direction and length as the straight line, and calculating an edge response of the straight line according to the difference includes:
said calculating being performed for each lineEdge directional response θ is located
The edge response of the line is:
when the calculated edge direction response theta is located at
The edge response of the line is:
wherein G is the edge response of the line,
is the coordinate of the center point of the straight line, L is the length of the straight line, theta is the direction of the edge response of the straight line, x is the abscissa of the center point of the straight line, y is the ordinate of the center point of the straight line, and F is the line integral of the straight line.
Optionally, the step of obtaining a vanishing line according to the edge response of each straight line and the non-maximum suppression method includes:
and selecting a straight line corresponding to the edge response which is locally maximum in space and angle from all the straight lines as a vanishing line according to the edge response of all the straight lines and the non-maximum value suppression method.
Optionally, the step of establishing a bayesian probability model of manhattan scene parameters of the image to be detected includes:
wherein the content of the first and second substances,
for random variables, selection
In the direction of the Manhattan vanishing line, eiFor the edge intensity response and the edge direction response of pixel i,
given Manhattan scene parameter phi, multi-scale edge
And corresponding Manhattan direction selection
After that, a pixel edge response e is observediThe conditional probability of (a) of (b),
observing multi-scale edge information after giving Manhattan scene parameters and corresponding Manhattan directions
The conditional probability of (a) of (b),
a priori selected for manhattan direction.
Optionally, the
Is defined as:
edge intensity response e of pixeliResponse to edge direction thetaiThe conditions are independent of each other,
the decomposition is as follows:
the above-mentioned
Is defined as:
wherein the pixel edge intensity response eiIs proportional to the intensity of the edge response after normalization, aeDefining the maximum value of the coefficient term, and defining the maximum value as the quantile of all edge response distribution histograms;
the above-mentioned
Is defined as:
wherein, delta theta is the gradient direction theta of the edge pixel and the multi-scale edge detection where the edge pixel is
Difference of zeta in linear direction, Pang(δ θ) increases with increasing absolute value of δ θ.
Optionally, the
Is defined as:
the above-mentioned
The decomposition is as follows:
the above-mentioned
Is defined as:
wherein the content of the first and second substances,
is multiscale edge detection
Beta is a predefined coefficient,
a response that is a multi-scale edge;
the above-mentioned
Is defined as:
where δ ζ represents an angle difference between the vanishing point to the edge detection center point and the edge detection direction, ρ is a parameter for adjusting the degree of smoothness, x is a horizontal coordinate of the straight center point, y is a vertical coordinate of the straight center point, ζ is a straight direction, and l is a length of the detected multi-scale edge response.
Optionally, the step of searching for the position of the vertical vanishing point in the image to be detected includes:
taking (w/2, h/2-10h) and (w/2, h/2+10h) as candidate points of the vanishing points in the vertical direction in the image to be detected, calculating the distance between the candidate points and the multi-scale edge detection result, taking the candidate points corresponding to the distance within a preset threshold range as the inner points of the vanishing points in the vertical direction, and obtaining the positions of the vanishing points in the vertical direction according to the edge response of all the inner points and least square fitting, wherein w is the width of the image to be detected, and h is the height of the image to be detected.
Optionally, the step of optimizing the initialized manhattan scene parameters based on the bayesian probability model and the simulated annealing algorithm to obtain the position of the vanishing point includes:
when the value of the Manhattan scene parameter is an initial value, setting the value of a tau parameter of the Bayesian probability model to be larger than a first preset value, setting the value of a rho parameter to be smaller than a second preset value, and determining the maximum value of the Manhattan scene parameter by using a non-gradient optimization method, wherein tau is the error tolerance of the side trend, and rho is a parameter for adjusting the smoothness degree;
and taking the maximum value as an initial value, reducing the value of the tau parameter according to a preset reduction amount, increasing the value of the rho parameter according to a preset increase amount, determining the maximum value of the Manhattan scene parameter by using a non-gradient optimization method, returning to the step of taking the maximum value as the initial value, and finishing optimization to obtain the position of a vanishing point until the Manhattan scene parameter meeting the preset precision requirement is obtained.
In a second aspect, the present application provides a detection apparatus for a man-hattan scene vanishing point and vanishing line, including:
the acquisition module is used for acquiring an image to be detected;
the vanishing line detection module is used for calculating the line integrals of all the straight lines in the image to be detected, calculating the difference value of the line integrals of two adjacent straight lines in the same direction and the same length as the straight line aiming at each straight line, calculating the edge response of the straight line according to the difference value, and obtaining the vanishing line according to the edge response of each straight line and a non-maximum suppression method;
the establishing module is used for establishing a Bayesian probability model of the Manhattan scene parameters of the image to be detected;
the initialization module is used for searching the position of a vertical vanishing point in the image to be detected, determining the position of a horizontal vanishing point in a Hough transform mode, and initializing the Manhattan scene parameters according to the position of the horizontal vanishing point;
and the vanishing point detection module is used for optimizing the initialized Manhattan scene parameters based on the Bayesian probability model and the simulated annealing algorithm to obtain the positions of the vanishing points.
Optionally, the vanishing line detecting module is specifically configured to:
for the two-dimensional data g (x, y) in the image to be detected, the method uses
Define a length L and a center point of
Normalized line integral with direction θ:
wherein x is the abscissa of the center point of the straight line, y is the ordinate of the center point of the straight line,
the coordinate gamma of the center point of the straight line is an integral variable element, meaning that the line integral obtains a sampling position along the straight line, g is two-dimensional image data, F is the line integral of the straight line, theta is the response direction of the edge of the straight line, and L is the length of the straight line
Optionally, the vanishing line detecting module is specifically configured to:
when the calculated edge direction response theta is located at the position of each straight line
The edge response of the line is:
when the calculated edge direction response theta is located at
The edge response of the line is:
wherein G is the edge response of the line,
is the coordinate of the center point of the straight line, L is the length of the straight line, theta is the direction of the edge response of the straight line, x is the abscissa of the center point of the straight line, y is the ordinate of the center point of the straight line, and F is the line integral of the straight line.
Optionally, the vanishing line detecting module is specifically configured to:
and selecting a straight line corresponding to the edge response which is locally maximum in space and angle from all the straight lines as a vanishing line according to the edge response of all the straight lines and the non-maximum value suppression method.
Optionally, the establishing module is specifically configured to:
the conditional probability of the pixel edge response in the image to be detected is as follows:
wherein the content of the first and second substances,
for random variables, selection
In the direction of the Manhattan vanishing line, eiFor the edge intensity response and the edge direction response of pixel i,
given Manhattan scene parameter phi, multi-scale edge
And corresponding Manhattan direction selection
After that, a pixel edge response e is observediThe conditional probability of (a) of (b),
observing multi-scale edge information after giving Manhattan scene parameters and corresponding Manhattan directions
The conditional probability of (a) of (b),
a priori selected for manhattan direction.
Optionally, the
Is defined as:
edge intensity response e of pixeliResponse to edge direction thetaiThe conditions are independent of each other,
the decomposition is as follows:
the above-mentioned
Is defined as:
wherein the pixel edge intensity response eiIs proportional to the intensity of the edge response after normalization, aeDefining the maximum value of the coefficient term, and defining the maximum value as the quantile of all edge response distribution histograms;
the above-mentioned
Is defined as:
wherein, delta theta is the gradient direction theta of the edge pixel and the multi-scale edge detection where the edge pixel is
Difference of zeta in linear direction, Pang(δ θ) increases with increasing absolute value of δ θ.
Optionally, the
Is defined as:
the above-mentioned
The decomposition is as follows:
the above-mentioned
Is defined as:
wherein the content of the first and second substances,
is multiscale edge detection
Beta is a predefined coefficient,
a response that is a multi-scale edge;
the above-mentioned
Is defined as:
where δ ζ represents an angle difference between the vanishing point to the edge detection center point and the edge detection direction, ρ is a parameter for adjusting the degree of smoothness, x is a horizontal coordinate of the straight center point, y is a vertical coordinate of the straight center point, ζ is a straight direction, and l is a length of the detected multi-scale edge response.
Optionally, the initialization module is specifically configured to:
taking (w/2, h/2-10h) and (w/2, h/2+10h) as candidate points of the vanishing points in the vertical direction in the image to be detected, calculating the distance between the candidate points and the multi-scale edge detection result, taking the candidate points corresponding to the distance within a preset threshold range as the inner points of the vanishing points in the vertical direction, and obtaining the positions of the vanishing points in the vertical direction according to the edge response of all the inner points and least square fitting, wherein w is the width of the image to be detected, and h is the height of the image to be detected.
Optionally, the vanishing point detecting module includes:
a first determining unit, configured to set, when the value of the manhattan scene parameter is an initial value, a value of a τ parameter of the bayesian probability model to be greater than a first preset value, and a value of a ρ parameter to be smaller than a second preset value, and determine a maximum value of the manhattan scene parameter by using a non-gradient optimization method, where τ is an error tolerance of a side trend, and ρ is a parameter for adjusting a smoothness degree;
and the second determining unit is used for taking the maximum value as an initial value, reducing the value of the tau parameter according to a preset reduction amount, increasing the value of the rho parameter according to a preset increase amount, determining the maximum value of the Manhattan scene parameter by using a non-gradient optimization method, returning to trigger, taking the maximum value as the initial value, and obtaining the position of a vanishing point after optimization is finished until the Manhattan scene parameter meeting the preset precision requirement is obtained.
In a third aspect, the present application provides a readable medium comprising executable instructions, which when executed by a processor of an electronic device, perform the method according to any of the first aspect.
In a fourth aspect, the present application provides an electronic device comprising a processor and a memory storing execution instructions, wherein when the processor executes the execution instructions stored in the memory, the processor performs the method according to any one of the first aspect.
According to the technical scheme, the image to be detected can be obtained, the line integrals of all straight lines in the image to be detected are calculated, the difference value of the line integrals of the two adjacent straight lines in the same direction and the same length as the straight lines is calculated for each straight line, the edge response of the straight line is calculated according to the difference value, the vanishing line is obtained according to the edge response of each straight line and a non-maximum suppression method, a Bayesian probability model of Manhattan scene parameters of the image to be detected is established, the position of the vanishing point in the vertical direction is searched in the image to be detected, the position of the vanishing point in the horizontal direction is determined in a Hough transform mode, the Manhattan scene parameters are initialized according to the position of the vanishing point in the horizontal direction, and the initialized Manhattan scene parameters are optimized based on the Bayesian probability model and a simulated annealing algorithm to obtain. In the technical scheme of the application, the multi-scale line integration technology is applied to straight line extraction, straight lines reflecting structural information in an image to be detected can be robustly detected, a Bayesian model considering all straight line pixel characteristics and integral Manhattan hypothesis constraint is adopted, optimization is carried out by a simulated annealing method, globally optimal vanishing points can be detected, and the situation that the vanishing points are locally optimal is avoided.
Further effects of the above-mentioned unconventional preferred modes will be described below in conjunction with specific embodiments.
Drawings
In order to more clearly illustrate the embodiments or prior art solutions of the present application, the drawings needed for describing the embodiments or prior art will be briefly described below, and it is obvious that the drawings in the following description are only some embodiments described in the present application, and that other drawings can be obtained by those skilled in the art without inventive exercise.
Fig. 1 is a schematic flow chart of a method for detecting a vanishing point and a vanishing line of a manhattan scene in an embodiment of the present application;
FIG. 2 is a schematic diagram of a vanishing point;
FIG. 3 is a graph illustrating the variation of the objective function with respect to τ;
fig. 4 is a schematic structural diagram of a device for detecting a vanishing point and a vanishing line of a manhattan scene in an embodiment of the present application;
fig. 5 is a schematic structural diagram of an electronic device according to an embodiment of the present application.
Detailed Description
In order to make the objects, technical solutions and advantages of the present application more apparent, the technical solutions of the present application will be described in detail and completely with reference to the following embodiments and accompanying drawings. It should be apparent that the described embodiments are only some of the embodiments of the present application, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.
The method aims to solve the problems that in the prior art, straight lines containing scene structure information cannot be extracted from a complex scene with low signal-to-noise ratio or a large amount of interference, and vanishing point detection falls into local optimization.
The application provides a method for detecting vanishing points and vanishing lines of a Manhattan scene, in the method, an image to be detected is obtained, line integrals of all straight lines in the image to be detected are calculated, the difference value of the line integrals of two adjacent straight lines in the same direction and the same length as the straight line is calculated for each straight line, calculating to obtain the edge response of the straight line according to the difference, obtaining a vanishing line according to the edge response of each straight line and a non-maximum suppression method, establishing a Bayesian probability model of the Manhattan scene parameters of the image to be detected, searching the position of a vertical vanishing point in an image to be detected, determining the position of a horizontal vanishing point in a Hough transform mode, initializing the Manhattan scene parameters according to the position of the horizontal vanishing point, and optimizing the initialized Manhattan scene parameters based on a Bayesian probability model and a simulated annealing algorithm to obtain the position of the vanishing point. In the technical scheme of the application, the multi-scale line integration technology is applied to straight line extraction, straight lines reflecting structural information in an image to be detected can be robustly detected, a Bayesian model considering all straight line pixel characteristics and integral Manhattan hypothesis constraint is adopted, optimization is carried out by a simulated annealing method, globally optimal vanishing points can be detected, and the situation that the vanishing points are locally optimal is avoided.
Various non-limiting embodiments of the present application are described in detail below with reference to the accompanying drawings.
Referring to fig. 1, a method for detecting a vanishing point and a vanishing line of a manhattan scene in an embodiment of the present application is shown. In this embodiment, the method is applied to an electronic device, and may include, for example, the steps of:
s101: and acquiring an image to be detected.
The manhattan assumption is a scene prior for artificial scenes, considering that edges in a scene are mainly distributed along three mutually perpendicular directions in a three-dimensional space, and the scene is composed of a plurality of mutually orthogonal planes, such as a cuboid-shaped building and most of indoor space. After the assumption is put forward, the method is widely applied to the fields of three-dimensional reconstruction of artificial scenes, scene understanding, camera calibration, camera self-positioning and the like.
The embodiment of the invention provides a method for detecting the vanishing points and the vanishing lines of a Manhattan scene on the basis of the Manhattan hypothesis.
S102: calculating the line integrals of all straight lines in the image to be detected, calculating the difference value of the line integrals of two adjacent straight lines with the same direction and the same length as the straight line for each straight line, calculating the edge response of the straight line according to the difference value, and obtaining the vanishing line according to the edge response of each straight line and a non-maximum value inhibition method.
The inventor finds that line integration can be applied to structured line detection in a low signal-to-noise ratio and a complex scene, so that the inventor applies the method to detection of a lost line and a lost point in a Manhattan scene to solve the problem that a line containing scene structure information cannot be extracted in a complex scene with a low signal-to-noise ratio or a large amount of interference in the prior art. Specifically, after the image to be detected is obtained, the line integrals of all straight lines in the image to be detected need to be calculated, and then the multi-scale line integrals are calculated.
Wherein, the above-mentioned line integral of all straight lines in calculating the image to be detected can include:
for two-dimensional data g (x, y) in the image to be detected, the method
Define a length L and a center point of
Normalized line integral with direction θ:
wherein x is the abscissa of the center point of the straight line, y is the ordinate of the center point of the straight line,
the coordinate of the center point of the straight line, gamma is an integral variable, meaning the sampling position of the line integral along the straight line, g is two-dimensional image data, F is the line integral of the straight line, theta is the response direction of the edge of the straight line, and L is the length of the straight line.
Therefore, the line integral calculation of all positions, directions and lengths on the image to be detected can be realized within a certain quantification error range within O (nlogn) (n is the number of pixels).
The calculation of the multi-scale line integral includes the following two steps:
1. edge detection by calculating the difference between line integrals
In the embodiment of the invention, the difference value of the integral of adjacent lines in the same direction and the same length is used for obtaining the edge response with the corresponding length in the image to be detected. Specifically, for each straight line, the difference of line integrals of two adjacent straight lines in the same direction and the same length as the straight line is calculated, and the edge response of the straight line is calculated according to the difference.
Illustratively, an edge response near one line integral may be defined as half the difference of the co-directional line integrals on adjacent sides. The distance between the line integrals on two spatially adjacent sides may be 2 pixels. In particular, the direction θ defining the edge response to be calculated is in
In between, considering the sign of the edge intensity response, the edge direction response in the range of 2 pi can be obtained.
That is, the calculating, for each straight line, a difference between line integrals of two adjacent straight lines having the same length and direction as the straight line, and calculating an edge response of the straight line according to the difference may include:
when the calculated edge direction response theta is located at the position of each straight line
The edge response of the line is:
when the calculated edge direction response theta is located at
The edge response of the line is:
wherein G is the edge response of the line,
is the coordinate of the center point of the straight line, L is the length of the straight line, theta is the direction of the edge response of the straight line, x is the abscissa of the center point of the straight line, y is the ordinate of the center point of the straight line, and F is the line integral of the straight line.
The edge responses described above are all edge responses at (x, y).
2. Non-maximum suppression
The multi-scale edges obtained only by the first step have similar high response values at similar positions, angles and lengths, which are not beneficial for subsequent calculation. Therefore, in the embodiment of the invention, by non-maximum suppression, only the spatially and angularly maximum edge intensity response value is reserved, and the others are set to be 0. That is, the obtaining of the vanishing line based on the edge response of each straight line and the non-maximum suppression method may include:
and selecting a straight line corresponding to the edge response which is locally maximum in space and angle from all the straight lines as a vanishing line according to the edge response of all the straight lines and the non-maximum value suppression method.
Illustratively, for adjacent edge strength responses of the same length and the same direction, if the edge strength responses of two adjacent sides are higher than the current edge strength response, the current edge strength response is not retained. And if the edge strength response of the center point with the same length in the adjacent direction of the same pixel point is higher than the current edge strength response, not retaining the current edge strength response. Through non-maximum suppression, the length, the position and the angle of a structured straight line in an image to be detected can be obtained more accurately.
S103: establishing a Bayesian probability model of Manhattan scene parameters of an image to be detected.
The multi-scale edge detection mode can robustly extract the straight line containing the main structural information in the image to be detected in a complex scene with low signal-to-noise ratio, but the corresponding angle of the straight line still has quantization error. This quantization error has a negative impact on the manhattan vanishing point estimate when performing manhattan vanishing point detection. In order to simultaneously guarantee the robustness and accuracy of the Manhattan vanishing point estimation, a multi-scale edge-pixel two-stage Bayes probability model is established in the embodiment of the invention, the observed multi-scale edge and pixel edge response is modeled, and the Manhattan vanishing point estimation and the position and angle of the multi-scale edge are jointly optimized.
Wherein, step S103 may include:
the conditional probability of the pixel edge response in the image to be detected is as follows:
wherein the content of the first and second substances,
for random variables, selection
In the direction of the Manhattan vanishing line,
for a given Manhattan scene parameter phi, multiple scalesEdge of degree
And corresponding Manhattan direction selection
After that, a pixel edge response e is observediThe conditional probability of (a) of (b),
observing multi-scale edge information after giving Manhattan scene parameters and corresponding Manhattan directions
The conditional probability of (a) of (b),
apriori selected for manhattan direction, eiFor the edge intensity response e of pixel iiAnd edge direction response.
The total likelihood probability for pixels in the image to be detected within the linear edge detection range is:
where E is the set of all edge pixels of the image, P (E | φ) is the conditional probability of observing E given the current Manhattan scene parameters, P (E [ + ])i| φ) is the probability of observing a single edge pixel response, φ is a Manhattan scene parameter, eiFor the edge intensity response e of pixel iiAnd edge directional response thetaiI.e. ei＝(ei，θi)。
The posterior probability corresponding to phi is:
P(φ|E)∝P(E|φ)P(φ)
where φ is a Manhattan scene parameter, E is a set of all edge pixels of the image, P (E | φ) is a conditional probability of observing E given the current Manhattan scene parameter, and P (φ) is a prior probability.
Manhattan scene parameters, namely three Manhattan main directions of a camera coordinate system. Since the three principal directions are perpendicular to each other, the manhattan scene can be represented by a rotation of the manhattan principal direction relative to the camera cartesian coordinate system, the rotation being represented by a three-dimensional euler angle (α, β, γ). If the focal length f of the camera is unknown, the parameters are represented by 4-dimensional vectors (alpha, beta, gamma, f) consisting of the Euler angles and the focal length.
Including length, line segment center position, angle.
Is that
The manhattan direction in which the hidden variable is selected,
respectively represent
From one of the three main directions of manhattan,
to represent
From other non-manhattan directions.
The following definitions of the likelihood and prior terms are provided: defining the likelihood and prior terms of the probability model.
wherein the edge intensity response e of pixel iiResponse to edge direction thetaiThe conditions are independent of each other,
given Manhattan scene parameter phi, multi-scale edge
And corresponding Manhattan direction selection
After that, a pixel edge response e is observediThe conditional probability of (a) of (b),
as a response of edge strength eiThe conditional probability of (a) of (b),
is an edge direction response thetaiThe conditional probability of (2).
Wherein the edge strength responds to eiThe conditional probability of (a) is:
i.e. the pixel edge intensity response e is observediIs proportional to the edge intensity response after normalization. Intuitively, the edge response term can be regarded as a coefficient term depending on the edge response. Alpha is alphaeThe maximum value of the coefficient term is defined as a constant, defined as the quantile of all edge response distribution histograms. In the present invention, a 0.90 quantile is used.
Edge directional response θiThe conditional probability of (a) is:
wherein, delta theta is the gradient direction theta of the edge pixel and the multi-scale edge detection where the edge pixel is
Difference in linear direction ζ. Intuitively, Pang(delta theta) is different between the two values by [ -T, T]When the absolute value of the difference between the two is [ T, 2T ]]When it is taken
Is a smooth function for the transition from t to e; the other ranges are then given as e. Pang(δ θ) increases with increasing absolute value of δ θ. τ is the error tolerance of the trend of the edge, and e and t are constants, and the probability can meet the normalization condition only by taking appropriate values.
The multiscale edge conditional probability term is defined as follows:
the response intensity items divided into multi-scale edges and other parameter items comprise the length, the position and the angle of the response intensity items,
is the response of the multi-scale edge.
The directional response is defined as follows:
fig. 2 is a schematic diagram of vanishing points, and fig. 2 shows the intuitive meaning of the above definition. δ ζ represents an angle difference between the vanishing point to the edge detection center point, and the direction of edge detection. ζ is the linear direction and l is the length of the detected multi-scale edge response. The directional response is related to the straight line segment center position (x, y), the straight line length l, and the straight line direction ζ. ρ is a parameter for adjusting the degree of smoothness of the term.
As can be seen in fig. 2, intuitively, the directional response is defined as the degree of deviation of the straight line segment to the vanishing point direction. This variable is related to the straight line segment center position (x, y), the straight line length l, and the straight line direction ζ. Gamma is a parameter for adjusting the degree of smoothness of the term.
Finally, a priori
Defined as a multi-term distribution resulting from statistical data.
1,2,3,4, the probability is 0.2,0.2,0.2,0.4, respectively. P (φ) is defined as a uniform distribution.
S104: searching the position of a vertical vanishing point in an image to be detected, determining the position of a horizontal vanishing point in a Hough transform mode, and initializing Manhattan scene parameters according to the position of the horizontal vanishing point.
The model in step S103 contains both continuous and discrete variables, and the optimization of the corresponding variables constitutes a complex non-convex optimization problem. Therefore, the embodiment of the invention provides a parameter initialization method, which obtains a better solution of all parameters, namely, searching the position of the vertical vanishing point in the image to be detected, determining the position of the horizontal vanishing point in the horizontal direction by means of hough transform, and initializing manhattan scene parameters according to the position of the horizontal vanishing point.
Specifically, the initialization process includes:
a. searching for vertical vanishing points:
considering that, in most natural images, the degree of coincidence between the vertical direction and the image vertical direction is high, the vertical direction vanishing point is searched first.
Specifically, the searching for the position of the vertical vanishing point in the image to be detected may include:
taking w/2, h/2-10h) and (w/2, h/2+10h) as candidate points of the vertical vanishing point in the image to be detected, and calculating the candidate points and the multi-scale edge detection result, namely the candidate points and the multi-scale edge detection result
Taking the candidate points corresponding to the distance within the preset threshold range as the inner points of the vertical vanishing point, and obtaining the position of the vertical vanishing point according to the edge response of all the inner points and least square fitting, wherein w is the width of the image to be detected, h is the height of the image to be detected, and δ ζ represents the angle difference between the vanishing point and the edge detection center point and the edge detection direction.
b. And for two vanishing points in the horizontal direction, obtaining candidate points of the horizontal vanishing points in a Hough transform mode.
Specifically, for vanishing points expressed in polar coordinates
Wherein the content of the first and second substances,
and r is the polar angle and the polar diameter of the polar coordinate respectively, and a histogram of the polar coordinate parameters is established. Wherein the content of the first and second substances,
the size of the histogram grid of (1) is 2 pi/n, and the size of the histogram grid of r is log (r)/m, namely, the histogram grid is uniformly divided in a logarithmic space. And m and n are constants and respectively control the size of the division grid. And counting the contributions of other edge pixels to the histogram except for the inner point edge of the vertical direction vanishing point, and obtaining the candidate point of the horizontal direction vanishing point.
In the process of obtaining candidate points of the horizontal direction vanishing points, constraints imposed by the manhattan scene on the positions of the three vanishing points, namely the positions of the vertical direction vanishing points and the positions of the two horizontal direction vanishing points, are further considered. Specifically, the vertical vanishing point and the horizontal vanishing point can be used to calculate the other horizontal vanishing point position. Therefore, the N vanishing point configurations with the maximum sum of the horizontal two vanishing point histogram counts are selected to initialize the N Manhattan scene parameters. In the next step, for each Manhattan scene parameter, a simulated annealing algorithm is operated to further optimize the parameter, and the parameter value with the maximum posterior probability is obtained as a final solution.
S105: and optimizing the initialized Manhattan scene parameters based on a Bayesian probability model and a simulated annealing algorithm to obtain the positions of vanishing points.
The better solutions of all the parameters are obtained in the step S104, and the objective function corresponding to the Bayesian probability model of the Manhattan parameters obtained in the step S104 is very uneven and has a large number of local optima. The embodiment of the invention adopts a simulated annealing algorithm to solve the problem. Namely, based on a Bayesian probability model and a simulated annealing algorithm, the initialized Manhattan scene parameters are optimized to obtain the positions of vanishing points.
The optimizing the initialized manhattan scene parameters based on the bayesian probability model and the simulated annealing algorithm to obtain the position of the vanishing point may include:
setting Bayesian probability model when value of Manhattan scene parameter is initial value
The value of the parameter tau in (1) is greater than a first preset value, setting
Determining the maximum value of the Manhattan scene parameters by using a non-gradient optimization method, wherein tau is the error tolerance of the edge trend, and rho is a parameter for adjusting the smoothness degree;
and taking the maximum value as an initial value, reducing the value of the tau parameter according to a preset reduction amount, increasing the value of the rho parameter according to a preset increase amount, determining the maximum value of the Manhattan scene parameter by using a non-gradient optimization method, returning to the step of taking the maximum value as the initial value, and finishing the optimization until the Manhattan scene parameter meeting the preset precision requirement is obtained, so as to obtain the position of the vanishing point.
Intuitively, in the initial stage of simulated annealing solution, the solution space curved surface formed by the objective function is smoothed by adjusting the parameters of the algorithm, so that the solution can escape from the local optimal value and more thoroughly cover the search space, and as the solution is gradually carried out, the parameters of the algorithm are adjusted to enable the objective function to approach the true value, so that the algorithm can converge to the global optimal solution.
In particular, by adjusting
Parameter of and
the p parameter in (2) to adjust the smoothness of the objective function.
Referring to fig. 3, fig. 3 is a graph illustrating the variation of the objective function with respect to τ. The left graph τ is 0.5, the middle graph τ is 0.05, and the right graph τ is 0.02. As can be seen from fig. 3, the objective function in the right graph is quite unsmooth, but its minimum point coincides with the middle graph. The objective function in the left graph is quite flat and smooth, but the minimum points have deviated from the middle and right graphs.
In summary, the method for detecting the vanishing points and the vanishing lines of the Manhattan scene can acquire an image to be detected, calculate the line integrals of all straight lines in the image to be detected, calculate the difference value of the line integrals of two adjacent straight lines in the same direction and the same length as the straight line for each straight line, calculating to obtain the edge response of the straight line according to the difference, obtaining a vanishing line according to the edge response of each straight line and a non-maximum suppression method, establishing a Bayesian probability model of the Manhattan scene parameters of the image to be detected, searching the position of a vertical vanishing point in an image to be detected, determining the position of a horizontal vanishing point in a Hough transform mode, initializing the Manhattan scene parameters according to the position of the horizontal vanishing point, and optimizing the initialized Manhattan scene parameters based on a Bayesian probability model and a simulated annealing algorithm to obtain the position of the vanishing point. In the technical scheme of the application, the multi-scale line integration technology is applied to straight line extraction, straight lines reflecting structural information in an image to be detected can be robustly detected, a Bayesian model considering all straight line pixel characteristics and integral Manhattan hypothesis constraint is adopted, optimization is carried out by a simulated annealing method, globally optimal vanishing points can be detected, and the situation that the vanishing points are locally optimal is avoided.
Meanwhile, the embodiment of the invention realizes an edge detection and non-maximum suppression mode of edge line integral response on the basis of line integration. And setting the definition of a multi-scale edge-pixel two-stage Bayesian probability model. An initialization solving mode that vertical direction vanishing points are searched firstly and then horizontal direction vanishing points are searched by utilizing a logarithm space construction histogram is proposed. By adjusting
Parameter of and
the rho parameter in the simulation model is used for adjusting the smoothness degree of the target function, and further an optimization mode based on simulated annealing solution is achieved.
Fig. 4 shows a specific embodiment of a device for detecting a vanishing point and a vanishing line of a manhattan scene according to the present application. The apparatus of this embodiment is a physical apparatus for executing the method of the above embodiment. The technical solution is essentially the same as that in the above embodiment, and the corresponding description in the above embodiment is also applicable to this embodiment. The device in this embodiment includes:
an obtaining module 401, configured to obtain an image to be detected;
a vanishing line detecting module 402, configured to calculate line integrals of all straight lines in the image to be detected, calculate, for each straight line, a difference between line integrals of two adjacent straight lines that are the same direction and length as the straight line, calculate an edge response of the straight line according to the difference, and obtain a vanishing line according to the edge response of each straight line and a non-maximum suppression method;
an establishing module 403, configured to establish a bayesian probability model of manhattan scene parameters of the image to be detected;
an initialization module 404, configured to search for a position of a vertical vanishing point in the image to be detected, determine a position of a horizontal vanishing point in a hough transform manner, and initialize the manhattan scene parameter according to the position of the horizontal vanishing point;
and a vanishing point detecting module 405, configured to optimize the initialized manhattan scene parameters based on the bayesian probability model and the simulated annealing algorithm to obtain a position of a vanishing point.
The device can obtain an image to be detected, line integrals of all straight lines in the image to be detected are calculated, for each straight line, the difference value of the line integrals of two adjacent straight lines in the same direction and the same length as the straight line is calculated, the edge response of the straight line is calculated according to the difference value, vanishing lines are obtained according to the edge response of each straight line and a non-maximum value suppression method, a Bayesian probability model of Manhattan scene parameters of the image to be detected is established, the position of a vertical vanishing point is searched in the image to be detected, the position of a horizontal vanishing point is determined in a Hough transform mode, the Manhattan scene parameters are initialized according to the position of the horizontal vanishing point, and the initialized Manhattan scene parameters are optimized based on the Bayesian probability model and a simulated annealing algorithm to obtain the position of the vanishing point. In the technical scheme of the application, the multi-scale line integration technology is applied to line extraction, the lines reflecting the structural information in the image to be detected can be robustly detected, the Bayesian model considering all line pixel characteristics and integral Manhattan hypothesis constraint is adopted to optimize by the simulated annealing method, the globally optimal vanishing point can be detected, and the situation that the image is trapped in the locally optimal vanishing point is avoided
In an implementation manner, the vanishing line detecting module 402 may be specifically configured to:
for the two-dimensional data g (x, y) in the image to be detected, the method uses
Define a length L and a center point of
Normalized line integral with direction θ:
wherein x is the abscissa of the center point of the straight line, y is the ordinate of the center point of the straight line,
is the coordinate of the center point of a straight line, gamma is an integral variable, meaning the sampling position of line integral along the straight line, g is two-dimensional image data, and F is the line of the straight lineIntegral, theta is the direction of the linear edge response, and L is the length of the line
In an implementation manner, the vanishing line detecting module 402 may be specifically configured to:
when the calculated edge direction response theta is located at the position of each straight line
The edge response of the line is:
when the calculated edge direction response theta is located at
The edge response of the line is:
wherein G is the edge response of the line,
is the coordinate of the center point of the straight line, L is the length of the straight line, theta is the direction of the edge response of the straight line, x is the abscissa of the center point of the straight line, y is the ordinate of the center point of the straight line, and F is the line integral of the straight line.
In an implementation manner, the vanishing line detecting module 402 may be specifically configured to:
and selecting a straight line corresponding to the edge response which is locally maximum in space and angle from all the straight lines as a vanishing line according to the edge response of all the straight lines and the non-maximum value suppression method.
In an implementation manner, the establishing module 403 may specifically be configured to:
the conditional probability of the pixel edge response in the image to be detected is as follows:
wherein the content of the first and second substances,
for random variables, selection
In the direction of the Manhattan vanishing line, eiFor the edge intensity response and the edge direction response of pixel i,
given Manhattan scene parameter phi, multi-scale edge
And corresponding Manhattan direction selection
After that, a pixel edge response e is observediThe conditional probability of (a) of (b),
observing multi-scale edge information after giving Manhattan scene parameters and corresponding Manhattan directions
The conditional probability of (a) of (b),
a priori selected for manhattan direction.
In one implementation, the
Is defined as:
edge intensity response e of pixeliResponse to edge direction thetaiThe conditions are independent of each other,
the decomposition is as follows:
the above-mentioned
Is defined as:
wherein the pixel edge intensity response eiIs proportional to the intensity of the edge response after normalization, aeDefining the maximum value of the coefficient term, and defining the maximum value as the quantile of all edge response distribution histograms;
the above-mentioned
Is defined as:
wherein, delta theta is the gradient direction theta of the edge pixel and the multi-scale edge detection where the edge pixel is
Difference of zeta in linear direction, Pang(δ θ) increases with increasing absolute value of δ θ.
In one implementation, the
Is defined as:
the above-mentioned
The decomposition is as follows:
the above-mentioned
Is defined as:
wherein the content of the first and second substances,
is multiscale edge detection
Beta is a predefined coefficient,
a response that is a multi-scale edge;
the above-mentioned
Is defined as:
where δ ζ represents an angle difference between the vanishing point to the edge detection center point and the edge detection direction, ρ is a parameter for adjusting the degree of smoothness, x is a horizontal coordinate of the straight center point, y is a vertical coordinate of the straight center point, ζ is a straight direction, and l is a length of the detected multi-scale edge response.
In an implementation manner, the initialization module 404 may be specifically configured to:
taking (w/2, h/2-10h) and (w/2, h/2+10h) as candidate points of the vanishing points in the vertical direction in the image to be detected, calculating the distance between the candidate points and the multi-scale edge detection result, taking the candidate points corresponding to the distance within a preset threshold range as the inner points of the vanishing points in the vertical direction, and obtaining the positions of the vanishing points in the vertical direction according to the edge response of all the inner points and least square fitting, wherein w is the width of the image to be detected, and h is the height of the image to be detected.
In one implementation, the vanishing point detecting module 405 may include:
a first determining unit, configured to set, when the value of the manhattan scene parameter is an initial value, a value of a τ parameter of the bayesian probability model to be greater than a first preset value, and a value of a ρ parameter to be smaller than a second preset value, and determine a maximum value of the manhattan scene parameter by using a non-gradient optimization method, where τ is an error tolerance of a side trend, and ρ is a parameter for adjusting a smoothness degree;
and the second determining unit is used for taking the maximum value as an initial value, reducing the value of the tau parameter according to a preset reduction amount, increasing the value of the rho parameter according to a preset increase amount, determining the maximum value of the Manhattan scene parameter by using a non-gradient optimization method, returning to trigger, taking the maximum value as the initial value, and obtaining the position of a vanishing point after optimization is finished until the Manhattan scene parameter meeting the preset precision requirement is obtained.
Fig. 5 is a schematic structural diagram of an electronic device according to an embodiment of the present application. On the hardware level, the electronic device comprises a processor and optionally an internal bus, a network interface and a memory. The Memory may include a Memory, such as a Random-Access Memory (RAM), and may further include a non-volatile Memory, such as at least 1 disk Memory. Of course, the electronic device may also include hardware required for other services.
The processor, the network interface, and the memory may be connected to each other via an internal bus, which may be an ISA (Industry Standard Architecture) bus, a PCI (Peripheral Component Interconnect) bus, an EISA (Extended Industry Standard Architecture) bus, or the like. The bus may be divided into an address bus, a data bus, a control bus, etc. For ease of illustration, only one double-headed arrow is shown in FIG. 5, but this does not indicate only one bus or one type of bus.
And the memory is used for storing the execution instruction. In particular, a computer program that can be executed by executing instructions. The memory may include both memory and non-volatile storage and provides execution instructions and data to the processor.
In a possible implementation manner, the processor reads the corresponding execution instruction from the nonvolatile memory to the memory and then runs the execution instruction, and may also obtain the corresponding execution instruction from other devices, so as to form a manhattan scene vanishing point and vanishing line detection apparatus on a logical level. The processor executes the execution instruction stored in the memory, so that the Manhattan scene vanishing point and vanishing line detection method provided by any embodiment of the application is realized through the executed execution instruction.
The method for detecting the vanishing points and vanishing lines of the manhattan scene provided by the embodiment shown in fig. 1 of the present application can be applied to a processor, or implemented by the processor. The processor may be an integrated circuit chip having signal processing capabilities. In implementation, the steps of the above method may be performed by integrated logic circuits of hardware in a processor or instructions in the form of software. The Processor may be a general-purpose Processor, including a Central Processing Unit (CPU), a Network Processor (NP), and the like; but also Digital Signal Processors (DSPs), Application Specific Integrated Circuits (ASICs), Field Programmable Gate Arrays (FPGAs) or other Programmable logic devices, discrete Gate or transistor logic devices, discrete hardware components. The various methods, steps, and logic blocks disclosed in the embodiments of the present application may be implemented or performed. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like.
The steps of the method disclosed in connection with the embodiments of the present application may be directly implemented by a hardware decoding processor, or implemented by a combination of hardware and software modules in the decoding processor. The software module may be located in ram, flash memory, rom, prom, or eprom, registers, etc. storage media as is well known in the art. The storage medium is located in a memory, and a processor reads information in the memory and completes the steps of the method in combination with hardware of the processor.
The embodiment of the present application further provides a readable storage medium, where the readable storage medium stores an execution instruction, and when the stored execution instruction is executed by a processor of an electronic device, the electronic device can execute the method for detecting a man-hattan scene vanishing point and a vanishing line provided in any embodiment of the present application.
The electronic device described in the foregoing embodiments may be a computer.
It will be apparent to those skilled in the art that embodiments of the present application may be provided as a method or computer program product. Accordingly, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects.
The embodiments in the present application are described in a progressive manner, and the same and similar parts among the embodiments can be referred to each other, and each embodiment focuses on the differences from the other embodiments. In particular, as for the apparatus embodiment, since it is substantially similar to the method embodiment, the description is relatively simple, and for the relevant points, reference may be made to the partial description of the method embodiment.
It should also be noted that the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other like elements in a process, method, article, or apparatus that comprises the element.
The above description is only an example of the present application and is not intended to limit the present application. Various modifications and changes may occur to those skilled in the art. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included in the scope of the claims of the present application.
Claims (10)
1. A detection method for a lost point and a lost line of a Manhattan scene is characterized by comprising the following steps:
acquiring an image to be detected;
calculating line integrals of all straight lines in the image to be detected, calculating the difference value of the line integrals of two adjacent straight lines in the same direction and the same length as the straight line for each straight line, calculating the edge response of the straight line according to the difference value, and obtaining a vanishing line according to the edge response of each straight line and a non-maximum suppression method;
establishing a Bayesian probability model of Manhattan scene parameters of the image to be detected;
searching the position of a vertical vanishing point in the image to be detected, determining the position of a horizontal vanishing point in a Hough transform mode, and initializing the Manhattan scene parameters according to the position of the horizontal vanishing point;
and optimizing the initialized Manhattan scene parameters based on the Bayesian probability model and the simulated annealing algorithm to obtain the positions of vanishing points.
2. The method according to claim 1, wherein the step of calculating the line integrals of all the straight lines in the image to be detected comprises:
for the two-dimensional data g (x, y) in the image to be detected, the method uses
Define a length L and a center point of
Normalized line integral with direction θ:
wherein x is the abscissa of the central point, y is the ordinate of the central point,
the coordinate of the center point of the straight line, gamma is an integral variable, meaning the sampling position of the line integral along the straight line, g is two-dimensional image data, F is the line integral of the straight line, theta is the response direction of the edge of the straight line, and L is the length of the straight line.
3. The method of claim 2, wherein the step of calculating, for each straight line, a difference between line integrals of two adjacent straight lines having the same direction and length as the straight line, and calculating an edge response of the straight line according to the difference comprises:
when the calculated edge direction response theta is located at the position of each straight line
The edge response of the line is:
when the calculated edge direction response theta is located at
The edge response of the line is:
wherein G is the edge response of the line,
is the coordinate of the center point of the straight line, L is the length of the straight line, theta is the direction of the edge response of the straight line, x is the abscissa of the center point of the straight line, y is the ordinate of the center point of the straight line, and F is the line integral of the straight line.
4. The method of claim 1, wherein the step of obtaining a vanishing line based on the edge response of each line and the non-maximum suppressing method comprises:
and selecting a straight line corresponding to the edge response which is locally maximum in space and angle from all the straight lines as a vanishing line according to the edge response of all the straight lines and the non-maximum value suppression method.
5. The method of claim 1, wherein the step of establishing a bayesian probabilistic model of manhattan scene parameters of the image to be detected comprises:
the conditional probability of the pixel edge response in the image to be detected is as follows:
wherein the content of the first and second substances,
for random variables, selection
In the direction of the Manhattan vanishing line, eiIs the edge intensity of pixel iThe response and the edge direction response are shown,
given Manhattan scene parameter phi, multi-scale edge
And corresponding Manhattan direction selection
After that, a pixel edge response e is observediThe conditional probability of (a) of (b),
observing multi-scale edge information after giving Manhattan scene parameters and corresponding Manhattan directions
The conditional probability of (a) of (b),
a priori selected for manhattan direction.
6. The method of claim 5, wherein the step of applying the coating comprises applying a coating to the substrate
Is defined as:
edge intensity response e of pixeliResponse to edge direction thetaiThe conditions are independent of each other,
the decomposition is as follows:
the above-mentioned
Is defined as:
wherein the pixel edge intensity response eiIs proportional to the intensity of the edge response after normalization, aeDefining the maximum value of the coefficient term, and defining the maximum value as the quantile of all edge response distribution histograms;
the above-mentioned
Is defined as:
wherein, delta theta is the gradient direction theta of the edge pixel and the multi-scale edge detection where the edge pixel is
Difference of zeta in linear direction, Pang(δ θ) increases with increasing absolute value of δ θ.
7. The method of claim 5, wherein the step of applying the coating comprises applying a coating to the substrate
Is defined as:
the above-mentioned
The decomposition is as follows:
the above-mentioned
Is defined as:
wherein the content of the first and second substances,
is multiscale edge detection
Beta is a predefined coefficient,
a response that is a multi-scale edge;
the above-mentioned
Is defined as:
where δ ζ represents an angle difference between the vanishing point to the edge detection center point and the edge detection direction, ρ is a parameter for adjusting the degree of smoothness, x is a horizontal coordinate of the straight center point, y is a vertical coordinate of the straight center point, ζ is a straight direction, and l is a length of the detected multi-scale edge response.
8. The method according to claim 1, wherein the step of searching the image to be detected for the position of the vertical vanishing point comprises:
taking (w/2, h/2-10h) and (w/2, h/2+10h) as candidate points of the vanishing points in the vertical direction in the image to be detected, calculating the distance between the candidate points and the multi-scale edge detection result, taking the candidate points corresponding to the distance within a preset threshold range as the inner points of the vanishing points in the vertical direction, and obtaining the positions of the vanishing points in the vertical direction according to the edge response of all the inner points and least square fitting, wherein w is the width of the image to be detected, and h is the height of the image to be detected.
9. The method according to claim 1, wherein the step of optimizing the initialized manhattan scene parameters based on the bayesian probability model and the simulated annealing algorithm to obtain the position of the vanishing point comprises:
when the value of the Manhattan scene parameter is an initial value, setting the value of a tau parameter of the Bayesian probability model to be larger than a first preset value, setting the value of a rho parameter to be smaller than a second preset value, and determining the maximum value of the Manhattan scene parameter by using a non-gradient optimization method, wherein tau is the error tolerance of the side trend, and rho is a parameter for adjusting the smoothness degree;
and taking the maximum value as an initial value, reducing the value of the tau parameter according to a preset reduction amount, increasing the value of the rho parameter according to a preset increase amount, determining the maximum value of the Manhattan scene parameter by using a non-gradient optimization method, returning to the step of taking the maximum value as the initial value, and finishing optimization to obtain the position of a vanishing point until the Manhattan scene parameter meeting the preset precision requirement is obtained.
10. A detection device for a lost point and a lost line of a Manhattan scene is characterized by comprising:
the acquisition module is used for acquiring an image to be detected;
the vanishing line detection module is used for calculating the line integrals of all the straight lines in the image to be detected, calculating the difference value of the line integrals of two adjacent straight lines in the same direction and the same length as the straight line aiming at each straight line, calculating the edge response of the straight line according to the difference value, and obtaining the vanishing line according to the edge response of each straight line and a non-maximum suppression method;
the establishing module is used for establishing a Bayesian probability model of the Manhattan scene parameters of the image to be detected;
the initialization module is used for searching the position of a vertical vanishing point in the image to be detected, determining the position of a horizontal vanishing point in a Hough transform mode, and initializing the Manhattan scene parameters according to the position of the horizontal vanishing point;
and the vanishing point detection module is used for optimizing the initialized Manhattan scene parameters based on the Bayesian probability model and the simulated annealing algorithm to obtain the positions of the vanishing points.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011632601.6A CN112598665B (en) | 2020-12-31 | 2020-12-31 | Method and device for detecting vanishing points and vanishing lines of Manhattan scene |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011632601.6A CN112598665B (en) | 2020-12-31 | 2020-12-31 | Method and device for detecting vanishing points and vanishing lines of Manhattan scene |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112598665A true CN112598665A (en) | 2021-04-02 |
| CN112598665B CN112598665B (en) | 2022-05-06 |
Family
ID=75206894
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011632601.6A Active CN112598665B (en) | 2020-12-31 | 2020-12-31 | Method and device for detecting vanishing points and vanishing lines of Manhattan scene |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112598665B (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117132499A (en) * | 2023-09-07 | 2023-11-28 | 石家庄开发区天远科技有限公司 | Background removing method and device for image recognition |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104063711A (en) * | 2014-06-23 | 2014-09-24 | 西北工业大学 | Corridor vanishing point rapid detection algorithm based on K-means method |
| CN104392451A (en) * | 2014-11-28 | 2015-03-04 | 河海大学 | Artificial scene image end point detection method |
| US20150199573A1 (en) * | 2014-01-10 | 2015-07-16 | Mitsubishi Electric Research Laboratories, Inc. | Global Scene Descriptors for Matching Manhattan Scenes using Edge Maps Associated with Vanishing Points |
| CN109446917A (en) * | 2018-09-30 | 2019-03-08 | 长安大学 | A kind of vanishing Point Detection Method method based on cascade Hough transform |
-
2020
- 2020-12-31 CN CN202011632601.6A patent/CN112598665B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20150199573A1 (en) * | 2014-01-10 | 2015-07-16 | Mitsubishi Electric Research Laboratories, Inc. | Global Scene Descriptors for Matching Manhattan Scenes using Edge Maps Associated with Vanishing Points |
| CN104063711A (en) * | 2014-06-23 | 2014-09-24 | 西北工业大学 | Corridor vanishing point rapid detection algorithm based on K-means method |
| CN104392451A (en) * | 2014-11-28 | 2015-03-04 | 河海大学 | Artificial scene image end point detection method |
| CN109446917A (en) * | 2018-09-30 | 2019-03-08 | 长安大学 | A kind of vanishing Point Detection Method method based on cascade Hough transform |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117132499A (en) * | 2023-09-07 | 2023-11-28 | 石家庄开发区天远科技有限公司 | Background removing method and device for image recognition |
| CN117132499B (en) * | 2023-09-07 | 2024-05-14 | 石家庄开发区天远科技有限公司 | Background removing method and device for image recognition |
Also Published As
| Publication number | Publication date |
|---|---|
| CN112598665B (en) | 2022-05-06 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US9303525B2 (en) | Method and arrangement for multi-camera calibration | |
| US8411966B2 (en) | Estimation of image relations from point correspondences between images | |
| EP2907082B1 (en) | Using a probabilistic model for detecting an object in visual data | |
| Manzanera et al. | Line and circle detection using dense one-to-one Hough transforms on greyscale images | |
| CN108875731B (en) | Target identification method, device, system and storage medium | |
| Vera et al. | Hough Transform for real-time plane detection in depth images | |
| Xu et al. | Distributed RANSAC for the robust estimation of three-dimensional reconstruction | |
| JP6202147B2 (en) | Curve detection method and curve detection apparatus | |
| CN106373128B (en) | Method and system for accurately positioning lips | |
| CN107025647B (en) | Image tampering evidence obtaining method and device | |
| CN111915657A (en) | Point cloud registration method and device, electronic equipment and storage medium | |
| CN112598665B (en) | Method and device for detecting vanishing points and vanishing lines of Manhattan scene | |
| Hofer et al. | Semi-Global 3D Line Modeling for Incremental Structure-from-Motion. | |
| WO2023016182A1 (en) | Pose determination method and apparatus, electronic device, and readable storage medium | |
| CN109840529B (en) | Image matching method based on local sensitivity confidence evaluation | |
| CN116310053A (en) | Determination method, rendering method, device and medium for light intensity distribution in scene rendering | |
| CN107464257B (en) | Wide base line matching method and device | |
| CN112446428B (en) | Image data processing method and device | |
| CN111681237B (en) | Image registration method, device, computer equipment and storage medium | |
| CN113658156A (en) | Sphere fitting method and device for removing local outliers in depth image | |
| CN109389032B (en) | Picture authenticity determining method and device, electronic equipment and readable storage medium | |
| CN114596320A (en) | Image segmentation method and device based on ALSLCV model | |
| CN113706505A (en) | Cylinder fitting method and device for removing local outliers in depth image | |
| Ziomek et al. | Evaluation of interest point detectors in presence of noise | |
| CN110728285B (en) | Rapid corner detection method based on dynamic frequency conversion |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |