CN115661267A - Monocular distance measurement model calibration method, electronic equipment and curtain wall robot - Google Patents
Monocular distance measurement model calibration method, electronic equipment and curtain wall robot Download PDFInfo
- Publication number
- CN115661267A CN115661267A CN202211403530.1A CN202211403530A CN115661267A CN 115661267 A CN115661267 A CN 115661267A CN 202211403530 A CN202211403530 A CN 202211403530A CN 115661267 A CN115661267 A CN 115661267A
- Authority
- CN
- China
- Prior art keywords
- line
- calibration
- horizontal
- calibration line
- vertical
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Landscapes
- Length Measuring Devices By Optical Means (AREA)
- Image Analysis (AREA)
Abstract
The invention provides a monocular distance measurement model calibration method, electronic equipment and a curtain wall robot, and relates to the technical field of robot vision, wherein a calibration image is obtained through a monocular camera of the curtain wall robot; obtaining an edge graph of a calibration line and linear line segment data of the edge of the calibration line based on the calibration image; determining a first calibration line and a second calibration line based on the linear segment data, obtaining a first parameter corresponding to the first calibration line based on the first calibration line, and obtaining a second parameter corresponding to the second calibration line based on the second calibration line; determining coordinate conversion model parameters based on the first parameters and the second parameters; and calculating the distance of each line segment based on the edge graph of the calibration line and the coordinate conversion model parameters, and obtaining a calibration result according to the distance of each line segment. The curtain wall robot can automatically identify the calibration line and calculate the model parameters after being adsorbed on the glass curtain wall, so that the data is more accurate and more suitable for the working environment; the parameters needing manual measurement are reduced, and the automation degree of the calibration process is increased.
Description
Technical Field
The invention relates to the technical field of robot vision, in particular to a monocular distance measurement model calibration method, electronic equipment and a curtain wall robot.
Background
Robots have gradually merged into people's lives, providing different types of services or functions. Such as a robot for cleaning glass curtain walls. In these robot sensing systems, monocular vision can provide a planar sensing area at a relatively low cost, and therefore, low-precision vision or multi-perception fusion is an option.
The monocular camera is calibrated, and a geometric model imaged by the camera is constructed according to the mutual relation between the three-dimensional geometric position of a certain point on the surface of the space object and the corresponding point in the image. As a technique of stereoscopic vision measurement, camera calibration directly affects subsequent distance measurement and even three-dimensional reconstruction, and thus an efficient and convenient calibration method is required.
The existing monocular distance measurement calibration method is mainly a Zhangyingyou calibration method, and the method calibrates parameters of each step in the process according to the process of converting a real world coordinate system into a pixel coordinate system of an image when a camera takes a picture. The world coordinate system is converted into a camera coordinate system through rigid body transformation, then converted into an image coordinate system through perspective projection, and finally converted into a pixel coordinate system through affine transformation, wherein the affine transformation and the perspective projection can be integrated into an internal reference matrix and a rigid body transformation external reference matrix of the camera.
The Zhangyingyou calibration method needs to prepare a black and white chessboard pattern calibration board, and take pictures of the calibration board at different positions and angles after the camera is fixed. And then calculating the posture of the calibration board according to the known size of the checkerboard by using MATLAB or an autonomously written program, and calculating each parameter on the internal reference matrix and the external reference matrix according to the pixel characteristics of the calibration board at different positions and different postures on the image.
However, such a calibration method has problems in the calibration of the curtain wall robot: firstly, the parameter calculation is complex, and a plurality of parameters of an internal parameter matrix and an external parameter matrix need to be calculated in the calibration process; the calibration process is high in professional, the quality of the photos needs to be evaluated, the photos of the corresponding postures of the calibration plate need to be supplemented and adjusted, an external parameter matrix corresponding to the plane of the glass curtain wall needs to be calculated according to information such as the installation height of a camera, the automation degree of the model parameter calibration process is low, and the requirement on the professional degree of workers is high; and secondly, the calibration process is carried out in a checkerboard calibration plate, which is not favorable for the situation that the curtain wall robot feeds back actual work.
Disclosure of Invention
The invention provides a monocular distance measurement model calibration method, electronic equipment and a curtain wall robot, and aims to solve the problems that in the prior art, the calibration process of the curtain wall robot is complex and the feedback of the curtain wall robot to actual work is not facilitated.
The invention provides a monocular distance measurement model calibration method of a curtain wall robot, which comprises the following steps: obtaining a calibration image through a monocular camera of the curtain wall robot; the calibration image comprises a preset calibration line on the glass curtain wall; obtaining an edge graph of a calibration line and linear line segment data of the edge of the calibration line based on the calibration image; determining a first calibration line and a second calibration line based on the linear segment data, obtaining a first parameter corresponding to the first calibration line based on the first calibration line, and obtaining a second parameter corresponding to the second calibration line based on the second calibration line; the first calibration line and the second calibration line form a preset angle; determining coordinate conversion model parameters based on the first parameters and the second parameters; and calculating the distance of each line segment based on the edge graph of the calibration line and the coordinate conversion model parameters, and obtaining a calibration result according to the distance of each line segment.
According to the monocular distance measurement model calibration method for the curtain wall robot, provided by the invention, based on linear segment data, a first calibration line and a second calibration line are determined, a first parameter corresponding to the first calibration line is obtained based on the first calibration line, and a second parameter corresponding to the second calibration line is obtained based on the second calibration line, and the monocular distance measurement model calibration method for the curtain wall robot comprises the following steps: determining a vertical calibration line based on the linear segment data, and obtaining a vertical parameter corresponding to the vertical calibration line based on the vertical calibration line; and determining a horizontal calibration line based on the linear segment data, and obtaining a horizontal parameter corresponding to the horizontal calibration line based on the horizontal calibration line.
According to the monocular distance measurement model calibration method for the curtain wall robot, provided by the invention, the vertical calibration line is determined based on the linear segment data, and the vertical parameter corresponding to the vertical calibration line is obtained based on the vertical calibration line, and the method comprises the following steps: judging straight line characteristics based on the straight line segment data, and determining a plurality of vertical line segments; carrying out weighted fusion on the plurality of vertical line segments to obtain vertical lines; and identifying the weighted and fused vertical line, determining the left vertical calibration line and the right vertical calibration line, and obtaining the slope of the left vertical calibration line, the intercept of the left vertical calibration line, the slope of the right vertical calibration line and the intercept of the right vertical calibration line.
According to the monocular distance measurement model calibration method for the curtain wall robot, provided by the invention, the vertical calibration line is determined based on the linear segment data, and the vertical parameter corresponding to the vertical calibration line is obtained based on the vertical calibration line, and the method comprises the following steps: obtaining two end points (x) in each straight line segment 1 ，y 1 ）、（x 2 ，y 2 ) Corresponding to a first difference on the y-axis; if the first difference is larger than a first preset value, the y-axis straight line segment corresponding to the first difference is a vertical line segment; if the slope difference value between the vertical line segments is smaller than a second preset value, determining the corresponding vertical line segment as the edge of the same calibration line; based on a weighted average method, fusing the edges of the same calibration line to obtain a vertical line; taking the straight line with the negative and maximum slope in the vertical lines as a left vertical calibration line, and obtaining the slope and intercept corresponding to the left vertical calibration line; and taking the straight line with the positive and minimum slope in the vertical lines as the right vertical calibration line, and obtaining the slope and the intercept corresponding to the right vertical calibration line.
According to the monocular distance measurement model calibration method for the curtain wall robot, provided by the invention, the horizontal calibration line is determined based on the linear segment data, and the horizontal parameter corresponding to the horizontal calibration line is obtained based on the horizontal calibration line, and the method comprises the following steps: judging straight line characteristics based on the straight line segment data, and determining a plurality of horizontal line segments; carrying out weighted fusion on the horizontal line segments to obtain a horizontal line; identifying the weighted and fused horizontal lines, determining a first horizontal calibration line and a second horizontal calibration line, and obtaining a y-axis value of the first horizontal calibration line and a y-axis value of the second horizontal calibration line; wherein the y-axis value of the first horizontal calibration line and the y-axis value of the second horizontal calibration line are horizontal parameters.
According to the monocular distance measurement model calibration method for the curtain wall robot, provided by the invention, the horizontal calibration line is determined based on the linear segment data, and the horizontal parameter corresponding to the horizontal calibration line is obtained based on the horizontal calibration line, and the method comprises the following steps: obtaining two end points (x) in each straight line segment 1 ，y 1 ）、（x 2 ，y 2 ) A first difference on the y-axis of (a); if the first difference is smaller than the third preset value, and x 1 、x 2 Within a preset range, determining a straight line segment corresponding to the first difference value as a horizontal line segment; obtaining a y-axis value of a horizontal line segment, an x-axis value and an x-axis difference between two end points; if the y-axis difference between the horizontal line segments is smaller than a fourth preset value, determining the corresponding horizontal line segment as the edge of the same calibration line; based on a weighted average method, fusing the edges of the same calibration line to obtain a horizontal line and a y-axis value corresponding to the horizontal line; taking the straight line with the maximum value of the y axis in the horizontal line as a first horizontal calibration line to obtain the value of the y axis of the first horizontal calibration line; and taking the straight line with the second largest value of the y axis in the horizontal line as a second horizontal calibration line to obtain the value of the y axis of the second horizontal calibration line.
According to the monocular distance measurement model calibration method for the curtain wall robot, provided by the invention, the coordinate transformation model parameters are determined based on the vertical parameters and the horizontal parameters, and the method comprises the following steps: determining a slope coefficient based on the slope of the left vertical calibration line and the slope of the right vertical calibration line; determining an intercept coefficient based on the intercept of the left vertical calibration line and the intercept of the right vertical calibration line; the coordinate conversion model parameters are determined based on the widths between the vertical calibration lines, the widths between the horizontal calibration lines, the slope coefficient, the intercept coefficient, and the horizontal parameters.
According to the monocular distance measurement model calibration method for the curtain wall robot, provided by the invention, the distance of each line segment is calculated based on the edge graph of the calibration line and the coordinate conversion model parameter, and the calibration result is obtained according to the distance of each line segment, and the method comprises the following steps: determining the horizontal distance between the horizontal line segment and the curtain wall robot and the vertical distance between the vertical line segment and the central axis of the curtain wall robot, and outputting the position and distance data of the linear line segment to obtain a simulated image; drawing a simulated calibration line position in the simulated image according to the vertical parameter and the horizontal parameter; and if the position of the simulation calibration line meets the first preset requirement and the simulation image meets the second preset requirement, determining that the monocular distance measuring model of the curtain wall robot is calibrated.
The invention also provides electronic equipment which comprises a memory, a processor and a computer program which is stored on the memory and can be run on the processor, wherein when the processor executes the program, the monocular distance measuring model calibration method of the curtain wall robot can be realized.
The invention also provides a curtain wall robot which comprises the monocular camera, the robot body and the electronic equipment.
According to the monocular distance measuring model calibration method, the electronic device and the curtain wall robot, the calibration image is obtained through the monocular camera of the curtain wall robot; the calibration image comprises a preset calibration line on the glass curtain wall; obtaining an edge graph of a calibration line and linear line segment data of the edge of the calibration line based on the calibration image; determining a first calibration line and a second calibration line based on the linear segment data, obtaining a first parameter corresponding to the first calibration line based on the first calibration line, and obtaining a second parameter corresponding to the second calibration line based on the second calibration line; determining coordinate conversion model parameters based on the first parameters and the second parameters; and calculating the distance of each line segment based on the edge graph of the calibration line and the coordinate conversion model parameters, and obtaining a calibration result according to the distance of each line segment. By the mode, the curtain wall robot can be calibrated, the robot can automatically identify the calibration line and calculate the model parameters after being adsorbed on the glass curtain wall, parameters needing manual measurement are reduced, the automation degree of the calibration process is increased, and the working efficiency of calibration during batch production is improved. The robot is calibrated in the running state of the glass curtain wall, so that the model parameters include the position deviation of the camera in the assembling process and the posture in the working state, and the data are more accurate and more accord with the working environment.
Drawings
In order to more clearly illustrate the technical solutions of the present invention or the prior art, the drawings needed for the description of the embodiments or the prior art will be briefly described below, and it is obvious that the drawings in the following description are some embodiments of the present invention, and those skilled in the art can also obtain other drawings according to the drawings without creative efforts.
FIG. 1 is a schematic flow chart of an embodiment of a monocular distance measurement model calibration method of a curtain wall robot according to the present invention;
FIG. 2 is a schematic diagram of an embodiment of a calibration image obtained after binarization processing according to the present invention;
FIG. 3 is a schematic diagram of an embodiment of an edge map of a calibration line according to the present invention;
FIG. 4 is a diagram illustrating an embodiment of a straight line extraction edge graph according to the present invention;
FIG. 5 is a schematic diagram of one embodiment of a simulated image according to the present invention;
fig. 6 is a schematic structural diagram of an embodiment of an electronic device according to the invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention clearer, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings, and it is obvious that the described embodiments are some, but not all embodiments of the present invention. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
The curtain wall robot is more and more applied, such as an overhead curtain wall cleaning robot, the outer vertical surface of a high-rise building can be cleaned in various types, the operation scene of the robot is different due to different building models cleaned each time, the calibration of the robot in the running state of the glass curtain wall can be realized by adopting the calibration method, and the data is more accurate and more suitable for the working environment.
Referring to fig. 1, fig. 1 is a schematic flowchart illustrating an embodiment of a monocular distance measuring model calibration method for a curtain wall robot according to the present invention, in this embodiment, the monocular distance measuring model calibration method for a curtain wall robot may include steps S110 to S150, which are as follows:
s110: and obtaining a calibration image through a monocular camera of the curtain wall robot.
The method of the embodiment is to place the curtain wall robot on the glass curtain wall for calibration. When the curtain wall robot is placed on the glass curtain wall, a calibration image can be obtained through a monocular camera of the curtain wall robot, wherein the calibration image comprises a preset calibration line on the glass curtain wall.
The preset calibration line is used for marking or pasting a calibration line and a check line on the glass curtain wall for testing and is used for automatically calibrating the precision of the model and the check model by the robot. After being set, the utility model can be used for a long time without repeated operation.
And arranging a calibration line for calibrating the model parameters by a user and a check line for checking whether the calibration result is accurate on the glass curtain wall. The test environment ensures an indoor light source with fixed position and brightness, and the environment has a single color tone. The calibration line segments and the inspection line segments may use colored tape or painted paint. The colors are red, green or blue, namely the colors corresponding to the three channels of R, G and B of the color image, so that the feature extraction is convenient. The two line segments are arranged according to the following principle.
For example, the calibration lines are two line segments perpendicular to the horizontal plane and one line segment parallel to the horizontal plane. Wherein the interval width of two perpendicular lines is the fuselage width of robot, and the moderate recognition effect in distance is better, also can provide direct position data for future direction of travel obstacle detection simultaneously. One horizontal line distance is from the blind area boundary of the monocular camera to 50-200mm forward, so that enough pixel characteristics are ensured.
For example, the check line is a line segment perpendicular to the horizontal plane and parallel to the horizontal plane. And the two inspection lines are line segments which keep a certain distance from the calibration line and are far away from the robot, and are used for completing the inspection ranging effect after the model calibration.
In addition, it should be noted that before obtaining the calibration image, the monocular camera principal point of the curtain wall robot needs to be measured. In some embodiments, the assay can be performed using MATLAB in conjunction with a checkerboard calibration plate. The camera principal point is the camera internal reference and is irrelevant to external conditions such as an installation position, so that the camera can be carried out after the camera arrives at a goods place, and the operation is not required to be carried out after the camera is installed on the curtain wall robot.
Measuring the camera principal point is equivalent to a simplified version of the zhangyingyou calibration method. After each monocular camera arrives at a good, the monocular camera is fixed, then the black and white checkerboard calibration plate is moved to different positions and different angles to take pictures, and then MATLAB is used for analyzing the pictures so as to calculate the camera principal point.
After the preset calibration line of the curtain wall robot is set and the camera principal point is determined, a worker prepares according to a normal working state, the worker prepares before completing the operation of the robot, aligns the robot to the calibration area, and after the robot is adsorbed on a glass curtain wall, the worker starts a calibration program, and the program automatically runs and feeds back a calibration result.
In such a way, the calibration process of the curtain wall robot is to simulate the working environment of the curtain wall robot, the data of the curtain wall robot in the running state of the glass curtain wall is recorded, the model parameters include the position deviation of the camera in the assembling process and the posture of the camera in the working state, and the data are more accurate and more accord with the working environment.
S120: and obtaining an edge graph of the calibration line and linear line segment data of the edge of the calibration line based on the calibration image.
Optionally, pixels corresponding to the preset calibration line color are extracted from the calibration image, and binarization processing is performed on the calibration image based on the pixels.
For example, pixels of corresponding colors are extracted according to the colors of the calibration lines, and binarization processing is performed, wherein the gray value of the pigment corresponding to the colors of the calibration lines is set to be 255, and the gray values of the pixels of other colors are set to be 0, so that the whole calibration image has a visual effect of only black and white. Referring to fig. 2, fig. 2 is a schematic diagram of an embodiment of a calibration image obtained after binarization processing according to the present invention.
For example, if the calibration line is red, red pixels with R values greater than 125, b values smaller than 100, and G values smaller than 100 are extracted. The image is then binarized, with the red pixels set to a maximum value of 255 and the remaining pixels set to a minimum value of 0.
Optionally, performing edge detection on the calibration image after the binarization processing to obtain an edge map of the calibration line. Alternatively, edge detection may be performed by the canny algorithm. The Canny algorithm can be divided into the following 5 steps: applying gaussian filtering to smooth the image with the aim of removing noise; finding an intensity gradient of the image; applying non-maximum suppression technology to eliminate edge false detection; applying a dual threshold approach to determine possible boundaries; the boundaries are tracked using a hysteresis technique. Referring to fig. 3, fig. 3 is a schematic diagram of an embodiment of an edge map of a calibration line according to the present invention.
Optionally, the edge map is subjected to straight line extraction. Optionally, a straight line may be extracted through a Hough transform algorithm, and the basic principle of Hough transform is to transform a curve (including a straight line) in an image space into a parameter space, and determine a description parameter of the curve by detecting an extreme point in the parameter space, so as to extract a regular curve in the image.
And obtaining the linear segment data of the edge of the calibration line after the linear extraction. Referring to fig. 4, fig. 4 is a schematic diagram of an embodiment of a graph of an edge after performing straight line extraction according to the present invention.
And dividing the extracted straight line segment into a vertical line segment and a horizontal line segment according to coordinate points at two ends of the extracted straight line segment.
S130: and determining a first calibration line and a second calibration line based on the linear segment data, obtaining a first parameter corresponding to the first calibration line based on the first calibration line, and obtaining a second parameter corresponding to the second calibration line based on the second calibration line.
The first calibration line and the second calibration line form a preset angle. Optionally, the first calibration line may be a vertical calibration line, and the second calibration line may be a horizontal calibration line, and a 90 ° angle is formed between the vertical calibration line and the horizontal calibration line.
For convenience of describing the embodiment, the vertical calibration line is taken as the first calibration line, and the horizontal calibration line is taken as the second calibration line. It is understood that when the first calibration line is not a vertical calibration line and the second calibration line is not a horizontal calibration line, the following manner is also applicable in the manner of calculation of the straight line angle, and it will be understood by those skilled in the art that the description is omitted here.
Optionally, the step of determining a vertical calibration line based on the straight line segment data, and obtaining a vertical parameter corresponding to the vertical calibration line based on the vertical calibration line includes:
in some embodiments, the step of determining a vertical calibration line based on the straight line segment data, and obtaining a vertical parameter corresponding to the vertical calibration line based on the vertical calibration line includes:
judging straight line characteristics based on the straight line segment data, and determining a plurality of vertical line segments; carrying out weighted fusion on the plurality of vertical line segments to obtain vertical lines; and identifying the weighted and fused vertical line, determining the left vertical calibration line and the right vertical calibration line, and obtaining the slope of the left vertical calibration line, the intercept of the left vertical calibration line, the slope of the right vertical calibration line and the intercept of the right vertical calibration line.
The straight line segment data comprises coordinates of two end points of the straight line segment, so that straight line characteristic judgment is carried out based on the straight line segment data, and a plurality of vertical line segments are determined; the step of performing weighted fusion on the vertical line segments to obtain the vertical line may include:
obtaining two end points (x) in each straight line segment 1 ，y 1 ）、（x 2 ，y 2 ) Corresponding to a first difference on the y-axis; if the first difference is larger than a first preset value, the straight line segment corresponding to the first difference is a vertical line segment; if the slope difference value between the vertical line segments is smaller than a second preset value, determining the corresponding vertical line segment as the edge of the same calibration line; and based on a weighted average method, fusing the edges of the same calibration line to obtain a vertical line.
It should be noted that, if the first difference is smaller than or equal to the first preset value, the y-axis straight line segment corresponding to the first difference is discarded; and if the slope difference value between the vertical line segments is greater than or equal to a second preset value, determining the corresponding vertical line segments as the edges of two different calibration lines.
Alternatively, the first preset value may be set to 5, and (x) is judged 1 ，y 1 ）、（x 2 ，y 2 ) And calculating the slope k, intercept l and length d of the straight line segment according to the following formulas:
and sequencing the vertical line segments according to the slope, and if the slope difference between the vertical line segments is smaller, namely the slope difference between the vertical line segments is smaller than a second preset value, judging that the vertical line segments are positioned at the edge of the same calibration line, and performing fusion.
Assuming that n vertical line segments to be fused in a certain slope interval are provided, the formula of the fused calibration line is as follows:
wherein the content of the first and second substances,
representing the slope of the nth vertical segment;
represents the length of the nth vertical segment;
the intercept of the nth vertical segment is shown.
Further, the step of identifying the weighted and fused vertical line, determining the left vertical calibration line and the right vertical calibration line, and obtaining the slope of the left vertical calibration line, the intercept of the left vertical calibration line, the slope of the right vertical calibration line and the intercept of the right vertical calibration line includes:
taking the straight line with the negative and maximum slope in the vertical lines as the left vertical calibration line, and obtaining the slope k corresponding to the left vertical calibration line l And intercept l l (ii) a Taking the straight line with the positive and minimum slope in the vertical lines as the right vertical calibration line, and obtaining the slope k corresponding to the right vertical calibration line r And intercept l r 。
Optionally, the step of determining a horizontal calibration line based on the linear segment data, and obtaining a horizontal parameter corresponding to the horizontal calibration line based on the horizontal calibration line includes:
in some embodiments, determining a horizontal calibration line based on the straight line segment data, and obtaining a horizontal parameter corresponding to the horizontal calibration line based on the horizontal calibration line includes:
judging straight line characteristics based on the straight line segment data, and determining a plurality of horizontal line segments; carrying out weighted fusion on the horizontal line segments to obtain a horizontal line; and identifying the weighted and fused horizontal lines, determining a first horizontal calibration line and a second horizontal calibration line, and obtaining a y-axis value of the first horizontal calibration line and a y-axis value of the second horizontal calibration line. Wherein the y-axis value of the first horizontal calibration line and the y-axis value of the second horizontal calibration line are horizontal parameters.
The straight line segment data comprises coordinates of two end points of the straight line segment, so that straight line characteristic judgment is carried out based on the straight line segment data, and a plurality of horizontal line segments are determined; carrying out weighted fusion on the plurality of horizontal line segments to obtain a horizontal line, wherein the weighted fusion comprises the following steps:
obtaining two end points (x) in each straight line segment 1 ，y 1 ）、（x 2 ，y 2 ) A first difference on the y-axis of (a); if the first difference is smaller than a third preset value, and x 1 、x 2 Determining a straight line segment corresponding to the first difference value as a horizontal line segment within a preset range; obtaining a y-axis value of a horizontal line segment, an x-axis value and an x-axis difference between two end points; if the y-axis difference between the horizontal line segments is less than a fourth preset value, judging the corresponding horizontal line segments as the same horizontal line segmentCalibrating the edge of the line; and based on a weighted average method, fusing the edges of the same calibration line to obtain a horizontal line and a y-axis numerical value corresponding to the horizontal line.
It should be noted that, if the first difference is greater than or equal to the third predetermined value, or x 1 、x 2 If the difference value is not within the preset range, the straight line segment corresponding to the first difference value is determined to be discarded; and if the y-axis difference between the horizontal line segments is greater than or equal to a fourth preset value, determining the corresponding horizontal line segment as the edge of two different calibration lines.
The third preset value may be set to 3, and the preset range may refer to 50% in the middle of the calibration image. Two end points (x) are determined 1 ，y 1 ）、（x 2 ，y 2 ) The difference in y values of (a) is less than 3, and the straight line segment of which x value is 50% in the middle of the image is a horizontal line segment. And recording the y-axis value, the x-axis value and the x-axis difference value Deltax between the two endpoints of the horizontal line segment.
Note that, if y 1 ≠y 2 Then the value of the y-axis may be y 1 And y 2 Average value of (a).
If the difference between the y values of two or more horizontal line segments is small (e.g., smaller than a fourth preset value), the corresponding horizontal line segment is determined as the edge of the same calibration line. If there are n horizontal line segments at the edge of the same calibration line, these horizontal line segments can be fused using a weighted average method. The fusion formula is as follows:
wherein, the first and the second end of the pipe are connected with each other,
a y-axis value representing the nth horizontal line segment;
the x-axis difference of the nth horizontal line segment is represented.
And calculating the y value of the fused horizontal line according to the formula.
According to the monocular distance measurement model calibration method for the curtain wall robot, provided by the invention, a horizontal line after weighted fusion is identified, a first horizontal calibration line and a second horizontal calibration line are determined, and a y-axis value of the first horizontal calibration line and a y-axis value of the second horizontal calibration line are obtained, and the method comprises the following steps:
taking the straight line with the maximum value of the y axis in the horizontal line as a first horizontal calibration line to obtain the value y' of the y axis of the first horizontal calibration line 1 (ii) a Taking the straight line with the second largest value of the y axis in the horizontal line as a second horizontal calibration line to obtain the value y' of the y axis of the second horizontal calibration line 2 。
S140: the coordinate conversion model parameters are determined based on the first parameters and the second parameters.
As can be seen from the above calculations, the vertical parameter may include the slope k of the left vertical calibration line l Slope of the right vertical calibration line
Intercept l of left vertical calibration line l And intercept of right vertical calibration line r . The horizontal parameter may comprise a y-axis value y' of the first horizontal calibration line 1 And the y-axis value y' of the second horizontal calibration line 2 。
Therefore, determining the coordinate conversion model parameters based on the vertical parameters and the horizontal parameters includes:
slope based on left vertical calibration line
And slope of the right vertical calibration line
Determining a slope coefficient a; intercept based on left vertical calibration line
And intercept of right vertical calibration line
Determining an intercept coefficient b; based on the width between perpendicular calibration lines
Width between horizontal calibration lines
Slope coefficient a, intercept coefficient b, and y-axis value y' of the first horizontal calibration line 1 And the y-axis value y' of the second horizontal calibration line 2 Determining parameters of coordinate transformation model
。
Specifically, the coordinate conversion model parameters are as follows:
it should be noted that the width W of the vertical calibration line may be determined by the distance between the left vertical calibration line and the right vertical calibration line; width between horizontal calibration lines
May be determined by the distance between the first and second horizontal calibration lines.
In the above, after the parameters of the coordinate conversion model are calculated, the parameters can be stored in the parameter file corresponding to the coordinate conversion model.
S150: and calculating the distance of each line segment based on the edge graph of the calibration line and the coordinate conversion model parameters, and obtaining a calibration result according to the distance of each line segment.
According to the monocular distance measurement model calibration method for the curtain wall robot, provided by the invention, the distance of each line segment is calculated based on the edge graph of the calibration line and the coordinate conversion model parameter, and the calibration result is obtained according to the distance of each line segment, and the method comprises the following steps:
determining the horizontal distance between the horizontal line segment and the curtain wall robot and the vertical distance between the vertical line segment and the central axis of the curtain wall robot, and outputting the position and distance data of the linear line segment to obtain a simulated image; drawing a simulated calibration line position in the simulated image according to the vertical parameter and the horizontal parameter; referring to fig. 5, fig. 5 is a schematic diagram of a simulated image according to an embodiment of the invention.
The numerical value on the line segment in the simulated image represents the distance between the line segment and the camera, wherein the numerical value on the horizontal line segment represents the horizontal distance between the horizontal line segment and the curtain wall robot, such as 277, 324, 414 and 467, and the unit of the numerical values is centimeter; the values on the vertical line segments represent the vertical distances of the vertical line segments from the central axis of the curtain wall robot, such as 2, 6, 11, 60, 100, 106, 158, 166, which are all in centimeters. The two straight lines intersecting in fig. 5 represent the simulated calibration lines. Wherein the simulated calibration line can also be understood as the calibration line recognized by the curtain wall robot.
It should be noted that each line segment in the simulated image may be displayed with a numerical value representing the distance between the line segment and the camera, which is not listed in fig. 5.
In addition, it should be noted that, since the numerical value on the vertical line segment represents the vertical distance between the vertical line segment and the central axis of the curtain wall robot, and the vertical line segment is not parallel to the central axis of the curtain wall robot in the simulation image, a plurality of numerical values, such as 100 and 106, may be included on one vertical line segment.
And if the position of the simulation calibration line meets the first preset requirement and the simulation image meets the second preset requirement, determining that the monocular distance measuring model of the curtain wall robot is calibrated.
The first preset requirement is used for limiting the position relation of the simulation calibration line, the horizontal calibration line and the vertical calibration line, and the second preset requirement is used for limiting the similarity of the simulation image and the setting field.
For example, if the identified calibration lines are on the inner sides of the two vertical calibration lines and the upper and lower sides of the horizontal calibration line (which satisfy the first predetermined requirement), and the identified widths of the respective inspection lines and the distances from the calibration line are consistent with those in the field setting (which satisfy the second predetermined requirement), the model calibration is considered to be successfully completed.
If the position of the simulation calibration line does not meet the first preset requirement, or the simulation image does not meet the second preset requirement, the calibration is unsuccessful; the daughter-in-law on the glass curtain wall, which needs to contact with the curtain wall robot, transfers the machine body up and down, and repeats the steps for calibration.
In summary, in the monocular distance measurement model calibration method for the curtain wall robot provided by this embodiment, pixels corresponding to the colors of the preset calibration line are extracted from the calibration image, and binarization processing is performed on the calibration image based on the pixels; performing edge detection on the calibration image after binarization processing to obtain an edge image of the calibration line; performing linear extraction on the edge graph to obtain linear segment data of the edge of the calibration line; determining a vertical calibration line and a horizontal calibration line based on the linear segment data, and obtaining a vertical parameter and a horizontal parameter; determining coordinate conversion model parameters based on the vertical parameters and the horizontal parameters; and calculating the distance of each line segment based on the edge graph of the calibration line and the coordinate conversion model parameters, and obtaining a calibration result according to the distance of each line segment. By the mode, the curtain wall robot can be calibrated, the robot can automatically identify the calibration line and calculate the model parameters after being adsorbed on the glass curtain wall, parameters needing manual measurement are reduced, the automation degree of the calibration process is increased, the working efficiency of calibration during batch production is improved, and the requirement on the professional degree of workers is low; in addition, the robot is calibrated in the running state of the glass curtain wall, so that the model parameters include the position deviation of the camera in the assembling process and the posture in the working state, and the data are more accurate and more accord with the working environment.
Fig. 6 shows a schematic structural diagram of an electronic device according to an embodiment of the present invention, where fig. 6 is a schematic structural diagram. In this embodiment, the electronic device may include a memory (memory) 620, a processor (processor) 610, and a computer program stored on the memory 620 and executable on the processor 610. When the processor 610 executes the program, the monocular distance measuring model calibration method of the curtain wall robot provided by the above methods is implemented.
Optionally, the electronic device may further include a communication bus 630 and a communication Interface (Communications Interface) 640, wherein the processor 610, the communication Interface 640, and the memory 620 are in communication with each other through the communication bus 630. The processor 610 may call logic instructions in the memory 620 to execute a monocular distance measuring model calibration method for a curtain wall robot, the method including:
obtaining a calibration image through a monocular camera of the curtain wall robot; the calibration image comprises a preset calibration line on the glass curtain wall; obtaining an edge graph of a calibration line and linear line segment data of the edge of the calibration line based on the calibration image; determining a first calibration line and a second calibration line based on the linear segment data, obtaining a first parameter corresponding to the first calibration line based on the first calibration line, and obtaining a second parameter corresponding to the second calibration line based on the second calibration line; the first calibration line and the second calibration line form a preset angle; determining coordinate conversion model parameters based on the first parameters and the second parameters; and calculating the distance of each line segment based on the edge graph of the calibration line and the coordinate conversion model parameters, and obtaining a calibration result according to the distance of each line segment.
In addition, the logic instructions in the memory 620 may be implemented in the form of software functional units and stored in a computer readable storage medium when the logic instructions are sold or used as a stand-alone product. Based on such understanding, the technical solution of the present invention may be embodied in the form of a software product, which is stored in a storage medium and includes instructions for causing a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the steps of the method according to the embodiments of the present invention. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk, or an optical disk, and various media capable of storing program codes.
On the other hand, the invention also provides a curtain wall robot, which comprises a monocular camera, a robot body and the electronic equipment.
The above-described embodiments of the apparatus are merely illustrative, and the units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of the present embodiment. One of ordinary skill in the art can understand and implement it without inventive effort.
Through the above description of the embodiments, those skilled in the art will clearly understand that each embodiment can be implemented by software plus a necessary general hardware platform, and certainly can also be implemented by hardware. With this understanding in mind, the above-described technical solutions may be embodied in the form of a software product, which can be stored in a computer-readable storage medium such as ROM/RAM, magnetic disk, optical disk, etc., and includes instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute the methods described in the embodiments or some parts of the embodiments.
Finally, it should be noted that: the above examples are only intended to illustrate the technical solution of the present invention, and not to limit it; although the present invention has been described in detail with reference to the foregoing embodiments, it should be understood by those of ordinary skill in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some technical features may be equivalently replaced; and such modifications or substitutions do not depart from the spirit and scope of the corresponding technical solutions of the embodiments of the present invention.
Claims (10)
1. A monocular distance measurement model calibration method of a curtain wall robot is characterized by comprising the following steps:
obtaining a calibration image through a monocular camera of the curtain wall robot; the calibration image comprises a preset calibration line on the glass curtain wall;
obtaining an edge graph of a calibration line and linear line segment data of the edge of the calibration line based on the calibration image;
determining a first calibration line and a second calibration line based on the linear segment data, obtaining a first parameter corresponding to the first calibration line based on the first calibration line, and obtaining a second parameter corresponding to the second calibration line based on the second calibration line; the first calibration line and the second calibration line form a preset angle;
determining coordinate conversion model parameters based on the first parameters and the second parameters;
and calculating the distance of each line segment based on the edge graph of the calibration line and the coordinate conversion model parameters, and obtaining a calibration result according to the distance of each line segment.
2. The monocular distance measuring model calibration method of a curtain wall robot as claimed in claim 1, wherein the determining a first calibration line and a second calibration line based on the straight line segment data, obtaining a first parameter corresponding to the first calibration line based on the first calibration line, and obtaining a second parameter corresponding to the second calibration line based on the second calibration line comprises:
determining a vertical calibration line based on the linear segment data, and obtaining a vertical parameter corresponding to the vertical calibration line based on the vertical calibration line;
and determining a horizontal calibration line based on the linear segment data, and obtaining a horizontal parameter corresponding to the horizontal calibration line based on the horizontal calibration line.
3. The calibration method for the monocular distance measuring model of the curtain wall robot as claimed in claim 2, wherein the determining a vertical calibration line based on the straight line segment data and obtaining a vertical parameter corresponding to the vertical calibration line based on the vertical calibration line comprises:
judging straight line characteristics based on the straight line segment data, and determining a plurality of vertical line segments;
performing weighted fusion on the plurality of vertical line segments to obtain a vertical line;
and identifying the weighted and fused vertical line, determining the left vertical calibration line and the right vertical calibration line, and obtaining the slope of the left vertical calibration line, the intercept of the left vertical calibration line, the slope of the right vertical calibration line and the intercept of the right vertical calibration line.
4. The method for calibrating the monocular distance measuring model of the curtain wall robot according to claim 3, wherein the determining a vertical calibration line based on the straight line segment data and obtaining a vertical parameter corresponding to the vertical calibration line based on the vertical calibration line comprises:
obtaining two end points (x) in each straight line segment 1 ，y 1 ）、（x 2 ，y 2 ) Corresponding to a first difference on the y-axis;
if the first difference is larger than a first preset value, a straight line segment corresponding to the first difference is a vertical line segment;
if the slope difference value between the vertical line segments is smaller than a second preset value, determining the corresponding vertical line segment as the edge of the same calibration line;
based on a weighted average method, fusing the edges of the same calibration line to obtain the vertical line;
taking the straight line with the negative and maximum slope in the vertical lines as the left vertical calibration line, and obtaining the slope and intercept corresponding to the left vertical calibration line;
and taking the straight line with the positive and minimum slope in the vertical lines as the right vertical calibration line, and obtaining the slope and the intercept corresponding to the right vertical calibration line.
5. The calibration method for the monocular distance measuring model of the curtain wall robot as claimed in claim 2, wherein the determining a horizontal calibration line based on the straight line segment data and obtaining a horizontal parameter corresponding to the horizontal calibration line based on the horizontal calibration line comprises:
judging straight line characteristics based on the straight line segment data, and determining a plurality of horizontal line segments;
carrying out weighted fusion on the horizontal line segments to obtain a horizontal line;
identifying the weighted and fused horizontal line, determining a first horizontal calibration line and a second horizontal calibration line, and obtaining a y-axis value of the first horizontal calibration line and a y-axis value of the second horizontal calibration line; wherein the y-axis value of the first horizontal calibration line and the y-axis value of the second horizontal calibration line are horizontal parameters.
6. The calibration method for the monocular distance measuring model of the curtain wall robot as recited in claim 5, wherein the determining a horizontal calibration line based on the straight line segment data and obtaining a horizontal parameter corresponding to the horizontal calibration line based on the horizontal calibration line comprises:
obtaining two end points (x) in each straight line segment 1 ，y 1 ）、（x 2 ，y 2 ) A first difference on the y-axis of (a);
if the first difference is smaller than a third preset value, and x 1 、x 2 Within a preset range, determining that the straight line segment corresponding to the first difference value is a horizontal line segment;
obtaining a y-axis value and an x-axis value of the horizontal line segment and an x-axis difference between two end points;
if the y-axis difference between the horizontal line segments is smaller than a fourth preset value, determining the corresponding horizontal line segment as the edge of the same calibration line;
based on a weighted average method, fusing the edges of the same calibration line to obtain the horizontal line and a y-axis numerical value corresponding to the horizontal line;
taking the straight line with the maximum y-axis value in the horizontal line as the first horizontal calibration line to obtain the y-axis value of the first horizontal calibration line;
and taking the straight line with the second largest value of the y axis in the horizontal line as the second horizontal calibration line to obtain the value of the y axis of the second horizontal calibration line.
7. The calibration method for the monocular distance measuring model of the curtain wall robot as in claim 4, wherein the determining the coordinate transformation model parameters based on the vertical parameters and the horizontal parameters comprises:
determining a slope coefficient based on the slope of the left vertical calibration line and the slope of the right vertical calibration line;
determining an intercept coefficient based on the intercept of the left vertically calibrated line and the intercept of the right vertically calibrated line;
determining the coordinate conversion model parameters based on the widths between the vertical calibration lines, the widths between the horizontal calibration lines, the slope coefficient, the intercept coefficient, and the horizontal parameters.
8. The calibration method for the monocular distance measuring model of the curtain wall robot as claimed in claim 6, wherein the calculating of the distance of each line segment based on the edge map of the calibration line and the coordinate transformation model parameter and the obtaining of the calibration result according to the distance of each line segment comprises:
determining the horizontal distance between the horizontal line segment and the curtain wall robot and the vertical distance between the vertical line segment and the central axis of the curtain wall robot, and outputting the position and distance data of the linear line segment to obtain a simulated image;
drawing a simulated calibration line position in the simulated image according to the vertical parameter and the horizontal parameter;
and if the position of the simulation calibration line meets a first preset requirement and the simulation image meets a second preset requirement, determining that the monocular distance measuring model of the curtain wall robot is calibrated.
9. An electronic device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, wherein the processor implements the monocular distance measuring model calibration method for a curtain wall robot according to any one of claims 1 to 8 when executing the program.
10. A curtain wall robot comprising a monocular camera, a robot body, and the electronic device of claim 9.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211403530.1A CN115661267B (en) | 2022-11-10 | 2022-11-10 | Monocular ranging model calibration method, electronic equipment and curtain wall robot |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211403530.1A CN115661267B (en) | 2022-11-10 | 2022-11-10 | Monocular ranging model calibration method, electronic equipment and curtain wall robot |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN115661267A true CN115661267A (en) | 2023-01-31 |
| CN115661267B CN115661267B (en) | 2023-04-25 |
Family
ID=85021275
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202211403530.1A Active CN115661267B (en) | 2022-11-10 | 2022-11-10 | Monocular ranging model calibration method, electronic equipment and curtain wall robot |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115661267B (en) |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US5790403A (en) * | 1994-07-12 | 1998-08-04 | Honda Giken Kogyo Kabushiki Kaisha | Lane image processing system for vehicle |
| JP2011126990A (en) * | 2009-12-17 | 2011-06-30 | Sumitomo Heavy Industries Process Equipment Co Ltd | Method and program for calibrating camera of coke oven wall observation apparatus |
| CN103729837A (en) * | 2013-06-25 | 2014-04-16 | 长沙理工大学 | Rapid calibration method of single road condition video camera |
| CN108292439A (en) * | 2015-11-30 | 2018-07-17 | 德尔福技术有限责任公司 | For calibrating installation to the method for the orientation of the video camera of vehicle |
| CN115223031A (en) * | 2022-09-20 | 2022-10-21 | 凌度(广东)智能科技发展有限公司 | Monocular frame distance measuring method and device, medium and curtain wall robot |
-
2022
- 2022-11-10 CN CN202211403530.1A patent/CN115661267B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US5790403A (en) * | 1994-07-12 | 1998-08-04 | Honda Giken Kogyo Kabushiki Kaisha | Lane image processing system for vehicle |
| JP2011126990A (en) * | 2009-12-17 | 2011-06-30 | Sumitomo Heavy Industries Process Equipment Co Ltd | Method and program for calibrating camera of coke oven wall observation apparatus |
| CN103729837A (en) * | 2013-06-25 | 2014-04-16 | 长沙理工大学 | Rapid calibration method of single road condition video camera |
| CN108292439A (en) * | 2015-11-30 | 2018-07-17 | 德尔福技术有限责任公司 | For calibrating installation to the method for the orientation of the video camera of vehicle |
| CN115223031A (en) * | 2022-09-20 | 2022-10-21 | 凌度(广东)智能科技发展有限公司 | Monocular frame distance measuring method and device, medium and curtain wall robot |
Non-Patent Citations (1)
| Title |
|---|
| 许嵩: "基于矩形的摄像机自标定几何方法" * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN115661267B (en) | 2023-04-25 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN104408460B (en) | A kind of lane detection and tracking detection method | |
| CN105279372B (en) | A kind of method and apparatus of determining depth of building | |
| CN109660783B (en) | Virtual reality parallax correction | |
| CN103745452B (en) | Camera external parameter assessment method and device, and camera external parameter calibration method and device | |
| CN110490936B (en) | Calibration method, device and equipment of vehicle camera and readable storage medium | |
| CN112991193B (en) | Depth image restoration method, device and computer-readable storage medium | |
| TWI709085B (en) | Method, device, computer readable storage medium and computing equipment for damage segmentation of vehicle damage image | |
| CN101697233A (en) | Structured light-based three-dimensional object surface reconstruction method | |
| JP2015184767A (en) | Information processor, information processing method, position attitude estimation device and robot system | |
| CN107016348A (en) | With reference to the method for detecting human face of depth information, detection means and electronic installation | |
| CN105147311A (en) | Visual equipment assisted scanning and positioning method and system applied to CT system | |
| CN111996883B (en) | Method for detecting width of road surface | |
| CN114359412B (en) | Automatic calibration method and system for external parameters of camera facing to building digital twins | |
| CN111192326B (en) | Method and system for visually identifying direct-current charging socket of electric automobile | |
| CN115439607A (en) | Three-dimensional reconstruction method and device, electronic equipment and storage medium | |
| CN110942092B (en) | Graphic image recognition method and recognition system | |
| CN110648362B (en) | Binocular stereo vision badminton positioning identification and posture calculation method | |
| CN115439840A (en) | Aviation piece slot area identification method, device, equipment and medium | |
| CN109146952A (en) | Estimate the method, apparatus and computer readable storage medium of compartment void volume | |
| CN113749646A (en) | Monocular vision-based human body height measuring method and device and electronic equipment | |
| CN106327438B (en) | A kind of pair of bloom and the augmented reality method and Crawl mat application for repeating texture elimination | |
| TWI462027B (en) | Image processing device and image processing method thereof | |
| CN115661267A (en) | Monocular distance measurement model calibration method, electronic equipment and curtain wall robot | |
| CN116125489A (en) | Indoor object three-dimensional detection method, computer equipment and storage medium | |
| TWI595446B (en) | Method for improving occluded edge quality in augmented reality based on depth camera |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |