CN115661267B - Monocular ranging model calibration method, electronic equipment and curtain wall robot - Google Patents
Monocular ranging model calibration method, electronic equipment and curtain wall robot Download PDFInfo
- Publication number
- CN115661267B CN115661267B CN202211403530.1A CN202211403530A CN115661267B CN 115661267 B CN115661267 B CN 115661267B CN 202211403530 A CN202211403530 A CN 202211403530A CN 115661267 B CN115661267 B CN 115661267B
- Authority
- CN
- China
- Prior art keywords
- line
- calibration
- horizontal
- vertical
- calibration line
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Abstract
The invention provides a monocular ranging model calibration method, electronic equipment and a curtain wall robot, and relates to the technical field of robot vision, wherein a calibration image is obtained through a monocular camera of the curtain wall robot; obtaining an edge map of a calibration line and straight line segment data of the edge of the calibration line based on the calibration image; determining a first calibration line and a second calibration line based on the linear segment data, obtaining a first parameter corresponding to the first calibration line based on the first calibration line, and obtaining a second parameter corresponding to the second calibration line based on the second calibration line; determining coordinate transformation model parameters based on the first parameter and the second parameter; and calculating the distance of each line segment based on the edge graph of the calibration line and the coordinate conversion model parameters, and obtaining a calibration result according to the distance of each line segment. The curtain wall robot disclosed by the invention can automatically identify the calibration line and calculate the model parameters after being adsorbed on the glass curtain wall, so that the data are more accurate and more in line with the working environment; parameters needing manual measurement are reduced, and the automation degree of the calibration process is increased.
Description
Technical Field
The invention relates to the technical field of robot vision, in particular to a monocular ranging model calibration method, electronic equipment and a curtain wall robot.
Background
Robots have gradually been incorporated into people's lives to provide different types of services or functions. Such as robots for cleaning glass curtain walls. In the perception system of these robots, monocular vision can provide a planar perception area at a relatively low cost, and thus becomes a choice for low-precision vision or multi-perception fusion.
The monocular camera is calibrated, and a geometric model imaged by the camera is constructed according to the interrelation between the three-dimensional geometric position of a certain point on the surface of the space object and the corresponding point in the image. As a technique for stereoscopic vision measurement, camera calibration directly affects subsequent distance measurement or even three-dimensional reconstruction, and thus an efficient and convenient calibration method is required.
The existing monocular distance measurement calibration method is mainly Zhang Zhengyou calibration method, and the method is used for calibrating parameters of each step in the process according to the process of converting a real world coordinate system into a pixel coordinate system of an image when a camera shoots. The world coordinate system is converted into a camera coordinate system through rigid transformation, then converted into an image coordinate system through perspective projection, and finally converted into a pixel coordinate system through affine transformation, wherein affine transformation and perspective projection can be integrated into an internal reference matrix of the camera, and the rigid transformation is converted into an external reference matrix.
Zhang Zhengyou calibration method requires preparing a black-and-white checkerboard calibration plate, fixing the camera, and taking pictures of the calibration plates at different positions and angles. And calculating the gesture of the calibration plate according to the known checkerboard size by using MATLAB or an autonomously written program, and calculating each parameter on the internal reference matrix and the external reference matrix according to the pixel characteristics of the calibration plates with different positions and different gestures on the image.
However, such a calibration method has problems in the calibration of curtain wall robots: firstly, the parameter calculation is complex, and the calibration process needs to calculate a plurality of parameters of an internal reference matrix and an external reference matrix; the calibration process has strong specialization, the photo quality needs to be evaluated, the photo of the corresponding posture of the calibration plate needs to be supplemented and adjusted, the external parameter matrix corresponding to the glass curtain wall plane needs to be calculated according to the information such as the installation height of a camera, and the like; and the calibration process is performed in the checkerboard calibration plate, so that the curtain wall robot is not beneficial to feeding back the actual working condition.
Disclosure of Invention
The invention provides a monocular ranging model calibration method, electronic equipment and a curtain wall robot, which are used for solving the problems that in the prior art, the calibration process of the curtain wall robot is complex and the feedback of the curtain wall robot to the actual work is not facilitated.
The invention provides a method for calibrating a monocular ranging model of a curtain wall robot, which comprises the following steps: obtaining a calibration image through a monocular camera of the curtain wall robot; the calibration image comprises a preset calibration line on the glass curtain wall; obtaining an edge map of a calibration line and straight line segment data of the edge of the calibration line based on the calibration image; determining a first calibration line and a second calibration line based on the linear segment data, obtaining a first parameter corresponding to the first calibration line based on the first calibration line, and obtaining a second parameter corresponding to the second calibration line based on the second calibration line; the first calibration line and the second calibration line form a preset angle; determining coordinate transformation model parameters based on the first parameter and the second parameter; and calculating the distance of each line segment based on the edge graph of the calibration line and the coordinate conversion model parameters, and obtaining a calibration result according to the distance of each line segment.
According to the monocular ranging model calibration method of the curtain wall robot, provided by the invention, a first calibration line and a second calibration line are determined based on linear line segment data, a first parameter corresponding to the first calibration line is obtained based on the first calibration line, a second parameter corresponding to the second calibration line is obtained based on the second calibration line, and the method comprises the following steps: determining a vertical calibration line based on the linear segment data, and obtaining a vertical parameter corresponding to the vertical calibration line based on the vertical calibration line; and determining a horizontal calibration line based on the linear segment data, and obtaining a horizontal parameter corresponding to the horizontal calibration line based on the horizontal calibration line.
According to the monocular ranging model calibration method of the curtain wall robot, provided by the invention, a vertical calibration line is determined based on linear line segment data, and vertical parameters corresponding to the vertical calibration line are obtained based on the vertical calibration line, and the method comprises the following steps: judging straight line characteristics based on the straight line segment data, and determining a plurality of vertical line segments; carrying out weighted fusion on a plurality of vertical line segments to obtain vertical lines; and identifying the vertical lines after weighted fusion, determining a left vertical calibration line and a right vertical calibration line, and obtaining the slope of the left vertical calibration line, the intercept of the left vertical calibration line, the slope of the right vertical calibration line and the intercept of the right vertical calibration line.
According to the monocular ranging model calibration method of the curtain wall robot, provided by the invention, a vertical calibration line is determined based on linear line segment data, and vertical parameters corresponding to the vertical calibration line are obtained based on the vertical calibration line, and the method comprises the following steps: obtaining two end points (x 1 ，y 1 ）、（x 2 ，y 2 ) Corresponding to a first difference on the y-axis; if the first difference value is larger than a first preset value, the y-axis straight line segment corresponding to the first difference value is a vertical line segment; if the slope difference between the vertical line segments is smaller than the second preset value, determining the corresponding vertical line segments as the sameCalibrating the edge of the line; based on a weighted average method, fusing the edges of the same calibration line to obtain a vertical line; taking the straight line with negative and maximum slope in the vertical line as a left vertical calibration line, and obtaining the corresponding slope and intercept of the left vertical calibration line; and taking the straight line with positive and minimum slope in the vertical line as a right vertical calibration line, and obtaining the corresponding slope and intercept of the right vertical calibration line.
According to the monocular ranging model calibration method of the curtain wall robot, provided by the invention, a horizontal calibration line is determined based on linear segment data, and horizontal parameters corresponding to the horizontal calibration line are obtained based on the horizontal calibration line, and the method comprises the following steps: judging straight line characteristics based on the straight line segment data, and determining a plurality of horizontal line segments; carrying out weighted fusion on a plurality of horizontal line segments to obtain a horizontal line; identifying the weighted and fused horizontal lines, determining a first horizontal calibration line and a second horizontal calibration line, and obtaining the y-axis value of the first horizontal calibration line and the y-axis value of the second horizontal calibration line; wherein the y-axis value of the first horizontal calibration line and the y-axis value of the second horizontal calibration line are horizontal parameters.
According to the monocular ranging model calibration method of the curtain wall robot, provided by the invention, a horizontal calibration line is determined based on linear segment data, and horizontal parameters corresponding to the horizontal calibration line are obtained based on the horizontal calibration line, and the method comprises the following steps: obtaining two end points (x 1 ，y 1 ）、（x 2 ，y 2 ) A first difference on the y-axis of (2); if the first difference is smaller than the third preset value and x 1 、x 2 In a preset range, determining a straight line segment corresponding to the first difference value as a horizontal line segment; obtaining a y-axis value of the horizontal line segment, an x-axis value and an x-axis difference value between two endpoints; if the y-axis difference value between the horizontal line segments is smaller than a fourth preset value, judging the corresponding horizontal line segments as the edges of the same calibration line; based on a weighted average method, fusing the edges of the same calibration line to obtain a horizontal line and a y-axis value corresponding to the horizontal line; taking a straight line with the largest y-axis value in the horizontal line as a first horizontal calibration line to obtain the y-axis value of the first horizontal calibration line; the y-axis in the horizontal line is countedThe second largest straight line is taken as a second horizontal calibration line, and the y-axis value of the second horizontal calibration line is obtained.
According to the monocular ranging model calibration method of the curtain wall robot, provided by the invention, the coordinate conversion model parameters are determined based on the vertical parameters and the horizontal parameters, and the method comprises the following steps: determining a slope coefficient based on the slope of the left vertical calibration line and the slope of the right vertical calibration line; determining an intercept coefficient based on the intercept of the left vertical calibration line and the intercept of the right vertical calibration line; coordinate transformation model parameters are determined based on the width between the vertical calibration lines, the width between the horizontal calibration lines, the slope coefficient, the intercept coefficient, and the horizontal parameter.
According to the monocular ranging model calibration method of the curtain wall robot, provided by the invention, the distances of all line segments are calculated based on the edge map and the coordinate conversion model parameters of the calibration line, and the calibration result is obtained according to the distances of all line segments, and the method comprises the following steps: determining the horizontal distance between the horizontal line segment and the curtain wall robot and the vertical distance between the vertical line segment and the central axis of the curtain wall robot, and outputting the position and distance data of the straight line segment to obtain a simulation image; drawing a simulation calibration line position in the simulation image according to the vertical parameter and the horizontal parameter; if the position of the simulated calibration line meets the first preset requirement and the simulated image meets the second preset requirement, the completion of the monocular ranging model calibration of the curtain wall robot is confirmed.
The invention also provides an electronic device, which comprises a memory, a processor and a computer program stored on the memory and capable of running on the processor, wherein the processor realizes the monocular ranging model calibration method of any curtain wall robot when executing the program.
The invention also provides a curtain wall robot which comprises the monocular camera, a robot body and the electronic equipment.
According to the monocular distance measurement model calibration method, the electronic equipment and the curtain wall robot, a calibration image is obtained through a monocular camera of the curtain wall robot; the calibration image comprises a preset calibration line on the glass curtain wall; obtaining an edge map of a calibration line and straight line segment data of the edge of the calibration line based on the calibration image; determining a first calibration line and a second calibration line based on the linear segment data, obtaining a first parameter corresponding to the first calibration line based on the first calibration line, and obtaining a second parameter corresponding to the second calibration line based on the second calibration line; determining coordinate transformation model parameters based on the first parameter and the second parameter; and calculating the distance of each line segment based on the edge graph of the calibration line and the coordinate conversion model parameters, and obtaining a calibration result according to the distance of each line segment. Through the mode, the curtain wall robot can be calibrated, the robot can automatically identify the calibration line and calculate the model parameters after being adsorbed on the glass curtain wall, parameters needing manual measurement are reduced, the automation degree of the calibration process is increased, and the working efficiency of calibration in mass production is improved. The invention performs calibration under the running state of the robot on the glass curtain wall, so that the model parameters comprise the position deviation of the camera in the assembly process and the condition of the pose in the working state, and the data are more accurate and more in line with the working environment.
Drawings
In order to more clearly illustrate the invention or the technical solutions of the prior art, the following description will briefly explain the drawings used in the embodiments or the description of the prior art, and it is obvious that the drawings in the following description are some embodiments of the invention, and other drawings can be obtained according to the drawings without inventive effort for a person skilled in the art.
FIG. 1 is a flow chart of an embodiment of a method for calibrating a monocular ranging model of a curtain wall robot according to the present invention;
FIG. 2 is a schematic diagram of an embodiment of a calibration image obtained after binarization processing according to the present invention;
FIG. 3 is a schematic diagram of one embodiment of an edge map of the present invention, marked lines;
FIG. 4 is a schematic diagram of an embodiment of a linear extraction trailing edge map according to the present invention;
FIG. 5 is a schematic representation of one embodiment of a simulated image of the present invention;
fig. 6 is a schematic structural diagram of an embodiment of the electronic device of the present invention.
Detailed Description
For the purpose of making the objects, technical solutions and advantages of the present invention more apparent, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings, and it is apparent that the described embodiments are some embodiments of the present invention, not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.
Curtain wall robots are increasingly applied, such as high-altitude curtain wall cleaning robots, which can realize cleaning of various types of high-rise facades, and the operation scenes of the robots are different because of different building models cleaned each time.
Referring to fig. 1, fig. 1 is a flow chart of an embodiment of a method for calibrating a monocular ranging model of a curtain wall robot according to the present invention, in this embodiment, the method for calibrating the monocular ranging model of the curtain wall robot may include steps S110 to S150, which specifically include the following steps:
s110: and obtaining a calibration image through a monocular camera of the curtain wall robot.
The method of the embodiment is to place the curtain wall robot on the glass curtain wall for calibration. When the curtain wall robot is placed on the glass curtain wall, a calibration image can be obtained through a monocular camera of the curtain wall robot, wherein the calibration image comprises a preset calibration line on the glass curtain wall.
The preset calibration line is a calibration line and a test line which are marked or stuck on the glass curtain wall for testing and are used for automatically calibrating the model and testing the model precision by a robot. Can be used for a long time after being set, and does not need repeated operation.
Setting a calibration line for calibrating model parameters by a user and a test line for checking whether the calibration result is accurate or not on the glass curtain wall. The test environment ensures a fixed position and brightness indoor light source and the environmental tone is single. The calibration line segment and the inspection line segment can use color adhesive tape or paint. The colors are red, green or blue, namely the colors corresponding to the R, G, B channels of the color image, so that feature extraction is convenient. The two line segments are arranged according to the following principle.
For example, the calibration line is two line segments perpendicular to the horizontal plane and one line segment parallel to the horizontal plane. The interval width of the two vertical lines is the width of the robot body, the distance is moderate, the recognition effect is good, and meanwhile, direct position data can be provided for future travel direction obstacle detection. The distance of one horizontal line is 50-200mm forward from the blind area boundary of the monocular camera, so that enough pixel characteristics are ensured.
For example, the inspection line is a line segment perpendicular to the horizontal plane and parallel to the horizontal plane. The two inspection lines are line segments which keep a certain distance from the calibration line and are far away from the robot, and are used for inspecting the ranging effect after the model calibration is completed.
In addition, it should be noted that before obtaining the calibration image, the principal point of the monocular camera of the curtain wall robot needs to be measured. In some embodiments, the determination may be performed using MATLAB in conjunction with a checkerboard calibration plate. The main point of the camera is the internal reference of the camera and is irrelevant to external conditions such as the installation position, and the like, so that the camera can be carried out after the camera arrives at the commodity, and the operation after the camera is installed on the curtain wall robot is not required.
Determining the camera principal point corresponds to a simplified version of the Zhang Zhengyou calibration method. After each monocular camera arrives, the monocular camera is fixed, then the black-white checkered calibration plate is moved to different positions and different angles to take pictures, and MATLAB is used for analyzing the pictures, so that a camera main point is calculated.
After the preset calibration line of the curtain wall robot is set and the main point of the camera is measured, a worker prepares according to a normal working state, prepares the robot before finishing the operation, aligns the robot to a calibration area, and after the robot is adsorbed on a glass curtain wall, the worker starts a calibration program, and the program automatically runs and feeds back a calibration result.
By the mode, the calibration process of the curtain wall robot is to simulate the working environment of the curtain wall robot, data of the curtain wall robot in the running state of the glass curtain wall are recorded, the model parameters comprise the position deviation of a camera in the assembly process and the condition of the posture in the working state, and the data are more accurate and more accord with the working environment.
S120: and obtaining an edge map of the calibration line and straight line segment data of the edge of the calibration line based on the calibration image.
Optionally, pixels corresponding to the colors of the preset calibration lines are extracted from the calibration image, and binarization processing is performed on the calibration image based on the pixels.
For example, pixels of the corresponding colors are extracted according to the colors of the calibration lines and binarized, wherein the gray values of the pigments corresponding to the colors of the calibration lines are set to 255, the gray values of the pixels of the other colors are set to 0, and the whole calibration image shows obvious visual effects of only black and white. Referring to fig. 2, fig. 2 is a schematic diagram of an embodiment of a calibration image obtained after binarization processing according to the present invention.
For example, the scale line is red, red pixels on the image with R values greater than 125, b values and G values less than 100 are extracted. The image is then binarized, the red pixel is set to a maximum value of 255 and the remaining pixels are set to a minimum value of 0.
Optionally, edge detection is carried out on the calibration image after the binarization processing, and an edge map of the calibration line is obtained. Alternatively, edge detection may be performed by a canny algorithm. The Canny algorithm can be divided into the following 5 steps: smoothing the image by applying gaussian filtering in order to remove noise; searching for an intensity gradient of the image; applying a non-maximum suppression technique to eliminate edge false detection; applying a double threshold approach to determine possible boundaries; the boundary is tracked using hysteresis techniques. Referring to fig. 3, fig. 3 is a schematic diagram of an embodiment of an edge map of the calibration line according to the present invention.
Optionally, the edge map is extracted linearly. Alternatively, a straight line can be extracted through a Hough transformation algorithm, and the basic principle of Hough transformation is that a curve (including the straight line) in an image space is transformed into a parameter space, and the description parameter of the curve is determined by detecting an extreme point in the parameter space, so that a regular curve in the image is extracted.
And obtaining the linear segment data of the edge of the calibration line after linear extraction. Referring to fig. 4, fig. 4 is a schematic diagram of an embodiment of an edge map after straight line extraction according to the present invention.
And dividing the extracted straight line segment into a vertical line segment and a horizontal line segment according to coordinate points at two ends of the extracted straight line segment.
S130: and determining a first calibration line and a second calibration line based on the linear segment data, obtaining a first parameter corresponding to the first calibration line based on the first calibration line, and obtaining a second parameter corresponding to the second calibration line based on the second calibration line.
The first calibration line and the second calibration line form a preset angle. Alternatively, the first calibration line may be a vertical calibration line and the second calibration line may be a horizontal calibration line, with 90 ° between the vertical calibration line and the horizontal calibration line.
For convenience of description of the solution of this embodiment, a description will be given below with a vertical calibration line as a first calibration line and a horizontal calibration line as a second calibration line. It will be appreciated that when the first calibration line is not a vertical calibration line and the second calibration line is not a horizontal calibration line, the manner of calculating the straight angle is equally applicable to the following manner, and those skilled in the art will understand that the description is not repeated here.
Optionally, the step of determining a vertical calibration line based on the line segment data and obtaining a vertical parameter corresponding to the vertical calibration line based on the vertical calibration line includes:
in some embodiments, the step of determining a vertical calibration line based on the line segment data and obtaining a vertical parameter corresponding to the vertical calibration line based on the vertical calibration line includes:
judging straight line characteristics based on the straight line segment data, and determining a plurality of vertical line segments; carrying out weighted fusion on a plurality of vertical line segments to obtain vertical lines; and identifying the vertical lines after weighted fusion, determining a left vertical calibration line and a right vertical calibration line, and obtaining the slope of the left vertical calibration line, the intercept of the left vertical calibration line, the slope of the right vertical calibration line and the intercept of the right vertical calibration line.
The straight line segment data comprises coordinates of two endpoints of the straight line segment, so that straight line characteristic judgment is carried out based on the straight line segment data, and a plurality of vertical line segments are determined; the step of weighting and fusing the plurality of vertical line segments to obtain the vertical line may include:
obtaining two end points (x 1 ，y 1 ）、（x 2 ，y 2 ) Corresponding to a first difference on the y-axis; if the first difference value is larger than a first preset value, the straight line segment corresponding to the first difference value is a vertical straight line segment; if the slope difference between the vertical line segments is smaller than a second preset value, judging the corresponding vertical line segments as the edges of the same calibration line; and fusing the edges of the same calibration line based on a weighted average method to obtain a vertical line.
It should be noted that if the first difference value is smaller than or equal to the first preset value, the y-axis line segment corresponding to the first difference value is discarded; if the slope difference between the vertical line segments is greater than or equal to a second preset value, the corresponding vertical line segments are judged to be the edges of two different calibration lines.
Alternatively, the first preset value may be set to 5, and the determination (x 1 ，y 1 ）、（x 2 ，y 2 ) A linear segment having y values differing by more than 5, and calculating the slope k, intercept l, and length d of the linear segment according to the following formula:
and ordering the vertical line segments according to the gradient, and if the gradient difference between the vertical line segments is smaller, namely the gradient difference between the vertical line segments is smaller than a second preset value, judging that the vertical line segments are positioned at the edge of the same calibration line, so that fusion can be carried out.
Assuming that n vertical line segments to be fused in a certain slope interval are provided, the fused calibration line formula is as follows:
wherein, the liquid crystal display device comprises a liquid crystal display device,
representing the slope of the nth vertical segment;
representing the length of the nth vertical line segment;
representing the intercept of the nth vertical segment.
Further, the step of identifying the weighted and fused vertical line, determining a left vertical calibration line and a right vertical calibration line, and obtaining a slope of the left vertical calibration line, an intercept of the left vertical calibration line, a slope of the right vertical calibration line, and an intercept of the right vertical calibration line, includes:
taking the straight line with negative and maximum slope in the vertical line as a left vertical calibration line, and obtaining the corresponding slope k of the left vertical calibration line l And intercept l l The method comprises the steps of carrying out a first treatment on the surface of the Taking the straight line with positive and minimum slope in the vertical line as a right vertical calibration line, and obtaining the corresponding slope k of the right vertical calibration line r And intercept l r 。
Optionally, the step of determining a horizontal calibration line based on the line segment data and obtaining a horizontal parameter corresponding to the horizontal calibration line based on the horizontal calibration line includes:
in some embodiments, determining a horizontal calibration line based on the line segment data, and obtaining a horizontal parameter corresponding to the horizontal calibration line based on the horizontal calibration line, includes:
judging straight line characteristics based on the straight line segment data, and determining a plurality of horizontal line segments; carrying out weighted fusion on a plurality of horizontal line segments to obtain a horizontal line; and identifying the weighted and fused horizontal lines, determining a first horizontal calibration line and a second horizontal calibration line, and obtaining the y-axis value of the first horizontal calibration line and the y-axis value of the second horizontal calibration line. Wherein the y-axis value of the first horizontal calibration line and the y-axis value of the second horizontal calibration line are horizontal parameters.
The straight line segment data comprises coordinates of two endpoints of the straight line segment, so that straight line feature judgment is carried out based on the straight line segment data, and a plurality of horizontal line segments are determined; carrying out weighted fusion on a plurality of horizontal line segments to obtain a horizontal line, wherein the method comprises the following steps:
obtaining two end points (x 1 ，y 1 ）、（x 2 ，y 2 ) A first difference on the y-axis of (2); if the first difference is smaller than the third preset value and x 1 、x 2 In a preset range, determining a straight line segment corresponding to the first difference value as a horizontal line segment; obtaining a y-axis value of the horizontal line segment, an x-axis value and an x-axis difference value between two endpoints; if the y-axis difference value between the horizontal line segments is smaller than a fourth preset value, judging the corresponding horizontal line segments as the edges of the same calibration line; and fusing the edges of the same calibration line based on a weighted average method to obtain a horizontal line and a y-axis value corresponding to the horizontal line.
It should be noted that if the first difference is greater than or equal to the third preset value, or x 1 、x 2 If the first difference value is not within the preset range, determining that the straight line segment corresponding to the first difference value is discarded; if the y-axis difference between the horizontal line segments is greater than or equal to a fourth preset value, the corresponding horizontal line segment is judged to be the edge of two different calibration lines.
The third preset value may be set to 3 and the preset range may refer to 50% in the middle of the calibration image. Determine two end points (x) 1 ，y 1 ）、（x 2 ，y 2 ) The y values of (c) differ by less than 3 and the x values are horizontal line segments with 50% of the straight line segments in the middle of the image. And the y-axis value of the horizontal line segment, the x-axis value and the x-axis difference Deltax between the two endpoints are recorded.
If y 1 ≠y 2 The y-axis value may be y 1 And y 2 Average value of (2).
If the y value difference between two or more horizontal line segments is smaller (for example, smaller than a fourth preset value), the corresponding horizontal line segments are judged to be the edges of the same calibration line. If there are n horizontal line segments at the edge of the same calibration line, these horizontal line segments can be fused using a weighted average method. The fusion formula is as follows:
wherein, the liquid crystal display device comprises a liquid crystal display device,
a y-axis value representing an nth horizontal line segment;
representing the x-axis difference of the nth horizontal line segment.
And calculating the y value of the fused horizontal line according to the formula.
According to the monocular ranging model calibration method of the curtain wall robot, provided by the invention, the weighted and fused horizontal lines are identified, the first horizontal calibration line and the second horizontal calibration line are determined, and the y-axis value of the first horizontal calibration line and the y-axis value of the second horizontal calibration line are obtained, and the method comprises the following steps:
taking a straight line with the largest y-axis value in the horizontal line as a first horizontal calibration line to obtain the y-axis value y' of the first horizontal calibration line 1 The method comprises the steps of carrying out a first treatment on the surface of the Taking a straight line with a second largest y-axis value in the horizontal line as a second horizontal calibration line to obtain a y-axis value y' of the second horizontal calibration line 2 。
S140: coordinate conversion model parameters are determined based on the first parameter and the second parameter.
From the above calculation, the vertical parameter may include the slope k of the left vertical calibration line l Slope of right vertical calibration line
Intercept l of left vertical calibration line l And the intercept l of the right vertical calibration line r . The horizontal parameter may include a y-axis value y' of the first horizontal calibration line 1 And a y-axis value y' of the second horizontal calibration line 2 。
Accordingly, determining coordinate conversion model parameters based on the vertical parameters and the horizontal parameters includes:
slope based on left vertical scaling line
And the slope of the right vertical calibration line
Determining a slope coefficient a; intercept based on left vertical calibration line
Intercept with right vertical calibration line
Determining an intercept coefficient b; based on the width between the vertical calibration lines
Width between horizontal calibration lines
The slope coefficient a, the intercept coefficient b and the y-axis value y' of the first horizontal calibration line 1 And a y-axis value y' of the second horizontal calibration line 2 Determining coordinate transformation model parameters
。
Specifically, the coordinate transformation model parameters are as follows:
the width W of the vertical calibration line may be determined by the distance between the left vertical calibration line and the right vertical calibration line; width between horizontal calibration lines
May be determined by the distance between the first horizontal calibration line and the second horizontal calibration line.
After the parameters of the coordinate conversion model are calculated, the parameters can be stored in the parameter file corresponding to the coordinate conversion model.
S150: and calculating the distance of each line segment based on the edge graph of the calibration line and the coordinate conversion model parameters, and obtaining a calibration result according to the distance of each line segment.
According to the monocular ranging model calibration method of the curtain wall robot, provided by the invention, the distances of all line segments are calculated based on the edge map and the coordinate conversion model parameters of the calibration line, and the calibration result is obtained according to the distances of all line segments, and the method comprises the following steps:
determining the horizontal distance between the horizontal line segment and the curtain wall robot and the vertical distance between the vertical line segment and the central axis of the curtain wall robot, and outputting the position and distance data of the straight line segment to obtain a simulation image; drawing a simulation calibration line position in the simulation image according to the vertical parameter and the horizontal parameter; referring to fig. 5, fig. 5 is a schematic diagram of an embodiment of a simulation image according to the present invention.
The values on the line segments in the simulated image represent the distance of the line segment from the camera, wherein the values on the horizontal line segment represent the horizontal distance of the horizontal line segment from the curtain wall robot, such as 277, 324, 414, 467, in centimeters; the values on the vertical line segments represent the vertical distance of the vertical line segments from the central axis of the curtain wall robot, e.g. 2, 6, 11, 60, 100, 106, 158, 166, all in cm. The two straight lines intersecting in fig. 5 represent simulated calibration lines. Wherein the simulated calibration line is also understood to be a calibration line identified by the curtain wall robot.
Note that, each line segment in the analog image may be displayed with a numerical value indicating a distance between the line segment and the camera, which is not illustrated in fig. 5.
Furthermore, it should be noted that since the values on the vertical line segment represent the vertical distance between the vertical line segment and the central axis of the curtain wall robot, and the vertical line segment is not parallel to the central axis of the curtain wall robot in the simulation image, a plurality of values, for example, 100 and 106, may be included on one vertical line segment.
If the position of the simulated calibration line meets the first preset requirement and the simulated image meets the second preset requirement, the completion of the monocular ranging model calibration of the curtain wall robot is confirmed.
The first preset requirement is used for limiting the position relation between the analog calibration line and the horizontal calibration line and the vertical calibration line, and the second preset requirement is used for limiting the similarity between the analog image and the setting site.
For example, if the identified calibration line is on the inner side of the two vertical calibration lines and on the upper and lower sides of the horizontal calibration line (meeting the first preset requirement), and the width of each identified inspection line and the distance from the calibration line are both consistent with the field setting (meeting the second preset requirement), then the model calibration is considered to be successfully completed.
If the position of the simulated calibration line does not meet the first preset requirement or the simulated image does not meet the second preset requirement, the calibration is unsuccessful; and (3) the robot needs to contact the curtain wall, is in contact with the glass curtain wall, turns the machine body up and down, and repeats the steps to calibrate.
In summary, according to the monocular ranging model calibration method of the curtain wall robot provided by the embodiment, pixels corresponding to the preset calibration line color are extracted from the calibration image, and binarization processing is performed on the calibration image based on the pixels; performing edge detection on the binarized calibration image to obtain an edge map of a calibration line; performing linear extraction on the edge map to obtain linear line segment data of the edge of the calibration line; determining a vertical calibration line and a horizontal calibration line based on the linear segment data, and obtaining a vertical parameter and a horizontal parameter; determining coordinate conversion model parameters based on the vertical parameters and the horizontal parameters; and calculating the distance of each line segment based on the edge graph of the calibration line and the coordinate conversion model parameters, and obtaining a calibration result according to the distance of each line segment. Through the mode, the curtain wall robot can be calibrated, the robot can automatically identify the calibration line and calculate the model parameters after being adsorbed on the glass curtain wall, the parameters needing manual measurement are reduced, the automation degree of the calibration process is increased, the working efficiency of calibration in mass production is improved, and the requirement on the professional degree of staff is lower; in addition, the invention performs calibration under the running state of the robot on the glass curtain wall, so that the model parameters comprise the position deviation of the camera in the assembly process and the condition of the pose in the working state, and the data are more accurate and more in line with the working environment.
The invention also provides an electronic device, refer to fig. 6, and fig. 6 is a schematic structural diagram of an embodiment of the electronic device. In this embodiment, the electronic device may include a memory 620, a processor 610, and a computer program stored on the memory 620 and executable on the processor 610. The processor 610 implements the method for calibrating the monocular ranging model of the curtain wall robot provided by the above methods when executing the program.
Optionally, the electronic device may further comprise a communication bus 630 and a communication interface (Communications Interface) 640, wherein the processor 610, the communication interface 640, and the memory 620 communicate with each other via the communication bus 630. The processor 610 may invoke logic instructions in the memory 620 to perform a method for monocular ranging model calibration of a curtain wall robot, the method comprising:
obtaining a calibration image through a monocular camera of the curtain wall robot; the calibration image comprises a preset calibration line on the glass curtain wall; obtaining an edge map of a calibration line and straight line segment data of the edge of the calibration line based on the calibration image; determining a first calibration line and a second calibration line based on the linear segment data, obtaining a first parameter corresponding to the first calibration line based on the first calibration line, and obtaining a second parameter corresponding to the second calibration line based on the second calibration line; the first calibration line and the second calibration line form a preset angle; determining coordinate transformation model parameters based on the first parameter and the second parameter; and calculating the distance of each line segment based on the edge graph of the calibration line and the coordinate conversion model parameters, and obtaining a calibration result according to the distance of each line segment.
Further, the logic instructions in the memory 620 described above may be implemented in the form of software functional units and may be stored in a computer readable storage medium when sold or used as a stand alone product. Based on this understanding, the technical solution of the present invention may be embodied essentially or in a part contributing to the prior art or in a part of the technical solution, in the form of a software product stored in a storage medium, comprising several instructions for causing a computer device (which may be a personal computer, a server, a network device, etc.) to perform all or part of the steps of the method according to the embodiments of the present invention. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read-Only Memory (ROM), a random access Memory (RAM, random Access Memory), a magnetic disk, or an optical disk, or other various media capable of storing program codes.
On the other hand, the invention also provides a curtain wall robot which comprises a monocular camera, a robot body and the electronic equipment.
The apparatus embodiments described above are merely illustrative, wherein the elements illustrated as separate elements may or may not be physically separate, and the elements shown as elements may or may not be physical elements, may be located in one place, or may be distributed over a plurality of network elements. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment. Those of ordinary skill in the art will understand and implement the present invention without undue burden.
From the above description of the embodiments, it will be apparent to those skilled in the art that the embodiments may be implemented by means of software plus necessary general hardware platforms, or of course may be implemented by means of hardware. Based on this understanding, the foregoing technical solution may be embodied essentially or in a part contributing to the prior art in the form of a software product, which may be stored in a computer readable storage medium, such as ROM/RAM, a magnetic disk, an optical disk, etc., including several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute the method described in the respective embodiments or some parts of the embodiments.
Finally, it should be noted that: the above embodiments are only for illustrating the technical solution of the present invention, and are not limiting; although the invention has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical scheme described in the foregoing embodiments can be modified or some technical features thereof can be replaced by equivalents; such modifications and substitutions do not depart from the spirit and scope of the technical solutions of the embodiments of the present invention.
Claims (8)
1. A method for calibrating a monocular ranging model of a curtain wall robot is characterized by comprising the following steps:
obtaining a calibration image through a monocular camera of the curtain wall robot; the calibration image comprises a preset calibration line on the glass curtain wall; the preset calibration line comprises two line segments perpendicular to a horizontal plane and one line segment parallel to the horizontal plane, wherein the interval width of the two line segments perpendicular to the horizontal plane is the body width of the curtain wall robot;
based on the calibration image, obtaining an edge map of a calibration line and straight line segment data of the edge of the calibration line;
determining a vertical calibration line based on the linear segment data, and obtaining a vertical parameter corresponding to the vertical calibration line based on the vertical calibration line;
determining a horizontal calibration line based on the linear segment data, and obtaining a horizontal parameter corresponding to the horizontal calibration line based on the horizontal calibration line;
determining coordinate conversion model parameters based on the vertical parameters and the horizontal parameters;
calculating the distance of each line segment based on the edge graph of the calibration line and the coordinate conversion model parameter, and obtaining a calibration result according to the distance of each line segment;
wherein the vertical parameter comprises the slope of a left vertical calibration line
Slope of right vertical calibration line +.>
Intercept of left vertical calibration line +.>
And the intercept of the right vertical calibration line +.>
；
The horizontal parameter comprises a y-axis value y' of a first horizontal calibration line 1 And a y-axis value y' of the second horizontal calibration line 2 ；
The determining coordinate transformation model parameters based on the vertical parameters and the horizontal parameters includes:
wherein a is a slope coefficient; b is the intercept coefficient；FThe parameters are coordinate transformation model parameters; width between vertical calibration lines
The width between the horizontal calibration lines is determined by the distance between the left-hand vertical calibration line and the right-hand vertical calibration line +.>
Is determined by the distance between the first horizontal calibration line and the second horizontal calibration line.
2. The method for calibrating a monocular ranging model of a curtain wall robot according to claim 1, wherein determining a vertical calibration line based on the straight line segment data, and obtaining a vertical parameter corresponding to the vertical calibration line based on the vertical calibration line, comprises:
performing linear characteristic judgment based on the linear segment data, and determining a plurality of vertical linear segments;
weighting and fusing the plurality of vertical line segments to obtain a vertical line;
and identifying the vertical lines after weighted fusion, determining a left vertical calibration line and a right vertical calibration line, and obtaining the slope of the left vertical calibration line, the intercept of the left vertical calibration line, the slope of the right vertical calibration line and the intercept of the right vertical calibration line.
3. The method for calibrating a monocular ranging model of a curtain wall robot according to claim 2, wherein determining a vertical calibration line based on the straight line segment data, and obtaining a vertical parameter corresponding to the vertical calibration line based on the vertical calibration line, comprises:
obtaining two end points (x 1 ，y 1 ）、（x 2 ，y 2 ) Corresponding to a first difference on the y-axis;
if the first difference value is larger than a first preset value, the straight line segment corresponding to the first difference value is a vertical line segment;
if the slope difference between the vertical line segments is smaller than a second preset value, judging the corresponding vertical line segments as the edges of the same calibration line;
based on a weighted average method, fusing the edges of the same calibration line to obtain the vertical line;
taking a straight line with negative and maximum slope in the vertical line as the left vertical calibration line, and obtaining the slope and intercept corresponding to the left vertical calibration line;
and taking a straight line with positive and minimum slope in the vertical line as the right vertical calibration line, and obtaining the slope and intercept corresponding to the right vertical calibration line.
4. The method for calibrating a monocular ranging model of a curtain wall robot according to claim 1, wherein determining a horizontal calibration line based on the straight line segment data, and obtaining a horizontal parameter corresponding to the horizontal calibration line based on the horizontal calibration line, comprises:
performing linear characteristic judgment based on the linear segment data, and determining a plurality of horizontal segments;
carrying out weighted fusion on the plurality of horizontal line segments to obtain a horizontal line;
identifying the weighted and fused horizontal lines, determining a first horizontal calibration line and a second horizontal calibration line, and obtaining a y-axis value of the first horizontal line and a y-axis value of the second horizontal line; wherein the y-axis value of the first horizontal line and the y-axis value of the second horizontal line are horizontal parameters.
5. The method for calibrating a monocular ranging model of a curtain wall robot according to claim 4, wherein determining a horizontal calibration line based on the straight line segment data and obtaining a horizontal parameter corresponding to the horizontal calibration line based on the horizontal calibration line comprises:
obtaining two end points (x 1 ，y 1 ）、（x 2 ，y 2 ) A first difference on the y-axis of (2);
if the first difference is smaller thanA third preset value, and x 1 、x 2 In a preset range, determining a straight line segment corresponding to the first difference value as a horizontal line segment;
obtaining a y-axis value of the horizontal line segment, an x-axis value and an x-axis difference value between two endpoints;
if the y-axis difference value between the horizontal line segments is smaller than a fourth preset value, judging the corresponding horizontal line segments as edges of the same calibration line;
based on a weighted average method, fusing the edges of the same calibration line to obtain the horizontal line and a y-axis value corresponding to the horizontal line;
taking a straight line with the largest y-axis value in the horizontal lines as the first horizontal line to obtain the y-axis value of the first horizontal line;
and taking the straight line with the second largest y-axis value in the horizontal line as the second horizontal line to obtain the y-axis value of the second horizontal line.
6. The method for calibrating a monocular ranging model of a curtain wall robot according to claim 5, wherein calculating the distance of each line segment based on the edge map of the calibration line and the coordinate transformation model parameter, and obtaining the calibration result according to the distance of each line segment, comprises:
determining the horizontal distance between a horizontal line segment and a curtain wall robot and the vertical distance between a vertical line segment and a central axis of the curtain wall robot, and outputting the position and distance data of a straight line segment to obtain a simulation image;
drawing a simulation calibration line position in the simulation image according to the vertical parameter and the horizontal parameter;
and if the position of the simulation calibration line meets the first preset requirement and the simulation image meets the second preset requirement, confirming that the monocular ranging model of the curtain wall robot is calibrated.
7. An electronic device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, characterized in that the processor implements the method for calibrating a monocular ranging model of a curtain wall robot according to any one of claims 1 to 6 when executing the program.
8. A curtain wall robot comprising a monocular camera, a robot body and the electronic device of claim 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211403530.1A CN115661267B (en) | 2022-11-10 | 2022-11-10 | Monocular ranging model calibration method, electronic equipment and curtain wall robot |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211403530.1A CN115661267B (en) | 2022-11-10 | 2022-11-10 | Monocular ranging model calibration method, electronic equipment and curtain wall robot |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN115661267A CN115661267A (en) | 2023-01-31 |
| CN115661267B true CN115661267B (en) | 2023-04-25 |
Family
ID=85021275
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202211403530.1A Active CN115661267B (en) | 2022-11-10 | 2022-11-10 | Monocular ranging model calibration method, electronic equipment and curtain wall robot |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115661267B (en) |
Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US5790403A (en) * | 1994-07-12 | 1998-08-04 | Honda Giken Kogyo Kabushiki Kaisha | Lane image processing system for vehicle |
Family Cites Families (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP5394908B2 (en) * | 2009-12-17 | 2014-01-22 | 住友重機械プロセス機器株式会社 | Coke oven wall observation device camera calibration method and program |
| CN103729837A (en) * | 2013-06-25 | 2014-04-16 | 长沙理工大学 | Rapid calibration method of single road condition video camera |
| EP3174007A1 (en) * | 2015-11-30 | 2017-05-31 | Delphi Technologies, Inc. | Method for calibrating the orientation of a camera mounted to a vehicle |
| CN115223031B (en) * | 2022-09-20 | 2022-12-20 | 凌度(广东)智能科技发展有限公司 | Monocular frame ranging method and device, medium and curtain wall robot |
-
2022
- 2022-11-10 CN CN202211403530.1A patent/CN115661267B/en active Active
Patent Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US5790403A (en) * | 1994-07-12 | 1998-08-04 | Honda Giken Kogyo Kabushiki Kaisha | Lane image processing system for vehicle |
Also Published As
| Publication number | Publication date |
|---|---|
| CN115661267A (en) | 2023-01-31 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN104408460B (en) | A kind of lane detection and tracking detection method | |
| CN109660783B (en) | Virtual reality parallax correction | |
| CN107507235B (en) | Registration method of color image and depth image acquired based on RGB-D equipment | |
| CN105279372B (en) | A kind of method and apparatus of determining depth of building | |
| US20170091996A1 (en) | Methods and Systems for Detecting and Combining Structural Features in 3D Reconstruction | |
| CN103745452B (en) | Camera external parameter assessment method and device, and camera external parameter calibration method and device | |
| CN110490936B (en) | Calibration method, device and equipment of vehicle camera and readable storage medium | |
| CN112991193B (en) | Depth image restoration method, device and computer-readable storage medium | |
| CN110473221B (en) | Automatic target object scanning system and method | |
| Mirzaalian Dastjerdi et al. | Measuring surface area of skin lesions with 2D and 3D algorithms | |
| CN110570435B (en) | Method and device for carrying out damage segmentation on vehicle damage image | |
| CN105835507A (en) | Method for attaching mobile phone cover lens to liquid crystal display | |
| CN105147311A (en) | Visual equipment assisted scanning and positioning method and system applied to CT system | |
| US20150256815A1 (en) | 3d camera calibration | |
| JP2022059013A (en) | Information processing apparatus, recognition support method, and computer program | |
| US11282180B1 (en) | Object detection with position, pose, and shape estimation | |
| CN111368717A (en) | Sight line determining method and device, electronic equipment and computer readable storage medium | |
| CN112017231A (en) | Human body weight identification method and device based on monocular camera and storage medium | |
| CN101996416A (en) | 3D face capturing method and equipment | |
| CN110648362A (en) | Binocular stereo vision badminton positioning identification and posture calculation method | |
| WO2012044308A1 (en) | Projection matrix | |
| CN115661267B (en) | Monocular ranging model calibration method, electronic equipment and curtain wall robot | |
| TWI462027B (en) | Image processing device and image processing method thereof | |
| CN107403448B (en) | Cost function generation method and cost function generation device | |
| TWI595446B (en) | Method for improving occluded edge quality in augmented reality based on depth camera |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |