CN114708318B - Unknown surface curvature measurement method based on depth camera - Google Patents
Unknown surface curvature measurement method based on depth camera Download PDFInfo
- Publication number
- CN114708318B CN114708318B CN202210380167.XA CN202210380167A CN114708318B CN 114708318 B CN114708318 B CN 114708318B CN 202210380167 A CN202210380167 A CN 202210380167A CN 114708318 B CN114708318 B CN 114708318B
- Authority
- CN
- China
- Prior art keywords
- curvature
- depth
- average value
- camera
- curve
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000691 measurement method Methods 0.000 title description 4
- 238000000034 method Methods 0.000 claims abstract description 21
- 239000011159 matrix material Substances 0.000 claims abstract description 14
- 238000001914 filtration Methods 0.000 claims abstract description 5
- 238000012545 processing Methods 0.000 claims description 8
- 238000007781 pre-processing Methods 0.000 abstract 1
- 238000004364 calculation method Methods 0.000 description 3
- 238000005259 measurement Methods 0.000 description 3
- 238000004458 analytical method Methods 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 238000011160 research Methods 0.000 description 2
- 238000013473 artificial intelligence Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000001514 detection method Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
- 238000005286 illumination Methods 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- 238000012544 monitoring process Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/60—Analysis of geometric attributes
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/70—Denoising; Smoothing
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/80—Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10028—Range image; Depth image; 3D point clouds
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Geometry (AREA)
- Image Analysis (AREA)
- Length Measuring Devices By Optical Means (AREA)
Abstract
An unknown surface curvature measuring method based on a depth camera firstly selects an RGB-D camera as acquisition equipment and calibrates the RGB-D camera; horizontally placing the cameras, continuously collecting depth images of a surface to be detected, and preprocessing by using a mean value filtering operation to obtain a relatively smooth depth image; converting the depth image by using a camera internal parameter matrix, and converting the depth image under the image pixel coordinate system into the depth image under the real world coordinate system; respectively calculating the average value of depth values of the depth images in the real coordinate system of the world according to the horizontal direction, the vertical direction and the time direction, performing curve fitting by using a least square method by taking three adjacent points as a group, calculating the average value and the variance of curvature in the two directions, and plotting, and judging whether the surface to be measured is flat or not according to the average value and the variance of the curvature, wherein the average value of the curvature is used as the curvature of the whole surface to be measured; the invention has the advantages of low cost, no contact, high precision and the like.
Description
Technical Field
The invention belongs to the technical field of image processing, and particularly relates to an unknown surface curvature measurement method based on a depth camera.
Background
In many application fields such as man-machine interaction, motion analysis, artificial intelligence, robots, unmanned operation, video monitoring and the like, depth information in a scene is main potential information, is an essential link for analysis and processing of depth images, and technology and application based on depth cameras are also continuously paid attention to.
At present, with the rapid development of technologies in related fields such as computer vision and image processing, research on shooting and processing of scene depth information is also becoming a research hotspot. A depth camera, as its name implies, is a generic term for cameras that can measure depth information in a scene. The existing depth cameras can be roughly divided into three types according to different shooting principles: ① A depth camera based on structured light principles; ② A depth camera based on time-of-flight principles; ③ A depth camera based on binocular stereoscopic vision principle. The depth camera based on the structured light principle is easy to be influenced by the smoothness and illumination of a shooting surface when shooting the depth image, so that the shot depth image is unclear; depth cameras based on the time-of-flight principle tend to have limited shooting ranges; the depth camera based on the binocular stereoscopic vision principle has the problem of left and right split vision, and needs to perform image alignment during shooting.
However, the existing methods for measuring the flatness and calculating the curvature of the surface to be measured mostly use a large number of complex measuring tools, meanwhile, because the error of manual measurement is large, the application is limited in the fields requiring high-precision measurement, and the flatness and curvature measurement cannot be accurately and rapidly performed, so that a low-cost measuring method for measuring the flatness and curvature of the surface is urgently needed.
Disclosure of Invention
In order to overcome the defects of the prior art, the invention aims to provide an unknown surface curvature measuring method based on a depth camera, which has the advantages of low cost, no contact, high precision and the like and meets the requirements of plane detection and curvature calculation.
In order to achieve the above purpose, the invention adopts the following technical scheme:
An unknown surface curvature measuring method based on a depth camera comprises the following steps:
step one: the RGB-D camera is selected as acquisition equipment, and the acquisition equipment is calibrated to obtain an internal parameter matrix H d of the depth camera:
Wherein, F xd is the horizontal component of the focal length of the depth camera, f yd is the vertical component of the focal length of the depth camera, c xd is the offset of the principal point of the depth camera in the horizontal direction, and c yd is the offset of the principal point of the depth camera in the vertical direction;
Step two: horizontally placing an RGB-D camera by using a level meter, continuously collecting N frames of depth images of a surface to be detected by using an RGB-D sensor, and processing by adopting an average filtering operation to obtain a relatively smooth depth image;
step three: converting the depth image by using a depth camera internal parameter matrix, and converting the depth image under the image pixel coordinate system into the depth image under the real world coordinate system;
Step four: respectively calculating the average value of the depth values of the depth images in the real coordinate system of the world according to the horizontal direction, the vertical direction and the time direction;
step five: and (3) performing curve fitting on the obtained average value of the depth values in the horizontal direction and the vertical direction by using a least square method by taking adjacent three points as a group, calculating and plotting the average value and the variance of the curvature in the two directions, judging whether the surface to be measured is flat or not according to the average value and the variance, and taking the average value of the curvature as the curvature of the whole surface to be measured.
The fifth concrete process comprises the following steps: let the current point of interest p coordinates be (x, y, z), then the coordinates of two points adjacent to the p point in the horizontal direction be (x 1,y1,z1) and (x 2,y2,z2), denoted as p 1 and p 2, and the coordinates of points adjacent to the p point in the vertical direction be (x 3,y3,z3) and (x 4,y4,z4), denoted as p 3 and p 4;
For three points p, p 1 and p 2 in the horizontal direction, curve fitting is performed by using a least square method, and if the fitted polynomial is set as y=a 0+a1x+a2x2, the sum of squares of distances from each point to the curve is minimum, namely the sum of squares of deviation At minimum, the fitted curve is closest to the original curve; the resulting matrix form of the curve is:
Then the curvature for the current point of interest is
For three points p, p 3 and p 4 in the vertical direction, curve fitting was also performed using the least squares method; assuming that the polynomial is set to y=b 0+b1x+b2x2, the sum of squares of points into the region of the curve is minimal, i.e. the sum of squares of deviations At the minimum, the fitted curve is closest to the original curve, and the matrix form of the obtained curve is as follows:
Then the curvature for the current point of interest is
Calculating the curvature of each point in the average depth, and calculating the curvature in the horizontal direction and the curvature in the vertical direction of each point; at the same time calculate the average value of the curvature in the horizontal directionAnd average value of curvature in vertical direction/>Variance of curvature in horizontal direction/>And variance of curvature in vertical direction/>Wherein use/>And/>Representing the curvature value of the current surface to be measured.
The invention has the advantages that:
(1) The invention only needs RGB-D camera as acquisition equipment, so it has the advantage of low hardware cost;
(2) The curvature of the unknown surface to be measured is measured by using the local curve fitting method, so that the method has the advantages of high precision, rapidness and stability.
Drawings
FIG. 1 is a flow chart of the method of the present invention.
Fig. 2 is a hardware platform of an embodiment of the invention.
Fig. 3 is a frame depth image acquired by the RGB-D camera of the embodiment.
Fig. 4 is a smoothed depth image obtained by mean filtering the depth image according to the embodiment.
Fig. 5 is a depth image of a real world coordinate system obtained by converting a smoothed depth map of a certain frame by an internal matrix of a depth camera according to an embodiment.
Fig. 6 is a block diagram illustrating the calculation of depth values in three directions based on consecutive depth images according to an embodiment.
Fig. 7 shows curvatures in two directions calculated based on depth values in the horizontal and vertical directions, (a) is curvature in the horizontal direction, and (b) is curvature in the vertical direction.
Detailed Description
The invention will now be described in detail with reference to the drawings and examples.
Referring to fig. 1 and 2, a depth camera-based unknown surface curvature measurement method includes the following steps:
Step one: an RGB-D camera is selected as acquisition equipment, and the acquisition equipment is calibrated by taking Azure Kinect as an example to obtain an internal reference matrix H d of the depth camera:
step two: horizontally placing an RGB-D camera by using a level meter, and continuously acquiring N frames of depth images of a surface to be detected by using an RGB-D sensor, wherein the depth images of a certain frame are shown in figure 3; the average filtering is used for processing the depth image, singular value points in the original depth image are removed, and a relatively smooth depth image is obtained, wherein the processing result of a certain frame of depth image is shown in fig. 4;
Step three: converting the depth image under the pixel image coordinate system into the depth image under the real world coordinate system by adopting the depth camera internal parameter matrix obtained in the step one, wherein the depth image under the real world coordinate system corresponding to the depth image under the pixel image coordinate system of a certain frame is shown in fig. 5;
Step four: respectively calculating the average value of the depth values of the depth images in the real coordinate system of the world according to the horizontal direction, the vertical direction and the time direction, wherein the horizontal direction is marked as x, the vertical direction is marked as y, the time direction is marked as z, and the average depth values of the three directions are shown in fig. 6;
Step five: performing curve fitting on the obtained average value of depth values in the horizontal direction and the vertical direction by using a least square method by taking adjacent three points as a group, calculating and plotting the average value and variance of curvature in the two directions, judging whether the surface to be measured is flat or not according to the average value and variance of the curvature, and taking the average value of the curvature as the curvature of the whole surface to be measured;
In the embodiment, for an average depth map, curvature calculation is performed on each pixel point; let the current point of interest p coordinates be (x, y, z), then the coordinates of two points adjacent to the p point in the horizontal direction be (x 1,y1,z1) and (x 2,y2,z2), denoted as p 1 and p 2, and the coordinates of points adjacent to the p point in the vertical direction be (x 3,y3,z3) and (x 4,y4,z4), denoted as p 3 and p 4;
For three points p, p 1 and p 2 in the horizontal direction, curve fitting is performed by using a least square method, and if the fitted polynomial is set as y=a 0+a1x+a2x2, the sum of squares of distances from each point to the curve is minimum, namely the sum of squares of deviation At minimum, the fitted curve is closest to the original curve; the resulting matrix form of the curve is:
Then the curvature for the current point of interest is
For three points p, p 3 and p 4 in the vertical direction, curve fitting was also performed using the least squares method; assuming that the polynomial is set to y=b 0+b1x+b2x2, the sum of squares of points into the region of the curve is minimal, i.e. the sum of squares of deviations At the minimum, the fitted curve is closest to the original curve, and the matrix form of the obtained curve is as follows:
Then the curvature for the current point of interest is Calculating the curvature of each point in the average depth, and calculating the curvature in the horizontal direction and the curvature in the vertical direction of each point; wherein the curvature in the horizontal direction is as in (a) of fig. 7, and the curvature in the vertical direction is as in (b) of fig. 7; at the same time calculate the average value of curvature in the horizontal direction/>And average value of curvature in vertical direction/>Variance of curvature in horizontal direction/>And variance of curvature in vertical direction/>Wherein use/>And/>Representing the curvature value of the current surface to be measured.
Claims (1)
1. The unknown surface curvature measuring method based on the depth camera is characterized by comprising the following steps of:
step one: the RGB-D camera is selected as acquisition equipment, and the acquisition equipment is calibrated to obtain an internal parameter matrix H d of the depth camera:
Wherein, F xd is the horizontal component of the focal length of the depth camera, f yd is the vertical component of the focal length of the depth camera, C xd is the offset of the principal point of the depth camera in the horizontal direction, and C yd is the offset of the principal point of the depth camera in the vertical direction;
Step two: horizontally placing an RGB-D camera by using a level meter, continuously collecting N frames of depth images of a surface to be detected by using an RGB-D sensor, and processing by adopting an average filtering operation to obtain a relatively smooth depth image;
step three: converting the depth image by using a depth camera internal parameter matrix, and converting the depth image under the image pixel coordinate system into the depth image under the real world coordinate system;
Step four: respectively calculating the average value of the depth values of the depth images in the real coordinate system of the world according to the horizontal direction, the vertical direction and the time direction;
Step five: performing curve fitting on the obtained average value of depth values in the horizontal direction and the vertical direction by using a least square method by taking adjacent three points as a group, calculating and plotting the average value and variance of curvature in the two directions, judging whether the surface to be measured is flat or not according to the average value and variance of the curvature, and taking the average value of the curvature as the curvature of the whole surface to be measured;
The fifth concrete process comprises the following steps: let the current point of interest p coordinates be (x, y, z), then the coordinates of two points adjacent to the p point in the horizontal direction be (x 1,y1,z1) and (x 2,y2,z2), denoted as p 1 and p 2, and the coordinates of points adjacent to the p point in the vertical direction be (x 3,y3,z3) and (x 4,y4,z4), denoted as p 3 and p 4;
For three points p, p 1 and p 2 in the horizontal direction, curve fitting is performed by using a least square method, and if the fitted polynomial is set as y=a 0+a1x+a2x2, the sum of squares of distances from each point to the curve is minimum, namely the sum of squares of deviation At minimum, the fitted curve is closest to the original curve; the resulting matrix form of the curve is:
Then the curvature for the current point of interest is
For three points p, p 3 and p 4 in the vertical direction, curve fitting was also performed using the least squares method; assuming that the polynomial is set to y=b 0+b1x+b2x2, the sum of squares of points into the region of the curve is minimal, i.e. the sum of squares of deviations At the minimum, the fitted curve is closest to the original curve, and the matrix form of the obtained curve is as follows:
Then the curvature for the current point of interest is
Calculating the curvature of each point in the average depth, and calculating the curvature in the horizontal direction and the curvature in the vertical direction of each point; at the same time calculate the average value of the curvature in the horizontal directionAnd average value of curvature in vertical direction/>Variance of curvature in horizontal direction/>And variance of curvature in vertical direction/>Wherein use/>And/>Representing the curvature value of the current surface to be measured.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210380167.XA CN114708318B (en) | 2022-04-12 | 2022-04-12 | Unknown surface curvature measurement method based on depth camera |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210380167.XA CN114708318B (en) | 2022-04-12 | 2022-04-12 | Unknown surface curvature measurement method based on depth camera |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN114708318A CN114708318A (en) | 2022-07-05 |
| CN114708318B true CN114708318B (en) | 2024-05-28 |
Family
ID=82174923
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210380167.XA Active CN114708318B (en) | 2022-04-12 | 2022-04-12 | Unknown surface curvature measurement method based on depth camera |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114708318B (en) |
Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| DE10247714A1 (en) * | 2001-10-24 | 2003-05-08 | Zeiss Carl | Low-pass filtering of measurement values acquired using a coordinate measurement instrument by use of a spline curve of any order fitted to the measurement values with the curve parameters defined by its own path length |
| DE102004030709A1 (en) * | 2004-06-25 | 2006-01-19 | Daimlerchrysler Ag | Curve path determination method in which the curve corresponding to a digital map route section, is determined and represented by route points not more than a defined distance from the curve |
| CN103389042A (en) * | 2013-07-11 | 2013-11-13 | 夏东 | Ground automatic detecting and scene height calculating method based on depth image |
| JP2017118396A (en) * | 2015-12-25 | 2017-06-29 | Kddi株式会社 | Program, device and method for calculating internal parameter of depth camera |
| CN109903277A (en) * | 2019-02-25 | 2019-06-18 | 电子科技大学 | A kind of scoliosis detection method based on polynomial curve fitting |
| CN110276791A (en) * | 2019-06-28 | 2019-09-24 | 北京航空航天大学 | A kind of depth camera emulation mode that parameter is configurable |
| KR102206108B1 (en) * | 2019-09-20 | 2021-01-21 | 광운대학교 산학협력단 | A point cloud registration method based on RGB-D camera for shooting volumetric objects |
| CN112595265A (en) * | 2020-12-07 | 2021-04-02 | 新拓三维技术(深圳)有限公司 | Method and equipment for measuring bending radius of cable |
| CN113607090A (en) * | 2021-07-23 | 2021-11-05 | 深圳市鹰眼在线电子科技有限公司 | Thickness measuring system, method and device for curved screen |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| EP2874118B1 (en) * | 2013-11-18 | 2017-08-02 | Dassault Systèmes | Computing camera parameters |
-
2022
- 2022-04-12 CN CN202210380167.XA patent/CN114708318B/en active Active
Patent Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| DE10247714A1 (en) * | 2001-10-24 | 2003-05-08 | Zeiss Carl | Low-pass filtering of measurement values acquired using a coordinate measurement instrument by use of a spline curve of any order fitted to the measurement values with the curve parameters defined by its own path length |
| DE102004030709A1 (en) * | 2004-06-25 | 2006-01-19 | Daimlerchrysler Ag | Curve path determination method in which the curve corresponding to a digital map route section, is determined and represented by route points not more than a defined distance from the curve |
| CN103389042A (en) * | 2013-07-11 | 2013-11-13 | 夏东 | Ground automatic detecting and scene height calculating method based on depth image |
| JP2017118396A (en) * | 2015-12-25 | 2017-06-29 | Kddi株式会社 | Program, device and method for calculating internal parameter of depth camera |
| CN109903277A (en) * | 2019-02-25 | 2019-06-18 | 电子科技大学 | A kind of scoliosis detection method based on polynomial curve fitting |
| CN110276791A (en) * | 2019-06-28 | 2019-09-24 | 北京航空航天大学 | A kind of depth camera emulation mode that parameter is configurable |
| KR102206108B1 (en) * | 2019-09-20 | 2021-01-21 | 광운대학교 산학협력단 | A point cloud registration method based on RGB-D camera for shooting volumetric objects |
| CN112595265A (en) * | 2020-12-07 | 2021-04-02 | 新拓三维技术(深圳)有限公司 | Method and equipment for measuring bending radius of cable |
| CN113607090A (en) * | 2021-07-23 | 2021-11-05 | 深圳市鹰眼在线电子科技有限公司 | Thickness measuring system, method and device for curved screen |
Non-Patent Citations (2)
| Title |
|---|
| 点云下地平面检测的RGB-D相机外参自动标定;孙士杰;宋焕生;张朝阳;张文涛;王璇;;中国图象图形学报;20180616(06);全文 * |
| 飞行时间深度相机和彩色相机的联合标定;周杰;安平;郑帅;严徐乐;左一帆;;信号处理;20170125(01);全文 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN114708318A (en) | 2022-07-05 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108613628B (en) | Overhead transmission line sag measurement method based on binocular vision | |
| CN112629441B (en) | 3D curved surface glass contour scanning detection method and system | |
| CN111288967B (en) | Remote high-precision displacement detection method based on machine vision | |
| CN102519400B (en) | Large slenderness ratio shaft part straightness error detection method based on machine vision | |
| CN108535321A (en) | A kind of building thermal technique method for testing performance based on three-dimensional infrared thermal imaging technique | |
| CN106996748A (en) | Wheel diameter measuring method based on binocular vision | |
| CN112949478A (en) | Target detection method based on holder camera | |
| CN111996883B (en) | Method for detecting width of road surface | |
| CN110879080A (en) | High-precision intelligent measuring instrument and measuring method for high-temperature forge piece | |
| CN110702343B (en) | Deflection measurement system and method based on stereoscopic vision | |
| CN114638909A (en) | Substation semantic map construction method based on laser SLAM and visual fusion | |
| Mi et al. | A vision-based displacement measurement system for foundation pit | |
| CN115880296B (en) | Machine vision-based prefabricated part quality detection method and device | |
| Wang et al. | Error analysis and improved calibration algorithm for LED chip localization system based on visual feedback | |
| CN113310433A (en) | Virtual binocular stereo vision measuring method based on line structured light | |
| CN113119129A (en) | Monocular distance measurement positioning method based on standard ball | |
| CN112595236A (en) | Measuring device for underwater laser three-dimensional scanning and real-time distance measurement | |
| CN116858139B (en) | Metal structure flatness measuring method based on binocular vision | |
| CN114708318B (en) | Unknown surface curvature measurement method based on depth camera | |
| CN116958092A (en) | Power transmission line sag rapid measurement method based on improved stereo matching algorithm | |
| CN114998422A (en) | High-precision rapid three-dimensional positioning system based on error compensation model | |
| CN115082555A (en) | High-precision displacement real-time measurement system and method of RGBD monocular camera | |
| CN210154538U (en) | Metal structure deformation measuring device based on machine vision | |
| CN112785647A (en) | Three-eye stereo image detection method and system | |
| CN111862327A (en) | Three-dimensional city modeling method for inspection robot |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |