CN113379824B - Quasi-circular fruit longitudinal and transverse diameter measuring method based on double-view-point cloud registration - Google Patents

Quasi-circular fruit longitudinal and transverse diameter measuring method based on double-view-point cloud registration Download PDF

Info

Publication number
CN113379824B
CN113379824B CN202110646831.6A CN202110646831A CN113379824B CN 113379824 B CN113379824 B CN 113379824B CN 202110646831 A CN202110646831 A CN 202110646831A CN 113379824 B CN113379824 B CN 113379824B
Authority
CN
China
Prior art keywords
point
point cloud
camera
fruit
calibration plate
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202110646831.6A
Other languages
Chinese (zh)
Other versions
CN113379824A (en
Inventor
饶秀勤
林洋洋
朱逸航
应义斌
徐惠荣
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Zhejiang University ZJU
Huanan Industrial Technology Research Institute of Zhejiang University
Original Assignee
Zhejiang University ZJU
Huanan Industrial Technology Research Institute of Zhejiang University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Zhejiang University ZJU, Huanan Industrial Technology Research Institute of Zhejiang University filed Critical Zhejiang University ZJU
Priority to CN202110646831.6A priority Critical patent/CN113379824B/en
Publication of CN113379824A publication Critical patent/CN113379824A/en
Application granted granted Critical
Publication of CN113379824B publication Critical patent/CN113379824B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/60—Analysis of geometric attributes
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/30—Determination of transform parameters for the alignment of images, i.e. image registration
    • G06T7/33—Determination of transform parameters for the alignment of images, i.e. image registration using feature-based methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30181—Earth observation
    • G06T2207/30188—Vegetation; Agriculture

Abstract

The invention discloses a method for measuring the longitudinal and transverse diameters of a similar round fruit based on double-view-point cloud registration. The method comprises the following steps: firstly, a double-visual-angle acquisition system is arranged, the poses of a main camera and an auxiliary camera are calibrated, and an initial conversion matrix for point cloud coarse registration is obtained; then, iterating to minimize the distance error of the correct matching point pair to obtain an optimized transformation matrix; and controlling the main camera and the auxiliary camera to synchronously acquire fruit point clouds, performing fine registration, down sampling, filtering and outlier removal on the acquired fruit point clouds by utilizing an optimized transformation matrix, and finally acquiring the longitudinal and transverse diameters of the quasi-circular fruits by adopting an OBB bounding box. The invention can avoid the damage to the quasi-round fruits; meanwhile, point cloud fine registration is carried out through an optimized conversion matrix obtained through iteration, the problems of large calculation amount of matching points, large number of point clouds, long time consumption and the like in a multi-view point cloud registration process are solved, time and labor can be saved, and online production of round-like fruits is facilitated.

Description

Quasi-circular fruit longitudinal and transverse diameter measuring method based on double-view-point cloud registration
Technical Field
The invention relates to a method for measuring the longitudinal and transverse diameters of a similar round fruit, in particular to a method for measuring the longitudinal and transverse diameters of the similar round fruit based on double-view-point cloud registration.
Background
China is a world with large fruit yield, and the total fruit yield reaches 25688.35 ten thousand tons in recent years, wherein the total fruit yields of common round-like fruits such as apples, oranges and melons respectively reach 4242.54 ten thousand tons, 4584.54 ten thousand tons and 1355.70 ten thousand tons. The quality grading of the quasi-circular fruits is beneficial to improving the commercialized quality of the quasi-circular fruits and promoting the qualification price and the high quality and the high price of the quasi-circular fruits in circulation. The longitudinal diameter and the transverse diameter of the quasi-circular fruits are one of the important parameters of the shape of the fruits (GB/T12947-. Compared with the traditional manual measurement, the method consumes time and labor, adopts the machine vision technology to measure the longitudinal and transverse diameters of the similar round fruits by the double-visual-angle point cloud registration method, has the advantages of non-contact, rapidness, no damage and the like, and has important practical application value.
The accurate longitudinal and transverse diameter of the quasi-circular fruit needs to be obtained by longitudinally and transversely cutting the quasi-circular fruit, but the adoption of the cutting method can cause serious damage to the quasi-circular fruit and cannot be used in the grading production process of the quasi-circular fruit.
Common methods for measuring the longitudinal and transverse diameter shapes of fruits by using two-dimensional images mainly comprise a minimum external method and a boundary point measurement method: octopus et al (2001) (Octopus, should be construed and bin. Low-level processing of apple images and size detection [ J ]. Zhejiang agricultural science, 2001(04):38-41.) after searching and refining contour lines by using a chain symbol method, the longitudinal diameter and the transverse diameter of an apple are calculated by using the minimum circumscribed rectangle, and the result shows that the correlation coefficient between the measured size and the actual size reaches 0.955; widehua et al (2011) (Shenbao, widehua, Yi Jianjun. apple diameter detection technology [ J ] based on minimum circumcircle method, agricultural research, 2011,33(12): 131-; gongchang et al (2013) (Gongchang, smart, Huangjie, poplar billow. research of a grapefruit size detection system based on machine vision [ J ] agricultural machinery research, 2013,35(11):22-25.) machine vision is utilized to obtain 4 boundary points of the upper, lower, left and right of a target image, and the longitudinal diameter and the transverse diameter of the grapefruit are estimated through the boundary points. The method for measuring the longitudinal and transverse diameters of the fruits based on the two-dimensional image can quickly realize the calculation of the longitudinal and transverse diameters of the fruits, but has higher requirements on the setting of a quasi-circular fruit illumination environment with a stronger light reflection characteristic on the surface and the like.
In recent years, with the development of three-dimensional reconstruction technology and consumer-level image acquisition equipment, researchers have measured the size or volume of a fruit by performing three-dimensional reconstruction in a manner of generating a fruit full-surface point cloud: yawe et al (2020) (Yawei, W, yifeic. front medical measurement Based on Three-Dimensional reconstraction. agromy, 2020,10,455) place a pear on a rotating table, obtain a Three-Dimensional point cloud picture of the pear using 9 pictures, and calculate its Three-Dimensional size. However, the method has the difficulties of large calculation amount of matching points, large number of point clouds and the like, takes long time and is not suitable for a production field. In addition, round-like fruits with smooth surfaces, such as honey pomelos and the like, have no obvious characteristic points and can be used for information matching.
Disclosure of Invention
In order to solve the problems and requirements in the background art, the invention provides a method for measuring the longitudinal and transverse diameters of a similar round fruit based on double-view-point cloud registration.
The technical scheme of the invention is as follows:
the invention comprises the following steps:
1) building a double-view acquisition system: the double-view-angle acquisition system comprises a storage platform, a quasi-circular fruit, two RGBD cameras, a synchronization line, a data transmission line, a calibration plate and a host;
the two RGBD cameras and the host are arranged on the object placing platform, the two RGBD cameras are connected through a synchronization line, the two RGBD cameras are respectively connected with the host through data transmission lines, the quasi-circular fruit or the calibration plate is arranged on the object placing platform, and the quasi-circular fruit or the calibration plate is arranged on one side of the two RGBD cameras and the host; the optical axes of the two RGBD cameras face towards the quasi-circular fruit or the calibration plate, and the optical axes of the two RGBD cameras form an included angle;
2) establishing a world coordinate system: recording one RGBD camera as a main camera C1The other RGBD camera is a sub-camera C2With the main camera C1The optical center of (A) is the origin of the world coordinate system and the main camera C1The optical axis of the camera is outward a Z axis of a world coordinate system, and a plane formed by an X axis and a Y axis of the world coordinate system is parallel to the main camera C1The X-axis is perpendicular to the Y-axis;
3) calibrating a main camera C1Auxiliary camera C2Pose: placing the calibration plate on the object placing platform, collecting the calibration plate by using the double-view collection system, and obtaining the main camera C1And a sub-camera C2Each of which isThe collected color image of the calibration plate and the corresponding depth image of the calibration plate are calibrated by a binocular calibration method for the main camera C1And a sub-camera C2Respectively acquiring color images of the calibration plates, processing the color images to obtain a secondary camera C2Coordinate conversion to main camera C1Initial transformation matrix T of coordinates12From the main camera C1And a sub-camera C2Respectively forming a main camera calibration plate point cloud and an auxiliary camera calibration plate point cloud by the collected calibration plate color images and the corresponding calibration plate depth images;
4) roughly registering and matching the point clouds of the calibration plates of the main camera and the auxiliary camera to obtain all correct matching point pairs of the point clouds of the rough registration calibration plates;
5) optimizing an initial transformation matrix T12；
6) Repeating the step 5) to iteratively optimize the transformation matrix T12Obtaining an optimized transformation matrix
Figure BDA0003110187960000021
7) Obtaining main and auxiliary point clouds of the quasi-circular fruits: placing the quasi-circular fruits on a storage platform, and controlling two RGBD cameras to synchronously acquire a main point cloud and an auxiliary point cloud of the quasi-circular fruits;
8) carrying out cloud fine registration on main points and auxiliary points of the quasi-circular fruits: using the optimized transformation matrix obtained in step 6)
Figure BDA0003110187960000031
Registering the main point cloud and the auxiliary point cloud of the quasi-circular fruit to obtain a precisely registered fruit point cloud of the quasi-circular fruit;
9) fine registration fruit point cloud down sampling: using a VoxelGrid filter to carry out down-sampling on the precisely registered fruit point cloud to obtain a precisely registered down-sampled fruit point cloud;
10) fine registration and downsampling fruit point cloud filtering: setting the value range of the X axis of the point cloud world coordinate system as (X)min，xmax) The value range of the Y axis is (Y)min，ymax) The value range of the Z axis is (Z)min，zmax) Utilizing a Conditionond algorithm according to the value ranges of the X axis, the Y axis and the Z axisRemoving external noise point clouds of the precisely registered and down-sampled fruit point clouds to obtain precisely registered and filtered fruit point clouds;
11) removing point cloud outliers of the fine registration filtering fruit: taking any one space point k in the fine registration filtering fruit point cloud, obtaining 50 space points adjacent to the current space point k, calculating the average distance between the current space point k and the adjacent 50 space points, selecting and removing the space points outside the range of 1.5 times of the average distance by Gaussian distribution as outliers of the current space point k, traversing all the space points in the fine registration filtering fruit point cloud, and obtaining the outlier-removed fine registration fruit point cloud;
12) measuring the longitudinal and transverse diameters: and (3) processing the fruit point cloud with the removed outliers and accurately registered by adopting an OBB algorithm with the directed minimum bounding box, drawing the minimum OBB bounding box of the quasi-circular fruit, and extracting the longitudinal diameter and the transverse diameter of the minimum OBB bounding box, which are consistent with the longitudinal diameter direction and the transverse diameter direction of the quasi-circular fruit in the axial direction, so as to obtain the longitudinal diameter H and the transverse diameter W of the quasi-circular fruit.
The step 4) is specifically as follows:
4.1) Using the initial transformation matrix T12Registering the auxiliary camera calibration plate point cloud and the main camera calibration plate point cloud to obtain a coarse registration calibration plate point cloud;
4.2) in the rough registration point cloud, arbitrarily selecting a space point in the main camera calibration plate point cloud, searching a space point with the distance between the space point and the current space point being smaller than an error threshold value d in the auxiliary camera calibration plate point cloud, and if the space point exists, taking the two space points as a correct matching point pair; traversing the point cloud of the calibration board of the auxiliary camera, and searching all correct matching point pairs corresponding to the current space point; if not, the current space point does not have a correct matching point pair;
4.3) traversing all the spatial points in the point cloud of the calibration plate of the main camera to obtain all correct matching point pairs of the point cloud of the rough registration calibration plate.
The step 5) is specifically as follows:
5.1) calculating the mass center of the main camera and the auxiliary camera point set: marking the main camera calibration plate point cloud and the auxiliary camera calibration plate point cloud in all correct matching points of the rough registration calibration plate point cloud as a main camera point set P and an auxiliary camera point set Q respectivelyEach of which satisfies the condition of P ═ P (P)1，p2，…，pi，…，pnAnd Q ═ Q (Q)1，q2，…，qi，…，qn) Wherein p isiRepresenting the ith spatial point, q, in the main camera point set PiRepresenting the ith space point in the auxiliary camera point set Q, n representing the logarithm of the correct matching point pair in the point cloud of the rough registration calibration plate, wherein the mass centers Q of the main camera point set P and the auxiliary camera point set Q are respectively as follows:
Figure BDA0003110187960000041
Figure BDA0003110187960000042
5.2) calculating the coordinates of the spatial points in the point set of the main camera and the auxiliary camera with the centroid as the origin: space point coordinate P 'with centroid P as origin in main camera point set P'iAnd space point coordinates Q 'with centroid Q as origin in the sub-camera point set Q'iRespectively as follows:
p′i＝pi-p
q′i＝qi-q
5.3) calculating the distance error e of the ith pair of correctly matched pointsiSatisfy ei＝|pi-(R·qi+ t) |, where R represents the rotation matrix of the current iteration and t represents the translation matrix of the current iteration, then the sum of squared distance errors E (R, t) for all correctly matched point pairs is:
Figure BDA0003110187960000043
5.4) carrying out SVD on the distance error square sum of all correct matching point pairs to obtain an optimized conversion matrix T'12。
The step 6) is specifically as follows:
continuously repeating the step 5) for iteration until the following value is reachedStopping the iteration when one of the iteration termination conditions is met; iteration termination condition 1: reaching the maximum iteration number, and the iteration termination condition 2: the sum of squared distance errors E (R, t) for all correctly matched pairs of points is less than the given termination error value; obtaining an optimized transformation matrix
Figure BDA0003110187960000044
The initial transformation matrix T12From an initial rotation matrix R12With the initial translation matrix t12And (4) forming.
The invention has the beneficial effects that:
the invention has low requirement on the illumination environment of the quasi-circular fruits with stronger light reflection characteristics on the surface, and can avoid the damage to the quasi-circular fruits; meanwhile, point cloud fine registration is carried out through an optimized conversion matrix obtained through iteration, the problems of large calculation amount of matching points, large number of point clouds, long time consumption and the like in the multi-view point cloud registration process are solved, time and labor can be saved, the measurement of the longitudinal and transverse diameters of the quasi-circular fruits is realized, and the online production of the quasi-circular fruits is facilitated.
Drawings
FIG. 1 is an overall flow chart of the present invention.
Fig. 2 is a layout of the dual view acquisition system of the present invention.
FIG. 3 is a calibration plate coarse registration point cloud plot of the calibration process of the present invention.
FIG. 4 is a calibration plate fine registration dot cloud for the calibration process of the present invention.
Figure 5 is a principal cloud of the present invention's roundlike fruit.
Figure 6 is a cloud of minor dots of the rounded-like fruit of the present invention.
Fig. 7 is a fine registration filtered fruit cloud of the present invention round-like fruit.
FIG. 8 is a cloud view of the present invention of outlier removed fine registration of a rounded fruit.
Figure 9 is a diagram of the minimal OBB bounding box for the roundlike fruit of the present invention.
In the figure: 1. the storage platform comprises 2 similar circular fruits, 3, RGBD cameras, 4, a synchronization line, 5, a data transmission line, 6 and a host.
Detailed Description
The invention is further illustrated by the following figures and examples.
The embodiment of selecting round-like fruit honey pomelos in the invention is as follows:
as shown in fig. 1, the present invention comprises the steps of:
1) as shown in fig. 2, a dual-view acquisition system was set up: the double-view-angle acquisition system comprises a storage platform (1), a quasi-circular fruit (2), two RGBD cameras (3), a synchronization line (4), a data transmission line (5), a calibration board and a host (6);
the two RGBD cameras (3) and the host (6) are both arranged on the storage platform (1), the two RGBD cameras (3) are connected through a synchronization line and used for controlling the two RGBD cameras (3) to synchronously acquire point clouds, the two RGBD cameras (3) are respectively connected with the host (6) through data transmission lines (5), the quasi-circular fruit (2) or the calibration plate is arranged on the storage platform (1), and the quasi-circular fruit (2) or the calibration plate is arranged on one side of the two RGBD cameras (3) and one side of the host (6); the optical axes of the two RGBD cameras (3) face towards the quasi-circular fruit (2) or the calibration plate, and the optical axes of the two RGBD cameras (3) form an included angle;
2) establishing a world coordinate system: one RGBD camera (3) is taken as a main camera C1The other RGBD camera (3) is a sub-camera C2With the main camera C1The optical center of (A) is the origin of the world coordinate system and the main camera C1The optical axis of the camera is outward a Z axis of a world coordinate system, and a plane formed by an X axis and a Y axis of the world coordinate system is parallel to the main camera C1The imaging plane of (1) is horizontally an X axis to the right and vertically a Y axis to the down;
3) calibrating a main camera C1Auxiliary camera C2Pose: placing the calibration plate on the object placing platform (1), collecting the calibration plate by using a double-view collection system, and obtaining a main camera C1And a sub-camera C2Respectively collecting the color image of the calibration plate and the depth image of the corresponding calibration plate, and calibrating the main camera C by a binocular calibration method1And a sub-camera C2Respectively acquiring color images of the calibration plates, processing the color images to obtain a secondary camera C2Coordinate conversion to main camera C1Initial transformation matrix T of coordinates12Initial transformation matrix T12From an initial rotation matrix R12With the initial translation matrix t12And (4) forming. By a main camera C1And a sub-camera C2Respectively forming a main camera calibration plate point cloud and an auxiliary camera calibration plate point cloud by the collected calibration plate color images and the corresponding calibration plate depth images;
initial transformation matrix T12Initial rotation matrix R12With the initial translation matrix t12Respectively as follows:
Figure BDA0003110187960000061
Figure BDA0003110187960000062
t12＝[-0.5935 0.00787 0.05856]
4) roughly registering and matching the point clouds of the calibration plates of the main camera and the auxiliary camera to obtain all correct matching point pairs of the point clouds of the rough registration calibration plates;
the step 4) is specifically as follows:
4.1) Using the initial transformation matrix T12Registering the auxiliary camera calibration plate point cloud and the main camera calibration plate point cloud to obtain a coarse registration calibration plate point cloud as shown in fig. 3;
4.2) in the rough registration point cloud, arbitrarily selecting a space point in the main camera calibration plate point cloud, searching a space point with the distance between the space point and the current space point being smaller than an error threshold value d in the auxiliary camera calibration plate point cloud, and if the space point exists, taking the two space points as a correct matching point pair; traversing the point cloud of the calibration board of the auxiliary camera, and searching all correct matching point pairs corresponding to the current space point; if not, the current space point does not have a correct matching point pair;
4.3) traversing all the spatial points in the point cloud of the calibration plate of the main camera to obtain all correct matching point pairs of the point cloud of the rough registration calibration plate.
5) Optimizing an initial transformation matrix T12；
The step 5) is specifically as follows:
5.1) calculating the mass center of the main camera and the auxiliary camera point set: respectively recording the primary camera calibration plate point cloud and the secondary camera calibration plate point cloud in all correct matching points of the rough registration calibration plate point cloud as a primary camera point set P and a secondary camera point set Q, and respectively satisfying the condition of P ═ P (P ═ Q)1，p2，…，pi，…，pnAnd Q ═ Q (Q)1，q2，…，qi，…，qn) Wherein p isiRepresenting the ith spatial point, q, in the main camera point set PiRepresenting the ith space point in the auxiliary camera point set Q, n representing the logarithm of the correct matching point pair in the point cloud of the rough registration calibration plate, wherein the mass centers Q of the main camera point set P and the auxiliary camera point set Q are respectively as follows:
Figure BDA0003110187960000063
Figure BDA0003110187960000071
5.2) calculating the coordinates of the spatial points in the point set of the main camera and the auxiliary camera with the centroid as the origin: space point coordinate P 'with centroid P as origin in main camera point set P'iAnd space point coordinates Q 'with centroid Q as origin in the sub-camera point set Q'iRespectively as follows:
p′i＝pi-p
q′i＝qi-q
5.3) calculating the distance error e of the ith pair of correctly matched pointsiSatisfy ei＝|pi-(R·qi+ t) |, where R represents the rotation matrix of the current iteration and t represents the translation matrix of the current iteration, then the sum of squared distance errors E (R, t) for all correctly matched point pairs is:
Figure BDA0003110187960000072
5.4) carrying out SVD on the distance error square sum of all correct matching point pairs to obtain an optimized conversion matrix T'12。
6) Repeating the step 5) to iteratively optimize the transformation matrix T12Obtaining an optimized transformation matrix
Figure BDA0003110187960000073
The step 6) is specifically as follows:
continuously repeating the step 5) to carry out iteration until one of the following iteration termination conditions is reached; iteration termination condition 1: reaching the maximum iteration number, and the iteration termination condition 2: the sum of squared distance errors E (R, t) for all correctly matched pairs of points is less than the given termination error value; obtaining an optimized transformation matrix
Figure BDA0003110187960000074
Optimizing a transformation matrix
Figure BDA0003110187960000075
By the optimized rotation matrix R*And an optimized translation matrix t*Composition is carried out; registering the auxiliary camera calibration plate point cloud and the main camera calibration plate point cloud to obtain a fine registration calibration plate point cloud, as shown in fig. 4;
optimizing a transformation matrix
Figure BDA0003110187960000076
Optimized rotation matrix R*And an optimized translation matrix t*Respectively as follows:
Figure BDA0003110187960000077
Figure BDA0003110187960000078
t*＝[-0.73497 0.016825 -0.03818]
7) obtaining main and auxiliary point clouds of the quasi-circular fruits: placing the quasi-circular fruits (2) on the storage platform (1), and controlling the two RGBD cameras (3) to synchronously acquire the main point cloud and the auxiliary point cloud of the quasi-circular fruits (2), as shown in FIGS. 5 and 6;
8) carrying out cloud fine registration on main points and auxiliary points of the quasi-circular fruits: using the optimized transformation matrix obtained in step 6)
Figure BDA0003110187960000081
Registering the main point cloud and the auxiliary point cloud of the quasi-circular fruit (2) to obtain a precisely registered fruit point cloud of the quasi-circular fruit (2);
9) fine registration fruit point cloud down sampling: using a VoxelGrid filter to carry out down-sampling on the fruit point cloud subjected to the fine registration so as to reduce the number of the point clouds and obtain the fruit point cloud subjected to the down-sampling of the fine registration;
10) fine registration and downsampling fruit point cloud filtering: setting the value range of the X axis of the point cloud world coordinate system as (X)min,xmax) The value range of the Y axis is (Y)min,ymax) The value range of the Z axis is (Z)min,zmax) Removing external noise point clouds of the precisely registered down-sampling fruit point clouds by using a conditionond algorithm according to the value ranges of the X axis, the Y axis and the Z axis to obtain precisely registered filtering fruit point clouds as shown in FIG. 7;
11) removing point cloud outliers of the fine registration filtering fruit: taking any one space point k in the fine registration filtering fruit point cloud, obtaining 50 space points adjacent to the current space point k, calculating the average distance between the current space point k and the adjacent 50 space points, selecting the space points out of the range of 1.5 times of the average distance by utilizing Gaussian distribution as outliers of the current space point k, removing the outliers, traversing all the space points in the fine registration filtering fruit point cloud, and obtaining the fine registration fruit point cloud with the outliers removed, as shown in FIG. 8;
12) measuring the longitudinal and transverse diameters: and (3) processing the fruit point cloud with the removed outliers and accurately registered by adopting an OBB algorithm with the directed minimum bounding box, and then drawing the minimum OBB bounding box of the quasi-circular fruit, as shown in fig. 9, extracting the longitudinal diameter and the transverse diameter of the minimum OBB bounding box, which are consistent with the longitudinal diameter direction and the transverse diameter direction of the quasi-circular fruit in the axial direction, to obtain the longitudinal diameter H and the transverse diameter W of the quasi-circular fruit.
Table 1 shows diameter data obtained by taking round balls with a diameter of 65mm according to the present example and performing 10 consecutive measurements, wherein the true value of the round ball is 65.24mm from the average value of 10 vernier caliper measurements.
Table 165 mm sphere diameter measurement data table
Figure BDA0003110187960000082
Figure BDA0003110187960000091

Claims (3)

1. A method for measuring the longitudinal and transverse diameters of a similar round fruit based on dual-view-point cloud registration is characterized by comprising the following steps:
1) building a double-view acquisition system: the double-view-angle acquisition system comprises an object placing platform (1), a quasi-circular fruit (2), two RGBD cameras (3), a synchronization line (4), a data transmission line (5), a calibration board and a host (6);
the two RGBD cameras (3) and the host (6) are both arranged on the storage platform (1), the two RGBD cameras (3) are connected through a synchronous line, the two RGBD cameras (3) are respectively connected with the host (6) through data transmission lines (5), the quasi-circular fruits (2) or the calibration plate are arranged on the storage platform (1), and the quasi-circular fruits (2) or the calibration plate are arranged on one sides of the two RGBD cameras (3) and the host (6); the optical axes of the two RGBD cameras (3) face towards the quasi-circular fruit (2) or the calibration plate, and the optical axes of the two RGBD cameras (3) form an included angle;
2) establishing a world coordinate system: one RGBD camera (3) is taken as a main camera C1The other RGBD camera (3) is a sub-camera C2With the main camera C1The optical center of (A) is the origin of the world coordinate system and the main camera C1The optical axis of the camera is outward a Z axis of a world coordinate system, and a plane formed by an X axis and a Y axis of the world coordinate system is parallel to the main camera C1The X-axis is perpendicular to the Y-axis;
3) calibrating a main camera C1Auxiliary camera C2Pose: placing the calibration plate on the object placing platform (1), collecting the calibration plate by using a double-view collection system, and obtaining a main camera C1And a sub-camera C2Respectively collecting the color image of the calibration plate and the depth image of the corresponding calibration plate, and calibrating the main camera C by a binocular calibration method1And a sub-camera C2Respectively acquiring color images of the calibration plates, processing the color images to obtain a secondary camera C2Coordinate conversion to main camera C1Initial transformation matrix T of coordinates12From the main camera C1And a sub-camera C2Respectively forming a main camera calibration plate point cloud and an auxiliary camera calibration plate point cloud by the collected calibration plate color images and the corresponding calibration plate depth images;
4) roughly registering and matching the point clouds of the calibration plates of the main camera and the auxiliary camera to obtain all correct matching point pairs of the point clouds of the rough registration calibration plates;
5) optimizing an initial transformation matrix T12；
The step 5) is specifically as follows:
5.1) calculating the mass center of the main camera and the auxiliary camera point set: respectively recording the primary camera calibration plate point cloud and the secondary camera calibration plate point cloud in all correct matching points of the rough registration calibration plate point cloud as a primary camera point set P and a secondary camera point set Q, and respectively satisfying the condition of P ═ P (P ═ Q)1，p2，…，pi，…，pnAnd Q ═ Q (Q)1，q2，…，qi，…，qn) Wherein p isiRepresenting the ith spatial point, q, in the main camera point set PiRepresenting the ith space point in the auxiliary camera point set Q, n representing the logarithm of the correct matching point pair in the point cloud of the rough registration calibration plate, wherein the mass centers Q of the main camera point set P and the auxiliary camera point set Q are respectively as follows:
Figure FDA0003504379100000021
Figure FDA0003504379100000022
5.2) calculating the coordinates of the spatial points in the point set of the main camera and the auxiliary camera with the centroid as the origin: space point coordinate P 'with centroid P as origin in main camera point set P'iAnd space point coordinates Q 'with centroid Q as origin in the sub-camera point set Q'iRespectively as follows:
p′i＝pi-p
q′i＝qi-q
5.3) calculating the distance error e of the ith pair of correctly matched pointsiSatisfy ei＝|pi-(R·qi+ t) |, where R represents the rotation matrix of the current iteration and t represents the translation matrix of the current iteration, then the sum of squared distance errors E (R, t) for all correctly matched point pairs is:
Figure FDA0003504379100000023
5.4) carrying out SVD on the distance error square sum of all correct matching point pairs to obtain an optimized conversion matrix T'12；
6) Repeating the step 5) to iteratively optimize the transformation matrix T12Obtaining an optimized transformation matrix
Figure FDA0003504379100000024
The step 6) is specifically as follows:
continuously repeating the step 5) to carry out iteration until one of the following iteration termination conditions is reached; iteration termination condition 1: reaching the maximum iteration number, and the iteration termination condition 2: the sum of squared distance errors E (R, t) for all correctly matched pairs of points is less than the given termination error value; obtaining an optimized transformation matrix
Figure FDA0003504379100000025
7) Obtaining main and auxiliary point clouds of the quasi-circular fruits: placing the quasi-circular fruits (2) on the storage platform (1), and controlling the two RGBD cameras (3) to synchronously acquire main point clouds and auxiliary point clouds of the quasi-circular fruits (2);
8) carrying out cloud fine registration on main points and auxiliary points of the quasi-circular fruits: using the optimized transformation matrix obtained in step 6)
Figure FDA0003504379100000026
Registering the main point cloud and the auxiliary point cloud of the quasi-circular fruit (2) to obtain a precisely registered fruit point cloud of the quasi-circular fruit (2);
9) fine registration fruit point cloud down sampling: using a VoxelGrid filter to carry out down-sampling on the precisely registered fruit point cloud to obtain a precisely registered down-sampled fruit point cloud;
10) fine registration and downsampling fruit point cloud filtering: setting the value range of the X axis of the point cloud world coordinate system as (X)min，xmax) The value range of the Y axis is (Y)min，ymax) The value range of the Z axis is (Z)min，zmax) Removing external noise point clouds of the precisely registered down-sampling fruit point clouds by using a conditionond algorithm according to the value ranges of an X axis, a Y axis and a Z axis to obtain precisely registered filtering fruit point clouds;
11) removing point cloud outliers of the fine registration filtering fruit: taking any one space point k in the fine registration filtering fruit point cloud, obtaining 50 space points adjacent to the current space point k, calculating the average distance between the current space point k and the adjacent 50 space points, selecting and removing the space points outside the range of 1.5 times of the average distance by Gaussian distribution as outliers of the current space point k, traversing all the space points in the fine registration filtering fruit point cloud, and obtaining the outlier-removed fine registration fruit point cloud;
12) measuring the longitudinal and transverse diameters: and (3) processing the fruit point cloud with the removed outliers and accurately registered by adopting an OBB algorithm with the directed minimum bounding box, drawing the minimum OBB bounding box of the quasi-circular fruit, and extracting the longitudinal diameter and the transverse diameter of the minimum OBB bounding box, which are consistent with the longitudinal diameter direction and the transverse diameter direction of the quasi-circular fruit in the axial direction, so as to obtain the longitudinal diameter H and the transverse diameter W of the quasi-circular fruit.
2. The method for measuring the longitudinal and transverse diameters of the similar-circular fruit based on the dual-view-point cloud registration according to claim 1, wherein the step 4) is specifically as follows:
4.1) Using the initial transformation matrix T12Registering the auxiliary camera calibration plate point cloud and the main camera calibration plate point cloud to obtain a coarse registration calibration plate point cloud;
4.2) in the rough registration point cloud, arbitrarily selecting a space point in the main camera calibration plate point cloud, searching a space point with the distance between the space point and the current space point being smaller than an error threshold value d in the auxiliary camera calibration plate point cloud, and if the space point exists, taking the two space points as a correct matching point pair; traversing the point cloud of the calibration board of the auxiliary camera, and searching all correct matching point pairs corresponding to the current space point; if not, the current space point does not have a correct matching point pair;
4.3) traversing all the spatial points in the point cloud of the calibration plate of the main camera to obtain all correct matching point pairs of the point cloud of the rough registration calibration plate.
3. The method for measuring the longitudinal and transverse diameters of the similar-circular fruit based on the dual-view-point cloud registration as claimed in claim 1, wherein the initial transformation matrix T is12From an initial rotation matrix R12With the initial translation matrix t12And (4) forming.
CN202110646831.6A 2021-06-10 2021-06-10 Quasi-circular fruit longitudinal and transverse diameter measuring method based on double-view-point cloud registration Active CN113379824B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202110646831.6A CN113379824B (en) 2021-06-10 2021-06-10 Quasi-circular fruit longitudinal and transverse diameter measuring method based on double-view-point cloud registration

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202110646831.6A CN113379824B (en) 2021-06-10 2021-06-10 Quasi-circular fruit longitudinal and transverse diameter measuring method based on double-view-point cloud registration

Publications (2)

Publication Number Publication Date
CN113379824A CN113379824A (en) 2021-09-10
CN113379824B true CN113379824B (en) 2022-04-01

Family

ID=77573428

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202110646831.6A Active CN113379824B (en) 2021-06-10 2021-06-10 Quasi-circular fruit longitudinal and transverse diameter measuring method based on double-view-point cloud registration

Country Status (1)

Country Link
CN (1) CN113379824B (en)

Families Citing this family (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111489433B (en) * 2020-02-13 2023-04-25 北京百度网讯科技有限公司 Method and device for positioning damage of vehicle, electronic equipment and readable storage medium
CN116912301A (en) * 2023-02-23 2023-10-20 浙江大学 Liver tumor alignment method and device, electronic equipment and medium

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101059452A (en) * 2007-05-29 2007-10-24 浙江大学 Fruit quality damage-free detection method and system based on multiple spectral imaging technique
CN111579511A (en) * 2020-06-15 2020-08-25 南京农业大学 Seed quality detection method and device based on structure hyperspectrum
CN111612850A (en) * 2020-05-13 2020-09-01 河北工业大学 Pig body size parameter measuring method based on point cloud
CN111968172A (en) * 2020-07-16 2020-11-20 中冶华天工程技术有限公司 Method and system for measuring volume of material in stock ground
EP3779357A1 (en) * 2019-08-12 2021-02-17 Leica Geosystems AG Localisation of a surveying instrument

Family Cites Families (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US10136646B2 (en) * 2013-06-26 2018-11-27 Indigo Ag, Inc. Agricultural endophyte-plant compositions, and methods of use

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101059452A (en) * 2007-05-29 2007-10-24 浙江大学 Fruit quality damage-free detection method and system based on multiple spectral imaging technique
EP3779357A1 (en) * 2019-08-12 2021-02-17 Leica Geosystems AG Localisation of a surveying instrument
CN111612850A (en) * 2020-05-13 2020-09-01 河北工业大学 Pig body size parameter measuring method based on point cloud
CN111579511A (en) * 2020-06-15 2020-08-25 南京农业大学 Seed quality detection method and device based on structure hyperspectrum
CN111968172A (en) * 2020-07-16 2020-11-20 中冶华天工程技术有限公司 Method and system for measuring volume of material in stock ground

Also Published As

Publication number Publication date
CN113379824A (en) 2021-09-10

Similar Documents

Publication Publication Date Title
CN104677305B (en) A kind of body surface three-dimensional method for reconstructing and system based on cross structure light
CN113379824B (en) Quasi-circular fruit longitudinal and transverse diameter measuring method based on double-view-point cloud registration
CN103913131B (en) Free curve method vector measurement method based on binocular vision
CN100498213C (en) Method for monitoring pig growth using binocular vision technology
CN109658398B (en) Part surface defect identification and evaluation method based on three-dimensional measurement point cloud
CN105716539B (en) A kind of three-dimentioned shape measurement method of quick high accuracy
CN109211198B (en) Intelligent target detection and measurement system and method based on trinocular vision
CN107977996B (en) Space target positioning method based on target calibration positioning model
CN110009690A (en) Binocular stereo vision image measuring method based on polar curve correction
CN103822581B (en) A kind of irregularly shaped object volume measuring method based on compressed sensing
CN112651968B (en) Wood board deformation and pit detection method based on depth information
CN107392929B (en) Intelligent target detection and size measurement method based on human eye vision model
CN101804720A (en) Vision-based online digital electronic engraving quality monitoring method and device
CN102221331A (en) Measuring method based on asymmetric binocular stereovision technology
CN108230402B (en) Three-dimensional calibration method based on triangular pyramid model
CN110648362B (en) Binocular stereo vision badminton positioning identification and posture calculation method
CN104050660A (en) Method for measuring workpiece round edges
CN112241964A (en) Light strip center extraction method for line structured light non-contact measurement
CN115375745A (en) Absolute depth measurement method based on polarization microlens light field image parallax angle
CN111583342A (en) Target rapid positioning method and device based on binocular vision
CN104614372B (en) Detection method of solar silicon wafer
CN111598172B (en) Dynamic target grabbing gesture rapid detection method based on heterogeneous depth network fusion
CN110653166B (en) Fruit detection and classification method and device
CN116883480A (en) Corn plant height detection method based on binocular image and ground-based radar fusion point cloud
CN115641326A (en) Sub-pixel size detection method and system for ceramic antenna PIN needle image

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant