CN115035184A - Honey pomelo volume estimation method based on lateral multi-view reconstruction - Google Patents
Honey pomelo volume estimation method based on lateral multi-view reconstruction Download PDFInfo
- Publication number
- CN115035184A CN115035184A CN202210662277.5A CN202210662277A CN115035184A CN 115035184 A CN115035184 A CN 115035184A CN 202210662277 A CN202210662277 A CN 202210662277A CN 115035184 A CN115035184 A CN 115035184A
- Authority
- CN
- China
- Prior art keywords
- camera
- honey
- point cloud
- volume
- honey pomelo
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 235000012907 honey Nutrition 0.000 title claims abstract description 105
- 238000000034 method Methods 0.000 title claims abstract description 31
- 240000000560 Citrus x paradisi Species 0.000 title claims abstract 33
- 238000005070 sampling Methods 0.000 claims abstract description 23
- 238000001914 filtration Methods 0.000 claims abstract description 15
- 230000011218 segmentation Effects 0.000 claims abstract description 15
- 238000011084 recovery Methods 0.000 claims abstract description 4
- NIXOWILDQLNWCW-UHFFFAOYSA-N acrylic acid group Chemical group C(C=C)(=O)O NIXOWILDQLNWCW-UHFFFAOYSA-N 0.000 claims description 20
- 238000005286 illumination Methods 0.000 claims description 16
- 239000011159 matrix material Substances 0.000 claims description 16
- 238000012545 processing Methods 0.000 claims description 6
- 230000003287 optical effect Effects 0.000 claims description 4
- 230000009466 transformation Effects 0.000 claims description 3
- 238000000638 solvent extraction Methods 0.000 claims description 2
- 238000000605 extraction Methods 0.000 claims 1
- 230000000630 rising effect Effects 0.000 claims 1
- 235000013399 edible fruits Nutrition 0.000 abstract description 23
- 238000005259 measurement Methods 0.000 abstract description 6
- 235000014443 Pyrus communis Nutrition 0.000 abstract description 4
- 244000276331 Citrus maxima Species 0.000 description 77
- 241000219109 Citrullus Species 0.000 description 4
- 235000012828 Citrullus lanatus var citroides Nutrition 0.000 description 4
- 241000220324 Pyrus Species 0.000 description 4
- 238000003384 imaging method Methods 0.000 description 4
- 238000010586 diagram Methods 0.000 description 3
- 235000001759 Citrus maxima Nutrition 0.000 description 2
- 235000003953 Solanum lycopersicum var cerasiforme Nutrition 0.000 description 2
- 240000003040 Solanum lycopersicum var. cerasiforme Species 0.000 description 2
- 238000011161 development Methods 0.000 description 2
- 230000018109 developmental process Effects 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 235000013305 food Nutrition 0.000 description 2
- 238000010801 machine learning Methods 0.000 description 2
- 238000011160 research Methods 0.000 description 2
- SUBDBMMJDZJVOS-UHFFFAOYSA-N 5-methoxy-2-{[(4-methoxy-3,5-dimethylpyridin-2-yl)methyl]sulfinyl}-1H-benzimidazole Chemical compound N=1C2=CC(OC)=CC=C2NC=1S(=O)CC1=NC=C(C)C(OC)=C1C SUBDBMMJDZJVOS-UHFFFAOYSA-N 0.000 description 1
- 235000011299 Brassica oleracea var botrytis Nutrition 0.000 description 1
- 235000017647 Brassica oleracea var italica Nutrition 0.000 description 1
- 240000003259 Brassica oleracea var. botrytis Species 0.000 description 1
- 241000227653 Lycopersicon Species 0.000 description 1
- 235000007688 Lycopersicon esculentum Nutrition 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 235000021028 berry Nutrition 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 235000020971 citrus fruits Nutrition 0.000 description 1
- 238000010276 construction Methods 0.000 description 1
- 238000013135 deep learning Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000000572 ellipsometry Methods 0.000 description 1
- 238000002474 experimental method Methods 0.000 description 1
- 235000011389 fruit/vegetable juice Nutrition 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 235000021017 pears Nutrition 0.000 description 1
- 230000000644 propagated effect Effects 0.000 description 1
- 238000013519 translation Methods 0.000 description 1
- 235000013311 vegetables Nutrition 0.000 description 1
- XLYOFNOQVPJJNP-UHFFFAOYSA-N water Substances O XLYOFNOQVPJJNP-UHFFFAOYSA-N 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/60—Analysis of geometric attributes
- G06T7/62—Analysis of geometric attributes of area, perimeter, diameter or volume
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T17/00—Three dimensional [3D] modelling, e.g. data description of 3D objects
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/70—Denoising; Smoothing
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/70—Determining position or orientation of objects or cameras
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10028—Range image; Depth image; 3D point clouds
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20092—Interactive image processing based on input by user
- G06T2207/20104—Interactive definition of region of interest [ROI]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30181—Earth observation
- G06T2207/30188—Vegetation; Agriculture
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Geometry (AREA)
- Computer Graphics (AREA)
- Software Systems (AREA)
- Length Measuring Devices By Optical Means (AREA)
Abstract
The invention discloses a honey pomelo volume estimation method based on lateral multi-view reconstruction. The method comprises the following steps: firstly, a multi-view image acquisition system is constructed, dense point cloud reconstruction of the honey pomelos is realized by utilizing the constructed system based on a motion recovery structure and a multi-view stereoscopic vision principle, then a closed convex hull is formed through dense point cloud segmentation, point cloud segmentation filtering, filtering point cloud down-sampling and down-sampling point cloud triangulation, and the volume of the closed convex hull is calculated to be used as a down-sampling point cloud volume estimation value. The method effectively solves the problem that the volume of the fruit is difficult to calculate, and is suitable for estimating the volume of the fruit with various shapes such as a drop shape, a spherical shape, an ellipsoidal shape, a pear shape and the like. Meanwhile, the image acquisition device with the uniform light environment constructed by the invention can well overcome a bright spot area formed by strong light reflection on the surface of the fruit. Furthermore, the invention can realize lossless and accurate measurement of the volume of the fruit and provide an important reference basis for quality grading of the fruit.
Description
Technical Field
The invention relates to a fruit volume estimation method, in particular to a honey pomelo volume estimation method based on lateral multi-view reconstruction.
Background
China is a world large fruit production country, and fruits have been developed into the third largest planting product of China, which is second to grains and vegetables. Meanwhile, as the countries with the largest planting areas of the pomelos in the world, the yield of the pomelos is the first in the world, the yield of the pomelos in 2019 is as high as 508 ten thousand tons, and the planting areas and the yield of the pomelos respectively account for 61.41 percent and 51.45 percent of the world. The honey pomelo has become one of the excellent representative varieties of pomelos at present after the cultivation history of nearly 500 years. At present, the quality grading of honey pomelos is mainly based on weight, yellow day rise and the like (2015) (yellow day rise, Zhudonghuang, Lin broccoli, Shenhong, Lijian. honey pomelo fruit grading standard research [ J ]. southern fruit tree in China, 2015,44(03):28-31+34.) research shows that the weight is more scientific by using the volume as a grading index, the correlation between the volume of the fruit and the internal quality is strongest, and the corresponding juice cytoplastation rate is the largest. Therefore, the estimation of the volume of the honey pomelos and the further commercial grading have important significance.
The traditional method for estimating the volume of the fruit is mainly manual measurement, such as measurement by using a drainage method, but the manual measurement mode has the defects of high labor intensity, low efficiency, long time consumption and the like. With the development of image processing techniques, estimation of external geometric characteristics of fruit, as represented by estimation of fruit volume and mass, has shown great advantages. Koc et al (2007) (Koc A B. determination of watermelons volume using the watermelon volume approximation and image processing [ J ]. Postharvest Biology and Technology,2007,45(3): 366) with watermelon as the study object, fitting watermelon volume by ellipsometry and image area estimation, respectively, found that the accuracy of the image area fitting method was higher. Omid et al (2010) (Omid M, Khojastehnazhand M, Tabatabaeefar A. estimating volume and mass of citrus fruit by image processing technique [ J ]. Journal of Food Engineering,2010,100(2): 315-. Nyalala et al (2019) (Nyalala I, Okinda C, Nyalala L, et al. tomato volume and mass estimation using computer vision and machine learning algorithms: Cherry tomato model [ J ]. Journal of Food Engineering,2019,263:288-298.) the method of combining machine vision and machine learning is used to estimate Cherry tomato volume and quality, wherein the accuracy of mass and volume estimation in RBF-SVM model reaches 0.9706.
In recent years, with the development of three-dimensional reconstruction technology and consumer-level image acquisition equipment, researchers have measured the size or volume of a fruit by performing three-dimensional reconstruction in a manner of generating a fruit full-surface point cloud: yawe et al (2020) (Yawei, W, Yifei C. front medical Measurement Based on Three-Dimensional reconstruction. agronomy, 2020,10,455) place a pear on a rotating table, obtain a Three-Dimensional point cloud chart of the pear using 9 pictures, and calculate its Three-Dimensional size. Ni and the like (2021) (Ni X, Li C, Jiang H, et al, three-dimensional photo graphing with deep learning interaction to extract berry from fruit stability trails [ J ]. ISPRS Journal of photo graphing and remove Sensing,2021,171: 297-.
However, the above objects are all ellipsoidal or spherical fruits, and it is difficult to apply the volume estimation of honey pomelos having various shapes such as a droplet shape, a spherical shape, an ellipsoidal shape, and a pear shape. In addition, because the honey pomelo has large fruit and the peel of the honey pomelo is covered with oil cells, the honey pomelo has the characteristic of easy light reflection, and a bright spot area is easy to form, the construction of a reconstruction device with illumination uniformity is also a great difficulty.
Disclosure of Invention
To solve the problems and needs in the background art, the present invention provides a honey pomelo volume estimation method based on lateral multi-view reconstruction.
The technical scheme of the invention is as follows:
1) and (3) building a multi-view image acquisition system: the multi-view image acquisition system comprises an illumination box, a rotating platform, a camera and a host, wherein the rotating platform is installed at the center inside the illumination box and used for placing honey pomelos, and the camera is composed of a downward looking camera C 1 Head-up camera C 2 And upward camera C 3 The circumference side surface of the illumination box is provided with a downward camera C from top to bottom 1 Head-up camera C 2 And upward camera C 3 Overlooking camera C 1 Head-up camera C 2 And upward camera C 3 All optical axes of the honey pomelo point to the overlooking camera C 1 Head-up camera C 2 And upward camera C 3 Are all connected with a host;
2) multi-view honey pomelo dense point cloud reconstruction: starting the rotating platform, looking down the camera C 1 Head-up camera C 2 And upward camera C 3 Synchronously shooting honey pomelos, and shooting by each camera to obtain a group of original honey pomelo images of the current honey pomelos; restoring the three-dimensional point coordinates of the current honey pomelos by utilizing a motion recovery structure and multi-view stereoscopic vision method according to the 3 groups of original honey pomelos images to obtain dense point clouds of the honey pomelos;
3) dense point cloud segmentation: removing background noise in the dense point cloud of the honey pomelos by adopting a conditionond algorithm according to the dense point cloud of the honey pomelos to obtain a partitioning point cloud of the honey pomelos;
4) segmentation point cloud filtering: for any point k in the honey pomelo segmentation point cloud, taking a plurality of adjacent points adjacent to the current point k, calculating the average distance between the current point k and the current plurality of adjacent points, then selecting outliers for removing by utilizing Gaussian distribution according to the average distance, traversing each point in the honey pomelo segmentation point cloud, removing all outliers of each point, and obtaining the currently obtained honey pomelo filtering point cloud;
5) filtering point cloud down sampling: randomly down-sampling the acquired honey pomelo filtering point cloud to obtain a honey pomelo down-sampling point cloud;
6) and (3) triangularization of down-sampling point cloud: triangularization processing is carried out on the honey pomelo down-sampling point cloud to form a closed convex hull;
7) volume estimation: the volume of the internal cavity of the closed convex hull is calculated and taken as the volume estimate of the current honey pomelo 3.
In the step 2), the rotating platform rotates at least one circle.
In the step 2), firstly, extracting feature points of all the original honey pomelo images by using a scale-invariant feature transformation algorithm, and matching the feature points among the original honey pomelo images to obtain each group of image matching pairs; then recovering an internal reference matrix and an external reference matrix corresponding to 3 cameras by utilizing an antipodal geometric principle according to the matching characteristic points of each group of image matching pairs; and finally, performing three-dimensional point cloud reconstruction on all original honey pomelo images by using a binocular stereo vision and multi-view stereo vision principle method according to the internal reference matrix and the external reference matrix corresponding to the 3 cameras to obtain dense point cloud of the honey pomelos.
In the step 4), points outside 0.5 times the average distance are taken as outliers.
And 5) randomly down-sampling the acquired honey pomelo filtering point cloud by adopting a point cloud random down-sampling method.
The illumination box in the step 1) comprises a transparent acrylic cylinder, a top LED light source, a bottom LED light source and a light guide film;
a rotating platform is arranged in the transparent acrylic cylinder, a camera groove is arranged on the circumferential side surface of the transparent acrylic cylinder, and a downward looking camera C is sequentially arranged on the circumferential side surface of the transparent acrylic cylinder from top to bottom 1 Head-up camera C 2 And upward camera C 3 The upper and lower end surfaces of the transparent acrylic cylinder are provided with light source mounting grooves, and the light sources on the upper and lower end surfaces are provided with light source mounting groovesThe top LED light source and the bottom LED light source are respectively embedded in the grooves, and the side surface of the outer circumference of the transparent acrylic cylinder is covered with a light guide film.
The invention has the beneficial effects that:
the method effectively solves the problem that the volume of the fruit is difficult to calculate, and is suitable for estimating the volume of the honey pomelos with various shapes such as water drops, spheres, ellipsoids, pears and the like. Meanwhile, the image acquisition device with the uniform light environment constructed by the invention can well overcome a bright spot area formed by strong light reflection on the surface of the fruit. Furthermore, the invention can realize lossless and accurate measurement of the volume of the fruit and provide an important reference basis for quality grading of the fruit.
Drawings
FIG. 1 is a flow chart of the method of the present invention.
Fig. 2 is a schematic structural diagram of an image acquisition system of the present invention.
Fig. 3 is a schematic view of the binocular vision system of the present invention.
Fig. 4 is a dense cloud-dotted graph of honey pomelos of the present invention.
Fig. 5 is a honey pomelo down-sampling cloud point of the present invention.
Fig. 6 is a convex hull diagram of a honey pomelo of the present invention.
Fig. 7 is a plot of a fit of the volume estimate of the present invention to an actual honey pomelo volume.
Fig. 8 is a schematic view of an illumination scheme of the present invention.
FIG. 9 is a graph of the V component of the present invention.
FIG. 10 is a ROI area luminance distribution diagram of the present invention.
In the figure: 1. illumination case, 2, rotary platform, 3, honey shaddock, 4, camera, 5, host computer, 6, display.
Detailed Description
The invention is further illustrated by the following figures and examples.
The embodiment of selecting honey pomelos in the invention is as follows:
as shown in fig. 1, the present invention comprises the steps of:
1) and (3) building a multi-view image acquisition system: as shown in FIG. 2, aThe view image acquisition system comprises an illumination box 1, a rotating platform 2, an industrial camera 4, a host 5 and a display 6, wherein the rotating platform 2 is installed at the center inside the illumination box 1, the rotating platform 2 is used for placing honey pomelos 3, and the camera 4 is composed of an overlook camera C 1 Head-up camera C 2 And upward camera C 3 The overlooking camera C is sequentially arranged along the circumferential side surface of the axial illumination box 1 from top to bottom 1 Head-up camera C 2 And upward camera C 3 Overlooking camera C 1 Head-up camera C 2 And upward camera C 3 All optical axes of (1) point to the honey pomelo 3, i.e. looking down camera C 1 Head-up camera C 2 And upward camera C 3 There is an angle between the optical axes. Downward camera C 1 Head-up camera C 2 And upward camera C 3 The position in the circumferential direction is not fixed, e.g. looking down at camera C 1 Head-up camera C 2 And upward camera C 3 May be arranged on the same axis or at equal intervals in the circumferential direction, looking down the camera C 1 Head-up camera C 2 And upward camera C 3 Are all connected with a host 5, the host 5 is connected with a display 6, and a camera C is overlooked 1 Head-up camera C 2 And upward camera C 3 Are connected with a host 5, and the host 5 is connected with a display 6;
the illumination box 1 in the step 1) comprises a transparent acrylic cylinder, a top LED light source, a bottom LED light source and a light guide film;
a rotating platform 2 is arranged in the transparent acrylic cylinder, a camera groove is arranged on the circumferential side surface of the transparent acrylic cylinder, and a downward looking camera C is sequentially arranged on the circumferential side surface of the transparent acrylic cylinder from top to bottom 1 Head-up camera C 2 And upward camera C 3 The light source mounting grooves are formed in the upper end face and the lower end face of the transparent acrylic cylinder, the top LED light source and the bottom LED light source are embedded in the light source mounting grooves of the upper end face and the lower end face respectively, and the light guide film covers the outer circumferential side face of the transparent acrylic cylinder.
2) Multi-view honey pomelo dense point cloud reconstruction: starting the rotating platform 2 looking down the camera C 1 Head-up camera C 2 And upward camera C 3 Synchronously shoot honey pomelos 3, the rotating platform 2 rotates at least oneIn the week, each camera shoots a group of honey pomelo original images of the current honey pomelo 3; restoring the three-dimensional point coordinates of the current honey pomelos by using a Motion recovery Structure (SFM) and Multi-View Stereo (MVS) method according to the 3 groups of original honey pomelos images to obtain dense point clouds of the honey pomelos;
in the step 2), firstly, extracting feature points of all honey pomelo original images by using a Scale-invariant feature transform (SIFT) algorithm, and matching the feature points among all the honey pomelo original images to obtain each group of image matching pairs; then recovering an internal reference matrix and an external reference matrix corresponding to 3 cameras by utilizing an antipodal geometric principle according to the matching characteristic points of each group of image matching pairs; and then calculating the three-dimensional coordinate position of the matched feature point, thereby obtaining the sparse three-dimensional point cloud of the honey pomelos. And finally, performing three-dimensional point cloud reconstruction on all original honey pomelo images by using a binocular stereo vision and multi-view stereo vision principle method according to the internal reference matrix and the external reference matrix corresponding to the 3 cameras to obtain dense point cloud of honey pomelos (figure 4).
In a specific embodiment, the rotation speed of the rotating platform 2 is set to 1 °/s, looking down the camera C 1 Head-up camera C 2 And upward camera C 3 All be connected with a trigger plate through the trigger control line, trigger plate is connected to on an NPN type laser sensor, through mutually supporting with camera host computer soft trigger, and the camera triggers the collection every interval 1s, gathers 3 groups, total 1080 honey shaddock original image.
As shown in fig. 3, two cameras corresponding to the image matching pair in step 2) are selected arbitrarily to form a binocular vision system, and the binocular vision system is recorded as O CL Is a left eye camera, O CR For the right eye camera, the focal length of the left eye camera is recorded as f l The focal length of the right eye camera is f r Arbitrary spatial point P (X) in the world coordinate system W ，Y W ，Z W ) The imaging point of the imaging system is p on the image coordinate system of the left eye camera l (x l ，y l ) The imaging point on the image coordinate system of the right eye camera is p r (x r ，y r ) The coordinate value under the coordinate system of the right eye camera is (X) WR ，Y WR ，Z WR ). In the specific implementation, the coordinate system of the left eye camera is used as the world coordinate system, so that the other two camera coordinate systems are converted into the current world coordinate system according to the internal reference matrix and the external reference matrix corresponding to the cameras.
Specifically, from the pinhole imaging model:
setting the rotation matrix R and the translation matrix t which are obtained by calculation in the step 2) and are converted from the coordinate system of the right eye camera to the coordinate system of the left eye camera as follows:
t T ＝[t 1 t 2 t 3 ] (4)
the transformation from the right eye camera coordinate system to the world coordinate system can be expressed as:
then, the above formulas are simultaneously obtained:
the above solutions are combined to obtain the product,
and traversing each group of image matching pairs to obtain three-dimensional point coordinates of all spatial points, wherein the spatial points form dense point clouds of the honey pomelos (figure 4).
3) Dense point cloud segmentation: setting the value range of the X axis of a point cloud world coordinate system as (X) according to the dense point cloud of the honey pomelos min ，x max ) The value range of the Y axis is (Y) min ，y max ) The value range of the Z axis is (Z) min ，Z max ) Removing background noise in dense point clouds of the honey pomelos by adopting a conditionond algorithm based on the value range of 3 coordinate axes to obtain honey pomelo segmentation point clouds;
4) segmentation point cloud filtering: for any point k in the honey pomelo segmentation point cloud, a plurality of adjacent points close to the current point k are taken, and in the specific implementation, 12 adjacent points are taken. Calculating the average distance between the current point k and a plurality of current adjacent points, then selecting points out of 0.5 times of the average distance by using Gaussian distribution as outliers for removing, traversing each point in the honey pomelo segmentation point cloud, removing all the outliers of each point, and obtaining the currently obtained honey pomelo filtering point cloud;
5) filtering point cloud down sampling: randomly down-sampling the obtained honey pomelo filtering point cloud by using a point cloud random down-sampling method to reduce the number of the point cloud, setting the number of the randomly down-sampled points to 4096, and obtaining the honey pomelo down-sampled point cloud (figure 5) after the down-sampling is finished;
6) and (3) triangularization of down-sampling point cloud: triangularization processing is carried out on the honey pomelo down-sampling point cloud to form a closed convex hull, as shown in fig. 6;
7) volume estimation: the closed convex hull is a closed cavity body, the volume of the inner cavity of the closed convex hull is calculated, and the volume of the inner cavity is used as the volume estimation value of the current honey pomelo 3.
As shown in fig. 8, in this embodiment, the top LED light source and the bottom LED light source are both soft strip light sources, the soft strip light source is 2835LED light source, when the 2835LED light source is lighted, the black arrow light source light is propagated along the transparent acrylic cylinder 1.8, and is refracted through the laser perforation point on the light guide film 1.5, and the gray conducting light is scattered inside the transparent acrylic cylinder 1.8, so as to finally form an internal illumination environment with excellent uniformity.
The original honey pomelo image is converted from the RGB space to the HSV space, and then the V component image is extracted, as shown in fig. 9. Then, a rectangular region of the V-component map is selected as an ROI, and the luminance values of all the pixels in the ROI are counted pixel by pixel to obtain an ROI luminance distribution map, as shown in fig. 10, the luminance MEAN and the standard deviation STD of the ROI are calculated. In this embodiment, MEAN is 196.1 and STD is 5.8.
In the experiment, 180 honey pomelos 3 are randomly selected, the steps 2 to 7 are repeated, and the average relative error between the point cloud volume estimated value sampled to 4096 points and the pomelo volume measured value of xi honey 3 is found to be 3.91% through statistics, and the R2 value is 0.942. Fig. 7 is a plot of the volume estimate fitted to the actual volume of honey pomelo 3.
Claims (6)
1. A honey pomelo volume estimation method based on lateral multi-view reconstruction is characterized by comprising the following steps:
1) and (3) building a multi-view image acquisition system: the multi-view image acquisition system comprises an illumination box (1), a rotating platform (2), a camera (4) and a host (5), wherein the rotating platform (2) is installed at the center inside the illumination box (1), the rotating platform (2) is used for placing honey pomelos (3), and the camera (4) is formed by overlooking a camera C 1 Head-up camera C 2 And upward camera C 3 The circumferential side surface of the illumination box (1) is provided with a downward looking camera C from top to bottom in sequence 1 Head-up camera C 2 And upward camera C 3 Overlooking camera C 1 Head-up camera C 2 And upward camera C 3 All optical axes of the honey pomelo (3) point to the overlook camera C 1 Head-up camera C 2 And upward camera C 3 Are all connected with a host (5);
2) multi-view honey pomelo dense point cloud reconstruction: starting the rotating platform (2) looking down the camera C 1 Head-up camera C 2 Rising and falling upwardVideo camera C 3 Synchronously shooting the honey pomelos (3), and shooting by each camera to obtain a group of original honey pomelo images of the current honey pomelos (3); recovering three-dimensional point coordinates of the current honey pomelos by using a motion recovery structure and multi-view stereoscopic vision method according to 3 groups of original images of the honey pomelos to obtain dense point clouds of the honey pomelos;
3) dense point cloud segmentation: removing background noise in the dense point cloud of the honey pomelos by adopting a conditionond algorithm according to the dense point cloud of the honey pomelos to obtain a partitioning point cloud of the honey pomelos;
4) segmentation point cloud filtering: for any point k in the honey pomelo segmentation point cloud, taking a plurality of adjacent points adjacent to the current point k, calculating the average distance between the current point k and the current plurality of adjacent points, then selecting outliers for removing by utilizing Gaussian distribution according to the average distance, traversing each point in the honey pomelo segmentation point cloud, removing all outliers of each point, and obtaining the currently obtained honey pomelo filtering point cloud;
5) filtering point cloud down sampling: randomly down-sampling the acquired honey pomelo filtering point cloud to obtain a honey pomelo down-sampling point cloud;
6) and (3) triangularization of the down-sampling point cloud: triangularization processing is carried out on the honey pomelo down-sampling point cloud to form a closed convex hull;
7) volume estimation: the volume of the internal cavity of the closed convex hull is calculated and taken as the volume estimate for the current honey pomelo 3.
2. A honey pomelo volume estimation method based on lateral multi-view reconstruction according to claim 1, characterized in that in step 2) the rotating platform (2) rotates at least one revolution.
3. The honey pomelo volume estimation method based on lateral multi-view reconstruction as claimed in claim 1, wherein in the step 2), firstly, feature point extraction is performed on all honey pomelo original images by using a scale invariant feature transformation algorithm, and feature points between each honey pomelo original image are matched to obtain each group of image matching pairs; then recovering an internal reference matrix and an external reference matrix corresponding to 3 cameras by utilizing an antipodal geometric principle according to the matching characteristic points of each group of image matching pairs; and finally, performing three-dimensional point cloud reconstruction on all original honey pomelo images by using a binocular stereoscopic vision and multi-view stereoscopic vision principle method according to the internal reference matrix and the external reference matrix corresponding to the 3 cameras to obtain dense point cloud of the honey pomelos.
4. The method for estimating volume of honey pomelos based on lateral multi-view reconstruction as claimed in claim 1, wherein in step 4), points outside 0.5 times of the average distance are taken as outliers.
5. The method for estimating the volume of the honey pomelos based on the lateral multi-view reconstruction as claimed in claim 1, wherein the random down-sampling method is adopted to randomly down-sample the obtained honey pomelo filter point clouds in the step 5).
6. The honey pomelo volume estimation method based on lateral multi-view reconstruction as claimed in claim 1, characterized in that the illumination box (1) in step 1) comprises a transparent acrylic cylinder, a top LED light source, a bottom LED light source and a light guiding film;
a rotating platform (2) is arranged in the transparent acrylic cylinder, a camera groove is arranged on the circumferential side surface of the transparent acrylic cylinder, and a downward looking camera C is sequentially arranged on the circumferential side surface of the transparent acrylic cylinder from top to bottom 1 Head-up camera C 2 And upward camera C 3 The light source mounting grooves are formed in the upper end face and the lower end face of the transparent acrylic cylinder, the top LED light source and the bottom LED light source are respectively embedded in the light source mounting grooves of the upper end face and the lower end face, and the light guide film covers the outer circumferential side face of the transparent acrylic cylinder.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210662277.5A CN115035184B (en) | 2022-06-13 | 2022-06-13 | Honey pomelo volume estimation method based on lateral multi-view reconstruction |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210662277.5A CN115035184B (en) | 2022-06-13 | 2022-06-13 | Honey pomelo volume estimation method based on lateral multi-view reconstruction |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN115035184A true CN115035184A (en) | 2022-09-09 |
| CN115035184B CN115035184B (en) | 2024-05-28 |
Family
ID=83125147
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210662277.5A Active CN115035184B (en) | 2022-06-13 | 2022-06-13 | Honey pomelo volume estimation method based on lateral multi-view reconstruction |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115035184B (en) |
Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104535582A (en) * | 2014-12-08 | 2015-04-22 | 昆明理工大学 | Horizontal swing magnetic tile detection device |
| US20180259496A1 (en) * | 2012-06-01 | 2018-09-13 | Agerpoint, Inc. | Systems and methods for monitoring agricultural products |
| US20210072391A1 (en) * | 2019-09-06 | 2021-03-11 | Volvo Car Corporation | Piece-wise network structure for long range environment perception |
| CN112686877A (en) * | 2021-01-05 | 2021-04-20 | 同济大学 | Binocular camera-based three-dimensional house damage model construction and measurement method and system |
| CN112884880A (en) * | 2021-01-20 | 2021-06-01 | 浙江大学 | Line laser-based honey pomelo three-dimensional modeling device and method |
| US20210383540A1 (en) * | 2020-04-22 | 2021-12-09 | Oxford University Innovation Limited | Partial volume estimation from surface reconstructions |
| CN113870179A (en) * | 2021-08-20 | 2021-12-31 | 浙江大学 | Honey pomelo longitudinal and transverse diameter measuring method based on multi-view profile map reconstruction |
| WO2022086739A2 (en) * | 2020-10-23 | 2022-04-28 | Argo AI, LLC | Systems and methods for camera-lidar fused object detection |
-
2022
- 2022-06-13 CN CN202210662277.5A patent/CN115035184B/en active Active
Patent Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20180259496A1 (en) * | 2012-06-01 | 2018-09-13 | Agerpoint, Inc. | Systems and methods for monitoring agricultural products |
| CN104535582A (en) * | 2014-12-08 | 2015-04-22 | 昆明理工大学 | Horizontal swing magnetic tile detection device |
| US20210072391A1 (en) * | 2019-09-06 | 2021-03-11 | Volvo Car Corporation | Piece-wise network structure for long range environment perception |
| US20210383540A1 (en) * | 2020-04-22 | 2021-12-09 | Oxford University Innovation Limited | Partial volume estimation from surface reconstructions |
| WO2022086739A2 (en) * | 2020-10-23 | 2022-04-28 | Argo AI, LLC | Systems and methods for camera-lidar fused object detection |
| CN112686877A (en) * | 2021-01-05 | 2021-04-20 | 同济大学 | Binocular camera-based three-dimensional house damage model construction and measurement method and system |
| CN112884880A (en) * | 2021-01-20 | 2021-06-01 | 浙江大学 | Line laser-based honey pomelo three-dimensional modeling device and method |
| CN113870179A (en) * | 2021-08-20 | 2021-12-31 | 浙江大学 | Honey pomelo longitudinal and transverse diameter measuring method based on multi-view profile map reconstruction |
Non-Patent Citations (4)
| Title |
|---|
| CHUNLONG ZHANG等: "A method for organs classification and fruit counting on pomegranate trees based on multi-features fusion and support vector machine by 3D point cloud", 《SCIENTIA HORTICULTURAE》, 1 February 2021 (2021-02-01) * |
| 吴铮铮;寇展;: "基于单目多视角影像的场景三维重建", 光学与光电技术, no. 05, 10 October 2020 (2020-10-10) * |
| 林洋洋: "基于三维重建的蜜柚外部几何特征估测研究", 《万方数据》, 23 February 2023 (2023-02-23) * |
| 胡燕威;王建军;范媛媛;卢云鹏;白崇岳;张荠匀;: "基于激光雷达的空间物体三维建模与体积计算", 中国激光, no. 05, 10 January 2020 (2020-01-10) * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN115035184B (en) | 2024-05-28 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN107818326B (en) | A kind of ship detection method and system based on scene multidimensional characteristic | |
| CN106356757B (en) | A kind of power circuit unmanned plane method for inspecting based on human-eye visual characteristic | |
| Liang et al. | Automatic plot-wise tree location mapping using single-scan terrestrial laser scanning | |
| CN110569796A (en) | Method for dynamically detecting lane line and fitting lane boundary | |
| CN112200854B (en) | Leaf vegetable three-dimensional phenotype measuring method based on video image | |
| Lou et al. | Accurate multi-view stereo 3D reconstruction for cost-effective plant phenotyping | |
| CN113470090A (en) | Multi-solid-state laser radar external reference calibration method based on SIFT-SHOT characteristics | |
| CN104240264A (en) | Height detection method and device for moving object | |
| CN114241031A (en) | Fish body ruler measurement and weight prediction method and device based on double-view fusion | |
| Malekabadi et al. | Disparity map computation of tree using stereo vision system and effects of canopy shapes and foliage density | |
| CN107560592A (en) | A kind of precision ranging method for optronic tracker linkage target | |
| CN112990063B (en) | Banana maturity grading method based on shape and color information | |
| CN110425983A (en) | A kind of monocular vision three-dimensional reconstruction distance measuring method based on polarization multi-spectrum | |
| CN108510544B (en) | Light strip positioning method based on feature clustering | |
| CN112465778A (en) | Underwater fish shoal observation device and method | |
| CN108320310B (en) | Image sequence-based space target three-dimensional attitude estimation method | |
| CN113379824B (en) | Quasi-circular fruit longitudinal and transverse diameter measuring method based on double-view-point cloud registration | |
| CN117409339A (en) | Unmanned aerial vehicle crop state visual identification method for air-ground coordination | |
| CN114298151A (en) | 3D target detection method based on point cloud data and image data fusion | |
| CN110889868A (en) | Monocular image depth estimation method combining gradient and texture features | |
| Xiang et al. | PhenoStereo: a high-throughput stereo vision system for field-based plant phenotyping-with an application in sorghum stem diameter estimation | |
| CN116883480A (en) | Corn plant height detection method based on binocular image and ground-based radar fusion point cloud | |
| CN116843738A (en) | Tree dumping risk assessment system and method based on TOF depth camera | |
| CN115035184A (en) | Honey pomelo volume estimation method based on lateral multi-view reconstruction | |
| CN114494586B (en) | Lattice projection deep learning network broadleaf branch and leaf separation and skeleton reconstruction method |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant |