CN112950787A - Target object three-dimensional point cloud generation method based on image sequence - Google Patents

Target object three-dimensional point cloud generation method based on image sequence Download PDF

Info

Publication number
CN112950787A
CN112950787A CN202110225992.8A CN202110225992A CN112950787A CN 112950787 A CN112950787 A CN 112950787A CN 202110225992 A CN202110225992 A CN 202110225992A CN 112950787 A CN112950787 A CN 112950787A
Authority
CN
China
Prior art keywords
image
image sequence
kpm
gray
pixel
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN202110225992.8A
Other languages
Chinese (zh)
Other versions
CN112950787B (en
Inventor
袁利毫
昝英飞
韩端锋
祝海涛
钟声华
秦浩
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Harbin Engineering University
Original Assignee
Harbin Engineering University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Harbin Engineering University filed Critical Harbin Engineering University
Priority to CN202110225992.8A priority Critical patent/CN112950787B/en
Publication of CN112950787A publication Critical patent/CN112950787A/en
Application granted granted Critical
Publication of CN112950787B publication Critical patent/CN112950787B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T17/00—Three dimensional [3D] modelling, e.g. data description of 3D objects
    • G06T17/20—Finite element generation, e.g. wire-frame surface description, tesselation
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F17/00—Digital computing or data processing equipment or methods, specially adapted for specific functions
    • G06F17/10—Complex mathematical operations
    • G06F17/16—Matrix or vector computation, e.g. matrix-matrix or matrix-vector multiplication, matrix factorization
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/10—Segmentation; Edge detection
    • G06T7/136—Segmentation; Edge detection involving thresholding
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10028—Range image; Depth image; 3D point clouds

Abstract

The invention discloses a target object three-dimensional point cloud generating method based on an image sequence, which is characterized in that on the basis of a segmented image, the segmented target image sequence is subjected to feature point detection, extraction and matching and error matching pair reduction to obtain an effective matching pair, and then three-dimensional point cloud reconstruction is carried out on the effective matching pair to obtain a target object three-dimensional point cloud model with higher quality and precision. The method can effectively improve the efficiency of the three-dimensional modeling process, improve the quality of the three-dimensional point cloud model, reduce the probability of occurrence of processing errors in each link, is suitable for development of intelligent robot development, and has important engineering value and theoretical guidance significance for the efficiency of intelligent robot operation, especially underwater robot operation.

Description

Target object three-dimensional point cloud generation method based on image sequence
Technical Field
The invention belongs to the technical field of computer vision simulation, and particularly relates to a method for three-dimensional reconstruction of a shot continuous image sequence.
Background
With the progress of image processing technology and the development of three-dimensional sensors, the demands of three-dimensional application scenes such as intelligent mobile robots, unmanned driving, AR and the like are rapidly increased, and the three-dimensional reconstruction technology is required. Different from two-dimensional images, the three-dimensional model has stronger sense of reality, can present more information of people, is also different from drawing modeling, and the three-dimensional scene reconstruction technology has higher requirements on reflecting the original scene situation in real time more quickly and better, wherein the three-dimensional modeling is realized only by an image sequence, although the economic cost is reduced, the three-dimensional model is difficult to ensure the quality, the precision and the accuracy of the three-dimensional model construction, while the reconstruction of continuous multiple images needs to consider the error accumulation generated by observation at different angles, and a process of optimizing or balancing the three-dimensional model exists. Therefore, in order to realize the real-scene three-dimensional model reconstruction quickly and well, the research on the three-dimensional reconstruction method based on the image sequence has important engineering value and practical significance.
At present, there are many methods for improving the three-dimensional reconstruction efficiency and precision of an image sequence, and the methods can be roughly divided into two types: one is that the image is divided to obtain a target image and a background image to respectively carry out three-dimensional reconstruction, and finally the reconstruction of an image three-dimensional model is realized through the splicing and fusion of point clouds; and the other type is that the three-dimensional reconstruction is directly carried out on the scene image, and then the obtained point cloud model segmentation is carried out on the optimization model.
Disclosure of Invention
The invention aims to solve the problems of low quality and poor accuracy of the existing three-dimensional point cloud model, and provides a target object three-dimensional point cloud generation method based on an image sequence.
The purpose of the invention is realized as follows:
a target object three-dimensional point cloud generation method based on an image sequence comprises the following steps:
acquiring a source image sequence, and calculating a gray level image sequence of the source image sequence;
performing threshold segmentation on each image in the gray image sequence, determining an optimal segmentation threshold, segmenting the gray image sequence according to the optimal segmentation threshold, reserving pixels larger than the optimal segmentation threshold as target object pixels to obtain a target gray image sequence, and correspondingly reserving the target object pixels in the source image sequence to obtain the target image sequence;
detecting characteristic points of the target image sequence, extracting and storing the characteristic points, and calculating the module value and the direction of the characteristic points;
and (3) an effective quasi-matching pair in the component target image sequence, wherein the module values and the directions of two characteristic points in the effective quasi-matching pair are equal, and the module values and the directions are recorded
Figure BDA0002956179770000011
A qth valid aligned pair of the pth image and the (p + 1) th image in the target image sequence; deleting the mismatching pairs from the effective quasi-matching pairs to obtain effective matching pairs;
calculating a basic matrix F between the p image and the p +1 image according to the obtained effective matching pairp(ii) a Acquiring an internal parameter matrix K of the camera; according to the obtained basic matrix FpComputing the eigenmatrix EpFor intrinsic matrix EpCalculating an extrinsic parameter rotation matrix R of the camera between the p-th image and the p + 1-th image by singular value decompositionpTranslation matrix Tp：
According to a rotation matrix RpTranslation matrix TpCalculating point cloud data of effective matching pairs between the p image and the p +1 image
Figure BDA0002956179770000025
Position:
Figure BDA0002956179770000021
wherein X ' ═ (X ', y ', 1) is the position of the feature point in the p +1 th image in the valid matching pair;
and calculating all the effective matching pairs to form a point cloud data set, transforming the obtained point cloud data set to the same coordinate system, and eliminating repeated data points to generate the three-dimensional point cloud of the target object.
The calculation method of the gray level image sequence of the source image sequence comprises the following steps:
performing gray value calculation on each pixel point of each image in the source image sequence:
Gray＝0.299R+0.587G+0.114B (2)
gray is a Gray value, and R, G, B are R, G, B three channel values of the pixel points respectively;
and storing the gray value of each pixel point in the pixel point of a new image sequence corresponding to the pixel point of the source image sequence, and taking the new image sequence as a gray image sequence.
The determination method of the optimal segmentation threshold comprises the following steps:
(a1) counting the number of pixels corresponding to each gray scale from 0 to 255 in the gray image sequence, and taking mu (t) (t is 0,1,2, … and 255) as the number of pixels corresponding to the gray value t;
(a2) calculating a mean of gray values of an image
Figure BDA0002956179770000022
Figure BDA0002956179770000023
Wherein p [ mu (t) ] is a probability function of the number of pixels mu (t) with the gray value of t;
(a3) calculating the pixel C of the target image of the object to be segmented in the image0Average gray level mu of1：
Figure BDA0002956179770000024
Where Pr (i | C0) indicates that the number of pixels having a gradation value i accounts for the pixel C of the divided object target image0Probability of (P)iThe probability that the pixel with the gray value i accounts for the total number of the pixels is obtained;
(a4) calculating the pixel C of the target image of the object to be segmented in the image0Number of pixels ofProportion omega1：
Figure BDA0002956179770000031
Wherein N is1Is a target image C0The number of pixels, Sum is the total number of image pixels;
(a5) calculating a background image pixel C in an image1Average gray level mu of2：
Figure BDA0002956179770000032
Where Pr (i | C1) indicates that the number of pixels with a gray level value of i accounts for background image pixel C1Probability of (P)iThe probability that the pixel with the gray value i accounts for the total number of the pixels is obtained;
(a6) calculating a background image pixel C in an image1Is the ratio omega of the number of pixels2：
Figure BDA0002956179770000033
Wherein N is2As a background image C1The number of pixels;
(a7) calculating an optimal segmentation threshold tbestSaid optimal segmentation threshold tbestTo maximize the value of t for the inter-class variance g (t),
Figure BDA0002956179770000034
wherein, the value of alpha belongs to (0,1) is adjusted according to each image segmentation binarization effect graph.
The detection method of the characteristic points comprises the following steps:
each image pixel value f (x, y) in the target grayscale image sequence is processed as follows:
(b1) the image pixel value f (x, y) is subjected to Gaussian filtering G under different Gaussian smoothing parameters sigmaσAnd (3) calculating:
Figure BDA0002956179770000035
(b2) DoG response image DoG f that calculates image pixel value f (x, y):
DoGf(x,y)＝DoG*f(x,y)＝g1(x,y)-g2(x,y) (10)
(b3) repeating the steps (b1) and (b2) to obtain Dogf1(x,y)、DoGf2(x,y)、DoGf3(x,y)；
For Dogf2All values in (x, y) are detected if a certain DoGf2(x, y) in DogfiIf the value of (x + n, y + m) (i is 1,2, 3; n is-1, 0, 1; and m is-1, 0,1) is maximum or minimum, marking the pixel as a characteristic point, and recording the characteristic point
Figure BDA0002956179770000036
The position of the characteristic point is (x, y) for the kth characteristic point of the mth image in the target image sequence.
And setting the module value of the characteristic point as M, wherein the calculation method comprises the following steps:
Figure BDA0002956179770000041
wherein the scale used for L is a characteristic point
Figure BDA0002956179770000042
The scale of the method is as follows:
L(x,y,σ)＝G(x,y,σ)*I(x,y) (12)
σ is an autonomously selected parameter and remains unchanged in all feature points.
Let the direction of the feature point be θ, the calculation method is:
θ(x,y)＝tan-1((L(x,y+1)-L(x,y-1))/(L(x+1,y)-L(x-1,y))) (13)。
the method for deleting the mismatching pairs comprises the following steps:
(c1) from valid quasi-matching pairs KPMq' inSelecting the first M non-collinear sample data, wherein M is more than or equal to 4, calculating an optimal homography matrix according to the formula, and marking the optimal homography matrix as a model M;
Figure BDA0002956179770000043
wherein (x, y), (x ', y') respectively represent valid quasi-matching pairs KPMq' the positions of two pairs of characteristic points in the equation, s is a scale parameter, and h is usually given as33Normalizing the matrix by 1;
(c2) calculating an effective quasi-matching pair KPM based on the model Mq' projection error of all data in e:
Figure BDA0002956179770000044
if e is less than 0.75, retaining the data, otherwise, removing the data;
(c3) let optimal valid matching pair KPM'best，KPM′bestInitial value of (1) ═ KPM1', if the current valid quasi-matching is to KPMq'the number of elements is greater than the optimal effective matching pair KPM'bestAnd then KPM'best＝KPMq', update the iteration number kk at the same time:
Figure BDA0002956179770000045
wherein, p is confidence coefficient, and takes value of 0.995, and w is KPM'bestNumber of elements/KPMq' number of elements, number of initial iterations kk0Middle KPM'bestThe number of the elements is 1;
(c4) if the iteration times are more than kk, exiting the loop and recording KPMqFor efficient matching of KPMq＝KPM′best(ii) a Otherwise, the number of iterations is increased by 1, and the above step (c3) is repeated.
The internal parameter matrix of the camera is as follows:
Figure BDA0002956179770000051
wherein f isx，fyIs the focal length, x0、y0Is the coordinate of the principal point, and s is the inclination parameter of the coordinate axis.
Basis matrix F between p image and p +1 imagepThe calculation method comprises the following steps:
Figure BDA0002956179770000052
wherein X is K-1(x,y,1),X′＝K-1(x ', y', 1) is an effective matching pair
Figure BDA0002956179770000053
The origin is located at the center of the image.
Compared with the prior art, the invention has the beneficial effects that:
the invention provides a target object three-dimensional point cloud generating method based on an image sequence, which takes target object three-dimensional point cloud generation based on the image sequence as a research object, deeply analyzes various process collocation of three-dimensional modeling and advantages and disadvantages of an algorithm, and designs the target object image sequence three-dimensional point cloud generating method based on image segmentation. The invention is suitable for development of intelligent robot development, and has important engineering value and theoretical guidance significance for the operation of the intelligent robot, especially the operation efficiency of the underwater robot.
Drawings
FIG. 1 is a flow chart of a method for generating a three-dimensional point cloud of a target object based on an image sequence;
FIG. 2 is a flow chart of a target object thresholding method based on an image sequence;
FIG. 3 is a schematic diagram of a feature point extraction algorithm model.
Detailed Description
The present invention will be described in further detail with reference to the accompanying drawings and specific embodiments.
As shown in fig. 1, the method for generating a three-dimensional point cloud of a target object based on an image sequence specifically includes the following steps:
1. acquiring a source image sequence from shooting equipment, and recording the source image sequence as S1; calculating a R, G, B three-channel value of each pixel point of each image in the source image sequence S1 according to the following formula (1), obtaining a Gray value Gray of each pixel point, storing the Gray value into the pixel points of a new image sequence corresponding to the pixel points of the source image sequence, and recording the new image sequence as a Gray image sequence S2.
Gray＝0.299R+0.587G+0.114B (1)
2. Each image in the grayscale image sequence S2 is threshold-segmented to determine an optimal segmentation threshold.
The flow chart of the threshold segmentation is shown in fig. 2, and the specific steps are as follows:
the number of pixels corresponding to each gradation level is counted from 0 to 255, and μ (t) (t is 0,1,2, …,255) is the number of pixels corresponding to the gradation value t.
(a1) Counting the number of pixels corresponding to each gray scale from 0 to 255 in the gray image sequence, and taking mu (t) (t is 0,1,2, … and 255) as the number of pixels corresponding to the gray value t;
(a2) calculating a mean of gray values of an image
Figure BDA0002956179770000061
Figure BDA0002956179770000062
Wherein p [ mu (t) ] is a probability function of the number of pixels mu (t) with the gray value of t; (ii) a
(a3) Calculating the pixel C of the target image of the object to be segmented in the image0Average gray level mu of1：
Figure BDA0002956179770000063
Where Pr (i | C0) indicates that the number of pixels having a gradation value i accounts for the pixel C of the divided object target image0Probability of (P)iIs the probability of the pixel with the gray value i occupying the total number of pixels.
(a4) Calculating the pixel C of the target image of the object to be segmented in the image0Is the ratio omega of the number of pixels1：
Figure BDA0002956179770000064
Wherein N is1Is a target image C0The number of pixels, Sum is the total number of image pixels;
(a5) calculating the average gray level μ of the background image pixel C1 in the image2：
Figure BDA0002956179770000065
Where Pr (i | C1) indicates that the number of pixels with a gray level value of i accounts for background image pixel C1Probability of (P)iIs the probability of the pixel with the gray value i occupying the total number of pixels.
(a6) Calculating a background image pixel C in an image1The ratio of the number of pixels:
Figure BDA0002956179770000066
wherein N is2As a background image C1The number of pixels;
(a7) calculating an optimal segmentation threshold tbestSaid optimal segmentation threshold tbestTo maximize the value of t for the inter-class variance g (t),
g(t)＝ω1 α-2(ω1μ-μ(t))2+ω2 α-2(μ(ω2-1)+μ(t))2 (7)
wherein, the value of alpha belongs to (0,1) is adjusted according to each image segmentation binarization effect graph.
According to the optimal segmentation threshold tbestThe gray image sequence S2 is divided to be less than tbestThe pixel of (A) is a background pixel C1Greater than tbestIs the target object pixel C0At this time, the target object pixel C0Reserving to obtain an object target gray level image sequence SAblackAnd correspondingly reserving pixel points in the source image sequence S1 to obtain a segmented target image sequence SA.
3. The method for detecting, extracting and storing the feature points of the target image sequence SA includes the following steps, and a flowchart of the method for extracting the feature points is shown in fig. 3:
for target gray image sequence SAblackIs processed as follows:
(b1) the image pixel value f (x, y) is subjected to Gaussian filtering G under different Gaussian smoothing parameters sigmaσAnd (3) calculating:
Figure BDA0002956179770000071
(b2) DoG response image DoG f that calculates image pixel value f (x, y):
DoGf(x,y)＝DoG*f(x,y)＝g1(x,y)-g2(x,y) (9)
(b3) repeating the steps (b1) and (b2) to obtain Dogf1(x,y)、DoGf2(x,y)、DoGf3(x,y)；
For Dogf2All values in (x, y) are detected if a certain DoGf2(x, y) in DogfiIf the value of (x + n, y + m) (i is 1,2, 3; n is-1, 0, 1; and m is-1, 0,1) is maximum or minimum, marking the pixel as a characteristic point, and recording the characteristic point
Figure BDA0002956179770000072
The position of the characteristic point is (x, y) for the kth characteristic point of the mth image in the target image sequence.
Calculating feature points from equation (10)
Figure BDA0002956179770000073
The modulus value M:
Figure BDA0002956179770000074
wherein the scale used for L is a characteristic point
Figure BDA0002956179770000075
The scale of the method is as follows:
L(x,y,σ)＝G(x,y,σ)*I(x,y) (11)
wherein the parameter sigma is selected autonomously and in all feature points
Figure BDA0002956179770000076
Is kept unchanged.
Calculating feature points from equation (12)
Figure BDA0002956179770000077
Direction θ of (c):
θ(x,y)＝tan-1((L(x,y+1)-L(x,y-1))/(L(x+1,y)-L(x-1,y))) (12)
4. for the characteristic points in the target image sequence SA
Figure BDA0002956179770000078
Are matched in pairs, i.e.
Figure BDA0002956179770000079
And
Figure BDA00029561797700000710
a pair of feature points KP with the middle module value M and the direction theta equal are effectively matched and recorded
Figure BDA0002956179770000081
For the q (q ═ 1,2,3, …) th valid quasi-matching pair of the p-th image and the p + 1-th image (p ═ 1,2,3, …) in the target image sequence SA, the mis-matching pair is deleted from the valid quasi-matching pair, resulting in a valid matching pair.
The specific steps for deleting the mismatching pairs are as follows:
(c1) from valid quasi-matching pairs KPMqSelecting the previous M non-collinear sample data, wherein M is more than or equal to 4, calculating an optimal homography matrix according to the formula, and marking the optimal homography matrix as a model M;
Figure BDA0002956179770000082
wherein (x, y), (x ', y') respectively represent valid quasi-matching pairs KPMq' the positions of two pairs of characteristic points in the equation, s is a scale parameter, and h is usually given as33Normalizing the matrix by 1;
(c2) calculating an effective quasi-matching pair KPM based on the model Mq' projection error of all data in e:
Figure BDA0002956179770000083
if e is less than 0.75, retaining the data, otherwise, removing the data;
(c3) let optimal valid matching pair KPM'best，KPM′bestInitial value of (1) ═ KPM1', if the current valid quasi-matching is to KPMq'the number of elements is greater than the optimal effective matching pair KPM'bestAnd then KPM'best＝KPMq', update the iteration number kk at the same time:
Figure BDA0002956179770000084
wherein, p is confidence coefficient, and takes value of 0.995, and w is KPM'bestNumber of elements/KPMq' number of elements, number of initial iterations kk0Middle KPM'bestThe number of the elements is 1;
(c4) if the iteration times are more than kk, exiting the loop and recording KPMqFor efficient matching of KPMq＝KPM′best(ii) a Otherwise, the number of iterations is increased by 1, and the above step (c3) is repeated.
5. Obtaining internal parameters of the camera by referring to relevant information of the camera or using a camera calibration method, wherein an internal parameter matrix K comprises:
Figure BDA0002956179770000085
wherein f isx，fyFor focal length, x is generally equal to0、y0Is the principal point coordinate (relative to the imaging plane) and s is the coordinate axis tilt parameter, ideally 0.
According to the obtained effective matching pairs
Figure BDA0002956179770000091
Calculating a basis matrix F between the p-th image and the p + 1-th image by equation (16)p。
Figure BDA0002956179770000092
Wherein X is K-1(x,y,1),X′＝K-1(x ', y', 1) is an effective matching pair
Figure BDA0002956179770000093
The origin is located at the center of the image.
According to the obtained basic matrix FpCalculating an extrinsic parameter rotation matrix R of the camera between the p-th image and the p + 1-th image by equation (18)pTranslation matrix Tp。
Fp＝C-TEpC-1，Ep＝Tp×Rp(18)
Where C is the vector of the position of the camera center in the world coordinate system given by itself, often let C equal the camera coordinate system of the p-th image, EpFor the intrinsic matrix of the camera between the p-th image and the p + 1-th image, by matching the intrinsic matrix EpThe exterior of the camera between the p-th image and the p + 1-th image can be calculated by Singular Value Decomposition (Singular Value Decomposition)Partial parameter rotation matrix RpTranslation matrix Tp。
6. Calculating the effective matching pair between the p-th image and the p + 1-th image by the formula (19)
Figure BDA0002956179770000094
Point cloud data of
Figure BDA0002956179770000095
Position:
Figure BDA0002956179770000096
will effectively match with the pair
Figure BDA0002956179770000097
All calculated and recorded as a point cloud data set Xp(p ═ 1,2,3, …). The obtained point cloud data set XpAnd transforming to the same coordinate system, and eliminating repeated data points to complete the generation of the three-dimensional point cloud of the target object.
In summary, the following steps: the invention discloses a target object three-dimensional point cloud generating method based on an image sequence, which is characterized in that on the basis of a segmented image, the segmented target image sequence is subjected to feature point detection, extraction and matching and error matching pair reduction to obtain an effective matching pair, and then the effective matching pair is subjected to three-dimensional point cloud reconstruction to obtain a target object three-dimensional point cloud model with higher quality and precision. The method can effectively improve the efficiency of the three-dimensional modeling process, improve the quality of the three-dimensional point cloud model, reduce the probability of occurrence of processing errors in each link, is suitable for development of intelligent robot development, and has important engineering value and theoretical guidance significance for the efficiency of intelligent robot operation, especially underwater robot operation.

Claims (9)

1. A target object three-dimensional point cloud generating method based on an image sequence is characterized by comprising the following steps:
acquiring a source image sequence, and calculating a gray level image sequence of the source image sequence;
performing threshold segmentation on each image in the gray image sequence, determining an optimal segmentation threshold, segmenting the gray image sequence according to the optimal segmentation threshold, reserving pixels larger than the optimal segmentation threshold as target object pixels to obtain a target gray image sequence, and correspondingly reserving the target object pixels in the source image sequence to obtain the target image sequence;
detecting characteristic points of the target image sequence, extracting and storing the characteristic points, and calculating the module value and the direction of the characteristic points;
and (3) an effective quasi-matching pair in the component target image sequence, wherein the module values and the directions of two characteristic points in the effective quasi-matching pair are equal, and the module values and the directions are recorded
Figure FDA0002956179760000013
A qth valid aligned pair of the pth image and the (p + 1) th image in the target image sequence; deleting the mismatching pairs from the effective quasi-matching pairs to obtain effective matching pairs;
calculating a basic matrix F between the p image and the p +1 image according to the obtained effective matching pairp(ii) a Acquiring an internal parameter matrix K of the camera; according to the obtained basic matrix FpComputing the eigenmatrix EpFor intrinsic matrix EpCalculating an extrinsic parameter rotation matrix R of the camera between the p-th image and the p + 1-th image by singular value decompositionpTranslation matrix Tp：
According to a rotation matrix RpTranslation matrix TpCalculating point cloud data of effective matching pairs between the p image and the p +1 image
Figure FDA0002956179760000012
Position:
Figure FDA0002956179760000011
wherein X ' ═ (X ', y ', 1) is the position of the feature point in the p +1 th image in the valid matching pair;
and calculating all the effective matching pairs to form a point cloud data set, transforming the obtained point cloud data set to the same coordinate system, and eliminating repeated data points to generate the three-dimensional point cloud of the target object.
2. The method for generating a three-dimensional point cloud of a target object based on an image sequence according to claim 1, wherein the calculation method of the gray-scale image sequence of the source image sequence is as follows:
performing gray value calculation on each pixel point of each image in the source image sequence:
Gray＝0.299R+0.587G+0.114B (2)
gray is a Gray value, and R, G, B are R, G, B three channel values of the pixel points respectively;
and storing the gray value of each pixel point in the pixel point of a new image sequence corresponding to the pixel point of the source image sequence, and taking the new image sequence as a gray image sequence.
3. The method for generating a three-dimensional point cloud of a target object based on an image sequence according to claim 1 or 2, wherein the optimal segmentation threshold is determined by:
(a1) counting the number of pixels corresponding to each gray scale from 0 to 255 in the gray image sequence, and taking mu (t) (t is 0,1,2, … and 255) as the number of pixels corresponding to the gray value t;
(a2) calculating a mean of gray values of an image
Figure FDA0002956179760000021
Figure FDA0002956179760000022
Wherein p [ mu (t) ] is a probability function of the number of pixels mu (t) with the gray value of t;
(a3) calculating the pixel C of the target image of the object to be segmented in the image0Average gray level mu of1：
Figure FDA0002956179760000023
Where Pr (i | C0) indicates that the number of pixels having a gradation value i accounts for the pixel C of the divided object target image0Probability of (P)iThe probability that the pixel with the gray value i accounts for the total number of the pixels is obtained;
(a4) calculating the pixel C of the target image of the object to be segmented in the image0Is the ratio omega of the number of pixels1：
Figure FDA0002956179760000024
Wherein N is1Is a target image C0The number of pixels, Sum is the total number of image pixels;
(a5) calculating a background image pixel C in an image1Average gray level mu of2：
Figure FDA0002956179760000025
Where Pr (i | C1) indicates that the number of pixels with a gray level value of i accounts for background image pixel C1Probability of (P)iThe probability that the pixel with the gray value i accounts for the total number of the pixels is obtained;
(a6) calculating a background image pixel C in an image1Is the ratio omega of the number of pixels2：
Figure FDA0002956179760000026
Wherein N is2As a background image C1The number of pixels;
(a7) calculating an optimal segmentation threshold tbestSaid optimal segmentation threshold tbestTo maximize the value of t for the inter-class variance g (t),
Figure FDA0002956179760000027
wherein, the value of alpha belongs to (0,1) is adjusted according to each image segmentation binarization effect graph.
4. The method for generating a three-dimensional point cloud of a target object based on an image sequence according to claim 1 or 2, wherein the method for detecting the feature points comprises:
each image pixel value f (x, y) in the target grayscale image sequence is processed as follows:
(b1) the image pixel value f (x, y) is subjected to Gaussian filtering G under different Gaussian smoothing parameters sigmaσAnd (3) calculating:
Figure FDA0002956179760000031
Figure FDA0002956179760000032
(b2) DoG response image DoG f that calculates image pixel value f (x, y):
DoGf(x,y)＝DoG*f(x,y)＝g1(x,y)-g2(x,y) (10)
(b3) repeating the steps (b1) and (b2) to obtain Dogf1(x,y)、DoGf2(x,y)、DoGf3(x,y)；
For Dogf2All values in (x, y) are detected if a certain DoGf2(x, y) in DogfiIf the value of (x + n, y + m) (i is 1,2, 3; n is-1, 0, 1; and m is-1, 0,1) is maximum or minimum, marking the pixel as a characteristic point, and recording the characteristic point
Figure FDA0002956179760000035
The position of the characteristic point is (x, y) for the kth characteristic point of the mth image in the target image sequence.
5. The method for generating a three-dimensional point cloud of a target object based on an image sequence according to claim 4, wherein the module value of the feature point is M, and the calculation method comprises:
Figure FDA0002956179760000033
wherein the scale used for L is a characteristic point
Figure FDA0002956179760000034
The scale of the method is as follows:
L(x,y,σ)＝G(x,y,σ)*I(x,y) (12)
σ is an autonomously selected parameter and remains unchanged in all feature points.
6. The method for generating a three-dimensional point cloud of a target object based on an image sequence according to claim 5, wherein the direction of the feature point is θ, and the calculation method comprises:
θ(x,y)＝tan-1((L(x,y+1)-L(x,y-1))/(L(x+1,y)-L(x-1,y))) (13)。
7. the method for generating the three-dimensional point cloud of the target object based on the image sequence according to claim 1 or 2, wherein the method for deleting the mismatching pairs comprises:
(c1) from valid quasi-matching pairs KPMqSelecting the previous M non-collinear sample data, wherein M is more than or equal to 4, calculating an optimal homography matrix according to the formula, and marking the optimal homography matrix as a model M;
Figure FDA0002956179760000041
wherein (x, y), (x ', y') respectively represent valid quasi-matching pairs KPMq' the positions of two pairs of feature points in the equation, s is a scale parameter, and h is usually given33Normalizing the matrix by 1;
(c2) calculating an effective quasi-matching pair KPM based on the model Mq' projection error of all data in e:
Figure FDA0002956179760000042
if e is less than 0.75, retaining the data, otherwise, removing the data;
(c3) let optimal valid matching pair KPM'best，KPM′bestInitial value of (1) ═ KPM1', if the current valid quasi-matching is to KPMq'the number of elements is greater than the optimal effective matching pair KPM'bestAnd then KPM'best＝KPMq', update the iteration number kk at the same time:
Figure FDA0002956179760000043
wherein, p is confidence coefficient, and takes value of 0.995, and w is KPM'bestNumber of elements/KPMq' number of elements, number of initial iterations kk0Middle KPM'bestThe number of the elements is 1;
(c4) if the iteration times are more than kk, exiting the loop and recording KPMqFor efficient matching of KPMq＝KPM′best(ii) a Otherwise, the number of iterations is increased by 1, and the above step (c3) is repeated.
8. The method of generating a three-dimensional point cloud of a target object based on an image sequence according to claim 1 or 2, wherein the internal parameter matrix of the camera is:
Figure FDA0002956179760000044
wherein f isx，fyIs the focal length, x0、y0Is the coordinate of the principal point, and s is the inclination parameter of the coordinate axis.
9. The method of claim 1 or 2, wherein the method comprisesBasis matrix F between p image and p +1 imagepThe calculation method comprises the following steps:
Figure FDA0002956179760000045
wherein X is K-1(x,y,1),X′＝K-1(x ', y', 1) is an effective matching pair
Figure FDA0002956179760000046
The origin is located at the center of the image.
CN202110225992.8A 2021-03-01 2021-03-01 Target object three-dimensional point cloud generation method based on image sequence Active CN112950787B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202110225992.8A CN112950787B (en) 2021-03-01 2021-03-01 Target object three-dimensional point cloud generation method based on image sequence

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202110225992.8A CN112950787B (en) 2021-03-01 2021-03-01 Target object three-dimensional point cloud generation method based on image sequence

Publications (2)

Publication Number Publication Date
CN112950787A true CN112950787A (en) 2021-06-11
CN112950787B CN112950787B (en) 2022-08-02

Family

ID=76246962

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202110225992.8A Active CN112950787B (en) 2021-03-01 2021-03-01 Target object three-dimensional point cloud generation method based on image sequence

Country Status (1)

Country Link
CN (1) CN112950787B (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116071520A (en) * 2023-03-31 2023-05-05 湖南省水务规划设计院有限公司 Digital twin water affair simulation test method

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106157373A (en) * 2016-07-27 2016-11-23 中测高科（北京）测绘工程技术有限责任公司 A kind of construction three-dimensional model building method and system
CN110517348A (en) * 2019-07-23 2019-11-29 西安电子科技大学 Target object three-dimensional point cloud method for reconstructing based on display foreground segmentation

Patent Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106157373A (en) * 2016-07-27 2016-11-23 中测高科（北京）测绘工程技术有限责任公司 A kind of construction three-dimensional model building method and system
CN110517348A (en) * 2019-07-23 2019-11-29 西安电子科技大学 Target object three-dimensional point cloud method for reconstructing based on display foreground segmentation

Non-Patent Citations (4)

* Cited by examiner, † Cited by third party
Title
ZHUHAITAO 等: "A feature-based Section Curve Reconstructing Strategy", 《KEY ENGINEERING MATERIALS》 *
朱冰琳等: "基于机器视觉的大田植株生长动态三维定量化研究", 《农业机械学报》 *
王晖等: "城市园林建设中花卉色彩图像匹配仿真研究", 《计算机仿真》 *
王欢 等: "利用Kinect深度信息的三维点云配准方法研究", 《计算机工程与应用》 *

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116071520A (en) * 2023-03-31 2023-05-05 湖南省水务规划设计院有限公司 Digital twin water affair simulation test method

Also Published As

Publication number Publication date
CN112950787B (en) 2022-08-02

Similar Documents

Publication Publication Date Title
CN109784333B (en) Three-dimensional target detection method and system based on point cloud weighted channel characteristics
CN110108258B (en) Monocular vision odometer positioning method
CN109740665B (en) Method and system for detecting ship target with occluded image based on expert knowledge constraint
CN108597009B (en) Method for detecting three-dimensional target based on direction angle information
CN110555908B (en) Three-dimensional reconstruction method based on indoor moving target background restoration
CN111862201A (en) Deep learning-based spatial non-cooperative target relative pose estimation method
CN111696196B (en) Three-dimensional face model reconstruction method and device
CN113052109A (en) 3D target detection system and 3D target detection method thereof
CN109325444B (en) Monocular texture-free three-dimensional object posture tracking method based on three-dimensional geometric model
CN111797688A (en) Visual SLAM method based on optical flow and semantic segmentation
CN113065546A (en) Target pose estimation method and system based on attention mechanism and Hough voting
CN113850865A (en) Human body posture positioning method and system based on binocular vision and storage medium
CN113888461A (en) Method, system and equipment for detecting defects of hardware parts based on deep learning
CN112541423A (en) Synchronous positioning and map construction method and system
CN112950787B (en) Target object three-dimensional point cloud generation method based on image sequence
CN115082498A (en) Robot grabbing pose estimation method, device, equipment and storage medium
CN112767478B (en) Appearance guidance-based six-degree-of-freedom pose estimation method
CN116977328B (en) Image quality evaluation method in active vision of vehicle bottom robot
CN112053385B (en) Remote sensing video shielding target tracking method based on deep reinforcement learning
CN112686202A (en) Human head identification method and system based on 3D reconstruction
CN116843738A (en) Tree dumping risk assessment system and method based on TOF depth camera
CN116862984A (en) Space pose estimation method of camera
CN116643291A (en) SLAM method for removing dynamic targets by combining vision and laser radar
CN111709269A (en) Human hand segmentation method and device based on two-dimensional joint information in depth image
CN113592947B (en) Method for realizing visual odometer by semi-direct method

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant
EE01 Entry into force of recordation of patent licensing contract

Application publication date: 20210611

Assignee: Enshobo (Guangzhou) Technology Co.,Ltd.

Assignor: HARBIN ENGINEERING University

Contract record no.: X2023980048395

Denomination of invention: A 3D point cloud generation method for target objects based on image sequences

Granted publication date: 20220802

License type: Common License

Record date: 20231127

EE01 Entry into force of recordation of patent licensing contract