CN106981043B - High-precision three-dimensional information rapid acquisition method based on random forest - Google Patents
High-precision three-dimensional information rapid acquisition method based on random forest Download PDFInfo
- Publication number
- CN106981043B CN106981043B CN201611037639.2A CN201611037639A CN106981043B CN 106981043 B CN106981043 B CN 106981043B CN 201611037639 A CN201611037639 A CN 201611037639A CN 106981043 B CN106981043 B CN 106981043B
- Authority
- CN
- China
- Prior art keywords
- training
- scene
- random
- camera
- random forest
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T1/00—General purpose image data processing
- G06T1/0007—Image acquisition
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10016—Video; Image sequence
- G06T2207/10021—Stereoscopic video; Stereoscopic image sequence
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20081—Training; Learning
Abstract
The invention provides a high-precision three-dimensional information rapid acquisition method based on a random forest, which does not need stereo matching and avoids the reduction of precision and acquisition rate caused by dense calculation. The method comprises three processes of learning model training data acquisition, random forest learning and deformation speckle optimal depth classification regression, wherein a hardware system consists of a digital projector, a camera and a detected scene, two working states of an initialization state and a running state exist, and three-dimensional information can be acquired rapidly with high precision in the running state.
Description
Technical Field
The invention relates to a three-dimensional information acquisition technology, in particular to a high-precision three-dimensional information rapid acquisition method based on a random forest.
Background
The three-dimensional information acquisition technology can simultaneously obtain the three-dimensional coordinates of a detected scene, and has important significance and wide application prospect in the fields of real object profiling, industrial detection, virtual reality, machine vision, intelligent interaction and the like. Three-dimensional information acquisition is mainly divided into two categories, namely a passive three-dimensional information acquisition method and an active three-dimensional information acquisition method. The passive three-dimensional information acquisition method is based on a stereo matching principle, two-dimensional images of a detected scene at different visual angles are acquired from multiple angles by using two or more camera systems, and when texture information of the detected scene is too simple or the detected scene has reflectivity in different directions, the calculation precision of the method is greatly reduced, the calculation complexity and the processing time are greatly increased, and high-precision three-dimensional information cannot be quickly acquired; the active three-dimensional information acquisition method mainly utilizes the form of structured light illumination, has the characteristic of high precision, but the precision of the method greatly depends on the conversion frame number of the illumination structured light and the optimization degree of a dynamic path during calculation, and the two just limit the acquisition speed of the active three-dimensional information acquisition method.
Disclosure of Invention
The invention aims to realize a high-precision and quick three-dimensional information acquisition method. In order to achieve the purpose, the invention provides a high-precision three-dimensional information rapid acquisition method based on a random forest, which does not need stereo matching and avoids the reduction of acquisition precision and acquisition rate caused by dense calculation. The method comprises three processes of obtaining learning model training data, learning random forests and classification regression of the optimal depth of the deformed speckles.
The hardware system of the invention consists of a digital projector, a camera and a measured scene, as shown in figure 1, the digital projector and the camera are precisely calibrated to project pseudo-random digital speckles on the surface of the measured scene, the camera synchronously acquires corresponding deformed speckles, and the focal length of the lens of the digital projector is the same as that of the lens of the camera and is marked asfThe optical centers of the two lenses are spaced bybAnd the working state remains unchanged.
The invention works in two working states: an initialization state and a runtime state. The working process is as shown in the attached figure 2, when the method works for the first time, an initialization state is firstly entered, a plurality of training scene depth values are obtained by using a high-precision structured light measurement method, and a classifier and a regressor of the random forest are trained and learned by combining a local binary feature label of each pixel of the deformed speckles; then the method enters a running state, a camera acquires deformation speckles of a detected scene, a random forest is used for classifying local binary features of each pixel to acquire local binary feature labels, and then the random deep forest is used for regressing the local binary feature labels to acquire optimal depth values corresponding to the deformation speckles. When the method does not work for the first time, the method directly enters a running state, and the deformed speckles are classified and the optimal depth is regressed. Due to the irrelevance between pixels of each point of the deformed speckles, in the running state of the method, the classification regression of the optimal depth of the deformed speckles of the detected scene is processed in parallel by using a Graphic Processing Unit (GPU).
In the learning model training data acquisition process, firstly, a high-precision structured light measurement method is used for acquiring the training dataTDepth values of training scenes, whereintThe depth value of the training scene is recorded asD t (x,y). According to the parameters of the digital projector and the camera, the depth value of the scene to be trained can be obtainedD t (x,y) Corresponding training scene disparity valuesd t (x,y)：
Wherein (A), (B), (C), (D), (C), (x,y) Coordinates of each point pixel in a pixel coordinate system are used for training scene depth values and training scene parallax values. Then, a digital projector projects a pseudo-random speckle image onto the training sceneR(x,y) The camera acquires corresponding deformed speckleI t (x,y) And is provided withM×NSliding window of pixels, traversing deformable speckleI t (x,y) Per dot pixel of (1) <x,y) Obtaining (A)x,y) Local binary characteristics of the points. To pairTThe same processing is carried out on each training scene to obtain the parallax value of the training scened t (x,y) And local binary characteristics are used as learning model training data, and the learning model training data is input into the random forest for learning of the random forest.
In the learning process of the random forest, firstly, the input deformed speckles are utilizedI t (x,y) Local binary characterization of pixels per point and digital projector projected speckle imagesR(x,y) Calculating the deformed speckleI t (x,y) The parallax label corresponding to each pixelc t (x,y). When the digital projector and the camera are accurately calibrated, the parallax labelc t (x,y) Satisfies the following conditions:
wherein (A), (B), (C), (D), (C), (x',y') As parallax labelsc t (x,y) In speckle imagesR(x,y) The corresponding pixel coordinates. By usingTTraining scene parallax value obtained by each training scened t (x,y) And corresponding parallax labelc t (x,y) Composing training set dataSAnd independently training a plurality of trees. In the present invention, each tree has in commonNLayer, frontkThe layer nodes are integers, so that the classification problem is solved; rear endN-kAnd the layer nodes are decimal numbers, and prediction data of sub-pixel precision is obtained through a regression function.
The learning process of the random forest of the invention starts from the root node of each tree and randomly sets a series of random discrimination parametersδUsing each of the discrimination parametersδWill train set dataSDivided into left training set dataS L (δ) And right training set dataS R (δ) While simultaneously computing the objective functionO(δ)：
Wherein the content of the first and second substances,E(S)、E(S L (δ))、E(S R (δ) Respectively training set dataSLeft training set dataS L (δ) And right training set dataS R (δ) The entropy of information of (1). Random discrimination parameterδIntermediate order objective functionO(δ) The maximum value is the final discrimination parameter of the node. For left training set dataS L (δ) And right training set dataS R (δ) Recursion is carried out in sequence until the training depth reaches the second of the treekLayer, or training set data is not re-separable. Training independent trees into decision trees, the decision trees forming a random forest for runtime stateF。
In the classification regression process of the optimal depth of the deformed speckles, as shown in the attached figure 3, a digital projector projects pseudo-random digital speckles on the surface of a measured scene, and the pixel arrangement and local amplification structure of the pseudo-random digital speckles are shown in the attached figure 4. Camera for acquiring deformation speckles of detected sceneI'(x,y) Using random forestsFBefore decision treekLayer, classifying each point pixel local binary characteristic to obtain local binary characteristic labelc' I (x,y) And then the random deep forest is recycledFAfter decision treeN-kLayer, regression is carried out to local binary characteristic label, obtains surveyed scene deformation speckleI'(x,y) Corresponding sub-pixel precision local binary feature labelc'(x,y) Because the digital projector and the camera are accurately calibrated, the optimal depth value of the detected sceneD'(x,y) Satisfies the following conditions:
in the invention, speckle is deformed due to the detected sceneI'(x,y) The processing of each pixel has irrelevancy, so that in the running state of the method, the classification regression of the optimal depth of the detected scene deformation speckles is carried out, and the GPU is used for carrying out parallel processing to obtain higher processing speed.
The invention provides a high-precision three-dimensional information rapid acquisition method based on a random forest. The method does not need stereo matching, avoids the reduction of precision and acquisition rate caused by dense calculation, and is a high-precision and rapid three-dimensional information acquisition method. The hardware system of the invention consists of a digital projector, a camera and a measured scene, and has two working states of an initialization state and a running state, and the three-dimensional information of the measured scene can be rapidly acquired with high precision in the running state.
Drawings
FIG. 1 is a hardware system diagram of a high-precision three-dimensional information rapid acquisition method based on random forest
FIG. 2 is a flow chart of the method of the present invention
FIG. 3 is a classification regression process of the optimal depth of deformed speckles in the present invention
FIG. 4 shows the pseudo-random digital speckle pixel arrangement and local magnification structure of the present invention
The reference numbers in the figures are:
1 digital projector, 2 camera, 3 measured scene, 4 measured scene deformation speckle, 5 random forest node, 6 random forest decision tree frontkLayer, 7 random forest decision TreeN-kLayer 8, optimal depth value of a detected scene, 9 pseudo-random digital speckle pixel arrangement and 10 pseudo-random digital speckle local amplification structure.
It should be understood that the above-described figures are merely schematic and are not drawn to scale.
Detailed Description
The following describes an exemplary embodiment of a method for fast obtaining three-dimensional information with high precision based on random forest according to the present invention in detail, and further describes the present invention in detail. It should be noted that the following examples are only for illustrative purposes and should not be construed as limiting the scope of the present invention, and that the skilled person in the art may make modifications and adaptations of the present invention without departing from the scope of the present invention.
The invention provides a high-precision three-dimensional information rapid acquisition method based on random forests.
The hardware system of the invention consists of a digital projector, a camera and a measured scene, as shown in figure 1, the digital projector and the camera are precisely calibrated to project pseudo-random digital speckles on the surface of the measured scene, the camera synchronously acquires corresponding deformed speckles, the focal length of the lens of the digital projector is the same as that of the lens of the camera, and the method is characterized in thatf =45mm, the optical center interval between the two lenses isb=570mm, and the operating state remains unchanged.
The invention works in two working states: an initialization state and a runtime state. The working process is as shown in the attached figure 2, when the method works for the first time, an initialization state is firstly entered, a plurality of training scene depth values are obtained by using a high-precision structured light measurement method, and a classifier and a regressor of the random forest are trained and learned by combining a local binary feature label of each pixel of the deformed speckles; then the method enters a running state, a camera acquires deformation speckles of a detected scene, a random forest is used for classifying local binary features of each pixel to acquire local binary feature labels, and then the random deep forest is used for regressing the local binary feature labels to acquire optimal depth values corresponding to the deformation speckles. When the method does not work for the first time, the method directly enters a running state, and the deformed speckles are classified and the optimal depth is regressed. Due to the irrelevance between pixels of each point of the deformed speckles, in the running state of the method, the classification regression of the optimal depth of the deformed speckles of the detected scene utilizes the GPU to perform parallel processing.
In the learning model training data acquisition process, firstly, a high-precision structured light measurement method is used for acquiring the training data togetherTDepth values of =13000 training scenes, wheretThe depth value of the training scene is recorded asD t (x,y) According to the parameters of the digital projector and the camera, the depth value of the scene to be trained can be obtainedD t (x,y) Corresponding training scene disparity valuesd t (x,y)：
Wherein (A), (B), (C), (D), (C), (x,y) Coordinates of each point pixel in a pixel coordinate system are used for training scene depth values and training scene parallax values. Then, a digital projector projects a pseudo-random speckle image onto the training sceneR(x,y) The camera acquires corresponding deformed speckleI t (x,y) And is provided withM×NSliding window of =32 × 32 pixels, traverse deformed speckleI t (x,y) Per dot pixel of (1) <x,y) Obtaining (A)x,y) Local binary characteristics of the points. 13000 training scenes are processed in the same way to obtain the parallax value of the training scenesd t (x,y) And local binary characteristics are used as learning model training data, and the learning model training data is input into the random forest for learning of the random forest.
In the learning process of the random forest, firstly, the input deformed speckles are utilizedI t (x,y) Local binary characterization of pixels per point and digital projector projected speckle imagesR(x,y) Calculating the deformed speckleI t (x,y) The parallax label corresponding to each pixelc t (x,y). When the digital projector and the camera are accurately calibrated, the parallax labelc t (x,y) Satisfies the following conditions:
wherein (A), (B), (C), (D), (C), (x',y') As parallax labelsc t (x,y) In speckle imagesR(x,y) The corresponding pixel coordinates. Training scene parallax value obtained by using 13000 training scenesd t (x,y) And corresponding parallax labelc t (x,y) Composing training set dataSAnd independently training a plurality of trees. In the present invention, each tree has in commonN=15 layers, frontkThe nodes of the layer =9 are integers, so that the classification problem is solved; rear endN-kAnd (4) =6 layers of nodes are decimal numbers, and prediction data of sub-pixel precision are obtained through a regression function.
The learning process of the random forest of the invention starts from the root node of each tree and randomly sets a series of random discrimination parametersδUsing each of the discrimination parametersδWill train set dataSDivided into left training set dataS L (δ) And right training set dataS R (δ) While simultaneously computing the objective functionO(δ)：
Wherein the content of the first and second substances,E(S)、E(S L (δ))、E(S R (δ) Respectively training set dataSLeft training set dataS L (δ) And right training set dataS R (δ) The entropy of information of (1). Random discrimination parameterδIntermediate order objective functionO(δ) The maximum value is the final discrimination parameter of the node. For left training set dataS L (δ) And right training set dataS R (δ) Recursion is carried out in sequence until the training depth reaches the second of the treekLayer, or training set data is not re-separable. Training independent trees into decision trees, the decision trees forming a random forest for runtime stateF。
In the classification regression process of the optimal depth of the deformed speckles, as shown in the attached figure 3, a digital projector projects pseudo-random digital speckles on the surface of a measured scene, and the pixel arrangement and local amplification structure of the pseudo-random digital speckles are shown in the attached figure 4. Camera for acquiring deformation speckles of detected sceneI'(x,y) Using random forestsFBefore decision treekLayer, classifying each point pixel local binary characteristic to obtain local binary characteristic labelc' I (x,y) And then the random deep forest is recycledFAfter decision treeN-kLayer, regression is carried out to local binary characteristic label, obtains surveyed scene deformation speckleI'(x,y) Corresponding sub-pixel precision local binary feature labelc'(x,y) Because the digital projector and the camera are accurately calibrated, the optimal depth value of the detected sceneD'(x,y) Satisfies the following conditions:
in the invention, speckle is deformed due to the detected sceneI'(x,y) The processing of each pixel has irrelevancy, so that in the running state of the method, the classification regression of the optimal depth of the detected scene deformation speckles is carried out, and the GPU is used for carrying out parallel processing to obtain higher processing speed.
Claims (1)
1. A high-precision three-dimensional information rapid acquisition method based on random forests is characterized by comprising three processes of acquisition of learning model training data, learning of random forests and classification regression of optimal depths of deformed speckles, wherein in the acquisition process of the learning model training data, firstly, the depth values of T training scenes are acquired by using a high-precision structured light measurement method, and the depth value of the T training scene is recorded as Dt(x, y), according to the parameters of the digital projector and the camera, the depth value D of the training scene can be obtainedt(x, y) corresponding training scene disparity values
Wherein, (x, y) is the coordinate of each pixel of the training scene depth value and the training scene parallax value in the pixel coordinate system, f is the focal length of the lens of the digital projector and the focal length of the lens of the camera, and b is the optical center interval of the lens of the digital projector and the camera; then, the digital projector projects a pseudo-random speckle image R (x, y) to the training scene, and the camera acquires the corresponding deformed speckle It(x, y) and setting a sliding window of M × N pixels to traverse the deformed speckle ItAcquiring local binary characteristics of the (x, y) point for each point pixel (x, y) on the (x, y); finally, the T training scenes are processed in the same way to obtain a training scene parallax value dt(x, y) and local binary characteristics are used as learning model training data, and the random forest is input for learning of the random forest; the learning process of the random forest is started from the root node of each tree, a series of random discrimination parameters delta are randomly set, and the training set data S is divided into left training set data S by utilizing each discrimination parameter deltaL(delta) and right training set data SR(δ) simultaneously calculating an objective function
Wherein E (S) and E (S)L(δ))、E(SR(δ)) training set data S and left training set data SL(delta) and right training set data SRThe information entropy of (delta) and the maximum value of the objective function O (delta) in the random discrimination parameters delta are the final discrimination parameters of the node; for left training set data SL(delta) and right training set data SR(delta) recursing the processes in sequence until the training depth reaches the kth layer of the tree or training set data is irreparable, training a plurality of independent trees into decision trees, and forming a random forest F for the running state by the decision trees; in the classification regression process of the optimal depth of the deformed speckles, a digital projector projects pseudo-random digital speckles on the surface of a detected scene, a camera obtains deformed speckles I '(x, y) of the detected scene, and each pixel local binary feature is classified by utilizing a front k layer of a random forest F decision tree to obtain a local binary feature label c'I(x, y), regressing the local binary characteristic label by utilizing an N-k layer behind a random forest F decision tree to obtain a sub-pixel precision local binary characteristic label c ' (x, y) corresponding to the detected scene deformation speckle I ' (x, y), wherein the optimal depth value D ' (x, y) of the detected scene meets the requirement of accurate calibration of a digital projector and a camera
The method comprises the following steps that f is the focal length of a lens of a digital projector and a lens of a camera, b is the optical center interval of the lens of the digital projector and the lens of the camera, and (x, y) is the coordinate of each pixel of a training scene depth value and a training scene parallax value in a pixel coordinate system.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201611037639.2A CN106981043B (en) | 2016-11-23 | 2016-11-23 | High-precision three-dimensional information rapid acquisition method based on random forest |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201611037639.2A CN106981043B (en) | 2016-11-23 | 2016-11-23 | High-precision three-dimensional information rapid acquisition method based on random forest |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN106981043A CN106981043A (en) | 2017-07-25 |
| CN106981043B true CN106981043B (en) | 2020-04-10 |
Family
ID=59340839
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201611037639.2A Active CN106981043B (en) | 2016-11-23 | 2016-11-23 | High-precision three-dimensional information rapid acquisition method based on random forest |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN106981043B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113674335B (en) * | 2021-08-19 | 2022-05-31 | 合肥的卢深视科技有限公司 | Depth imaging method, electronic device and storage medium |
Family Cites Families (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CA2855440A1 (en) * | 2013-06-28 | 2014-12-28 | Kelly J. Stegman | Tissue displacement estimation by ultrasound speckle tracking |
| CN103400391B (en) * | 2013-08-09 | 2016-08-10 | 北京博思廷科技有限公司 | A kind of multi-object tracking method based on the random forest improved and device |
| CN104598915B (en) * | 2014-01-24 | 2017-08-11 | 深圳奥比中光科技有限公司 | A kind of gesture identification method and device |
-
2016
- 2016-11-23 CN CN201611037639.2A patent/CN106981043B/en active Active
Also Published As
| Publication number | Publication date |
|---|---|
| CN106981043A (en) | 2017-07-25 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109685141B (en) | Robot article sorting visual detection method based on deep neural network | |
| CN109816703B (en) | Point cloud registration method based on camera calibration and ICP algorithm | |
| Fanello et al. | Ultrastereo: Efficient learning-based matching for active stereo systems | |
| CN104899563B (en) | Two-dimensional face key feature point positioning method and system | |
| US20180091798A1 (en) | System and Method for Generating a Depth Map Using Differential Patterns | |
| US20180089501A1 (en) | Computer implemented method of detecting the distance of an object from an image sensor | |
| CN111524233B (en) | Three-dimensional reconstruction method of static scene dynamic target | |
| CN102445165B (en) | Stereo vision measurement method based on single-frame color coding grating | |
| CN108615244A (en) | A kind of image depth estimation method and system based on CNN and depth filter | |
| CN109711472B (en) | Training data generation method and device | |
| CN102884397A (en) | Structured-light measuring method and system | |
| CN112001973B (en) | Quick three-dimensional human head measuring method based on digital speckle correlation | |
| CN112184793B (en) | Depth data processing method and device and readable storage medium | |
| TW201944291A (en) | Face recognition method | |
| CN104182968A (en) | Method for segmenting fuzzy moving targets by wide-baseline multi-array optical detection system | |
| CN111105451B (en) | Driving scene binocular depth estimation method for overcoming occlusion effect | |
| CN104634323B (en) | A kind of multistage shooting tri-dimensional photographing system and method | |
| CN112465778A (en) | Underwater fish shoal observation device and method | |
| Ophoff et al. | Improving real-time pedestrian detectors with RGB+ depth fusion | |
| CN113674335A (en) | Depth imaging method, electronic device, and storage medium | |
| CN106981043B (en) | High-precision three-dimensional information rapid acquisition method based on random forest | |
| CN111369611B (en) | Image pixel depth value optimization method, device, equipment and storage medium thereof | |
| KR101852085B1 (en) | Depth map acquisition device and depth map acquisition method | |
| CN108665448B (en) | Obstacle detection method based on binocular vision | |
| CN108090930A (en) | Barrier vision detection system and method based on binocular solid camera |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |