CN104504690B - A kind of object support relationship estimating method based on Kinect - Google Patents
A kind of object support relationship estimating method based on Kinect Download PDFInfo
- Publication number
- CN104504690B CN104504690B CN201410766797.6A CN201410766797A CN104504690B CN 104504690 B CN104504690 B CN 104504690B CN 201410766797 A CN201410766797 A CN 201410766797A CN 104504690 B CN104504690 B CN 104504690B
- Authority
- CN
- China
- Prior art keywords
- region
- scene
- formula
- image
- scs
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/50—Depth or shape recovery
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/12—Edge-based segmentation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/97—Determining parameters from multiple pictures
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10024—Color image
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10028—Range image; Depth image; 3D point clouds
Landscapes
- Engineering & Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Image Analysis (AREA)
Abstract
The invention discloses a kind of object support relationship estimating method based on Kinect, including：Image is split using the hierarchical Segmentation Algorithm based on RGB D images, scene structure Grade Model is built, the scene supporting relation model based on structural class, supporting relation constraint rule, seeks using linear programming scene supporting relation, stability is carried out to object in scene using scene supporting relation infers several steps.The present invention is effectively utilized the depth data of Kinect acquisitions, and the innovative supporting relation being extracted in scene is capable of the work of effective auxiliary robot vision and target identification.
Description
Technical field
The present invention relates to scene analysis field, specifically a kind of object support relationship estimating method based on Kinect.
Background technology
With the development of camera shooting collecting device, have been able to collect the image information comprising depth data at present.How
Depth information is made good use of, makes it in robot vision, the fields such as object identification play the effect of bigger at research hotspot.Closely
In the past few years, the scene analysis based on RGB-D data is largely studied, and wherein the analysis of scene structure is still short of, and
Supporting relation is obtained by scene structure and can be good at auxiliary robot vision, is differentiated the stability of object in scene, also can
Novel identification feature enough is provided for target identification, important component in scene understanding will be become in the future.
In current existing scene structure extraction method, bounding box, the scene for mainly extracting the object in scene are whole
The structural class information of structural information, important line information and different zones, the research of the contact between object in scene
Still it is weak.Current scene structure extraction method still has following shortcoming：
The extraction for laying particular emphasis on the overall structure information of scene is merely able to sketch the contours of simple scene entirety three-dimensional information, and
In the case where that can collect depth information at present, scene overall structure information cannot meet the need of scene structure extraction
It wants.
The extraction of object bounding box is merely able to the bounding box of extraction single body in scene, has ignored different objects between scene
Between contact, do so the contact that can not be embodied well between object.
The research of existing supporting relation extraction algorithm is less, in existing technology, in scene to the scene of core about
The extraction of beam relationship is not comprehensive enough, causes the accuracy of algorithm not high.
Invention content
The object of the present invention is to provide a kind of object support relationship estimating method based on Kinect, to solve the prior art
The deficiency of the defects of scene structure extraction and existing supporting relation extraction algorithm.
In order to achieve the above object, the technical solution adopted in the present invention is：
(1), the image that scene to be analyzed is shot using Kinect cameras, including the depth image and cromogram of scene
Picture；Difference operation is carried out to depth image, obtains the detailed depth data of scene；By stereoscopic vision algorithm by depth image and coloured silk
Color image is registrated, to generate the RGB-D images of scene；
(2), RGB-D pre-processing image datas include the following steps：
(2.1), the normal information of each pixel in depth image is extracted：Pixel normal information is using around pixel
The depth information in the regions 8*8 is chosen in the region and pixel gray value is not much different in all the points in 5 point, extraction field
It is fitted a plane, using the normal vector of the plane as pixel normal vector；
(2.2), scene coordinate system and world coordinate system are registrated：The depth number of its surrounding pixel is utilized to each pixel
According to seeking plane parameter by least square method；By the mean- for extracting straight line and gauging surface normal vector from image
Shift models choose a certain amount of principal direction candidate, the triple orthogonal main directions are calculated by formula (1)-formula (3)
Scoring：
In formula (1)-formula (3), v1,v2,v3It is three principal directions, for convenience, formula (1)-formula is represented in (3) with j
Three directions of v, i.e. j=1,2,3.N in formula (1)jRepresent the scoring of pixel on the directions j, LjRepresent the directions j up-sampling
The scoring of line, the two scorings are defined in formula (2) and formula (3).W in formula (2)nIt is sampled point normal vector vector weight
Parameter.NNRepresent the number at plane midpoint.σ represents regulating constant, is generally set to σ=0.01.I represents the volume of pixel in plane
Number.NiRepresent pixel normal line vector in plane, vjJ-th candidates direction vector is represented, the two dot product is a number.Formula (3)
In, wlIt is the weight parameter of rectilinear direction vector.NLRepresent the number of plane cathetus.σ and vjDefinition and formula (2) in phase
Together.I represents the number of plane cathetus, LiRepresent linear vector in plane.Pass through commenting for formula (1) every group of orthogonal line segment of calculating
Point, the reference axis as indoor scene coordinate system of highest scoring is selected, then indoor scene can be sat by spin matrix
Mark system and world coordinate system are registrated；
(3), the layering segmentation based on auxiliary information, includes the following steps：
(3.1), watershed algorithm carries out over-segmentation to image, gradient image is converted images into first, such as formula (4)-
(6) shown in：
G (i, y)=dx(i, y)+dy(i, y) (4)
Wherein l (i, j) represents coordinate as i, and the pixel value of the point of y passes through the picture to each pixel in the x and y direction
Plain value change rate obtains G (x, y), and generates gradient image, and over-segmentation image is generated by watershed algorithm；
(3.2), edge strength is predicted, after giving a region, obtains edge strength function, strong by minimizing edge
Degree can classify to different edges, and final segmentation image is finally obtained by the method for successive ignition；
(4), the scene supporting relation extraction based on constraint rule, includes the following steps：
(4.1), scene structure Grade Model defines：
The foundation of the image for being divided into R region for one, bolster model is as follows：One is assigned to each region to hide
Variable SCi, which region wherein i expressions belong to, as shown in formula (7)：
(4.2), scene supporting relation model defines：
The foundation of the image for being divided into R region for one, supporting relation model is as follows：
1) hidden variable SCS is usedi：I=1 ..., R indicates the supporting zone of a region i：(a) by visual in image
A certain region supports, SCSi∈ { 1 ..., R }；(b) by object support not visible in image, SCSi=h；(c) it need not prop up
Support, this also illustrates that the region is ground, SCSi=g；
2) SCT is usediIndicate the supporting type of region i：(a) by region SCiIt is supported from bottom, SCTi=0 (b) quilt
Region SCiIt is supported from behind, SCTi=1；
Pay attention to：It is as defined above middle SCiRepresent the type of target area i, SCSiRepresent the type in the regions support i, SCTiGeneration
Table is by the type of i region supports.
(4.3), scene structure grade and supporting type are classified：
The SIFT feature for choosing the region of each scene generates the profiler in the region, uses logistic regression later
Grader is classified, as shown in formula (8) and formula (9)：
α in formulaiFor the weights distribution that maximal possibility estimation is sought, SiFor the SIFT feature describer of sample data, pass through instruction
Practice sample data and the method using logistic regression classifier, obtains the structural class and supporting type in each region in scene
Classification；
(4.4), constraint rule is supported：For correctly seeking to supporting relation, it is proposed that following several constraint rules,
1) structural class bound term ψSCS, indicate the constraints in the region of region i and supporting zone i, wherein SCiAnd SCSi
Step (4.2) is shown in definition.Its formula (10) is as follows：
WhereinWithThe minimum point of region i and the height coordinate of peak are represented,WithRepresent Support
The minimum point and peak in domain,The minimum point of region j is represented, wherein region j is arbitrary region.
2) adjacent bound term ψSCT, it is ensured that it is adjacent to be supported region and supporting zone, and formula (11) is as follows：
WithRespectively represent the minimum point and supporting zone SCS in the regions iiPeak, D2(i, SCSi) represent i
Region and supporting zone SCSiMinimum horizontal distance；
3) bound term is supported, it is ensured that other regions at least have a support in addition to ground in scene：Its formula (12) is such as
Under：
Wherein every definition is see in step (4.4) 1) part, in addition to ground does not have in constraint specific definition scene
Support is outer, other regions in scene are required to a support, and in scene coordinate system, higher than ground；
(4.5), energy function is established：
Two initial terms of energy function can be obtained by step (4.3)：Regional structure grade and supporting type grader, it is complete
Whole energy function is as follows：
E (SC, SCT, SCS)=ESC(SC, SCT)+ESCS(SCS)+EPS(SC, SCT, SCS) (13)
Wherein, R is the total quantity in region,For region i and SCSiSupport feature, DspFor the support described in step 4.3
Type sorter,For the structure feature of region i, DscFor the structural class grader in step 4.3, EpsTo be propped up in step 4.4
Support constraint rule；
(4.6), linear restriction solves：
The solution of least energy function can be solved with integer programming, be firstly introduced into boolean's indicator variable sr
Indicate supporting zone and supporting type, sc indicates structural class SC, and the sum in region is that the sum of visibility region adds in scene
One zone of ignorance, uses R*=R+1 indicates the sum in region in scene；
Enable Boolean variable sri,j:1≤i≤N,1≤j≤2N*+ 1 indicates interregional supporting relation --- while indicating
SCiAnd SCTi, as 1≤j≤R*, indicate that region i is supported by region j from bottom, i.e. SCTi=0；Work as R*+1≤j≤2R*,
sri,j=1 expression region i is supported by region j from back, i.e. SCTi=1,Expression region i is ground (SCi=
1)；
Enable variable sci,λThe structural class of region i is indicated, if sci,λ=1 indicates that the structural class of region i is λ；
Indicator variableIndicate sri,j=1, sci,λ=1, scj,γ=1；
Utilize above-mentioned complete boolean's representation method, you can by the minimum energy function problem of formula (13), be converted into as
The integer programming problem of lower formula：Approximately convert the solution of energy function to a linear restriction equation：
WhereinIt is the integer coding coefficient of respective items respectively, in order to make each single item become integer.
Constraints：
By solving the linear restriction equation, the SC in each region is obtainediParameter is found out according to step (4.2) analysis
The supporting zone and supporting type in each region.
The present invention has more excellent image Segmentation Technology, while establishing more perfect constraints mechanism, carries
The high extraction accuracy of existing supporting relation algorithm.
Compared with the prior art, the present invention has the beneficial effect that：
1, effectively depth data is utilized in the present invention, and the registration by depth data coordinate system and world coordinate system is effective
The terrestrial information extracted in scene, the plane normal vector of each pixel is extracted under the auxiliary of depth data, is successfully carried
Plane information in scene is taken.
2, the present invention establishes a kind of effective scene structure Grade Model, which can effectively describe in scene not
With the structural class of object, which effectively summarizes the structural class characteristic of different objects in indoor scene.
3, the present invention on the basis of scene structural class model, close by the scene support that further establishing can solve
It is model, facilitates solution.
4, the present invention solves supporting relation by the method for minimizing energy function, and is 3 by energy function formalization
Component part so that former problem can solve.
5, the present invention establishes the tied mechanism between more perfect object scene, the mechanism energy on the basis of original method
Enough effective raising supporting relation accuracys rate.
6, present invention employs the methods of linear programming, are a linear programming equation by energy function formalization, pass through
The method of approximate solution can quickly solve the supporting relation in scene between different objects.
Specific implementation mode
A kind of object support relationship estimating method based on Kinect, includes the following steps：
Step 1, the image that scene to be analyzed is shot using Kinect cameras can collect scene by Kinect cameras
Depth image, in order to obtain more accurate depth image, we to depth image carry out difference operation.It is provided according to Kinect
Camera parameter depth image and coloured image are registrated, obtain the complete RGB-D images of scene.Preferably training data
Collection, the secondary left and right RGB-D images of a general scene capture 20.
Step 2, RGB-D pre-processing image datas：
Step 2.1 extracts the normal information of each pixel in depth image.It is selected in the region of 8*8 around each pixel
Depth value and point of this depth value difference within 5 are taken, a plane is fitted, using the plane normal vector after fitting as this
The normal vector of pixel.It can obtain the normal information of all pixels point.
Step 2.2 utilizes each pixel the depth data of its surrounding pixel, and plane ginseng is sought by least square method
Number.Method for registering is assumed based on the Manhattan world：Most of visible planes are orthogonal to one of three principal direction.By from figure
The mean-shift models that straight line and gauging surface normal vector are extracted as in, it is candidate to choose a certain amount of principal direction.For each
With candidate direction similar in Y direction, we sample the orthogonal to that candidate direction of any two, are calculated by following formula
The scoring of the triple orthogonal main directions.
V1 in formula, v2, v3 are as principal direction, and N is the plane parameter of pixel, and L is the principal direction of the line of sampling, NN
It is the number of Points And lines in plane respectively with NL, wn and wl are the weight parameters of normal vector and rectilinear direction vector.
As above scored all orthogonal normal vectors in scene, select highest scoring as indoor scene coordinate system
Then indoor scene coordinate system and world coordinate system can be registrated by reference axis by spin matrix.It selects after co-registration
The minimum plane of average coordinates in world coordinate system is selected as main supporting plane.
Layering segmentation of the step 3 based on auxiliary information
Step 3.1 watershed algorithm carries out over-segmentation to image, gradient image is converted images into first, such as formula institute
Show：
G (x, y)=dxi+dyj (4)
Wherein l (i, j) represents coordinate as i, and the pixel value of the point of j passes through the picture to each pixel in the x and y direction
Plain value change rate obtains G (x, y), and generates gradient image, and over-segmentation image is generated by watershed algorithm；
Step 3.2 edge strength is predicted, is given a region and is obtained edge strength function, can by minimizing edge strength
To classify to different edges.Final segmentation image is finally obtained by the method for successive ignition.
The area just outside a city gate system of scene of the step 4 based on constraint rule extracts
Step 4.1 scene structure Grade Model defines：
The image for being divided into R region for one supports the foundation of bolster model as follows：One is assigned to each region
Hidden variable SCi, wherein i expression which region belonged to.
Step 4.2 scene supporting relation model defines：
The foundation of the image for being divided into R region for one, supporting relation model is as follows：1) hidden variable is used
Si：I=1 ..., R indicates the supporting zone of a region i：(a) by a certain region supports visual in image, Si ∈ 1 ...,
R}；(b) by object support not visible in image, Si=h；(c) it need not support, this also illustrates that the region is ground, Si=
g.2) STi is used to indicate the supporting type of region i：(a) be supported from bottom by region Si, STi=0 (b) by region Si from
It is supported behind, STi=1.
Step 4.3 scene structure grade is classified with supporting type：
Different zones in collected more secondary RGB-D images are demarcated, the SIFT for choosing the region of each scene is special
Sign, generates the profiler in the region, the logistic regression classifier of the different zones in the scene is calculated according to calibration result,
As shown by the equation.Image-region to be detected is predicted, the structural class prediction result in each region is obtained.
P=α0+α1S1+α2S2+…+αnSn (9)
α in formulaiFor the weights distribution that maximal possibility estimation is sought, SiFor the SIFT feature describer of sample data.Pass through instruction
Practice sample data and seek logistic regression classifier, each region in RGB-D scenes to be extracted is obtained using the grader
Structural class is classified with supporting type prediction.
Step 4.4 supports constraint rule：For correctly seeking to supporting relation, it is proposed that following several constraint rule
Then.
1) structural class bound term ψscs：Constraint of the structural class in region to supporting zone.By analyzing in indoor scene
Supporting relation and structural class find that the structural class in region has its supporting zone important influence, have not
Negligible constraint.Region for belonging to furniture (SCi=2) and structure (SCi=4) in indoor scene is merely able to by ground
It is supported, supporting zone is all lower than any region.And ground (SCi=1) no any support, in entire scene coordinate system
In, it is in extreme lower position.For wisp (SCi=3), since it can arbitrarily be moved, the range of choice of its supporting zone
It is very wide, but the spatial altitude that must satisfy it has to be lower than supporting zone.Its formula is as follows：
WhereinWithThe minimum point of the peak and region j of region i is indicated respectively.
2) adjacent bound term ψSST：Supporting zone constraint adjacent with region is supported.In normal indoor scene, support
Region and it is supported region under force, it is typically adjacent.Its formula is as follows：
WithThe peak of the minimum point and the regions si in the regions i is respectively represented, D (i, Si) represents the regions i and the regions Si
Minimum horizontal distance.
3) bound term is supported：Other than ground does not support, other regions in scene are required to support, and are sat in scene
It is higher than ground in mark system.Its formula is as follows：
In scene other than ground does not support, other regions in scene are required to a support, and in scene coordinate
It is higher than ground in system.Step 4.5 energy function is established.By step 4.3 we can obtain two of energy function it is initial
：Regional structure grade and supporting type grader.Complete energy function is as follows：
E (S, ST, SC)=Es(S,ST)+ESC(SC)+EPS(S,ST,SC) (13)
Wherein,For the support feature of region i and Si, DspFor the supporting type grader described in step 4.3.For area
The structure feature of domain i, DscFor the structural class grader in step 4.3.EpsTo support constraint rule in step 4.4.
Step 4.6 linear restriction solves
Approximately convert the solution of energy function to a linear restriction equation：
Constraints：
By solving the linear restriction equation, the Si parameters in each region are obtained, are found out according to step 4.2 analysis each
The supporting zone and supporting type in region.Boolean variable sri,j:1≤i≤R,1≤j≤2R*+ 1 indicates that interregional support is closed
System, while indicating Si and STi.As 1≤j≤R*, sri,j=1 expression region i is supported (STi=0) by region j from bottom；
Work as R*≤j≤2R*, sri,j=1 expression region i is supported (STi=1) by region j from back.Indicate region i
For ground (Si=g).With variable sci,λThe structural class of region i is indicated, if sci,λ=1 indicates that the structural class of region i is
λ.Indicator variableIndicate sri,j=1, sci,λ=1, scj,γ=1.By solving the linear restriction equation, each area is obtained
The Si parameters in domain find out the supporting zone and supporting type in each region according to step 4.2 analysis.
Embodiment：
In order to verify the effect of algorithm in the present invention, the present invention is to No. 1001-1200 in NYU-Depth2 databases
RGB-D images are tested.It is extracted the depth information and colour information of image, while utilizing the training point of existing calibration result
Class device finally uses the correct supporting zone for evaluating each region for evaluation condition, and seeks algorithm with existing supporting relation
It compares.
Since when judging supporting relation correctness, correctly whether there is also strive for the structural class classification of supporting zone
View, therefore all shown here by the accuracy of band structure grade and without the accuracy structure of structural class, concrete outcome
As shown in table 1,2.
As can be seen from the above table, method proposed by the present invention being capable of not jljl in more accurate and effective extraction scene
The supporting relation of body.
Claims (1)
1. a kind of object support relationship estimating method based on Kinect, it is characterised in that：Include the following steps：
(1), the image that scene to be analyzed is shot using Kinect cameras, including the depth image and coloured image of scene；
Difference operation is carried out to depth image, obtains the detailed depth data of scene；By stereoscopic vision algorithm by depth image and colour
Image is registrated, to generate the RGB-D images of scene；
(2), RGB-D pre-processing image datas include the following steps：
(2.1), the normal information of each pixel in depth image is extracted：Pixel normal information utilizes 8*8 around pixel
The depth information in region, in the region, selection and pixel gray value are not much different is fitted in all the points in 5 point, extraction field
One plane, using the normal vector of the plane as pixel normal vector；
(2.2), scene coordinate system and world coordinate system are registrated：The depth data of its surrounding pixel is utilized to each pixel,
Plane parameter is sought by least square method；By the mean-shift moulds for extracting straight line and gauging surface normal vector from image
Type chooses a certain amount of principal direction candidate, the scoring of the triple orthogonal main directions is calculated by formula (1)-formula (3)：
In formula (1)-formula (3), v1,v2,v3It is three principal directions, for convenience, represents v's with j in formula (1)-formula (3)
Three directions, i.e. j=1,2,3, the N in formula (1)jRepresent the scoring of pixel on the directions j, LjRepresent the directions j up-sampling line
Scoring, the two scorings are defined in formula (2) and formula (3), w in formula (2)nIt is sampled point normal vector vector weight ginseng
Number, NNThe number at plane midpoint is represented, σ represents regulating constant, is generally set to σ=0.01, and i represents the volume of pixel in plane
Number, NiRepresent pixel normal line vector in plane, vjJ-th candidates direction vector is represented, the two dot product is a number, formula (3)
In, wlIt is the weight parameter of rectilinear direction vector, NLRepresent the number of plane cathetus, σ and vjDefinition and formula (2) in phase
Together, i represents the number of plane cathetus, LiLinear vector in plane is represented, commenting for formula (1) every group of orthogonal line segment of calculating is passed through
Point, the reference axis as indoor scene coordinate system of highest scoring is selected, then indoor scene can be sat by spin matrix
Mark system and world coordinate system are registrated；
(3), the layering segmentation based on auxiliary information, includes the following steps：
(3.1), watershed algorithm carries out over-segmentation to image, gradient image is converted images into first, such as formula (4)-(6)
It is shown：
G (x, y)=dx(i, j)+dy(i, j) (4)
Wherein l (i, j) represents coordinate as i, and the pixel value of the point of j passes through the pixel value to each pixel in the x and y direction
Change rate obtains G (x, y), and generates gradient image, and over-segmentation image is generated by watershed algorithm；
(3.2), edge strength is predicted, after giving a region, obtains edge strength function, can by minimizing edge strength
To classify to different edges, final segmentation image is finally obtained by the method for successive ignition；
(4), the scene supporting relation extraction based on constraint rule, includes the following steps：
(4.1), scene structure Grade Model defines：
The foundation of the image for being divided into R region for one, bolster model is as follows：A hidden variable is assigned to each region
SCi, which region wherein i expressions belong to, as shown in formula (7)：
(4.2), scene supporting relation model defines：
The foundation of the image for being divided into R region for one, supporting relation model is as follows：
1) hidden variable SCS is usedi：I=1 ..., R indicates the supporting zone of a region i：(a) by visual a certain in image
Region supports, SCSi∈ { 1 ..., R }；(b) by object support not visible in image, SCSi=h；(c) it need not support, this
Also illustrate that the region is ground, SCSi=g；
2) SCT is usediIndicate the supporting type of region i：(a) by region SCiIt is supported from bottom, SCTi=0 (b) is by region
SCiIt is supported from behind, SCTi=1；
Pay attention to：It is as defined above middle SCiRepresent the type of target area i, SCSiRepresent the type in the regions support i, SCTiRepresent quilt
The type of i region supports,
(4.3), scene structure grade and supporting type are classified：
The SIFT feature for choosing the region of each scene is generated the profiler in the region, is classified later using logistic regression
Device is classified, as shown in formula (8) and formula (9)：
α in formulaiFor the weights distribution that maximal possibility estimation is sought, SiFor the SIFT feature describer of sample data, pass through training sample
Notebook data and the method for using logistic regression classifier obtain the structural class in each region and supporting type point in scene
Class；
(4.4), constraint rule is supported：For correctly seeking to supporting relation, it is proposed that following several constraint rules,
1) structural class bound term ψSCS, indicate the constraints in the region of region i and supporting zone i, wherein SCiAnd SCSiDefinition
See that step (4.2), formula (10) are as follows：
WhereinWithThe minimum point of region i and the height coordinate of peak are represented,WithRepresent supporting zone most
Low spot and peak,The minimum point of region j is represented, wherein region j is arbitrary region,
2) adjacent bound term ψSCT, it is ensured that it is adjacent to be supported region and supporting zone, and formula (11) is as follows：
WithRespectively represent the minimum point and supporting zone SCS in the regions iiPeak, D2(i, SCSi) represent the regions i
With supporting zone SCSiMinimum horizontal distance；
3) bound term is supported, it is ensured that other regions at least have a support in addition to ground in scene：Its formula (12) is as follows：
Wherein every definition is see in step (4.4) 1) part, in addition to ground does not support in constraint specific definition scene
Outside, other regions in scene are required to a support, and in scene coordinate system, higher than ground；
(4.5), energy function is established：
Two initial terms of energy function can be obtained by step (4.3)：Regional structure grade and supporting type grader, complete energy
Flow function is as follows：
E (SC, SCT, SCS)=ESC(SC, SCT)+ESCS(SCS)+EPS(SC, SCT, SCS) (13)
Wherein, R is the total quantity in region,For region i and SCSiSupport feature, DspFor the supporting type described in step 4.3
Grader,For the structure feature of region i, DscFor the structural class grader in step 4.3, EpsTo be supported about in step 4.4
Beam rule；
(4.6), linear restriction solves：
The solution of least energy function can be solved with integer programming, boolean's indicator variable sr is firstly introduced into and indicates
Supporting zone and supporting type, sc indicate structural class SC, and the sum in region is that the sum of visibility region adds one in scene
Zone of ignorance uses R*=R+1 indicates the sum in region in scene；
Enable Boolean variable sri,j:1≤i≤N,1≤j≤2N*+ 1 indicates interregional supporting relation --- while indicating SCiWith
SCTi, as 1≤j≤R*, indicate that region i is supported by region j from bottom, i.e. SCTi=0；Work as R*+1≤j≤2R*, sri,j=1
Indicate that region i is supported by region j from back, i.e. SCTi=1,Expression region i is ground (SCi=1)；
Enable variable sci,λThe structural class of region i is indicated, if sci,λ=1 indicates that the structural class of region i is λ；
Indicator variableIndicate sri,j=1, sci,λ=1, scj,γ=1；
Utilize above-mentioned complete boolean's representation method, you can by the minimum energy function problem of formula (13), be converted into following public affairs
The integer programming problem of formula：Approximately convert the solution of energy function to a linear restriction equation：
WhereinIt is the integer coding coefficient of respective items respectively, in order to make each single item become integer, constrains item
Part：
By solving the linear restriction equation, the SC in each region is obtainediParameter finds out each area according to step (4.2) analysis
The supporting zone and supporting type in domain.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201410766797.6A CN104504690B (en) | 2014-12-12 | 2014-12-12 | A kind of object support relationship estimating method based on Kinect |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201410766797.6A CN104504690B (en) | 2014-12-12 | 2014-12-12 | A kind of object support relationship estimating method based on Kinect |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN104504690A CN104504690A (en) | 2015-04-08 |
| CN104504690B true CN104504690B (en) | 2018-07-27 |
Family
ID=52946084
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201410766797.6A Active CN104504690B (en) | 2014-12-12 | 2014-12-12 | A kind of object support relationship estimating method based on Kinect |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN104504690B (en) |
Families Citing this family (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110889397B (en) * | 2018-12-28 | 2023-06-20 | 南京大学 | Visual relation segmentation method based on human body |
| CN113240723A (en) * | 2021-05-18 | 2021-08-10 | 中德（珠海）人工智能研究院有限公司 | Monocular depth estimation method and device and depth evaluation equipment |
Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102682477A (en) * | 2012-05-16 | 2012-09-19 | 南京邮电大学 | Regular scene three-dimensional information extracting method based on structure prior |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| FR2962834A1 (en) * | 2010-07-13 | 2012-01-20 | Commissariat Energie Atomique | METHOD OF DETECTING A TARGET IN STEREOSCOPIC IMAGES BY LEARNING AND STATISTICAL CLASSIFICATION FROM A LAW OF PROBABILITY. |
-
2014
- 2014-12-12 CN CN201410766797.6A patent/CN104504690B/en active Active
Patent Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102682477A (en) * | 2012-05-16 | 2012-09-19 | 南京邮电大学 | Regular scene three-dimensional information extracting method based on structure prior |
Also Published As
| Publication number | Publication date |
|---|---|
| CN104504690A (en) | 2015-04-08 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN107871119B (en) | Target detection method based on target space knowledge and two-stage prediction learning | |
| CN110175576A (en) | A kind of driving vehicle visible detection method of combination laser point cloud data | |
| Ling et al. | Land cover change mapping at the subpixel scale with different spatial-resolution remotely sensed imagery | |
| CN105869178B (en) | A kind of complex target dynamic scene non-formaldehyde finishing method based on the convex optimization of Multiscale combination feature | |
| CN105069746B (en) | Video real-time face replacement method and its system based on local affine invariant and color transfer technology | |
| CN104715238B (en) | A kind of pedestrian detection method based on multi-feature fusion | |
| Ge et al. | Enhanced subpixel mapping with spatial distribution patterns of geographical objects | |
| CN106056155B (en) | Superpixel segmentation method based on boundary information fusion | |
| Recky et al. | Windows detection using k-means in cie-lab color space | |
| CN104850850B (en) | A kind of binocular stereo vision image characteristic extracting method of combination shape and color | |
| CN105225226B (en) | A kind of cascade deformable part model object detection method based on image segmentation | |
| CN109711288A (en) | Remote sensing ship detecting method based on feature pyramid and distance restraint FCN | |
| CN109359684A (en) | Fine granularity model recognizing method based on Weakly supervised positioning and subclass similarity measurement | |
| CN108009509A (en) | Vehicle target detection method | |
| CN108875911A (en) | One kind is parked position detecting method | |
| CN110119728A (en) | Remote sensing images cloud detection method of optic based on Multiscale Fusion semantic segmentation network | |
| CN104134200B (en) | Mobile scene image splicing method based on improved weighted fusion | |
| CN104050481B (en) | Multi-template infrared image real-time pedestrian detection method combining contour feature and gray level | |
| CN106096542B (en) | Image video scene recognition method based on distance prediction information | |
| CN108647602B (en) | A kind of aerial remote sensing images scene classification method determined based on image complexity | |
| CN102509104B (en) | Confidence map-based method for distinguishing and detecting virtual object of augmented reality scene | |
| CN105488809A (en) | Indoor scene meaning segmentation method based on RGBD descriptor | |
| CN105427309A (en) | Multiscale hierarchical processing method for extracting object-oriented high-spatial resolution remote sensing information | |
| CN112396619B (en) | Small particle segmentation method based on semantic segmentation and internally complex composition | |
| CN106557740B (en) | The recognition methods of oil depot target in a kind of remote sensing images |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |