CN102436657A - Active light depth measurement value modifying method based on application of the internet of things - Google Patents

Abstract

The invention discloses an active light depth measurement value modifying method based on an application of an internet of things. Two light depth map measurement value modifying methods are discloses. In the method, most of recovered depth map can be regarded as a value in a Zc coordinate direction of a camera coordinate system XcYcZc by analyzing an active light depth map imaging principle; generally speaking, the difference between different image coordinates is small, and the non-uniformity is low; for simplifying the storage, the significant modification can be obtained by storing only one group of modification data. By using a correction method of a depth value, a comparatively accurate depth measurement value can be obtained, and the method plays a great role in reducing the error in the sequential evaluation.

Description

Active optical depth measured value modification method based on the Internet of Things application

Technical field

The present invention relates to a kind of active optical depth measured value modification method of using based on Internet of Things.

Background technology

Modern intelligent video monitoring system (Intelligence video surveillance) all adopts the method for computer vision to come the information in the video is analyzed; Extract the various various actions main bodys in the video, and the behavior of behavioral agent is analyzed.But present intelligent video monitoring system all also is faced with various problems; Stable detection such as the partial occlusion target; Shadow removal; The detection of the not obvious target of contrast; And to accurately cutting apart of foreground target etc. all be the difficult point problem in the current intelligent video monitoring, these problems why difficulty that all be because can't obtain the mutual spatial relationship between the object in the scene, make the performance of intelligent video monitoring system be difficult to the performance that reaches higher to the disappearance of each pixel depth information.

Depth information in the video monitoring scene is vital information, shines upon toward the two dimensional image coordinate system but the common video monitor procedure all is the three-dimensional world coordinate system, in obtaining the process of video, has all lost range information.Because human eye and brain have extremely strong self calibrating function; And has a considerable extra priori; Thereby also can be good at understanding the content in the image for the video that does not have direct depth information, also can distinguish the relative position relation of various piece in the image.But it is impossible adopting manpower to come video content analysis fully always, the epoch that particularly the monitoring camera number is numerous and human cost is increasingly high now.

In other intelligence system fields, all need the environmental information of system around can perception such as fields such as robot application, particularly to the accurate perception of range information between object in the surrounding environment and the system.Thereby for various intelligence systems, the depth information of perception surrounding environment all is the key that can system give full play to should have effect.Owing in process for imaging, all lost depth information, the depth information that wants to regain surrounding environment is a very process of trouble.The current method of obtaining picture depth information mainly combines the mode of certain priori or used for multi-vision visual to obtain through camera calibration simultaneously; But these methods all need more prior imformation; And the calibration process trouble, also need demarcate in the mode installation process of used for multi-vision visual recovery depth information camera.

Primesense company is the global leader of 3D field of machine vision, and its depth image based on the structure light coding technology can directly obtain the picture depth on each pixel orientation, and it is extremely convenient to use.But the ranging information of primesense has certain structure property error, and this has caused in to scene object accurately to locate and survey in the high subsequent applications at three-dimensional real world having error.Thereby through makeover process, the depth information that collects is at first revised, and then carried out follow-up processing, can guarantee the various precision in the computation process.

In order to obtain the update information of degree of depth camera accurately; Need carry out accurately mapping, modeling to the relation between actual distance and measuring distance and the orientation; After the mapping relations of having set up between actual distance and measuring distance and the orientation; Just can revise, and then these revised depth informations are used for subsequent calculations the measured real-time depth information.The degree of depth correction algorithm that this paper designed is applicable to various degree of depth cameras, particularly PS company designed based on structured light degree of depth camera series.

Summary of the invention

The objective of the invention is to overcome deficiency of the prior art, the active optical depth measured value modification method of using based on Internet of Things is provided.

Based on the active optical depth measured value modification method that Internet of Things is used, concrete grammar is following:

Definition one, depth map; Depth map is to be exported by depth camera, and forgives the image of target range camera range information on each orientation.Be its at coordinate u, the image value f at v place (u v) representes u, on the v sight line apart from the distance value of camera closest approach.

Definition two, active optical depth figure; Having obtained of depth image is multiple, and a kind of is to utilize binocular or used for multi-vision visual to obtain the image that parallax obtains, and a kind of in addition is the method for utilizing active light to throw light on and find range then.The mentioned active ligh-ranging method of the present invention comprises the depth distance image that adopts following method to obtain: interferometric method, structure light coding method, time-of-flight method etc.

Definition three, index plane; A large-area plane is such as wall;

Definition four, linear interpolation; A kind of mathematics estimation method, the functional value of two points carries out Linear Estimation to the functional value of intermediate point about utilization, $f\left(x_2\right)=\frac{x_2-x_0}{x_1-x_0}f\left(x_1\right)+\frac{x_1-x_2}{x_1-x_0}f\left(x_0\right)\mathrm{For}{x}_0<x_2<x_1.$

Definition five, look-up table, English lookup table, a kind of data storage method becomes an array or similar data structure conveniently to calculate calculative data storage.

Definition six, arc cosine method; Be used for after the angle of knowing adjacent side and adjacent side and hypotenuse, calculating the triangle hypotenuse; Promptly

wherein a be adjacent side length; C is a hypotenuse length, and θ is the angle of adjacent side and hypotenuse.

Based on the active optical depth measured value modification method that Internet of Things is used, concrete steps are following:

(1) at first degree of depth camera is fixedly mounted on the horizontal slide rail;

(2) with the some index planes of camera perpendicular alignmnet, the typical case is like wall;

(3) camera is moved along slide rail, and note camera apart from the actual range of metope as nominal range, and the target depth image value that simultaneously camera is measured is as measuring distance;

(4) combine the field size information calculations of camera to go out the accurate distance of camera and metope on each pixel orientation through the arc cosine method;

(5) set up look-up table table, size is M * N * 2 * Mark, and M * N is the image size, and each pixel is set up look-up table in the relation that measures between distance and the actual distance on the different distance, and so far preliminary work finishes;

(6) obtain the depth map of a width of cloth through degree of depth camera collection;

(7) read pixel orientation u, the last measuring distance of v is as the x in the linear interpolation ₂, and on relevant this pixel orientation among the look-up table table, promptly look-up table tabel (u, v ...) and in x ₂Two immediate measured values are as x ₀x ₁, and read measuring value x ₀x ₁Nominal range as f (x ₀) and f (x ₁);

(8) utilize two nominal range f (x ₀) and f (x ₁) depth value to current x ₂Measure distance and carry out the estimation f (x that linear interpolation obtains current accurate depth value ₂);

(9) travel through all pixels, each pixel is repeated (7) and (8) to obtain complete Corrected Depth image;

This LUT Method that each pixel is all set up a plurality of different nominal ranges and measuring distance can be replied very accurately; And based on the method that complete measurement information is revised again need set up all pixels at the accurate look-up table corresponding relation between the degree of depth and the both sides degree of depth on each nominal depth; This theoretical method is clear; Method is simple, and the recovery precision of picture depth is higher.But a plurality of positions of each pixel of needs storage, need relatively large storage space;

Through the analysis to active optical depth figure image-forming principle, most of depth map that recovers can be regarded camera coordinate system X as _cY _cZ _cZ _cThe value of coordinate direction, and in general the difference between the different images coordinate is less, and heterogeneity is lower, thereby in order to simplify storage, can only store the correction that one group of correction data can obtain quite big degree.

Below the other a kind of method of our brief account:

A kind of in addition modification method is based on the revised law of model; We find that through after a large amount of measurements the distance that degree of depth camera provides is the Z axle measuring value under the camera coordinate system, but the measuring value of Z axle has discrepancy with real Z-direction coordinate figure really; Need revise; The same a kind of method of method of revising has similar place, such as all being to select an index plane, with camera and alignment criteria face; And then measure nominal value and video camera distance apart from metope; Because degree of depth camera is that the Z axle is carried out to picture, thus can utilize measured value on the more index plane on average obtain the corresponding relation between actual measured value and the nominal value, set up the nominal value of one dimension and the look-up table of measured value;

Trimming process is following:

(1): at first degree of depth camera is fixedly mounted on the horizontal slide rail;

(2): with the some index planes of camera perpendicular alignmnet, the typical case is like wall;

(3): camera is moved along slide rail, and note camera apart from the actual range of metope as nominal range, and the image value that simultaneously camera is measured is on average as measuring distance;

(4): set up one group of relevant average measurement Z _cValue and nominal Z _cThe look-up table table of value (being the distance of video camera) apart from metope _AverageSize is 2 * Mark, and what Mark represented is the number of nominal value.

(5): obtain a width of cloth depth map depth, big or small M * N;

(6): the angular distance of calculating current pixel point distance center pixel; The calculating of angular distance need be known the visual field size and the sum of all pixels of video camera;

${\mathrm{\&theta;}}_u=(u-\frac{M}{2})\&times;\left(\frac{{\mathrm{fov}}_u}{M}\right),$ ${\mathrm{\&theta;}}_v=(v-\frac{N}{2})\&times;\left(\frac{{\mathrm{fov}}_v}{N}\right)$

In the following formula: fov _uWith fov _vExpression is the visual field size of the both direction of video camera respectively, and M and N represent video camera u respectively, the number of pixels on the v direction;

(7): data calibration, read measuring distance on each pixel orientation as the x in the linear interpolation through linear interpolation to the depth image value of input ₂, and to look-up table table _AverageIn with x ₂Two immediate measured values are as x ₀x ₁, and read measuring value x ₀x ₁Nominal range as f (x ₀) and f (x ₁); Carrying out interpolation correction through linear interpolation again, to obtain measured value be x ₂Corrected value f (x ₂), proofread and correct back image depth as the first step _{Modified_table}(u, value v);

(8): angularity correction, the depth value behind the 8th step process look-up table data calibration is carried out the correction of angle information: depth according to the pixel index value again _Modified(u, v)=depth _{Modified_table}(u, v)/(cos (θ _u) * cos (θ _v));

(9): travel through all pixels and obtain through the image depth behind the data calibration _Modified

Beneficial effect:

The invention provides two kinds degree of depth camera carried out the depth value method of correcting, through the bearing calibration of depth value, we can obtain comparatively accurate depth measurement, have so also brought into play greatly and act on for we reduce error in the subsequent calculations.

Embodiment

Be easy to understand and understand in order to make technological means of the present invention, creation characteristic, workflow, method of application reach purpose and effect,, further set forth the present invention below in conjunction with specific embodiment.

Based on the active optical depth measured value modification method that Internet of Things is used, concrete grammar is following:

Embodiment 1:

Definition one, depth map; Depth map is to be exported by depth camera, and forgives the image of target range camera range information on each orientation.Be its at coordinate u, the image value f at v place (u v) representes u, on the v sight line apart from the distance value of camera closest approach.

Definition two, active optical depth figure; Having obtained of depth image is multiple, and a kind of is to utilize binocular or used for multi-vision visual to obtain the image that parallax obtains, and a kind of in addition is the method for utilizing active light to throw light on and find range then.The mentioned active ligh-ranging method of the present invention comprises the depth distance image that adopts following method to obtain: interferometric method, structure light coding method, time-of-flight method etc.

Definition three, index plane; A large-area plane is such as wall;

Definition four, linear interpolation; A kind of mathematics estimation method, the functional value of two points carries out Linear Estimation to the functional value of intermediate point about utilization, $f\left(x_2\right)=\frac{x_2-x_0}{x_1-x_0}f\left(x_1\right)+\frac{x_1-x_2}{x_1-x_0}f\left(x_0\right)\mathrm{For}{x}_0<x_2<x_1.$

Definition five, look-up table, English lookup table, a kind of data storage method becomes an array or similar data structure conveniently to calculate calculative data storage.

Definition six, arc cosine method; Be used for after the angle of knowing adjacent side and adjacent side and hypotenuse, calculating the triangle hypotenuse; Promptly wherein a be adjacent side length; C is a hypotenuse length, and θ is the angle of adjacent side and hypotenuse.

Based on the active optical depth measured value modification method that Internet of Things is used, concrete steps are following:

(1) at first degree of depth camera is fixedly mounted on the horizontal slide rail;

(2) with the some index planes of camera perpendicular alignmnet, the typical case is like wall;

(3) camera is moved along slide rail, and note camera apart from the actual range of metope as nominal range, and the target depth image value that simultaneously camera is measured is as measuring distance;

(4) combine the field size information calculations of camera to go out the accurate distance of camera and metope on each pixel orientation through the arc cosine method;

(5) set up look-up table table, size is M * N * 2 * Mark, and M * N is the image size, and each pixel is set up look-up table in the relation that measures between distance and the actual distance on the different distance, and so far preliminary work finishes;

(6) obtain the depth map of a width of cloth through degree of depth camera collection;

(7) read pixel orientation u, the last measuring distance of v is as the x in the linear interpolation ₂, and on relevant this pixel orientation among the look-up table table, promptly look-up table tabel (u, v ...) and in x ₂Two immediate measured values are as x ₀x ₁, and read measuring value x ₀x ₁Nominal range as f (x ₀) and f (x ₁);

(8) utilize two nominal range f (x ₀) and f (x ₁) depth value to current x ₂Measure distance and carry out the estimation f (x that linear interpolation obtains current accurate depth value ₂);

(9) travel through all pixels, each pixel is repeated (7) and (8) to obtain complete Corrected Depth image;

This LUT Method that each pixel is all set up a plurality of different nominal ranges and measuring distance can be replied very accurately; And based on the method that complete measurement information is revised again need set up all pixels at the accurate look-up table corresponding relation between the degree of depth and the both sides degree of depth on each nominal depth; This theoretical method is clear; Method is simple, and the recovery precision of picture depth is higher.But a plurality of positions of each pixel of needs storage, need relatively large storage space; Through the analysis to active optical depth figure image-forming principle, most of depth map that recovers can be regarded camera coordinate system X as _cY _cZ _cZ _cThe value of coordinate direction, and in general the difference between the different images coordinate is less, and heterogeneity is lower, thereby in order to simplify storage, can only store the correction that one group of correction data can obtain quite big degree.

Embodiment 2:

We find that through after a large amount of measurements the distance that degree of depth camera provides is the Z axle measuring value under the camera coordinate system, but the measuring value of Z axle has discrepancy with real Z-direction coordinate figure really; Need revise; The same a kind of method of method of revising has similar place, such as all being to select an index plane, with camera and alignment criteria face; And then measure nominal value and video camera distance apart from metope; Because degree of depth camera is that the Z axle is carried out to picture, thus can utilize measured value on the more index plane on average obtain the corresponding relation between actual measured value and the nominal value, set up the nominal value of one dimension and the look-up table of measured value;

Trimming process is following:

(1): at first degree of depth camera is fixedly mounted on the horizontal slide rail;

(2): with the some index planes of camera perpendicular alignmnet, the typical case is like wall;

(3): camera is moved along slide rail, and note camera apart from the actual range of metope as nominal range, and the image value that simultaneously camera is measured is on average as measuring distance;

(4): set up one group of relevant average measurement Z _cValue and nominal Z _cThe look-up table table of value (being the distance of video camera) apart from metope _AverageSize is 2 * Mark, and what Mark represented is the number of nominal value.

(5): obtain a width of cloth depth map depth, big or small M * N;

(6): the angular distance of calculating current pixel point distance center pixel; The calculating of angular distance need be known the visual field size and the sum of all pixels of video camera;

${\mathrm{\&theta;}}_u=(u-\frac{M}{2})\&times;\left(\frac{{\mathrm{fov}}_u}{M}\right),$ ${\mathrm{\&theta;}}_v=(v-\frac{N}{2})\&times;\left(\frac{{\mathrm{fov}}_v}{N}\right)$

In the following formula: fov _uWith fov _vExpression is the visual field size of the both direction of video camera respectively, and M and N represent video camera u respectively, the number of pixels on the v direction;

(7): data calibration, read measuring distance on each pixel orientation as the x in the linear interpolation through linear interpolation to the depth image value of input ₂, and to look-up table table _AverageIn with x ₂Two immediate measured values are as x ₀x ₁, and read measuring value x ₀x ₁Nominal range as f (x ₀) and f (x ₁); Carrying out interpolation correction through linear interpolation again, to obtain measured value be x ₂Corrected value f (x ₂), proofread and correct back image depth as the first step _{Modified_table}(u, value v);

(8): angularity correction, the depth value behind the 8th step process look-up table data calibration is carried out the correction of angle information: depth according to the pixel index value again _Modified(u, v)=depth _{Modified_table}(u, v)/(cos (θ _u) * cos (θ _v));

(9): travel through all pixels and obtain through the image depth behind the data calibration _Modified

More than show and described ultimate principle of the present invention and principal character and advantage of the present invention.The technician of the industry should understand; The present invention is not restricted to the described embodiments; That describes in the foregoing description and the instructions just explains principle of the present invention; Under the prerequisite that does not break away from spirit and scope of the invention, the present invention also has various changes and modifications, and these variations and improvement all fall in the scope of the invention that requires protection.The present invention requires protection domain to be defined by appending claims and equivalent thereof.

Claims (2)

1. the active optical depth measured value modification method of using based on Internet of Things is characterized in that, may further comprise the steps:

Definition one, depth map; Depth map is to be exported by depth camera, and forgives the image of target range camera range information on each orientation, and promptly it is at coordinate u, the image value f at v place (u v) representes u, on the v sight line apart from the distance value of camera closest approach,

Definition two, active optical depth figure; Having obtained of depth image is multiple, and a kind of is to utilize binocular or used for multi-vision visual to obtain the image that parallax obtains, and a kind of in addition is the method for utilizing active light to throw light on and find range then,

Definition three, index plane; A large-area plane is such as wall;

Definition four, linear interpolation; A kind of mathematics estimation method, the functional value of two points carries out Linear Estimation to the functional value of intermediate point about utilization, $f\left(x_2\right)=\frac{x_2-x_0}{x_1-x_0}f\left(x_1\right)+\frac{x_1-x_2}{x_1-x_0}f\left(x_0\right)\mathrm{For}{x}_0<x_2<x_1,$

Definition five, look-up table, a kind of data storage method becomes an array or similar data structure conveniently calculating calculative data storage,

Definition six, arc cosine method; Be used for after the angle of knowing adjacent side and adjacent side and hypotenuse, calculating the triangle hypotenuse; Promptly

wherein a be adjacent side length; C is a hypotenuse length, and θ is the angle of adjacent side and hypotenuse.

Based on the active optical depth measured value modification method that Internet of Things is used, concrete steps are following:

(1) at first degree of depth camera is fixedly mounted on the horizontal slide rail;

(2) with the some index planes of camera perpendicular alignmnet, the typical case is like wall;

(3) camera is moved along slide rail, and note camera apart from the actual range of metope as nominal range, and the target depth image value that simultaneously camera is measured is as measuring distance;

(4) combine the field size information calculations of camera to go out the accurate distance of camera and metope on each pixel orientation through the arc cosine method;

(5) set up look-up table table, size is M * N * 2 * Mark, and M * N is the image size, and each pixel is set up look-up table in the relation that measures between distance and the actual distance on the different distance, and so far preliminary work finishes;

(6) obtain the depth map of a width of cloth through degree of depth camera collection;

(7) read pixel orientation u, the last measuring distance of v is as the x in the linear interpolation ₂, and on relevant this pixel orientation among the look-up table table, promptly look-up table tabel (u, v ...) and in x ₂Two immediate measured values are as x ₀x ₁, and read measuring value x ₀x ₁Nominal range as f (x ₀) and f (x ₁);

(8) utilize two nominal range f (x ₀) and f (x ₁) depth value to current x ₂Measure distance and carry out the estimation f (x that linear interpolation obtains current accurate depth value ₂);

(9) travel through all pixels, each pixel is repeated (7) and (8) to obtain complete Corrected Depth image;

This LUT Method that each pixel is all set up a plurality of different nominal ranges and measuring distance can be replied very accurately; And based on the method that complete measurement information is revised again need set up all pixels at the accurate look-up table corresponding relation between the degree of depth and the both sides degree of depth on each nominal depth; This theoretical method is clear; Method is simple, and the recovery precision of picture depth is higher.But a plurality of positions of each pixel of needs storage, need relatively large storage space; Through the analysis to active optical depth figure image-forming principle, most of depth map that recovers can be regarded camera coordinate system X as _cY _cZ _cZ _cThe value of coordinate direction, and in general the difference between the different images coordinate is less, and heterogeneity is lower, thereby in order to simplify storage, can only store the correction that one group of correction data can obtain quite big degree.

2. the active optical depth measured value modification method of using based on Internet of Things is characterized in that a kind of in addition modification method is following:

The distance that degree of depth camera provides is the Z axle measuring value under the camera coordinate system; But the measuring value of Z axle has discrepancy with real Z-direction coordinate figure really, need revise, and the same a kind of method of the method for correction has similar place; Such as all being index plane of selection; With camera and alignment criteria face, and then measure nominal value and the distance of video camera, owing to degree of depth camera is that the Z axle is carried out to picture apart from metope; Thereby can utilize measured value on the more index plane on average obtain the corresponding relation between actual measured value and the nominal value, set up the nominal value of one dimension and the look-up table of measured value;

Concrete steps are following:

(1): at first degree of depth camera is fixedly mounted on the horizontal slide rail;

(2): with the some index planes of camera perpendicular alignmnet, the typical case is like wall;

(3): camera is moved along slide rail, and note camera apart from the actual range of metope as nominal range, and the image value that simultaneously camera is measured is on average as measuring distance;

(4): set up one group of relevant average measurement Z _cValue and nominal Z _cThe look-up table table of value (being the distance of video camera) apart from metope _AverageSize is 2 * Mark, and what Mark represented is the number of nominal value.

(5): obtain a width of cloth depth map depth, big or small M * N;

(6): the angular distance of calculating current pixel point distance center pixel; The calculating of angular distance need be known the visual field size and the sum of all pixels of video camera;

${\mathrm{\&theta;}}_u=(u-\frac{M}{2})\&times;\left(\frac{{\mathrm{fov}}_u}{M}\right),$ ${\mathrm{\&theta;}}_v=(v-\frac{N}{2})\&times;\left(\frac{{\mathrm{fov}}_v}{N}\right)$

In the following formula: fov _uWith fov _vExpression is the visual field size of the both direction of video camera respectively, and M and N represent video camera u respectively, the number of pixels on the v direction;

(7): data calibration, read measuring distance on each pixel orientation as the x in the linear interpolation through linear interpolation to the depth image value of input ₂, and to look-up table table _AverageIn with x ₂Two immediate measured values are as x ₀x ₁, and read measuring value x ₀x ₁Nominal range as f (x ₀) and f (x ₁); Carrying out interpolation correction through linear interpolation again, to obtain measured value be x ₂Corrected value f (x ₂), proofread and correct back image depth as the first step _{Modified_table}(u, value v);

(8): angularity correction, the depth value behind the 8th step process look-up table data calibration is carried out the correction of angle information: depth according to the pixel index value again _Modified(u, v)=depth _{Modified_table}(u, v)/(cos (θ _u) * cos (θ _v));

(9): travel through all pixels and obtain through the image depth behind the data calibration _Modified