CN105551020A - Method and device for detecting dimensions of target object - Google Patents

Abstract

The invention discloses a method and a device for detecting dimensions of a target object. A baseline of a stereo camera and a reference plane are parallel, optical axes of the stereo camera are parallel, and the reference plane is contained in the visual field. The method comprises steps that pre-calibrated inner and outer parameters of the stereo camera are utilized, and epipolar correction on a first image and a second image of images shot through the stereo camera is carried out; a target object area of the first image is determined; the boundary of a target object in the target object area is determined after epipolar correction; the parallax of each row of pixel points of the target object area after epipolar correction in the second image after epipolar correction is determined; and the dimensions of the target object are determined according to the parallax and the boundary. The method solves problems that a laser distance measurement detection mode has disadvantages of high mounting complexity level and high hardware cost and cannot guarantee measurement precision.

Description

A kind of method and device detecting object size

Technical field

The present invention relates to stereoscopic vision field, particularly relate to a kind of method and the device that detect object size.

Background technology

In many scenes, there is the detection demand to object size.Such as in road traffic scene, consider that the vehicle of overlength, superelevation, ultra-wide can increase traffic hazard probability of happening in motion, threaten to traffic safety, whether needing transfinites to the length, width and height of vehicle is detected.

The means of detection object size common are at present laser ranging modes.Still to detect vehicle length, which needs to install at same position to organize laser range finder more, measures the length of vehicle.

The mode of laser ranging detects object size, and at least there are the following problems:

Installation complexity is high and hardware cost is high.

Be difficult to effectively split forward and backward vehicle, be difficult to ensure measuring accuracy; Especially, when heavy traffic, measuring accuracy will decline greatly.

Summary of the invention

The object of this invention is to provide a kind of method and the device that detect object size, install with the detection mode solving laser ranging that complexity is high, hardware cost is high and cannot ensure the problem of measuring accuracy.

The object of the invention is to be achieved through the following technical solutions:

Embodiments provide a kind of method detecting object size, the baseline of stereoscopic camera is parallel with reference surface, and each optical axis of stereoscopic camera is parallel and comprise described reference surface in the visual field, and the method comprises:

Use and demarcate the inside and outside parameter of the described stereoscopic camera obtained in advance, EP point correction is carried out to the first image of the image pair that described stereoscopic camera is taken and the second image; And determine the target object area in described first image;

The border of object is determined in described target object area after EP point corrects; And determine the parallax in second image of the often row pixel of the described target object area after EP point correction after EP point corrects;

The size of described object is determined according to described parallax and described border.

Preferably, described determine EP point correct after the second image after EP point corrects of the often row pixel of described target object area in the step of parallax comprise:

Determine the Matching power flow space G of the pixel in the described target object area after EP point correction _u-V-D, described G _u-V-Din value represent EP point correct after described target object area in coordinate be (u, v) pixel EP point correction after the second image in parallax be the Matching power flow of d, d be not less than 0 integer;

By described G _u-V-Dboil down to V-D space G _vD;

Based on described G _vDdetermine the parallax in second image of the often row pixel of the described target object area after EP point correction after EP point corrects.

Preferably, described by described G _u-V-Dboil down to V-D space G _vDstep comprise:

The each row pixel imposed a condition is met, by described G in described target object area after correcting based on EP point _u-V-Dboil down to V-D space G _vD; Describedly impose a condition as two boundary pixel point distances in same a line pixel reach setting threshold value; Described G _vDin value represent EP point correct after described target object area in meet parallax in the one-row pixels point that imposes a condition the second image after EP point corrects be the Mean match cost of d.

Preferably, described based on described G _vDdetermine that the step of the parallax in second image of the often row pixel of the described target object area after EP point correction after EP point corrects realizes based on following formula:

$d_v^{*}\left(j\right)={\arg }_{d}min\{I_{v-d}(d,j)+S_S\left(d_v\right(j),d_v(j+1\left)\right)\}$

Wherein:

$S_S\left(d_v\right(j),d_v(j+1\left)\right)=\left\{\begin{array}{c}0,|d_v\left(j\right)-d_v(j+1)|\≤{\mathrm{th}}_{ds}\\ {\mathrm{th}}_{Ss},|d_v\left(j\right)-d_v(j+1)|>{\mathrm{th}}_{ds}\end{array}\right\}$

Wherein: d _v ^*(j) represent EP point correct after the second image after EP point corrects of the jth row pixel of target object area in parallax, the value of j is from 1 to the line number of described first image; I _v-d(d, j) represent EP point correct after target object area jth row pixel EP point correct after the second image in parallax be d time Matching power flow; S _s(d _v(j), d _v(j+1)) for measure EP point correct after the jth row pixel of target object area and parallax corresponding to jth+1 row pixel degree of suddenling change; Th _dsfor setting threshold value; Th _ssfor setting value.

Preferably, determine that the step on the border of object comprises in described described target object area after EP point corrects:

Based on described G _u-V-D, determine EP point correct after described target object area in each pixel be the degree of confidence of object pixel;

According to the degree of confidence determined, determine the border of the often row pixel in the described target object area of described object after EP point corrects.

Based on same inventive concept, the embodiment of the present invention additionally provides a kind of device detecting object size, and the baseline of stereoscopic camera is parallel with reference surface, and each optical axis of stereoscopic camera is parallel and comprise described reference surface in the visual field, and this device comprises:

EP point correction module, for using the inside and outside parameter of demarcating the described stereoscopic camera obtained in advance, carries out EP point correction to the first image of the image pair that described stereoscopic camera is taken and the second image;

Target object area determination module, for determining the target object area in described first image;

Object border determination module, for determining the border of object in the described target object area after EP point corrects;

Parallax determination module, for determining the parallax in second image of the often row pixel of the described target object area after EP point correction after EP point corrects;

Object size determination module, for determining the size of described object according to described parallax and described border.

Preferably, in order to determine the parallax in second image of the often row pixel of the described target object area after EP point correction after EP point corrects, described parallax determination module is used for:

Determine the Matching power flow space G of the pixel in the described target object area after EP point correction _u-V-D, described G _u-V-Din value represent EP point correct after described target object area in coordinate be (u, v) pixel EP point correction after the second image in parallax be the Matching power flow of d, d be not less than 0 integer;

By described G _u-V-Dboil down to V-D space G _vD;

Based on described G _vDdetermine the parallax in second image of the often row pixel of the described target object area after EP point correction after EP point corrects.

Preferably, in order to by described G _u-V-Dboil down to V-D space G _vD, described parallax determination module also for:

The each row pixel imposed a condition is met, by described G in described target object area after correcting based on EP point _u-V-Dboil down to V-D space G _vD; Describedly impose a condition as two boundary pixel point distances in same a line pixel reach setting threshold value; Described G _vDin value represent EP point correct after described target object area in meet parallax in the one-row pixels point that imposes a condition the second image after EP point corrects be the Mean match cost of d.

Preferably, in order to based on described G _vDdetermine the parallax in second image of the often row pixel of the described target object area after EP point correction after EP point corrects, described parallax determination module realizes based on following formula:

$d_v^{*}\left(j\right)={\arg }_{d}min\{I_{v-d}(d,j)+S_S\left(d_v\right(j),d_v(j+1\left)\right)\}$

Wherein:

$S_S\left(d_v\right(j),d_v(j+1\left)\right)=\left\{\begin{array}{c}0,|d_v\left(j\right)-d_v(j+1)|\≤{\mathrm{th}}_{ds}\\ {\mathrm{th}}_{Ss},|d_v\left(j\right)-d_v(j+1)|>{\mathrm{th}}_{ds}\end{array}\right\}$

Wherein: d _v* (j) represent EP point correct after the second image after EP point corrects of the jth row pixel of target object area in parallax, the value of j is from 1 to the line number of described first image; I _v-d(d, j) represent EP point correct after target object area jth row pixel EP point correct after the second image in parallax be d time Matching power flow; S _s(d _v(j), d _v(j+1)) for measure EP point correct after the jth row pixel of target object area and parallax corresponding to jth+1 row pixel degree of suddenling change; Th _dsfor setting threshold value; Th _ssfor setting value.

Preferably, in order to determine the border of object in the described target object area after EP point corrects, described object border determination module is used for:

Based on described G _u-V-D, determine EP point correct after described target object area in each pixel be the degree of confidence of object pixel;

According to the degree of confidence determined, determine the border of the often row pixel in the described target object area of described object after EP point corrects.

The method that the embodiment of the present invention proposes and device, only need at scene to be detected to set up stereoscopic camera, and do not need to install at same position to organize laser range finder more.The hardware cost of stereoscopic camera lower than the hardware cost organizing laser range finder more, and it installs complexity also lower than the installation complexity of laser range finder, and the technical scheme that therefore embodiment of the present invention provides can reduce installs complexity and hardware cost.In addition, in the embodiment of the present invention, after stereoscopic camera collects image, the size of object can be detected by image procossing.In the image processing process that the embodiment of the present invention provides, it is in order to the follow-up size accurately calculating object that EP point corrects; Determining that target object area is to reduce subsequent arithmetic amount, improving treatment effeciency, in target object area, determining that the border of object is then the size in order to accurately detect object.Consider that object is three-dimensional, the size of object accurately cannot be expressed in the border determined in single picture, and the technical scheme that therefore embodiment of the present invention provides determines parallax further, obtains object size in conjunction with parallax and boundary accurate.Visible, the technical scheme that the embodiment of the present invention provides, efficiently, accurately can detect the size of object.

Accompanying drawing explanation

The method flow diagram of the detection object size that Fig. 1 provides for the embodiment of the present invention;

Fig. 2 is in the embodiment of the present invention, the image pair that stereoscopic camera gathers;

Fig. 3 is in the embodiment of the present invention, the image pair after EP point corrects;

Fig. 4 is in the embodiment of the present invention, the right boundary of vehicle region;

Fig. 5 is in the embodiment of the present invention, the V-D image I of vehicle region _v-d;

Fig. 6 is in the embodiment of the present invention, the parallax straight line of the parallax curve that vehicle region is corresponding and ground region;

The device schematic diagram of the detection object size that Fig. 7 provides for the embodiment of the present invention.

Embodiment

The following condition of stereoscopic camera demand fulfillment of erection: the baseline of stereoscopic camera is parallel to reference surface, each optical axis is parallel and comprise reference surface in the visual field.And carry out the demarcation of following stereoscopic camera inside and outside parameter: the baseline B of stereoscopic camera, the focal distance f of stereoscopic camera, the coordinate (u of image center in image coordinate system ₀, v ₀), apart from floor level h _c, pitching angle theta.In addition, by the corresponding relation of the image ordinate v and parallax d that calculate reference surface region in image, d can also be designated as _g(v).

When needing to know object that image detects relative position in the environment, needing the reference surface determined in environment, and setting up camera coordinates system and world coordinate system based on the reference surface determined.

Below to reference surface, camera coordinates system, world coordinate system, image coordinate system, and the embodiment of the present invention based on system architecture and camera parameter demarcate be described.

Reference surface: to need in Environment static, be arranged in the visual field of stereoscopic camera, and the larger face of area is as such as, with reference to face, ground level, desktop etc.

Camera coordinates system: the coordinate system taking the photocentre of certain camera (the left camera of such as binocular solid camera) in stereoscopic camera as initial point foundation.The optical axis that the coordinate axis (being called the z-axis of camera coordinates system in the embodiment of the present invention) of camera coordinates system is this camera; Another coordinate axis (being called the x-axis of camera coordinates system in the embodiment of the present invention) is parallel with reference surface, another coordinate axis (being called the y-axis of camera coordinates system in the embodiment of the present invention) and x-axis and z-axis place plane orthogonal.

Image coordinate system: with certain pixel (being generally left upper apex) in image for true origin, the line direction of image is transverse axis u, and the column direction of image is the coordinate system that longitudinal axis v sets up.The transverse axis of image coordinate system is corresponding with the x-axis of camera coordinates system, and the longitudinal axis of image coordinate system is corresponding with the y-axis of camera coordinates system.

World coordinate system: the coordinate system set up for initial point with certain the environment point on reference surface.The coordinate axis (in the embodiment of the present invention be called the Z axis of world coordinate system) that be projected as world coordinate system of z-axis on reference surface of camera coordinates system; The X-axis of world coordinate system (is called the X-axis of world coordinate system) on reference surface in the embodiment of the present invention; The Y-axis of world coordinate system is perpendicular to reference surface.

Below in conjunction with accompanying drawing, the technical scheme that the embodiment of the present invention provides is described in detail.

Embodiments provide a kind of method detecting object size, the method is based on the stereoscopic camera set up according to above-mentioned condition, and as shown in Figure 1, the method comprises following operation:

Step 100, use and demarcate the inside and outside parameter of the above-mentioned stereoscopic camera obtained in advance, EP point correction is carried out to the first image of the image pair that this stereoscopic camera is taken and the second image; And determine the target object area in this first image.

It should be pointed out that the restriction carried out EP point correction and determine not have between target object area in sequential.

Wherein: if stereoscopic camera is binocular camera, then the first image and the second image are the image that two cameras of binocular camera are taken simultaneously respectively.If stereoscopic camera is three order cameras or more object cameras, then the first image is the image of selected camera, and the second image is the image having arbitrary camera in the public visual field simultaneously to take with selected camera in stereoscopic camera.After EP point corrects, the ordinate of the pixel that same environment is corresponding o'clock in the first image is identical with the ordinate of pixel corresponding in the second image.Ordinate is herein the ordinate in image coordinate system.

Wherein, the binocular image comprising this object of vehicle is obtained to use binocular solid camera, when determining the target object area in the first image, can be specifically the car plate coordinate information identifying vehicle in the first image, then obtain target vehicle approximate region I in the picture according to the motion detection algorithm that car plate coordinate information combines based on background filtering _v(i.e. target object area).

The border of object is determined in step 110, above-mentioned target object area after EP point corrects; And determine the parallax in second image of the often row pixel of this target object area after EP point correction after EP point corrects.

It should be pointed out that the border of determining object and determine the restriction that do not exist between parallax in sequential.

Wherein, the one-row pixels point of target area refers to the one-row pixels point on the u direction of image coordinate system.

Wherein, the difference of the horizontal ordinate of the pixel that the horizontal ordinate of the pixel that same environment is corresponding o'clock in the first image is corresponding with in the second image is called parallax.

Step 120, determine the size of above-mentioned object according to above-mentioned parallax and border.

The method that the embodiment of the present invention proposes, only needs at scene to be detected to set up stereoscopic camera, and does not need to install at same position to organize laser range finder more.The hardware cost of stereoscopic camera lower than the hardware cost organizing laser range finder more, and it installs complexity also lower than the installation complexity of laser range finder, and the technical scheme that therefore embodiment of the present invention provides can reduce installs complexity and hardware cost.In addition, in the embodiment of the present invention, after stereoscopic camera collects image, the size of object can be detected by image procossing.In the image processing process that the embodiment of the present invention provides, it is in order to the follow-up size accurately calculating object that EP point corrects; Determining that target object area is to reduce subsequent arithmetic amount, improving treatment effeciency, in target object area, determining that the border of object is then the size in order to accurately detect object.Consider that object is three-dimensional, the size of object accurately cannot be expressed in the border determined in single picture, and the technical scheme that therefore embodiment of the present invention provides determines parallax further, obtains object size in conjunction with parallax and boundary accurate.Visible, the technical scheme that the embodiment of the present invention provides, efficiently, accurately can detect the size of object.

In the embodiment of the present invention, determine in above-mentioned steps 110 that the implementation of parallax has multiple, such as, parallax can be determined based on Matching power flow space.Concrete: the Matching power flow space G determining the pixel in the above-mentioned target object area after EP point correction _u-V-D, this G _u-V-Din value represent EP point correct after target object area in coordinate be (u, v) pixel EP point correction after the second image in parallax be the Matching power flow of d, d be not less than 0 integer; By G _u-V-Dboil down to V-D space G _vD; Based on this G _vDdetermine the parallax in second image of the often row pixel of the target object area after EP point correction after EP point corrects.

The technical scheme that the embodiment of the present invention provides, determines parallax based on Matching power flow space, specifically V-D space G _vDdetermine parallax, get around point-to-point coupling, thus do not need the optimal match point searching for each pixel, speed is faster; Avoid the interference of Mismatching point to measurement result simultaneously, improve the accuracy of algorithm.

Wherein, determine that the implementation in the Matching power flow space of target object area is the Matching power flow of each pixel calculated in target object area when parallax is d in the second image, thus obtain Matching power flow space.After polar curve correction, the central point of the first image and the second image is generally unequal, so, if the camera of shooting the first image is in the left side of the camera of shooting second image, and the pixel I in the first image _vthe pixel that (u, v) is corresponding in the second image can be expressed as I _v ²((u-d+u ₀₂-u ₀₁), v), if the camera of shooting the first image is on the right side of the camera of shooting second image, the pixel I in the first image _vthe pixel that (u, v) is corresponding in the second image can be expressed as I _v ²(u+d+u ₀₂-u ₀₁), v).Wherein, the measurement mode of Matching power flow has multiple, such as, weighed by SAD or NCC.

Wherein, by G _u-V-Dboil down to V-D space G _vDimplementation have multiple.Such as: in the target object area after correcting based on EP point, meet each row pixel imposed a condition, by G _u-V-Dboil down to V-D space G _vD.Wherein, impose a condition as two boundary pixel point distances in same a line pixel reach setting threshold value; G _vDin value represent EP point correct after target object area in meet parallax in the one-row pixels point that imposes a condition the second image after EP point corrects be the Mean match cost of d.Wherein, in target object area, meet whole pixels that the one-row pixels point that imposes a condition does not refer to this line, and refer to both sides circle pixel and between pixel.

Wherein, based on G _vDdetermine that the implementation of the parallax in second image of the often row pixel of the target object area after EP point correction after EP point corrects has multiple.Such as determine based on following formula:

$d_v^{*}\left(j\right)={\arg }_{d}min\{I_{v-d}(d,j)+S_S\left(d_v\right(j),d_v(j+1\left)\right)\}$

Wherein:

$S_S\left(d_v\right(j),d_v(j+1\left)\right)=\left\{\begin{array}{c}0,|d_v\left(j\right)-d_v(j+1)|\≤{\mathrm{th}}_{ds}\\ {\mathrm{th}}_{Ss},|d_v\left(j\right)-d_v(j+1)|>{\mathrm{th}}_{ds}\end{array}\right\}$

Wherein: d _v ^*(j) represent EP point correct after the second image after EP point corrects of the jth row pixel of target object area in parallax, the value of j is from the line number of 1 to the first image; I _v-d(d, j) represent EP point correct after target object area jth row pixel EP point correct after the second image in parallax be d time Matching power flow; S _s(d _v(j), d _v(j+1)) be the level and smooth item in cost function, for measure EP point correct after the jth row pixel of target object area and parallax corresponding to jth+1 row pixel degree of suddenling change; Th _dsfor setting threshold value; Th _ssfor setting value.

Due to target (i.e. object, as automobile) to be measured outside surface be continually varying, generally can not undergo mutation, so as parallax (d corresponding to adjacent rows (jth row and jth+1 row) _v(j) and d _v(j+1) difference) is less than threshold value th _ds, this level and smooth item (or being called penalty term) cost is 0, when the difference of parallax is greater than threshold value th _dstime, punishment cost is value th _ss.As by I _v-dmiddle cost value normalizes to 0 ~ 255, th _ssdesirable 50, estimate to obtain one to encourage algorithm level and smooth curve.

In the embodiment of the present invention, determine in step 110 that the implementation on the border of object has multiple.Such as, based on above-mentioned G _u-V-D, determine EP point correct after target object area in each pixel be the degree of confidence of object pixel; According to the degree of confidence determined, determine the border of the often row pixel in the target object area of object after EP point corrects.

Wherein, determine that the implementation of degree of confidence has multiple, such as can based on following formulae discovery degree of confidence:

$B(i,j)=min\{{\mathrm{\Σ}}_{k=W_s}^{i}C(k,j,d_g(j\left)\right),{\mathrm{\Σ}}_{K=i}^{W_e}C(k,j,d_g(j\left)\right)\}$

Wherein, B (i, j) for EP point correct after described target object area in the degree of confidence of coordinate to be the pixel of i, j be object pixel; W _ifor the maximum occurrences of described target object area on u direction; C (k, j, d _g(j)) be G _u-V-Dmatching power flow in space.

Wherein, determine that the boundary pixel point of object in every row pixel is determined in the border of often row pixel namely.Its implementation has multiple, such as, add up the left margin of every a line pixel in target object area for being greater than threshold value th in this row _vBleftmost side location of pixels U _vlj (), right margin is the rightmost side location of pixels U being greater than this threshold value _vr(j):

$U_{vl}\left(j\right)=\underset{i}{\arg min}\{B(i,j)>{\mathrm{th}}_{vB}\}$

$U_{vr}\left(j\right)=\underset{i}{\arg \max }\{B(i,j)>{\mathrm{th}}_{vB}\}$

Wherein, th _vBfor setting threshold value.

Known based on above-mentioned formula, the above-mentioned satisfied each row pixel imposed a condition refers to and allly meets U _vr(j) >U _vl(j)+th _vwimage line, V-D image I is calculated to these image lines _v-dand be normalized to the gray level image of 0 ~ 255, thus obtain VD space.

Based on above-mentioned any embodiment, the specific implementation of step 120 has multiple.Wherein, both can determine in camera coordinates system object size and as final Output rusults; Also can determine in world coordinate system object size and as final Output rusults; The portion size of object can also be determined in camera coordinates system, in world coordinate system, determine another part size of object, and using determination result as final Output rusults.Certainly, also can using the size conversion determined in a coordinate system in another coordinate system as final Output rusults.Illustrate below:

For every a line of target object area, calculate three-dimensional coordinate and the object width of camera coordinates system lower boundary point:

$W_{cj}=\frac{B\left(U_{vr}\right(j\left)\right)-U_{vl}\left(j\right))}{d_v^{*}\left(j\right)}$

$y_{cj}=\frac{B(j-v_{0l})}{d_v^{*}\left(j\right)}$

$Z_{cj}=\frac{Bf}{d_v^{*}\left(j\right)}$

By the height h of camera _cand pitching angle theta, by { y _cjand { z _cjbe transformed into world coordinate system under:

y _wj＝cosθ×y _cj+sinθ×z _cj-h _c

z _wj＝-sinθ×y _cj+cosθ×z _cj

Then the length of object is:

$L_v\left(j\right)=\underset{j}{\max }\left\{z_{wj}\right\}-\underset{j}{\min }\left\{z_{wj}\right\}$

The width of object is:

$W_v=\underset{j}{max}\left\{W_{cj}\right\}$

The height of object is:

$H_v=\underset{j}{max}\left\{y_{wj}\right\}$

Based on above-mentioned any means embodiment, after the size of object being detected, just can carry out subsequent treatment according to the size detected.The present invention does not limit the application of size.Be detected as example with vehicle dimension, the vehicle length, width and height information that measurement can be obtained and vehicle overload standard value compare, and judge whether to transfinite, if transfinite, then send warning in conjunction with license board information to control desk.

Below in conjunction with embody rule scene, the technical scheme that the embodiment of the present invention provides is described in detail:

In the present embodiment, choose the camera that two, left and right is installed side by side, reference surface chooses ground level, and the baseline of two cameras in left and right is parallel to ground level, and the optical axis of two cameras is parallel, and comprises above-mentioned reference surface in the visual field.The focal length of stereoscopic camera is f=5594, image center is (u _0l, v _0l)=(1223.814,282.181), (u _0r, v _0r)=(1223.814,282.181), base length B=0.5026, apart from floor level h _c=7.5694, pitching angle theta=0.1257.By the corresponding relation of the image ordinate v and parallax d that calculate ground region in image, for:

d _g(v)＝0.06587×v+93.35928

Above-mentioned binocular solid camera is used to take, obtain an image pair, this image is to comprising left camera image (i.e. the first image) and right camera image (i.e. the second image), obtain license board information and the position of vehicle from the Car license recognition module that camera system is integrated, then the motion detection algorithm combined based on background filtering obtains target vehicle approximate region I in the picture _v, in the embodiment of the present invention, suppose I _vat right camera image, as shown in Figure 2.

Use and demarcate the binocular camera inside and outside parameter obtained in advance, EP point correction is carried out to binocular image, obtain the stereographic map pair after correcting.After EP point corrects, same three-dimensional environment point is at the subpoint p of stereographic map centering _l(u _l, v _l) and p _r(u _r, v _r) meet v _l=v _r; As shown in Figure 3.

Calculating chart photo I _v(u, v) Matching power flow when parallax is d in left figure, after polar curve corrects, in stereo-picture, left and right two figure central points are generally unequal, and corresponding left figure image sheet can be expressed as I _v ^l(u+d+u _0l-u _0r), v), Matching power flow is specifically weighed by SAD or NCC, obtains I like this _vwith I _v ^lu-V-D space C (u, v, d) of Matching power flow;

For image sheet I _vin each pixel p (i, j), calculate p and belong to the degree of confidence of vehicle region:

$B(i,j)=min\{{\mathrm{\Σ}}_{k=W_s}^{i}C(k,j,d_g(j\left)\right),{\mathrm{\Σ}}_{K=i}^{W_e}C(k,j,d_g(j\left)\right)\}$

Statistical graph photo I _vin the right boundary of each driving, left margin is for being greater than threshold value th in this row _vBthe leftmost side location of pixels of=800, right margin is the rightmost side location of pixels being greater than this threshold value:

$U_{vl}\left(j\right)=\underset{i}{\arg min}\{B(i,j)>{\mathrm{th}}_{vB}\}$

$U_{vr}\left(j\right)=\underset{i}{\arg \max }\{B(i,j)>{\mathrm{th}}_{vB}\}$

As shown in Figure 4, be the right boundary of vehicle region.

U is met to all _vr(j) >U _vl(j)+th _vwimage line, calculate V-D image I _v-dand be normalized to the gray level image of 0 ~ 255, I _v-dbe defined as:

$I_{v-d}(i,j)=\underset{k=U_{vl}\left(j\right)}{\overset{U_{vr}\left(j\right)}{\Σ}}C(k,j,i)$

As shown in Figure 5, be the V-D image I of vehicle region _v-d.

Based on I _v-dthe parallax that in Image estimation image, the every a line of vehicle region is corresponding:

$d_v^{*}\left(j\right)={\arg }_{d}min\{I_{v-d}(d,j)+S_S\left(d_v\right(j),d_v(j+1\left)\right)\}$

Wherein:

$S_S\left(d_v\right(j),d_v(j+1\left)\right)=\left\{\begin{array}{c}0,|d_v\left(j\right)-d_v(j+1)|\≤{\mathrm{th}}_{ds}\\ {\mathrm{th}}_{Ss},|d_v\left(j\right)-d_v(j+1)|>{\mathrm{th}}_{ds}\end{array}\right\}$

As shown in Figure 6, be parallax curve (right side) corresponding to vehicle region and the parallax straight line (left side) of ground region.

For every a line of vehicle region, calculate three-dimensional coordinate and the vehicle width of vehicle frontier point under camera coordinates system:

$W_{cj}=\frac{B\left(U_{vr}\right(j)-U_{vl}(j\left)\right)}{d_v^{*}\left(j\right)}$

$y_{cj}=\frac{B(j-v_{0l})}{d_v^{*}\left(j\right)}$

$Z_{cj}=\frac{Bf}{d_v^{*}\left(j\right)}$

By the height h of camera _cand pitching angle theta, by { y _cjand { z _cjbe transformed into world coordinate system under:

y _wj＝cosθ×y _cj+sinθ×z _cj-h _c

z _wj＝-sinθ×y _cj+cosθ×z _cj

Then the length of target vehicle is:

$L_v\left(j\right)=\underset{j}{\max }\left\{z_{wj}\right\}-\underset{j}{\min }\left\{z_{wj}\right\}$

The width of target vehicle is:

$W_v=\underset{j}{max}\left\{W_{cj}\right\}$

The height of target vehicle is:

$H_v=\underset{j}{max}\left\{y_{wj}\right\}$

Based on the inventive concept same with method, the embodiment of the present invention also provides the device detecting object size, and the baseline of stereoscopic camera is parallel with reference surface, and each optical axis of stereoscopic camera is parallel and comprise above-mentioned reference surface in the visual field, as shown in Figure 7, comprising::

EP point correction module 701, for using the inside and outside parameter of demarcating the described stereoscopic camera obtained in advance, carries out EP point correction to the first image of the image pair that described stereoscopic camera is taken and the second image;

Target object area determination module 702, for determining the target object area in described first image;

Object border determination module 703, for determining the border of object in the described target object area after EP point corrects;

Parallax determination module 704, for determining the parallax in second image of the often row pixel of the described target object area after EP point correction after EP point corrects;

Object size determination module 705, for determining the size of described object according to described parallax and described border.

The device that the embodiment of the present invention proposes, only needs at scene to be detected to set up stereoscopic camera, and does not need to install at same position to organize laser range finder more.The hardware cost of stereoscopic camera lower than the hardware cost organizing laser range finder more, and it installs complexity also lower than the installation complexity of laser range finder, and the technical scheme that therefore embodiment of the present invention provides can reduce installs complexity and hardware cost.In addition, in the embodiment of the present invention, after stereoscopic camera collects image, the size of object can be detected by image procossing.In the image processing process that the embodiment of the present invention provides, it is in order to the follow-up size accurately calculating object that EP point corrects; Determining that target object area is to reduce subsequent arithmetic amount, improving treatment effeciency, in target object area, determining that the border of object is then the size in order to accurately detect object.Consider that object is three-dimensional, the size of object accurately cannot be expressed in the border determined in single picture, and the technical scheme that therefore embodiment of the present invention provides determines parallax further, obtains object size in conjunction with parallax and boundary accurate.Visible, the technical scheme that the embodiment of the present invention provides, efficiently, accurately can detect the size of object.

Optionally, based on above-mentioned any embodiment, in order to determine the parallax in second image of the often row pixel of the described target object area after EP point correction after EP point corrects, described parallax determination module is used for:

Determine the Matching power flow space G of the pixel in the described target object area after EP point correction _u-V-D, described G _u-V-Din value represent EP point correct after described target object area in coordinate be (u, v) pixel EP point correction after the second image in parallax be the Matching power flow of d, d be not less than 0 integer;

By described G _u-V-Dboil down to V-D space G _vD;

Based on described G _vDdetermine the parallax in second image of the often row pixel of the described target object area after EP point correction after EP point corrects.

Optionally, based on above-mentioned any embodiment, in order to by described G _u-V-Dboil down to V-D space G _vD, described parallax determination module also for:

The each row pixel imposed a condition is met, by described G in described target object area after correcting based on EP point _u-V-Dboil down to V-D space G _vD; Describedly impose a condition as two boundary pixel point distances in same a line pixel reach setting threshold value; Described G _vDin value represent EP point correct after described target object area in meet parallax in the one-row pixels point that imposes a condition the second image after EP point corrects be the Mean match cost of d.

Optionally, based on above-mentioned any embodiment, in order to based on described G _vDdetermine EP point correct after described target object area often row pixel EP point correct after the second image in parallax, described parallax determination module also for:

$d_v^{*}\left(j\right)={\arg }_{d}min\{I_{v-d}(d,j)+S_S\left(d_v\right(j),d_v(j+1\left)\right)\}$

Wherein:

$S_S\left(d_v\right(j),d_v(j+1\left)\right)=\left\{\begin{array}{c}0,|d_v\left(j\right)-d_v(j+1)|\≤{\mathrm{th}}_{ds}\\ {\mathrm{th}}_{Ss},|d_v\left(j\right)-d_v(j+1)|>{\mathrm{th}}_{ds}\end{array}\right\}$

Optionally, based on above-mentioned any embodiment, in order to determine the border of object in the described target object area after EP point corrects, described object border determination module is used for:

Based on described G _u-V-D, determine EP point correct after described target object area in each pixel be the degree of confidence of object pixel;

According to the degree of confidence determined, determine the border of the often row pixel in the described target object area of described object after EP point corrects.

Those skilled in the art should understand, embodiments of the invention can be provided as method, system or computer program.Therefore, the present invention can adopt the form of complete hardware embodiment, completely software implementation or the embodiment in conjunction with software and hardware aspect.And the present invention can adopt in one or more form wherein including the upper computer program implemented of computer-usable storage medium (including but not limited to magnetic disk memory, CD-ROM, optical memory etc.) of computer usable program code.

The present invention describes with reference to according to the process flow diagram of the method for the embodiment of the present invention, equipment (system) and computer program and/or block scheme.Should understand can by the combination of the flow process in each flow process in computer program instructions realization flow figure and/or block scheme and/or square frame and process flow diagram and/or block scheme and/or square frame.These computer program instructions can being provided to the processor of multi-purpose computer, special purpose computer, Embedded Processor or other programmable data processing device to produce a machine, making the instruction performed by the processor of computing machine or other programmable data processing device produce device for realizing the function of specifying in process flow diagram flow process or multiple flow process and/or block scheme square frame or multiple square frame.

These computer program instructions also can be stored in can in the computer-readable memory that works in a specific way of vectoring computer or other programmable data processing device, the instruction making to be stored in this computer-readable memory produces the manufacture comprising command device, and this command device realizes the function of specifying in process flow diagram flow process or multiple flow process and/or block scheme square frame or multiple square frame.

These computer program instructions also can be loaded in computing machine or other programmable data processing device, make on computing machine or other programmable devices, to perform sequence of operations step to produce computer implemented process, thus the instruction performed on computing machine or other programmable devices is provided for the step realizing the function of specifying in process flow diagram flow process or multiple flow process and/or block scheme square frame or multiple square frame.

Although describe the preferred embodiments of the present invention, those skilled in the art once obtain the basic creative concept of cicada, then can make other change and amendment to these embodiments.So claims are intended to be interpreted as comprising preferred embodiment and falling into all changes and the amendment of the scope of the invention.

Obviously, those skilled in the art can carry out various change and modification to the present invention and not depart from the spirit and scope of the present invention.Like this, if these amendments of the present invention and modification belong within the scope of the claims in the present invention and equivalent technologies thereof, then the present invention is also intended to comprise these change and modification.

Claims (10)

1. detect a method for object size, it is characterized in that, the baseline of stereoscopic camera is parallel with reference surface, and each optical axis of stereoscopic camera is parallel and comprise described reference surface in the visual field, and the method comprises:

Use and demarcate the inside and outside parameter of the described stereoscopic camera obtained in advance, EP point correction is carried out to the first image of the image pair that described stereoscopic camera is taken and the second image; And determine the target object area in described first image;

The border of object is determined in described target object area after EP point corrects; And determine the parallax in second image of the often row pixel of the described target object area after EP point correction after EP point corrects;

The size of described object is determined according to described parallax and described border.

2. method according to claim 1, is characterized in that, described determine EP point correct after the second image after EP point corrects of the often row pixel of described target object area in the step of parallax comprise:

Determine the Matching power flow space G of the pixel in the described target object area after EP point correction _u-V-D, described G _u-V-Din value represent EP point correct after described target object area in coordinate be (u, v) pixel EP point correction after the second image in parallax be the Matching power flow of d, d be not less than 0 integer;

By described G _u-V-Dboil down to V-D space G _vD;

Based on described G _vDdetermine the parallax in second image of the often row pixel of the described target object area after EP point correction after EP point corrects.

3. method according to claim 2, is characterized in that, described by described G _u-V-Dboil down to V-D space G _vDstep comprise:

The each row pixel imposed a condition is met, by described G in described target object area after correcting based on EP point _u-V-Dboil down to V-D space G _vD; Describedly impose a condition as two boundary pixel point distances in same a line pixel reach setting threshold value; Described G _vDin value represent EP point correct after described target object area in meet parallax in the one-row pixels point that imposes a condition the second image after EP point corrects be the Mean match cost of d.

4. method according to claim 2, is characterized in that, described based on described G _vDdetermine that the step of the parallax in second image of the often row pixel of the described target object area after EP point correction after EP point corrects realizes based on following formula:

Wherein:

Wherein: d _v ^*(j) represent EP point correct after the second image after EP point corrects of the jth row pixel of target object area in parallax, the value of j is from 1 to the line number of described first image; I _v-d(d, j) represent EP point correct after target object area jth row pixel EP point correct after the second image in parallax be d time Matching power flow; S _s(d _v(j), d _v(j+1)) for measure EP point correct after the jth row pixel of target object area and parallax corresponding to jth+1 row pixel degree of suddenling change; Th _dsfor setting threshold value; Th _ssfor setting value.

5. the method according to any one of claim 2 ~ 4, is characterized in that, determines that the step on the border of object comprises in described described target object area after EP point corrects:

Based on described G _u-V-D, determine EP point correct after described target object area in each pixel be the degree of confidence of object pixel;

According to the degree of confidence determined, determine the border of the often row pixel in the described target object area of described object after EP point corrects.

6. detect a device for object size, it is characterized in that, the baseline of stereoscopic camera is parallel with reference surface, and each optical axis of stereoscopic camera is parallel and comprise described reference surface in the visual field, and this device comprises:

EP point correction module, for using the inside and outside parameter of demarcating the described stereoscopic camera obtained in advance, carries out EP point correction to the first image of the image pair that described stereoscopic camera is taken and the second image;

Target object area determination module, for determining the target object area in described first image;

Object border determination module, for determining the border of object in the described target object area after EP point corrects;

Parallax determination module, for determining the parallax in second image of the often row pixel of the described target object area after EP point correction after EP point corrects;

Object size determination module, for determining the size of described object according to described parallax and described border.

7. device according to claim 6, is characterized in that, in order to determine the parallax in second image of the often row pixel of the described target object area after EP point correction after EP point corrects, described parallax determination module is used for:

Determine the Matching power flow space G of the pixel in the described target object area after EP point correction _u-V-D, described G _u-V-Din value represent EP point correct after described target object area in coordinate be (u, v) pixel EP point correction after the second image in parallax be the Matching power flow of d, d be not less than 0 integer;

By described G _u-V-Dboil down to V-D space G _vD;

Based on described G _vDdetermine the parallax in second image of the often row pixel of the described target object area after EP point correction after EP point corrects.

8. device according to claim 7, is characterized in that, in order to by described G _u-V-Dboil down to V-D space G _vD, described parallax determination module also for:

The each row pixel imposed a condition is met, by described G in described target object area after correcting based on EP point _u-V-Dboil down to V-D space G _vD; Describedly impose a condition as two boundary pixel point distances in same a line pixel reach setting threshold value; Described G _vDin value represent EP point correct after described target object area in meet parallax in the one-row pixels point that imposes a condition the second image after EP point corrects be the Mean match cost of d.

9. device according to claim 7, is characterized in that, in order to based on described G _vDdetermine the parallax in second image of the often row pixel of the described target object area after EP point correction after EP point corrects, described parallax determination module realizes based on following formula:

Wherein:

Wherein: d _v ^*(j) represent EP point correct after the second image after EP point corrects of the jth row pixel of target object area in parallax, the value of j is from 1 to the line number of described first image; I _v-d(d, j) represent EP point correct after target object area jth row pixel EP point correct after the second image in parallax be d time Matching power flow; S _s(d _v(j), d _v(j+1)) for measure EP point correct after the jth row pixel of target object area and parallax corresponding to jth+1 row pixel degree of suddenling change; Th _dsfor setting threshold value; Th _ssfor setting value.

10. the device according to any one of claim 7 ~ 9, is characterized in that, in order to determine the border of object in the described target object area after EP point corrects, described object border determination module is used for:

Based on described G _u-V-D, determine EP point correct after described target object area in each pixel be the degree of confidence of object pixel;

According to the degree of confidence determined, determine the border of the often row pixel in the described target object area of described object after EP point corrects.