CN106909929A - Pedestrian's distance detection method and device - Google Patents

Pedestrian's distance detection method and device Download PDF

Info

Publication number
CN106909929A
CN106909929A CN201510968247.7A CN201510968247A CN106909929A CN 106909929 A CN106909929 A CN 106909929A CN 201510968247 A CN201510968247 A CN 201510968247A CN 106909929 A CN106909929 A CN 106909929A
Authority
CN
China
Prior art keywords
pedestrian
lane line
monocular camera
image
coordinate system
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN201510968247.7A
Other languages
Chinese (zh)
Inventor
姜波
黄忠伟
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
BYD Co Ltd
Original Assignee
BYD Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by BYD Co Ltd filed Critical BYD Co Ltd
Priority to CN201510968247.7A priority Critical patent/CN106909929A/en
Publication of CN106909929A publication Critical patent/CN106909929A/en
Pending legal-status Critical Current

Links

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00Pattern recognition
    • G06F18/20Analysing
    • G06F18/285Selection of pattern recognition techniques, e.g. of classifiers in a multi-classifier system
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06VIMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Data Mining & Analysis (AREA)
  • General Physics & Mathematics (AREA)
  • Artificial Intelligence (AREA)
  • Evolutionary Biology (AREA)
  • Evolutionary Computation (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • General Engineering & Computer Science (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Human Computer Interaction (AREA)
  • Multimedia (AREA)
  • Image Analysis (AREA)

Abstract

The invention discloses a kind of pedestrian's distance detection method and device, wherein, the method is comprised the following steps:The rectangular characteristic of the pedestrian image that monocular-camera shoots is calculated, and rectangular characteristic is trained, to obtain multiple Weak Classifiers;Multiple Weak Classifiers are combined by Cascade algorithms, to constitute strong classifier;The pixel coordinate of pedestrian in pedestrian image is obtained by strong classifier;The pixel coordinate of parameter and pedestrian according to monocular-camera calculates the distance between monocular-camera and pedestrian.Method according to embodiments of the present invention, it is possible to increase the real-time and robustness of pedestrian's distance detection.

Description

Pedestrian distance detection method and device
Technical Field
The invention relates to the technical field of intelligent vehicles, in particular to a pedestrian distance detection method and device.
Background
With the development of intelligent vehicle technology, the intelligent degree of modern vehicles is continuously improved. In the driving process of the vehicle, the vehicle can acquire surrounding image information through machine vision, so that information helpful to driving can be analyzed according to the image information. For example, the user can be reminded or directly braked emergently according to the analyzed obstacle information to ensure the driving safety.
Most vehicles collect image information through a monocular camera, and if the distance between the vehicles and pedestrians can be analyzed from the image information collected by the monocular camera, the driving safety can be improved undoubtedly.
At present, some element in the image can be recognized in the related art, and thus, the recognition of the pedestrian can be provided with assistance to some extent. However, the pedestrian images collected by the monocular camera are affected by the surrounding environment, and have other elements such as traffic signs, license plates, buildings and the like, and it is difficult to accurately distinguish pedestrians from such images in the related art. In addition, the related art has a slow speed of identifying the pedestrian in the image, so that the robustness and the real-time performance of detecting the distance of the pedestrian are poor, and the related technology is difficult to be applied to actual products.
Disclosure of Invention
The present invention is directed to solving, at least to some extent, one of the technical problems in the related art. Therefore, an object of the present invention is to provide a method for detecting a pedestrian distance, which can improve the real-time performance and robustness of the detection of the pedestrian distance.
A second object of the present invention is to provide a pedestrian distance detection apparatus.
A pedestrian distance detection method according to an embodiment of a first aspect of the invention includes the steps of: calculating the rectangular features of the pedestrian images shot by the monocular camera, and training the rectangular features to obtain a plurality of weak classifiers; combining the plurality of weak classifiers by a cascade algorithm to form a strong classifier; acquiring pixel coordinates of the pedestrian in the pedestrian image through the strong classifier; and calculating the distance between the monocular camera and the pedestrian according to the parameters of the monocular camera and the pixel coordinates of the pedestrian.
According to the pedestrian distance detection method provided by the embodiment of the invention, the plurality of weak classifiers are obtained by training the rectangular features of the pedestrian image, the strong classifiers are formed by the plurality of weak classifiers through a cascade algorithm, then the pixel coordinates of the pedestrian in the pedestrian image are obtained through the strong classifiers, and then the distance between the monocular camera and the pedestrian is calculated according to the parameters of the monocular camera and the pixel coordinates of the pedestrian, so that the pedestrian in the pedestrian image can be rapidly and accurately identified, the real-time performance and robustness of the pedestrian distance detection can be improved, and the popularization of the pedestrian distance detection technology is promoted.
A pedestrian distance detection apparatus according to an embodiment of the second aspect of the invention includes: the training module is used for calculating the rectangular features of the pedestrian images shot by the monocular camera and training the rectangular features to obtain a plurality of weak classifiers; a combination module for combining the plurality of weak classifiers by a cascade algorithm to constitute a strong classifier; the first acquisition module is used for acquiring the pixel coordinates of the pedestrian in the pedestrian image through the strong classifier; and the calculation module is used for calculating the distance between the monocular camera and the pedestrian according to the parameters of the monocular camera and the pixel coordinates of the pedestrian.
According to the pedestrian distance detection device provided by the embodiment of the invention, the plurality of weak classifiers are obtained by training the rectangular features of the pedestrian image, the strong classifiers are formed by the plurality of weak classifiers through the cascade algorithm, then the pixel coordinates of the pedestrian in the pedestrian image are obtained through the strong classifiers, and then the distance between the monocular camera and the pedestrian is calculated according to the parameters of the monocular camera and the pixel coordinates of the pedestrian, so that the pedestrian in the pedestrian image can be rapidly and accurately identified, the real-time performance and robustness of the pedestrian distance detection can be improved, and the popularization of the pedestrian distance detection technology is promoted.
Additional aspects and advantages of the invention will be set forth in part in the description which follows and, in part, will be obvious from the description, or may be learned by practice of the invention.
Drawings
FIG. 1 is a flow chart of a pedestrian distance detection method according to one embodiment of the invention;
FIG. 2 is a schematic diagram of a rectangular feature template according to one embodiment of the invention;
FIG. 3 is a schematic diagram of a rectangular feature template applied to a face image according to one embodiment of the present invention;
FIG. 4 is a diagram illustrating the relationship between an integral graph and pixels in a rectangle according to an embodiment of the present invention;
FIG. 5 is a schematic diagram of a plurality of coordinate systems according to one embodiment of the present invention;
FIG. 6 is a schematic diagram of the position relationship between camera coordinates and a world coordinate system during calibration of a monocular camera according to one embodiment of the present invention;
FIG. 7 is a schematic diagram of the intersection of two lane lines on the left and right according to one embodiment of the present invention;
FIG. 8 is a schematic diagram of the relationship between the left lane line and the right lane line and the world coordinate according to an embodiment of the present invention;
FIG. 9 is a geometric relationship diagram representing the distance between a monocular camera and a pedestrian according to one embodiment of the present invention;
fig. 10 is a block diagram showing the construction of a pedestrian distance detecting apparatus according to an embodiment of the present invention.
Detailed Description
Reference will now be made in detail to embodiments of the present invention, examples of which are illustrated in the accompanying drawings, wherein like or similar reference numerals refer to the same or similar elements or elements having the same or similar function throughout. The embodiments described below with reference to the drawings are illustrative and intended to be illustrative of the invention and are not to be construed as limiting the invention.
A pedestrian distance detection method and apparatus according to an embodiment of the present invention will be described below with reference to the accompanying drawings.
Fig. 1 is a flowchart of a pedestrian distance detection method according to an embodiment of the present invention.
As shown in fig. 1, the pedestrian distance detection method according to the embodiment of the present invention includes the following steps:
s101, calculating the rectangular features of the pedestrian images shot by the monocular camera, and training the rectangular features to obtain a plurality of weak classifiers.
Fig. 2 shows four rectangular feature templates a-d, each containing white rectangles and black rectangles. In one image, regions of different features may be represented by white and black rectangles of a rectangular feature template. For example, as shown in fig. 3, in the facial image, the color of the human eye region is darker than that of the cheek region, and therefore, the two regions can be represented by the rectangular feature template a in fig. 2.
In one embodiment of the invention, the rectangular feature may be defined as the sum of the pixels in the white rectangle minus the sum of the pixels in the black rectangle in the rectangular feature template. Wherein the sum of pixels in the white rectangles and the sum of pixels in the black rectangles can be calculated by the integral map, respectively. For example, in the four rectangles A-D shown in FIG. 4, since the integral map i1 at point 1 is equal to the sum of the pixels in rectangle A, the integral map i2 at point 2 is equal to the sum of the pixels in rectangles A and B, the integral map i3 at point 3 is equal to the sum of the pixels in rectangles A and C, and the integral map i4 at point 4 is equal to the sum of the pixels in rectangles A-D, the sum of the pixels in rectangle D is i4+ i1-i2-i 3.
After the plurality of rectangular features of the pedestrian image are calculated by the method, the rectangular features can be trained to select the rectangular features with strong classification characteristics, so that a plurality of weak classifiers are obtained. In one embodiment of the invention, the rectangular features may be trained by the Adaptive Boosting algorithm (a modified algorithm of the Adaptive Boosting algorithm).
Specifically, a plurality of rectangular features are used as training samples and can be respectively labeled with corresponding class labels. Wherein, can be xiRepresents the ith training sample in yiClass label representing ith training sampleThus, a training sample set S { (x) can be formed1,y1),(x2,y2),...(xm,ym) Wherein i is a positive integer, xi∈X,yi∈Y,yiThe training samples for a total of k classes are represented by {1,2,3 … k }. In one embodiment of the invention, the weighted error for each training sample may be calculated by a weak learning algorithm. The weight may be initialized: w is a1.i=D1(i) 1/m, i 1.. T, where the distribution D of each training sample may be 1/m, and the number of iterations performed by the weak learning algorithm may be T, i.e., T1, 2.. T. In particular, for ht=L(D,wt) Can assume htX → Y, where L is a weak learning algorithm, and the weighted error of each training sample after each iteration is calculatedhtFor training sample xiIs also expressed as ht(xi). If it is notT-1 and exits the iterative process. Then orderAnd update the weight toWherein,is a normalization constant. Therefore, a plurality of training samples with low weighting errors can be finally determined and used as weak classifiers obtained by training.
M1 contains various kinds of labels, and the errors involved in the algorithm are weighted errors. In the embodiment of the invention, the rectangular features are trained through an Adaboost. M1 algorithm, so that target elements can be distinguished from various elements in an image, and the accuracy of pedestrian distance detection is greatly improved.
S102, combining the weak classifiers by a cascade algorithm to form a strong classifier.
For t weak classifiers h obtained by the above stepsj(xi) The combination can be performed by a cascade algorithm to form a strong classifier:
and S103, acquiring the pixel coordinates of the pedestrian in the pedestrian image through the strong classifier.
In an embodiment of the invention, the coordinate system concerned may comprise: an image pixel coordinate system, an image physical coordinate system, a camera coordinate system, and a world coordinate system. Fig. 5 is a schematic diagram of the coordinate systems.
In FIG. 5, up、vpThe axes represent the image pixel coordinate system with the upper left corner of the image as the origin and u to the right and downwards, respectivelypShaft and vpPositive direction of axis, the coordinate system is the coordinate (u) of each pixel in units of pixelsp,vp) Representing the number of rows and columns of the pixel on the image.
The x and y axes represent an image physical coordinate system, the image physical coordinate system takes the intersection o of the optical axis and the image plane as an origin, and the x axis and the y axis are respectively parallel to and consistent with the u axis and the v axis of the image pixel coordinate system, and the coordinate system is taken as a unit of millimeter.
Xc、Yc、ZcThe axes represent the camera coordinate system with the lens optical center OcIs the origin, Z thereofcThe axis coinciding with the optical axis and being perpendicular to the image plane and taking the direction of the photograph as positive, XcAxis and YcThe axes are parallel to the x-axis and y-axis of the physical coordinate system of the image, respectively, it being understood that the figures areO in 5co may represent the camera focal length.
Xw、Yw、ZwThe axes represent a world coordinate system, which is a selected one of the reference coordinate systems in the measurement environment to describe the relationship between the camera and the object.
The pixel coordinates of the pedestrian are the coordinates of the pixel of the pedestrian in the image pixel coordinate system. In one embodiment of the invention, the detection windows of the strong classifier can be scaled up layer by layer, the pedestrian image is detected in a plurality of scales respectively to generate detection results in a plurality of scales, and the pixel of the pedestrian in the pedestrian image is determined according to the detection results in the plurality of scales to obtain the pixel coordinate (u, v) of the pedestrian.
It should be understood that the training of the rectangular features characterizing the pixels, and the resulting strong classifier, can better distinguish the pixels of the pedestrian from the pixels of other elements in the pedestrian image. The pedestrian image is detected through a plurality of scales to obtain a plurality of detection results, and the positions of the pixels of the pedestrians in the pedestrian image can be determined more accurately by integrating the plurality of detection results, so that the pixel coordinates (u, v) of the pedestrians are further obtained. In one embodiment of the invention, the size of the pedestrian image is kept unchanged, and only the size of the detection window of the strong classifier is changed, so that the integral image is calculated only once, the calculation time can be greatly reduced, and the speed of acquiring the pixel coordinates of the pedestrian is improved.
And S104, calculating the distance between the monocular camera and the pedestrian according to the parameters of the monocular camera and the pixel coordinates of the pedestrian.
In one embodiment of the invention, the parameters of the monocular camera may include internal parameters of the monocular camera and external parameters of the monocular camera, wherein the internal parameters may include an equivalent focal length β expressed in pixels in an axial direction representing the number of pixel columns in an image physical coordinate system, and an intersection coordinate (u) of an optical axis and an image plane in an image pixel coordinate system0,v0) External parameters may be wrappedThe pitch angle phi and the installation height h of the monocular camera are included.
When the monocular camera is installed on the vehicle, the parameters of the monocular camera can be calibrated. The internal parameters can be obtained by using a Matrix Laboratory (Matlab) calibration kit. The tool box adopts a black and white chessboard as a calibration reference object, a monocular camera is used for collecting a plurality of template images of the black and white chessboard from different distances and angles during calibration, 20 of the template images are extracted to carry out calibration experiments on internal parameters of the camera, and then an optimization tool of the tool box is used for optimization.
In one embodiment of the present invention, the external parameters of the monocular camera may be obtained by: acquiring a left lane line and a right lane line of a vehicle in a road image shot by a monocular camera; and acquiring external parameters of the monocular camera according to the left lane line and the right lane line of the vehicle.
Specifically, the coordinates of the intersection of the left and right lane lines of the vehicle in the image pixel coordinate system, the slopes of the left and right lane lines in the image pixel coordinate system, and the distance between the left and right lane lines may be acquired.
In one embodiment of the present invention, if the coordinate of the point M in the world coordinate system is [ X ]w,Yw,Zw]TThe coordinates in the image pixel coordinate system are [ u, v ]]TThen the coordinate transformation relationship of the point can be expressed as:
wherein Z iscα and β are equivalent focal lengths expressed in units of pixels in an image physical coordinate system in the axial direction respectively expressing the number of rows and columns of pixels, P is a perspective transformation matrix, A is an internal parameter matrix of the camera, and t is [ t [ [ t ]x,ty,tz]TRepresenting the worldThe origin in the coordinate system is the coordinate in the camera coordinate system. The rotation matrix R is a combination of the direction cosines of the camera optical axis relative to the world coordinate system coordinate axes. If the spatial point M follows a winding Xw,Yw,ZwThe order of the axes is rotated by angles psi, theta, phi, respectively, and R can be expressed as:
as shown in fig. 6, for the vehicle-mounted monocular camera, the yaw angle and the roll angle of the rotation matrix R may be set to zero, that is, ψ ═ θ ═ 0, the camera pitch angle Φ, and the camera mounting height h. Under the model, only the pitch angle phi and the installation height h of the camera are used for calibrating parameters needing to be solved. From the geometric relationship, the coordinates of the origin of the world coordinate system in the camera coordinate system are (0, hcos φ, 0). The parameters R and t in formula (1) may be, respectively, according to the definition of formula (1):
since α and β are closer in actual calibration, f can be definedrα, β, combining formula (1) and formula (3), the mapping relationship between the coordinates in the image pixel coordinate system and the coordinates in the world coordinate system is as follows:
when Z isw→ ∞ time, the two lane lines in the road image intersect at the point (u)e,ve) Wherein
same as thatCan obtain u
Fig. 7 is a schematic diagram of intersection of the left lane line and the right lane line, and fig. 8 is a schematic diagram of a relationship between the left lane line and the right lane line and world coordinates. In fig. 8, L denotes a left lane line, and R denotes a right lane line. Theta is L and the world coordinate system ZwAngle of axis, dLAnd dRThe distances from the origin of the world coordinate system to L and R are respectively, so that the distance W between the left lane line and the right lane line is dL+dR. As can be seen from FIG. 8, the slope of the line L is cot θ and the intercept is dLSin θ, therefore, the equation for the line for L is as follows:
in one embodiment of the present invention, in the image pixel coordinate system, the plane of the point where the monocular camera is located may intersect with the left and right lane lines of the vehicle, respectively at (u)1,v1) And (u)2,v2). When Z iswWhen equal to 0, Xw=-dL[ cos θ ] can be substituted by (u) of the formula (4)1,v1) The coordinates are:
similarly, (u) can be obtained2,v2) The coordinates of (a).
Is prepared from (u)1,v1) And (u)2,v2) The slope of the left lane line L is obtained from the coordinates of (a):
likewise, the slope of the right lane line R is:
and calculating the pitch angle phi of the monocular camera according to the coordinates and the internal parameters of the intersection point of the left lane line and the right lane line of the vehicle in the image pixel coordinate system.
From equation (5), the pitch angle of the monocular camera is:
and calculating the installation height h of the monocular camera according to the slope of the left lane line and the right lane line in the image pixel coordinate system, the distance between the left lane line and the right lane line and the internal parameter.
From the geometric relationship:
therefore, the installation height of the monocular camera is:
therefore, the calibration of the external parameters of the monocular camera is completed through the left lane line and the right lane line of the vehicle, and the calibration is simple and convenient.
Fig. 9 shows the geometrical relationship between the distance d between the monocular camera and the pedestrian and the monocular camera parameters, the pixel coordinates (u, v) of the pedestrian. As shown in fig. 9, since only the distance in the vehicle traveling direction between the monocular camera and the pedestrian is calculated, only v in the pixel coordinates of the pedestrian can be usedpCoordinates v of the axis and v in coordinates of intersection of the optical axis and the image planepCoordinate v of axis0. F in fig. 9 is the focal length of the monocular camera.In the image physical coordinate system, the relationship between the equivalent focal length β expressed in units of pixels and the focal length f of the monocular camera in the axial direction indicating the number of pixel columns is β ═ f/dyWherein d isyIs the physical size of each pixel in the y-axis direction in the physical coordinate system of the image. As can be seen from fig. 9, the distance between the monocular camera and the pedestrian can be calculated by the following formula:
according to the pedestrian distance detection method provided by the embodiment of the invention, the plurality of weak classifiers are obtained by training the rectangular features of the pedestrian image, the strong classifiers are formed by the plurality of weak classifiers through a cascade algorithm, then the pixel coordinates of the pedestrian in the pedestrian image are obtained through the strong classifiers, and then the distance between the monocular camera and the pedestrian is calculated according to the parameters of the monocular camera and the pixel coordinates of the pedestrian, so that the pedestrian in the pedestrian image can be rapidly and accurately identified, the real-time performance and robustness of the pedestrian distance detection can be improved, and the popularization of the pedestrian distance detection technology is promoted.
In order to realize the embodiment, the invention further provides a pedestrian distance detection device.
Fig. 10 is a block diagram showing the construction of a pedestrian distance detecting apparatus according to an embodiment of the present invention.
As shown in fig. 10, the pedestrian distance detection apparatus according to the embodiment of the present invention includes: a training module 10, a combining module 20, a first acquisition module 30 and a calculation module 40.
The training module 10 is configured to calculate a rectangular feature of a pedestrian image captured by the monocular camera, and train the rectangular feature to obtain a plurality of weak classifiers.
Fig. 2 shows four rectangular feature templates a-d, each containing white rectangles and black rectangles. In one image, regions of different features may be represented by white and black rectangles of a rectangular feature template. For example, as shown in fig. 3, in the facial image, the color of the human eye region is darker than that of the cheek region, and therefore, the two regions can be represented by the rectangular feature template a in fig. 2.
In one embodiment of the invention, the rectangular feature may be defined as the sum of the pixels in the white rectangle minus the sum of the pixels in the black rectangle in the rectangular feature template. Wherein the sum of pixels in the white rectangles and the sum of pixels in the black rectangles can be calculated by the integral map, respectively. For example, in the four rectangles A-D shown in FIG. 4, since the integral map i1 at point 1 is equal to the sum of the pixels in rectangle A, the integral map i2 at point 2 is equal to the sum of the pixels in rectangles A and B, the integral map i3 at point 3 is equal to the sum of the pixels in rectangles A and C, and the integral map i4 at point 4 is equal to the sum of the pixels in rectangles A-D, the sum of the pixels in rectangle D is i4+ i1-i2-i 3.
After the plurality of rectangular features of the pedestrian image are calculated by the method, the rectangular features can be trained to select the rectangular features with strong classification characteristics, so that a plurality of weak classifiers are obtained. In one embodiment of the invention, the training module 10 may train the rectangular feature through the adaboost. m1 algorithm.
Specifically, a plurality of rectangular features are used as training samples and can be respectively labeled with corresponding class labels. Wherein, can be xiRepresents the ith training sample in yiA class label indicating the ith training sample can constitute a training sample set S { (x)1,y1),(x2,y2),...(xm,ym) Wherein i is a positive integer, xi∈X,yi∈Y,yiThe training samples for a total of k classes are represented by {1,2,3 … k }. In one embodiment of the invention, the weighted error for each training sample may be calculated by a weak learning algorithm. The weight may be initialized: w is a1.i=D1(i) 1/m, i1, m, where the distribution D of each training sample may be 1/m, and the number of iterations performed by the weak learning algorithm may be T, i.e., T ═ m1, 2. In particular, for ht=L(D,wt) Can assume htX → Y, where L is a weak learning algorithm, and the weighted error of each training sample after each iteration is calculatedhtFor training sample xiIs also expressed as ht(xi). If it is notT-1 and exits the iterative process. Then orderAnd update the weight toWherein,is a normalization constant. Therefore, a plurality of training samples with low weighting errors can be finally determined and used as weak classifiers obtained by training.
M1 contains various kinds of labels, and the errors involved in the algorithm are weighted errors. In the embodiment of the invention, the rectangular features are trained through an Adaboost. M1 algorithm, so that target elements can be distinguished from various elements in an image, and the accuracy of pedestrian distance detection is greatly improved.
The combining module 20 is used to combine a plurality of weak classifiers by a cascade algorithm to constitute a strong classifier.
For t weak classifiers h obtained by training module 10j(xi) The combining module 20 may combine through a cascading algorithm to form a strong classifier:
the first obtaining module 30 is used for obtaining the pixel coordinates of the pedestrian in the pedestrian image through the strong classifier.
In an embodiment of the invention, the coordinate system concerned may comprise: an image pixel coordinate system, an image physical coordinate system, a camera coordinate system, and a world coordinate system. Fig. 5 is a schematic diagram of the coordinate systems.
In FIG. 5, up、vpThe axes represent the image pixel coordinate system with the upper left corner of the image as the origin and u to the right and downwards, respectivelypShaft and vpPositive direction of axis, the coordinate system is the coordinate (u) of each pixel in units of pixelsp,vp) Representing the number of rows and columns of the pixel on the image.
The x and y axes represent an image physical coordinate system, the image physical coordinate system takes the intersection o of the optical axis and the image plane as an origin, and the x axis and the y axis are respectively parallel to and consistent with the u axis and the v axis of the image pixel coordinate system, and the coordinate system is taken as a unit of millimeter.
Xc、Yc、ZcThe axes represent the camera coordinate system with the lens optical center OcIs the origin, Z thereofcThe axis coinciding with the optical axis and being perpendicular to the image plane and taking the direction of the photograph as positive, XcAxis and YcThe axes are parallel to the x-axis and y-axis of the physical coordinate system of the image, respectively, it being understood that O in FIG. 5co may represent the camera focal length.
Xw、Yw、ZwThe axes represent a world coordinate system, which is a selected one of the reference coordinate systems in the measurement environment to describe the relationship between the camera and the object.
The pixel coordinates of the pedestrian are the coordinates of the pixel of the pedestrian in the image pixel coordinate system. In an embodiment of the present invention, the first obtaining module 30 may proportionally enlarge the detection windows of the strong classifier layer by layer, detect the pedestrian image in a plurality of scales respectively to generate detection results in a plurality of scales, and determine the pixel of the pedestrian in the pedestrian image according to the detection results in the plurality of scales to obtain the pixel coordinate (u, v) of the pedestrian.
It should be understood that the training of the rectangular features characterizing the pixels, and the resulting strong classifier, can better distinguish the pixels of the pedestrian from the pixels of other elements in the pedestrian image. The pedestrian image is detected through a plurality of scales to obtain a plurality of detection results, and the positions of the pixels of the pedestrians in the pedestrian image can be determined more accurately by integrating the plurality of detection results, so that the pixel coordinates (u, v) of the pedestrians are further obtained. In one embodiment of the invention, the size of the pedestrian image is kept unchanged, and only the size of the detection window of the strong classifier is changed, so that the integral image is calculated only once, the calculation time can be greatly reduced, and the speed of acquiring the pixel coordinates of the pedestrian is improved.
The calculation module 40 is used for calculating the distance between the monocular camera and the pedestrian according to the parameters of the monocular camera and the pixel coordinates of the pedestrian.
In one embodiment of the invention, the parameters of the monocular camera may include internal parameters of the monocular camera and external parameters of the monocular camera, wherein the internal parameters may include an equivalent focal length β expressed in pixels in an axial direction representing the number of pixel columns in an image physical coordinate system, and an intersection coordinate (u) of an optical axis and an image plane in an image pixel coordinate system0,v0) The external parameters may include the pitch angle phi and the mounting height h of the monocular camera.
When the monocular camera is installed on the vehicle, the parameters of the monocular camera can be calibrated. Wherein, the internal parameters can be obtained by adopting a Matlab calibration tool box. The tool box adopts a black and white chessboard as a calibration reference object, a monocular camera is used for collecting a plurality of template images of the black and white chessboard from different distances and angles during calibration, 20 of the template images are extracted to carry out calibration experiments on internal parameters of the camera, and then an optimization tool of the tool box is used for optimization.
In an embodiment of the invention, a second acquisition module for acquiring external parameters of the monocular camera may be further included, and the second acquisition module is used for acquiring two lane lines of the left and right of the vehicle in the road image captured by the monocular camera; and acquiring external parameters of the monocular camera according to the left lane line and the right lane line of the vehicle.
Specifically, the second obtaining module may obtain coordinates of an intersection point of two lane lines on the left and right of the vehicle in the image pixel coordinate system, obtain slopes of the two lane lines on the left and right in the image pixel coordinate system, and obtain a distance between the two lane lines on the left and right.
In one embodiment of the present invention, if the coordinate of the point M in the world coordinate system is [ X ]w,Yw,Zw]TThe coordinates in the image pixel coordinate system are [ u, v ]]TThen the coordinate transformation relationship of the point can be expressed as:
wherein Z iscα and β are equivalent focal lengths expressed in units of pixels in an image physical coordinate system in the axial direction respectively expressing the number of rows and columns of pixels, P is a perspective transformation matrix, A is an internal parameter matrix of the camera, and t is [ t [ [ t ]x,ty,tz]TRepresenting the coordinates of the origin in the world coordinate system in the camera coordinate system. The rotation matrix R is a combination of the direction cosines of the camera optical axis relative to the world coordinate system coordinate axes. If the spatial point M follows a winding Xw,Yw,ZwThe order of the axes is rotated by angles psi, theta, phi, respectively, and R can be expressed as:
as shown in fig. 6, for the vehicle-mounted monocular camera, the yaw angle and the roll angle of the rotation matrix R may be set to zero, that is, ψ ═ θ ═ 0, the camera pitch angle Φ, and the camera mounting height h. Under the model, only the pitch angle phi and the installation height h of the camera are used for calibrating parameters needing to be solved. From the geometric relationship, the coordinates of the origin of the world coordinate system in the camera coordinate system are (0, hcos φ, 0). The parameters R and t in formula (1) may be, respectively, according to the definition of formula (1):
since α and β are closer in actual calibration, f can be definedrα, β, combining formula (1) and formula (3), the mapping relationship between the coordinates in the image pixel coordinate system and the coordinates in the world coordinate system is as follows:
when Z isw→ ∞ time, the two lane lines in the road image intersect at the point (u)e,ve) Wherein
similarly, u can be obtained
Fig. 7 is a schematic diagram of intersection of the left lane line and the right lane line, and fig. 8 is a schematic diagram of a relationship between the left lane line and the right lane line and world coordinates. In fig. 8, L denotes a left lane line, and R denotes a right lane line. Theta is L and the world coordinate system ZwAngle of axis, dLAnd dRThe distances from the origin of the world coordinate system to L and R are respectively, so that the distance W between the left lane line and the right lane line is dL+dR. As can be seen from FIG. 8, the slope of the line L is cot θ and the intercept is dLSin θ, therefore, the equation for the line for L is as follows:
in one embodiment of the present invention, in the image pixel coordinate system, the plane of the point where the monocular camera is located may intersect with the left and right lane lines of the vehicle, respectively at (u)1,v1) And (u)2,v2). When Z iswWhen equal to 0, Xw=-dL[ cos θ ] can be substituted by (u) of the formula (4)1,v1) The coordinates are:
similarly, (u) can be obtained2,v2) The coordinates of (a).
Is prepared from (u)1,v1) And (u)2,v2) The slope of the left lane line L is obtained from the coordinates of (a):
likewise, the slope of the right lane line R is:
the second acquisition module can calculate the pitch angle phi of the monocular camera according to the coordinates and the internal parameters of the intersection point of the left lane line and the right lane line of the vehicle in the image pixel coordinate system.
From equation (5), the pitch angle of the monocular camera is:
the second acquisition module can also calculate the installation height h of the monocular camera according to the slope of the left lane line and the right lane line in the image pixel coordinate system, the distance between the left lane line and the right lane line and the internal parameters.
From the geometric relationship:
therefore, the installation height of the monocular camera is:
therefore, the calibration of the external parameters of the monocular camera is completed through the left lane line and the right lane line of the vehicle, and the calibration is simple and convenient.
Fig. 9 shows the geometrical relationship between the distance d between the monocular camera and the pedestrian and the monocular camera parameters, the pixel coordinates of the pedestrian. As shown in fig. 9, since only the distance in the vehicle traveling direction between the monocular camera and the pedestrian is calculated, only v in the pixel coordinates of the pedestrian is usedpCoordinates v of the axis and v in coordinates of intersection of the optical axis and the image planepCoordinate v of axis0In the image physical coordinate system, the relationship between the focal length f of the monocular camera and the equivalent focal length β expressed in units of pixels in the axial direction indicating the number of pixel columns is β ═ f/dyWherein d isyIs the physical size of each pixel in the y-axis direction in the physical coordinate system of the image. As can be seen from fig. 9, the distance between the monocular camera and the pedestrian can be calculated by the following formula:
according to the pedestrian distance detection device provided by the embodiment of the invention, the plurality of weak classifiers are obtained by training the rectangular features of the pedestrian image, the strong classifiers are formed by the plurality of weak classifiers through the cascade algorithm, then the pixel coordinates of the pedestrian in the pedestrian image are obtained through the strong classifiers, and then the distance between the monocular camera and the pedestrian is calculated according to the parameters of the monocular camera and the pixel coordinates of the pedestrian, so that the pedestrian in the pedestrian image can be rapidly and accurately identified, the real-time performance and robustness of the pedestrian distance detection can be improved, and the popularization of the pedestrian distance detection technology is promoted.
In the description of the present invention, it is to be understood that the terms "central," "longitudinal," "lateral," "length," "width," "thickness," "upper," "lower," "front," "rear," "left," "right," "vertical," "horizontal," "top," "bottom," "inner," "outer," "clockwise," "counterclockwise," "axial," "radial," "circumferential," and the like are used in the orientations and positional relationships indicated in the drawings for convenience in describing the invention and to simplify the description, and are not intended to indicate or imply that the referenced device or element must have a particular orientation, be constructed and operated in a particular orientation, and are not to be considered limiting of the invention.
Furthermore, the terms "first", "second" and "first" are used for descriptive purposes only and are not to be construed as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of the present invention, "a plurality" means two or more unless specifically defined otherwise.
In the present invention, unless otherwise expressly stated or limited, the terms "mounted," "connected," "secured," and the like are to be construed broadly and can, for example, be fixedly connected, detachably connected, or integrally formed; can be mechanically or electrically connected; either directly or indirectly through intervening media, either internally or in any other relationship. The specific meanings of the above terms in the present invention can be understood by those skilled in the art according to specific situations.
In the present invention, unless otherwise expressly stated or limited, the first feature "on" or "under" the second feature may be directly contacting the first and second features or indirectly contacting the first and second features through an intermediate. Also, a first feature "on," "over," and "above" a second feature may be directly or diagonally above the second feature, or may simply indicate that the first feature is at a higher level than the second feature. A first feature being "under," "below," and "beneath" a second feature may be directly under or obliquely under the first feature, or may simply mean that the first feature is at a lesser elevation than the second feature.
In the description herein, references to the description of the term "one embodiment," "some embodiments," "an example," "a specific example," or "some examples," etc., mean that a particular feature, structure, material, or characteristic described in connection with the embodiment or example is included in at least one embodiment or example of the invention. In this specification, the schematic representations of the terms used above are not necessarily intended to refer to the same embodiment or example. Furthermore, the particular features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples. Furthermore, various embodiments or examples and features of different embodiments or examples described in this specification can be combined and combined by one skilled in the art without contradiction.
Although embodiments of the present invention have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting the present invention, and that variations, modifications, substitutions and alterations can be made to the above embodiments by those of ordinary skill in the art within the scope of the present invention.

Claims (14)

1. A pedestrian distance detection method is characterized by comprising the following steps:
calculating the rectangular features of the pedestrian images shot by the monocular camera, and training the rectangular features to obtain a plurality of weak classifiers;
combining the plurality of weak classifiers by a cascade algorithm to form a strong classifier;
acquiring pixel coordinates of the pedestrian in the pedestrian image through the strong classifier;
and calculating the distance between the monocular camera and the pedestrian according to the parameters of the monocular camera and the pixel coordinates of the pedestrian.
2. The pedestrian distance detection method according to claim 1, wherein the rectangular feature is trained by an Adaboost.M1 algorithm.
3. The pedestrian distance detection method according to claim 1, wherein the parameters of the monocular camera include internal parameters of the monocular camera and external parameters of the monocular camera, wherein the internal parameters include an equivalent focal length β expressed in units of pixels in an axial direction representing the number of pixel columns in an image physical coordinate system, and an intersection coordinate (u) of an optical axis and an image plane in an image pixel coordinate system0,v0) And the external parameters comprise a pitch angle phi and a mounting height h of the monocular camera.
4. The pedestrian distance detection method according to claim 3, wherein the external parameter is obtained by:
acquiring a left lane line and a right lane line of a vehicle in a road image shot by a monocular camera;
and acquiring external parameters of the monocular camera according to the left lane line and the right lane line of the vehicle.
5. The pedestrian distance detection method according to claim 3 or 4, wherein the acquiring the external parameters of the monocular camera according to the left and right lane lines of the vehicle includes:
acquiring coordinates of intersection points of a left lane line and a right lane line of the vehicle in an image pixel coordinate system, acquiring slopes of the left lane line and the right lane line in the image pixel coordinate system, and acquiring a distance between the left lane line and the right lane line;
calculating the pitch angle phi of the monocular camera according to the coordinates of the intersection point of the left lane line and the right lane line of the vehicle in an image pixel coordinate system and the internal parameters;
and calculating the installation height h of the monocular camera according to the slope of the left lane line and the right lane line in the image pixel coordinate system, the distance between the left lane line and the right lane line and the internal parameter.
6. The method according to claim 1, wherein the obtaining of the pixel coordinates of the pedestrian in the pedestrian image by the strong classifier specifically comprises:
amplifying the detection windows of the strong classifiers in an equal ratio layer by layer, and detecting the pedestrian images in multiple scales respectively to generate detection results in multiple scales;
and determining the pixel of the pedestrian in the pedestrian image according to the detection results of the multiple scales to obtain the pixel coordinate (u, v) of the pedestrian.
7. The pedestrian distance detection method according to claim 3 or 5, characterized in that the distance between the monocular camera and the pedestrian is calculated by the following formula:
d = h * β c o s φ - ( v - v 0 ) s i n φ ( v - v 0 ) c o s φ + β s i n φ .
8. a pedestrian distance detection device characterized by comprising:
the training module is used for calculating the rectangular features of the pedestrian images shot by the monocular camera and training the rectangular features to obtain a plurality of weak classifiers;
a combination module for combining the plurality of weak classifiers by a cascade algorithm to constitute a strong classifier;
the first acquisition module is used for acquiring the pixel coordinates of the pedestrian in the pedestrian image through the strong classifier;
and the calculation module is used for calculating the distance between the monocular camera and the pedestrian according to the parameters of the monocular camera and the pixel coordinates of the pedestrian.
9. The pedestrian distance detection device according to claim 8, wherein the training module trains the rectangular feature by an Adaboost.M1 algorithm.
10. The pedestrian distance detection device according to claim 8, wherein the parameters of the monocular camera include an internal parameter of the monocular camera and an external parameter of the monocular camera, wherein the internal parameter includes an equivalent focal length β expressed in units of pixels in an axial direction representing the number of pixel columns in an image physical coordinate system, and an intersection coordinate (u) of an optical axis and an image plane in an image pixel coordinate system0,v0) And the external parameters comprise a pitch angle phi and a mounting height h of the monocular camera.
11. The pedestrian distance detection device according to claim 10, characterized by further comprising:
and the second acquisition module is used for acquiring the left lane line and the right lane line of the vehicle in the road image shot by the monocular camera and acquiring the external parameters of the monocular camera according to the left lane line and the right lane line of the vehicle.
12. The pedestrian distance detection device according to claim 10 or 11, wherein the second acquisition module is configured to:
acquiring coordinates of intersection points of a left lane line and a right lane line of the vehicle in an image pixel coordinate system, acquiring slopes of the left lane line and the right lane line in the image pixel coordinate system, and acquiring a distance between the left lane line and the right lane line;
calculating the pitch angle phi of the monocular camera according to the coordinates of the intersection point of the left lane line and the right lane line of the vehicle in an image pixel coordinate system and the internal parameters;
and calculating the installation height h of the monocular camera according to the slope of the left lane line and the right lane line in the image pixel coordinate system, the distance between the left lane line and the right lane line and the internal parameter.
13. The pedestrian distance detection device according to claim 8, wherein the first acquisition module is configured to:
amplifying the detection windows of the strong classifiers in an equal ratio layer by layer, and detecting the pedestrian images in multiple scales respectively to generate detection results in multiple scales;
and determining the pixel of the pedestrian in the pedestrian image according to the detection results of the multiple scales to obtain the pixel coordinate (u, v) of the pedestrian.
14. The pedestrian distance detection device according to claim 10 or 13, wherein the calculation module is configured to calculate the distance between the monocular camera and the pedestrian by the following formula:
d = h * β c o s φ - ( v - v 0 ) s i n φ ( v - v 0 ) c o s φ + β s i n φ .
CN201510968247.7A 2015-12-22 2015-12-22 Pedestrian's distance detection method and device Pending CN106909929A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201510968247.7A CN106909929A (en) 2015-12-22 2015-12-22 Pedestrian's distance detection method and device

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201510968247.7A CN106909929A (en) 2015-12-22 2015-12-22 Pedestrian's distance detection method and device

Publications (1)

Publication Number Publication Date
CN106909929A true CN106909929A (en) 2017-06-30

Family

ID=59200560

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201510968247.7A Pending CN106909929A (en) 2015-12-22 2015-12-22 Pedestrian's distance detection method and device

Country Status (1)

Country Link
CN (1) CN106909929A (en)

Cited By (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108961838A (en) * 2018-08-16 2018-12-07 大连民族大学 Road pedestrian categorizing system
CN109165591A (en) * 2018-08-16 2019-01-08 大连民族大学 road pedestrian classification method
CN109272482A (en) * 2018-07-20 2019-01-25 浙江浩腾电子科技股份有限公司 A kind of urban road crossing vehicle queue detection system based on sequence image
CN109920001A (en) * 2019-03-14 2019-06-21 大连民族大学 Method for estimating distance based on pedestrian head height
CN109949355A (en) * 2019-03-14 2019-06-28 大连民族大学 The method of half fan-shaped equidistant line model is established in monocular vision pedestrian's distance estimations
CN109959919A (en) * 2017-12-22 2019-07-02 比亚迪股份有限公司 Automobile and monocular cam distance measuring method, device
CN110197104A (en) * 2018-02-27 2019-09-03 杭州海康威视数字技术股份有限公司 Distance measuring method and device based on vehicle

Citations (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103204123A (en) * 2013-03-25 2013-07-17 中国电子科技集团公司第三十八研究所 Vehicle-pedestrian detecting, tracking and early-warning device and method

Patent Citations (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103204123A (en) * 2013-03-25 2013-07-17 中国电子科技集团公司第三十八研究所 Vehicle-pedestrian detecting, tracking and early-warning device and method

Non-Patent Citations (3)

* Cited by examiner, † Cited by third party
Title
周勇: "智能车辆中的几个关键技术研究", 《中国博士学位论文全文数据库工程科技Ⅱ辑》 *
李欣: "基于单目视觉的车辆前方障碍物测距", 《中国优秀硕士学位论文全文数据库工程科技Ⅱ辑》 *
苏云鹏: "基于车载辅助驾驶系统的行人检测模块设计", 《中国优秀硕士学位论文全文数据库信息科技辑》 *

Cited By (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109959919A (en) * 2017-12-22 2019-07-02 比亚迪股份有限公司 Automobile and monocular cam distance measuring method, device
CN109959919B (en) * 2017-12-22 2021-03-26 比亚迪股份有限公司 Automobile and monocular camera ranging method and device
CN110197104A (en) * 2018-02-27 2019-09-03 杭州海康威视数字技术股份有限公司 Distance measuring method and device based on vehicle
CN110197104B (en) * 2018-02-27 2022-03-29 杭州海康威视数字技术股份有限公司 Distance measurement method and device based on vehicle
CN109272482A (en) * 2018-07-20 2019-01-25 浙江浩腾电子科技股份有限公司 A kind of urban road crossing vehicle queue detection system based on sequence image
CN109272482B (en) * 2018-07-20 2021-08-24 浙江浩腾电子科技股份有限公司 Urban intersection vehicle queuing detection system based on sequence images
CN108961838A (en) * 2018-08-16 2018-12-07 大连民族大学 Road pedestrian categorizing system
CN109165591A (en) * 2018-08-16 2019-01-08 大连民族大学 road pedestrian classification method
CN109920001A (en) * 2019-03-14 2019-06-21 大连民族大学 Method for estimating distance based on pedestrian head height
CN109949355A (en) * 2019-03-14 2019-06-28 大连民族大学 The method of half fan-shaped equidistant line model is established in monocular vision pedestrian's distance estimations

Similar Documents

Publication Publication Date Title
CN106909929A (en) Pedestrian's distance detection method and device
CN107230218B (en) Method and apparatus for generating confidence measures for estimates derived from images captured by vehicle-mounted cameras
CN104573646B (en) Chinese herbaceous peony pedestrian detection method and system based on laser radar and binocular camera
CN109583267B (en) Vehicle target detection method, vehicle target detection device, and vehicle
CN107424116B (en) Parking space detection method based on side surround view camera
EP2237988B1 (en) Object detection and recognition system
Zhou et al. A novel lane detection based on geometrical model and gabor filter
WO2021227645A1 (en) Target detection method and device
CN109919074B (en) Vehicle sensing method and device based on visual sensing technology
US9064157B2 (en) Pedestrian detection systems and methods
CN110490936B (en) Calibration method, device and equipment of vehicle camera and readable storage medium
CN111967360B (en) Target vehicle posture detection method based on wheels
CN110745140A (en) Vehicle lane change early warning method based on continuous image constraint pose estimation
EP2224743B1 (en) Vehicle periphery monitoring device, vehicle, and vehicle periphery monitoring program
CN107796373B (en) Distance measurement method based on monocular vision of front vehicle driven by lane plane geometric model
CN108645375B (en) Rapid vehicle distance measurement optimization method for vehicle-mounted binocular system
CN110659548B (en) Vehicle and target detection method and device thereof
CN113256739B (en) Self-calibration method and device for vehicle-mounted BSD camera and storage medium
CN112740225A (en) Method and device for determining road surface elements
EP2312550B1 (en) Vehicle periphery monitoring device
CN105678287A (en) Ridge-measure-based lane line detection method
US20190213427A1 (en) Detection and Validation of Objects from Sequential Images of a Camera
CN113033441B (en) Pedestrian collision early warning method based on wide-angle imaging
CN117333846A (en) Detection method and system based on sensor fusion and incremental learning in severe weather
CN111238490A (en) Visual positioning method and device and electronic equipment

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
RJ01 Rejection of invention patent application after publication
RJ01 Rejection of invention patent application after publication

Application publication date: 20170630