JP4062306B2 - Image processing apparatus and method - Google Patents

Description

  The present invention relates to an image processing apparatus and method for processing an image captured by a camera mounted on a vehicle.

  The following pedestrian detection device is known from Patent Document 1. In this pedestrian detection device, an image captured by a pair of stereo cameras arranged in parallel in the horizontal direction is subjected to image processing, pedestrians existing in the vicinity are detected, and a distance to the pedestrian is calculated.

JP 2002-24986 A

  However, when a stereo camera is used as in the conventional apparatus, a calibration work between the cameras is required, which causes a problem that the cost of the camera increases.

The present invention performs image processing on an image in front of the host vehicle imaged by an imaging unit, calculates the lateral movement speed and movement direction of an edge detected from each pixel on the image, and calculates the calculated image. Based on the speed , the pedestrian approaching from the side of the estimated traveling path of the own vehicle is detected, the height and the stride in the detected pedestrian image are calculated, and the calculated height in the pedestrian image is calculated. The pedestrian's stride in the real space is estimated based on the ratio between the stride and the ratio between the predetermined height and the stride that is set in advance based on the human walking posture. The distance from the own vehicle to the pedestrian is estimated based on the stride.

According to the present invention, the image in front of the host vehicle captured by the imaging unit is subjected to image processing , and the lateral movement speed and the movement direction of the edge detected from each pixel on the image are calculated as the image speed. Based on the obtained image speed , a pedestrian approaching from the side of the estimated traveling path of the own vehicle is detected. In addition, the pedestrian's stride in real space is estimated, and the distance from the vehicle to the pedestrian is estimated based on the estimated pedestrian's stride. This makes it possible to detect a pedestrian and estimate the distance to the pedestrian based on an image captured by one camera, which is advantageous in terms of cost.

-First embodiment-
FIG. 1 is a block diagram illustrating a configuration example of an embodiment of an image processing apparatus according to the first embodiment. The image processing apparatus 100 is mounted on a vehicle, and a camera 101 that captures the front of the vehicle, an image memory 102 that stores an image captured by the camera 101, and an image captured by the camera 101 and stored in the image memory 102 are described later. And a counter memory 104 for storing a count value of a pixel counter, which will be described later.

  The camera 101 is a high-speed camera having an image sensor such as a CCD or a CMOS, for example, and continuously images the front of the vehicle at a very small fixed time Δt interval, for example, 2 ms interval while the host vehicle is running. To the image memory 102. As shown in FIG. 2, the camera 101 is installed in front of the interior of the vehicle, its visual axis direction Z is directed to the front front direction of the vehicle, and the horizontal axis X of the imaging surface is parallel to the ground surface. The vertical axis Y of the imaging surface is set to be perpendicular to the ground surface. The continuous images captured by the camera 101 are output to the image memory 102 and stored. A specific example of an image captured by the camera 101 is shown in FIG. The captured image 3a includes a stationary object 3b constituting the background and a moving object moving in a direction crossing the front of the host vehicle from the side of the road, that is, a pedestrian approaching from the side of the estimated traveling path of the host vehicle. 3c exists.

  The control device 103 calculates the moving speed in the horizontal direction (horizontal direction) in the image of each object existing in the captured image 3a and the speed information having the speed direction indicating the moving direction as a speed component as the image speed. Based on the calculated image speed, a pedestrian 3c moving in a direction crossing the front of the host vehicle from the roadside is detected from objects existing in the captured image 3a. Then, the detected stride of the pedestrian 3c is estimated, and the distance between the pedestrian 3c and the host vehicle is calculated based on the estimated stride. Specifically, the processing is as follows.

  First, an image picked up by the camera 101 is taken from the image memory 102, and the picked-up picked-up image 3a is binarized using a predetermined threshold value, thereby extracting an edge of an object present in the image. Then, thinning processing is performed on each edge in the extracted image to accurately obtain the center of the edge. Then, the thinned edge is expanded so that the edge width becomes a constant width, for example, a width corresponding to three pixels. By normalizing the extracted edges in this way, an edge image in which each edge has a uniform width can be obtained.

  FIG. 4 is a diagram showing a specific example of each process performed to obtain the edge image by normalizing the extracted edge. That is, the thinning process is performed on the edge obtained by binarization shown in FIG. 4A to obtain the thinned edge shown in FIG. 4B. Then, the thinned edge is expanded to give the edge a certain width as shown in FIG.

  Thereafter, among the pixel counters stored in the counter memory 104, the count value of the pixel counter corresponding to the pixel having an edge in the current edge image is updated. The pixel counter is a counter corresponding to each pixel of the edge image, and 1 is added to the count value of the pixel counter corresponding to the pixel where the edge exists, and the count value of the pixel counter corresponding to the pixel where the edge does not exist is Initialize with 0. By performing the updating process of the counter value for each frame continuously captured by the camera 101, a pixel having a long edge existence time has a large count value, and a pixel having a short edge existence time has a small count value.

  In the edge image, the difference in the edge existence time in each pixel is calculated by taking the difference in the count value of the pixel counter corresponding to each pixel adjacent in the horizontal direction, and the edge moves one pixel. Get the time it takes. Then, by obtaining the reciprocal of this value, the lateral speed in the image space at each pixel can be calculated. The horizontal speed in the image space of each pixel corresponds to the horizontal movement speed of the edge included in each pixel. Further, the edge is moving in the speed direction of the calculated moving speed. Thereby, the moving direction and moving speed of the horizontal edge in each pixel on the image, that is, the image speed can be calculated.

  Based on the image speed calculated in the above-described processing, among the pixels having the horizontal speed component existing in the captured image 3a, an edge constituting a moving object moving in the direction crossing the front of the host vehicle from the roadside Extract only the pixels that contain. That is, in general, in the captured image 3a continuously captured when the host vehicle is traveling, as shown in FIG. 5, the stationary object 3b constituting the background in the image is the FOE ( (Focus of Expansion) 5a has a velocity direction radially toward the outside of the captured image 3a. That is, when the host vehicle is traveling straight, the stationary object 3b moves from the center of the image toward the outside of the image. On the other hand, the pedestrian 3c moving in the direction crossing the front of the vehicle from the side of the road has a moving speed in the horizontal direction from the outside of the image toward the center of the image.

  In consideration of this, by extracting only pixels having a moving speed in the horizontal direction from the captured image 3a toward the center of the image, only pixels including edges that move in the direction crossing the front of the vehicle from the roadside are extracted. Can be extracted. If the extracted pixels are present in a certain range at a predetermined density or more and the speed components of the pixels in the range are approximate, the pixels having the speed component present in the range are , It is determined that an individual moving object is configured. That is, it is determined that there is an individual moving object that can be identified, for example, a pedestrian 3c, within this range.

Next, the stride of the pedestrian 3c in the real space is estimated based on the aspect ratio (aspect ratio) on the image of the pedestrian 3c detected from the captured image 3a. That is, as shown in FIG. 6, the vertical length (height) H _D and the horizontal length (step length) W _D on the image of the pedestrian 3 c detected in the captured image 3 a are calculated, and W _D / a _{H D} is calculated as the aspect ratio. The calculated aspect ratio of the pedestrian 3c in the captured image 3a is the same in the real space. That is, the height of the H _R in the real _space, if the stride and W _R, holds the relationship expressed by a following equation (1).
H _D : W _D = H _R : W _R (1)

Also, in general, the human body height H _R and stride W _R in the real space, there is a linear relationship as shown in FIG. 7, it is known that the relationship expressed by a following expression (2) holds .
W _R (m) = a · H _R (m) + b (2)
The variables a and b vary depending on the age and sex of the pedestrian.
Therefore, stride _{W R} in the formula (1) and (2) from, the real space of the pedestrian 3c can be estimated by the following equation (3).
W _R = (− b · W _D / a) / (H _D −W _D / a) (3)

For example, in a general pedestrian and general adult in the real space, based on the known ratio between the height H _R and footstep W _R that is set based on the walking posture, in the formula (2), a = since 1, b = -0.9 and the following equation relational expression (4) is satisfied, estimated by the formula (3), stride length _{W R} in the real space of the pedestrian. 3c, by the following equation (5) Is possible.
_{W R (m) = H R} (m) -0.9 (m) ··· (4)
_{W R (m) = 0.9 ·} W D / (H D -W D) ··· (5)

The positional relationship in real space between the pedestrian 3c having the stride estimated by the equation (5) and the own vehicle is as shown in FIG. In this case, the estimated distance L between the own vehicle and the pedestrian 3c in the real space can be schematically represented as shown in FIG. 8B, where F is the focal length of the camera 101, It can be calculated by the following equation (6).
L = F · W _R / W _D (6)
As a result, the stride of the pedestrian 3c moving in the direction crossing the front of the host vehicle from the side of the road is estimated from among the objects present in the captured image 3a, and the pedestrian 3c and the self are determined based on the estimated stride. An estimated distance from the vehicle can be calculated.

  FIG. 9 is a flowchart illustrating processing of the image processing apparatus 100 according to the first embodiment. The processing shown in FIG. 9 is executed by the control device 103 when the ignition switch of the vehicle is turned on and the power of the image processing device 100 is turned on. In step S <b> 10, an image 3 a in front of the host vehicle captured by the camera 101 is captured from the image memory 102. Thereafter, the process proceeds to step S20, and as described above, edge extraction processing is performed on the image 3a to generate an edge image in which the contour of the object existing in the captured image 3a is extracted. Then, the image speed is calculated based on the count value of the pixel counter in each pixel.

Thereafter, the process proceeds to step S30, where only the pixels having a lateral movement speed toward the center of the image are extracted in the captured image 3a, the pedestrian 3c existing in the captured image 3a is detected, and the process proceeds to step S40. In step S40, the aspect ratio of the pedestrian 3c detected in the captured image 3a is calculated, and the process proceeds to step S50. Based on the calculated aspect ratio, the above-described equation (5) is used to calculate the pedestrian 3c in the real space. to estimate the stride _{W R.} Thereafter, the process proceeds to step S60.

In step S60, the pedestrian stride on the captured image 3a, footstep W _R of the pedestrian in the real _space, and on the basis of the focal length F of the camera 101, by the above-mentioned equation (6) and the vehicle pedestrian 3c and An estimated distance L is calculated. Then, it progresses to step S70, it is judged whether the ignition switch of the own vehicle was turned off, and when it judges that it is not turned off, it returns to step S10 and repeats a process. On the other hand, if it is determined that the ignition switch of the host vehicle has been turned off, the process is terminated.

According to the first embodiment described above, the following operational effects can be obtained.
(1) to calculate the aspect ratio of the pedestrian in the captured image 3a, calculated aspect ratio, and based on the relationship between the general pedestrian height and stride, the stride W _R of the pedestrian in the real space It was decided to estimate. Thus, a typical pedestrian may be height and stride by adding to become linear relationship to estimate a stride W _R of accurately pedestrians.
(2) estimated pedestrian stride W _R in the real space, footstep W D of the pedestrian in the captured image _3a, and on the basis of the focal length F of the camera 101, the estimated distance between the host vehicle and pedestrian L was to be calculated. This makes it possible to estimate the distance L between the host vehicle and the pedestrian without using a special measuring device for measuring the distance to the pedestrian.
(3) A moving object composed of pixels having a moving speed in the horizontal direction toward the center of the image in the captured image 3a is detected as a pedestrian 3c existing in the captured image 3a. Thus, in general, when the host vehicle is traveling straight, the background of a stationary object or the like has a speed component from the center of the image to the outside of the image, whereas the pedestrian approaching the host vehicle from the side of the road. In consideration of having a speed component from the outside of the image toward the center of the image, only a pedestrian moving in a direction crossing the front of the host vehicle from the roadside can be detected accurately.

-Second embodiment-
In the second embodiment, taking into account that an elderly person has an aspect ratio different from that of a general pedestrian such as the general adult described in the first embodiment, the stride of the pedestrian detected in the image. And the estimated distance from the vehicle to the pedestrian is calculated. 1 to 5 are the same as those in the first embodiment, and the description thereof is omitted.

As shown in FIG. 10 (a), elderly 10a, it generally adult relative stride _{W D2} than 10b is small in general. Therefore, the aspect ratio of the elderly 10a, that is, W _D2 / H _D2 is smaller than that of the general adult 10b. For this reason, when the aspect ratio of the pedestrian moving in the direction crossing the front of the vehicle from the side of the road detected from the captured image 3a is equal to or less than a predetermined value set in advance, the control device 103 It is determined that the pedestrian is an elderly person.

The aspect ratio of the elderly 10a as described above from different than the general adult 10b, as shown in FIG. 10 (b), the relationship between the height _{H R2} and stride _{W R2} in the real space of the elderly 10a is a different linear relationship 10c is a linear relationship 10d between height _{H R} and stride _{W R} in the general adult 10b. Therefore, when the pedestrian detected in the captured image 3a is an elderly person, walking in the real space according to a relational expression different from the expression (5) in the first embodiment, that is, the following expression (7): It is necessary to estimate the person's stride.
W _R (m) = a ′ · H _R (m) + b ′ (7)

For example, in the height _{H R2} and stride _{W R2} elderly 10a in the real space, in equation (7), a'= 1/ 2, b'= -0.5 and the following relational expression (8) since There are known to be established, the equation (3) in the first embodiment, footstep W _R2 in the real space of the elderly 10a can be estimated by the following equation (9).
W _R2 (m) = H _R2 (m) /2-0.5 (m) (8)
W _R2 (m) = W _D2 / (H _D2 −2 · W _D2 ) (9)

Thus, the estimated distance L between the host vehicle and the elderly 10a in the real space, based on the stride W _R2 elderly 10a calculated by the equation (9), wherein in the first embodiment (6) Therefore, it is computable by following Formula (10).
L = F · W _R2 / W _D2 (10)
Thereby, even when the pedestrian moving in the direction crossing the front of the host vehicle from the side of the road detected from the captured image 3a is an elderly person, the stride is estimated and the distance between the elderly person 10a and the host vehicle is estimated. An estimated distance can be calculated.

  FIG. 11 is a flowchart illustrating processing of the image processing apparatus 100 according to the second embodiment. The processing shown in FIG. 11 is executed by the control device 103 when the ignition switch of the vehicle is turned on and the image processing apparatus 100 is turned on. Note that the same step numbers are assigned to the processes common to those in FIG. 9 in the first embodiment, and differences will be mainly described.

In step S41, it is determined whether or not the calculated pedestrian aspect ratio is equal to or less than a predetermined value set in advance. If it is determined that the aspect ratio is greater than the predetermined value, the process proceeds to step S42. In step S42, it is determined that the detected pedestrian is a general adult, and the process proceeds to step S51, where the pedestrian is calculated according to the general adult stride estimation formula, that is, the formula (5) described above in the first embodiment. estimating a stride _{W R.} Thereafter, the process proceeds to step S60.

On the other hand, if it is determined that the aspect ratio is equal to or less than the predetermined value, the process proceeds to step S43. In step S43, it is determined that the detected pedestrian is an elderly person, the process proceeds to step S52, and the pedestrian's stride _WR2 is estimated by a step estimation formula for elderly persons, that is, equation (9). Thereafter, the process proceeds to step S60.

According to the second embodiment described above, the following effects can be obtained in addition to the functions and effects of the first embodiment.
(1) It is determined whether the pedestrian detected in the captured image is a general adult or an elderly person, and the step length estimation method is changed based on the determination result. Thus, an elderly person can estimate the stride with higher accuracy in consideration of the difference in walking posture from the general adult that the stride is smaller than that of the general adult.
(2) When the aspect ratio of the pedestrian detected in the captured image is equal to or less than a predetermined value set in advance, the pedestrian is determined to be an elderly person. Accordingly, it is possible to accurately determine whether the pedestrian is a general adult or an elderly person in consideration of the difference in walking posture between the elderly and the general adult.

-Modification-
Note that the image processing apparatus according to the above-described embodiment can be modified as follows.
(1) In the above-described first and second embodiments, the example in which the image processing apparatus 100 according to the present invention is mounted on a vehicle has been described. However, the present invention is not limited to this, and the image processing apparatus 100 may be mounted on another moving body. Good.

(2) In the first and second embodiments described above, after performing edge extraction processing on a captured image to obtain an edge image, the pixel counter of the pixel in which the edge exists is updated, The example in which the velocity component of the edge in each pixel on the image is calculated based on the count value of the pixel counter has been described. However, the present invention is not limited to this, and the optical flow may be calculated using, for example, a general method such as a gradient method or block matching.

(3) In the first and second embodiments described above, the pedestrian's stride is estimated based on the aspect ratio of the pedestrian and the linear relationship between the height and the stride. And the example which calculates the estimated distance between the own vehicle and a pedestrian based on the estimated stride was demonstrated. However, the present invention is not limited to this, and the pedestrian's height is estimated based on the pedestrian's aspect ratio and the linear relationship between the height and the stride, and the estimated distance between the vehicle and the pedestrian is calculated based on the estimated height. It may be calculated.

(4) In the second embodiment described above, the pedestrian detected in the captured image is determined to be either a general adult or an elderly person based on the aspect ratio, and the general adult and the elderly The stride length is estimated by applying a formula for estimating the stride length corresponding to each person. However, the present invention is not limited to this, and as a type of pedestrian, other types of pedestrians that can be determined based on differences in aspect ratio, such as children and men and women, are distinguished, and a calculation formula for stride estimation corresponding to each type is provided. You may make it estimate a stride by applying.

  Note that the present invention is not limited to the configurations in the above-described embodiments as long as the characteristic functions of the present invention are not impaired.

  The correspondence between the constituent elements of the claims and the embodiment will be described. The camera 101 corresponds to imaging means, and the control device 103 corresponds to speed information calculation means, pedestrian detection means, calculation means, stride estimation means, distance estimation means, and pedestrian determination means. This correspondence is an example, and the correspondence is different depending on the configuration of the embodiment.

It is a block diagram which shows the structural example of one Embodiment of an image processing apparatus. It is a figure which shows the example of installation to the vehicle of the camera 101. FIG. It is a figure which shows the specific example of the image imaged with the camera. It is a figure which shows the specific example of each process performed in order to normalize the extracted edge and to obtain an edge image. It is a figure which shows the specific example of the speed direction detected by each object in the captured image continuously imaged when the own vehicle is drive | working. It is a diagram illustrating the vertical length (height) H _D, and the lateral length (stride) W _D on pedestrian image detected in the captured image. It is a graph showing the linear relationship of the human vertical length (height) H _R and stride W _R in the real space. It is a figure which shows the specific example of the positional relationship on the real space of a pedestrian and the own vehicle. It is a flowchart figure which shows the process of the image processing apparatus 100 in 1st Embodiment. It is a figure showing the difference in the aspect ratio of the general adult and elderly person in 2nd Embodiment. It is a flowchart figure which shows the process of the image processing apparatus 100 in 2nd Embodiment.

Explanation of symbols

DESCRIPTION OF SYMBOLS 100 Image processing apparatus 101 Camera 102 Image memory 103 Control apparatus 104 Counter memory

Claims (3)

Imaging means for capturing an image in front of the host vehicle;
Image speed calculation means for performing image processing on an image captured by the imaging means , and calculating a lateral movement speed of an edge detected from each pixel on the image, and a movement direction as an image speed ;
Pedestrian detection means for detecting a pedestrian approaching from the side of the estimated traveling path of the host vehicle based on the image speed calculated by the image speed calculation means;
Calculating means for calculating the height and stride in the image of the pedestrian detected by the pedestrian detecting means;
Based on the ratio between the height and the stride in the image of the pedestrian calculated by the calculating means, and the ratio between the predetermined height and the stride that is preset based on the human walking posture, the pedestrian in the real space Stride estimation means for estimating stride,
An image processing apparatus comprising: a distance estimating unit that estimates a distance from the host vehicle to a pedestrian based on a pedestrian's stride in real space estimated by the stride estimating unit. The image processing apparatus according to claim 1.
Pedestrian determination means for determining the type of pedestrian detected by the pedestrian detection means based on the ratio of the height and the stride in the image of the pedestrian calculated by the calculation means,
The image processing apparatus according to claim 1, wherein the stride estimation means estimates a stride of a pedestrian by an estimation method according to a pedestrian type determined by the pedestrian determination means. Image processing is performed on the image in front of the host vehicle captured by the imaging unit, and the lateral movement speed of the edge detected from each pixel on the image and the movement direction are calculated as the image speed ,
Based on the calculated image speed , detect a pedestrian approaching from the side of the estimated traveling path of the vehicle,
Calculate the height and stride in the detected pedestrian image,
Estimate the pedestrian's stride in real space based on the ratio between the height and the stride in the calculated pedestrian image and the ratio between the predetermined stature and the stride set in advance based on the human walking posture. ,
An image processing method characterized by estimating a distance from a host vehicle to a pedestrian based on a pedestrian's stride in an estimated real space.

Images