JP2008242571A - Object detection device - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an object detection device for finding out a pedestrian who runs out of the shade of a static vehicle in an early stage. <P>SOLUTION: This object detection device is provided with: a camera which photographs an image in the periphery of its own vehicle within a predetermined image pickup range; a static object detection means for detecting a static vehicle 30 existing in the image pickup range based on the image photographed by the camera and the speed of its own vehicle; a processing region setting means for setting a processing region 35 which is smaller than the image pickup region at the edge of the side close to its own vehicle in the static vehicle 30; and a pedestrian detection means for judging whether or not the object picked up in the processing region 35 is a pedestrian. <P>COPYRIGHT: (C)2009,JPO&INPIT

Description

  The present invention relates to an object detection device.

A method has been proposed in which an image around a vehicle is taken with a camera and a pedestrian is detected by matching processing or the like (see, for example, Patent Document 1).
When detecting a pedestrian by image processing, as a method for improving detection performance, a pedestrian candidate is detected in each frame, the pedestrian candidate is tracked (tracked) over several frames, and is continuously the same for a predetermined number of frames or more. There is known a method of finally detecting a pedestrian candidate as a pedestrian when a pedestrian candidate is detected.
Japanese Patent Laid-Open No. 2003-9140

However, since each frame detects a pedestrian candidate when the pedestrian's entire body is visible, the method of tracking the pedestrian candidate over several frames detects the pedestrian for a predetermined frame after the pedestrian's entire body is visible. Will not be. Therefore, when there is a pedestrian who tries to jump out from the shadow of a stationary vehicle, there is a problem that even if a part of the body is visible, tracking starts after the whole body is visible, and detection is delayed. .
In the method of judging a pedestrian from information on a part of the body, the discrimination ability is lowered as compared with the discrimination method using the whole body.

  Therefore, an object of the present invention is to provide an object detection device that can quickly find a pedestrian who is about to jump out of the shadow of an object.

  In order to solve the above-mentioned problem, the invention according to claim 1 is directed to an imaging unit (for example, the camera 5 in the embodiment) that captures an image around the own vehicle (for example, the own vehicle 2 in the embodiment) within a predetermined imaging range. ) And an image captured by the imaging unit and the speed of the host vehicle, a stationary object detection unit (for example, the embodiment) that detects a stationary object (for example, the stationary vehicle 30 in the embodiment) that exists in the imaging range. Stationary object detecting means 12) and processing area setting means (for example, processing area 35 in the embodiment) smaller than the imaging range at the end of the stationary object closer to the vehicle. , Processing area setting means 14) in the embodiment, and pedestrian detection for determining whether the object imaged in the processing area is a pedestrian (for example, the pedestrian 40 in the embodiment) Stage (e.g., a pedestrian detecting unit 16 in the embodiment) and, characterized by comprising a.

  According to a second aspect of the present invention, the pedestrian detection unit includes a complementation availability determination unit (for example, a complementation availability determination unit in the embodiment) that determines whether an object photographed in the processing area can be complemented as a pedestrian. It is characterized by that.

  According to the first aspect of the present invention, since the processing area is set at the end of the stationary object that is closer to the own vehicle, it is possible to reliably find a pedestrian who is about to jump out of the stationary object in the course of the own vehicle. Can do. In addition, since a processing area smaller than the imaging range is set, the calculation load can be reduced and a pedestrian trying to jump out can be found early.

  According to the invention which concerns on Claim 2, even when a part of pedestrian's body is hidden behind a stationary object, it becomes possible to complement the object in the image as a pedestrian. Thereby, it becomes possible to start tracking of a pedestrian candidate at an early stage, and a pedestrian who tries to jump out from the shadow of an object can be discovered early.

Hereinafter, embodiments of the present invention will be described with reference to the drawings.
(Object detection device)
FIG. 1 is a schematic configuration diagram of an object detection apparatus according to the present embodiment. The object detection device 1 according to the present embodiment is based on a camera (imaging means) 5 that captures an image around the host vehicle 2 within a predetermined imaging range, an image captured by the camera 5, and the speed of the host vehicle 2. A stationary object detection means 12 for detecting a stationary vehicle (stationary object) existing in the imaging range; a processing area setting means 14 for setting a processing area smaller than the imaging range at an end of the stationary vehicle on the side close to the own vehicle 2; Pedestrian detection means 16 for determining whether an object photographed in the processing area is a pedestrian. The stationary object detection means 12, the processing area setting means 14, and the pedestrian detection means 16 are constructed in a computer 10 mounted on the host vehicle 2.

  The object detection device 1 includes a camera 5 (stereo cameras 5L and 5R) that captures an image in front of the vehicle 2 within a predetermined imaging range. The stereo cameras 5L and 5R are composed of a CCD camera or the like, and are attached to the ceiling portion inside the windshield. The left camera 5L is disposed in the vicinity of the rear-view mirror at the center in the vehicle width direction, and the right camera 5R is disposed at the right end in the vehicle width direction. The right camera 5R may be disposed near the rear-view mirror in the center in the vehicle width direction, and the left camera 5L may be disposed at the left end in the vehicle width direction. The distance between the left camera 5L and the right camera 5R is set to about 30 to 40 cm.

  Images taken by the stereo cameras 5L and 5R are input to the computer 10 mounted on the vehicle. The computer 10 includes a stationary object detection unit 12 that detects a stationary vehicle existing in the imaging range based on an image captured by the camera 5 and the speed of the host vehicle 2. The stationary object detection unit 12 detects a vehicle by a template matching method or the like after binarizing the captured image of the camera 5. Further, the movement vector of the detected vehicle is extracted, and it is determined whether or not the vehicle is a stationary vehicle.

  Further, in order to detect a pedestrian who is about to jump out from behind a stationary vehicle, a processing region setting means 14 is provided for setting a processing region in the vicinity of the detected stationary vehicle. The processing area setting means 14 sets a processing area smaller than the imaging range at the end of the stationary vehicle on the side close to the own vehicle 2. That is, the processing area 35 is set at the right end of the stationary vehicle 30 existing on the left side (traveling lane) of the own vehicle and at the left end of the stationary vehicle existing on the right side (opposing lane) of the own vehicle. To do. The processing area is rectangular, and its size is set equal to the extension and width of the pedestrian.

  Moreover, the pedestrian detection means 16 which judges whether the object image | photographed within the process area | region is a pedestrian is provided. The pedestrian detection unit 16 includes a complementation possibility determination unit 17 that determines whether an object photographed in the processing area can be complemented as a pedestrian. Even when a part of the pedestrian's body is hidden behind a stationary vehicle, the complementation propriety judging means 17 complements the part to generate an image of the whole body of the pedestrian. And when such a complement is possible, it can be estimated that a pedestrian candidate exists in a process area | region.

  Moreover, the pedestrian detection means 16 is provided with the pedestrian identification means 18 which judges whether the object can be identified as a pedestrian, when the object in a process area | region can be complemented. When a circumscribed rectangle equivalent to the circumscribed rectangle for the pedestrian is set for the object, the pedestrian identifying unit 18 extracts the object as a pedestrian. It also detects the uneven shape on the surface of pedestrian candidates, and detects pedestrians with relatively large unevenness on the surface, wheels, doors, windows, street trees, etc. with relatively small unevenness on the surface. Identify. When the pedestrian can be identified, a pedestrian candidate can be detected in the image frame.

  Furthermore, the pedestrian detection means 16 tracks (tracks) a pedestrian candidate over a plurality of temporally continuous image frames. When the same pedestrian candidate is detected continuously for a predetermined number of frames or more, and the whole body of the pedestrian candidate is detected in the latest frame, it is finally determined whether it can be identified as a pedestrian. If it can be identified as a pedestrian, the pedestrian candidate is finally detected as a pedestrian.

(Object detection method)
Next, an object detection method using the object detection device will be described.
FIG. 2 is a flowchart of the object detection method. First, an image around the host vehicle 2 is taken by the camera 5 within a predetermined imaging range (S10). FIG. 3 is an example of a photographed image.
Next, the stationary object detection unit 12 determines whether there is a parked vehicle 30 in the imaging range (S12). The determination of the presence or absence of the vehicle 30 can be performed by, for example, a template matching method. The template matching method is a method for detecting an object having a predetermined shape by performing comparison while moving a pattern (template) having a predetermined shape on an image. Various vehicle shapes serving as templates are registered in the stationary object detection unit 12 after binarization processing in advance.

  As a specific procedure of the template matching method, first, the image is binarized. Next, a part of the binarized image data and the template are compared for each pixel, and when the number of matched pixels is equal to or greater than a predetermined number, it is determined that the matching has been performed. Such matching processing is performed for the entire image data and for all templates. Note that matching processing is performed in the same manner for an enlarged / reduced or inclined template of an existing template. If the matching process is successful, the template-shaped vehicle 30 is detected.

  Next, it is determined whether the vehicle 30 is a stationary vehicle based on the detected relative speed between the vehicle 30 and the host vehicle. The determination of whether or not the vehicle is a stationary vehicle is performed by extracting the detected movement vector of the vehicle 30. In general, an optical flow is known as one of methods for extracting a motion of an object from a moving image. The optical flow is a movement vector 32 of the object 30 and is calculated from the amount of movement of a point having the same level of luminance on the assumption that the luminance change of the same object is small between two temporally continuous images. .

In general, the host vehicle 2 travels toward a vanishing point (Focus of Expansion; hereinafter referred to as “FOE”). In this case, the optical flow 92 of the surrounding stationary object (for example, the roadside tree 90) is generated so as to spring out from the FOE. Therefore, when the optical flow calculated for the vehicle 30 is generated so as to spring out of the FOE, it can be determined that the vehicle 30 is a stationary vehicle.
Note that the stationary vehicle 30 is detected for each of the images taken by the stereo cameras 5R and 5L. When it is determined in S12 that no stationary vehicle exists (when the determination is No), the process returns to S10, a new image is captured by the camera 5, and the stationary vehicle detection process is repeated.

  Next, the processing area 35 for performing a pedestrian detection process is set by the processing area setting means 14 (S14). In a pedestrian detection process to be described later, a pedestrian who is about to jump out of the stationary vehicle 30 into the course of the own vehicle is detected. Therefore, a processing area 35 for pedestrian detection processing is set at the end of the stationary vehicle 30 on the side close to the own vehicle. That is, the processing area 35 is set at the right end of the stationary vehicle 30 existing on the left side (traveling lane) of the own vehicle and at the left end of the stationary vehicle existing on the right side (opposing lane) of the own vehicle. To do. Thus, by setting the processing area 35 smaller than the imaging range, it is possible to efficiently detect a pedestrian to be monitored.

The processing area 35 has a rectangular shape and is set to a height and width equivalent to those of a pedestrian. In the image, the lower end coordinate of the pedestrian 40 that is behind the stationary vehicle is substantially equal to the lower end coordinate v _b of the stationary vehicle, so the lower end coordinate −v _b of the stationary vehicle is set as the lower end coordinate of the processing area 35. Further, in addition to the upper end coordinate v _t of the processing area 35, the left end coordinate a and the right end coordinate b (width width between both) are set as follows.

FIG. 4A is an explanatory diagram of a method for setting the upper end coordinates of the processing region. Let h be the height of the camera mounted on the vehicle relative to the road surface. Let L be the distance from the camera to the pedestrian 40 behind the stationary vehicle. Then, if the focal length of the camera is f, the following relationship is obtained from the similar relationship between the lower end coordinates (f, −v _b ) of the pedestrian on the image and the lower end coordinates (L, −h) of the actual pedestrian 40. The formula holds.

これをＬについて解けば、次式が得られる。

Solving this for L yields:

Furthermore, if the height of the pedestrian is p (for example, 1.8 m), the upper end coordinates (f, v _t ) of the pedestrian on the image and the upper end coordinates (L, _ph ) of the actual pedestrian 40 From the similarity relationship, the following equation holds.

これをｖ_ｔについて解き、さらに数式２を代入すれば、次式が得られる。

Solving this with respect to v _t and further substituting Equation 2 yields:

  FIG. 4B is an explanatory diagram of a method for setting the upper end coordinates of the processing area 35 when the host vehicle is pitching. Assuming that the pitching angle θ of the host vehicle is sufficiently small, the following equation is obtained by the same procedure as that in the case where the pitching is not performed.

以上により、処理領域３５の上端座標ｖ_ｔが設定される。

As described above, the upper end coordinate v _t of the processing area 35 is set.

  On the other hand, the following equation is established from the similar relationship between the width width of the pedestrian on the image and the width W of the actual pedestrian 40.

これをwidthについて解き、さらに数式２を代入すれば、次式が得られる。

Solving this for width and further substituting Equation 2 yields:

  Returning to FIG. 3, the position of the processing region 35 in the width direction is set around the end of the stationary vehicle 30 on the side close to the own vehicle. That is, if the U direction (horizontal direction) coordinate of the end of the stationary vehicle 30 near the host vehicle is c, the left end coordinate a and the right end coordinate b of the processing region 35 are expressed by the following equations.

以上により、処理領域３５の左端座標ａおよび右端座標ｂが設定される。

As described above, the left end coordinate a and the right end coordinate b of the processing region 35 are set.

  Next, whether or not the image in the processing area 35 can be complemented by the pedestrian sample image is judged by the complementation possibility judging means 17 (S16). As described above, since the end of the stationary vehicle 30 is set in the processing area 35, there is a possibility that a part of the pedestrian's body is hidden behind the stationary vehicle 30 in the image in the processing area 35. is there. Therefore, it is determined whether or not a part of the image can be complemented to generate an image of the whole pedestrian.

  5 and 6 are explanatory diagrams of a method for complementing images in the processing area. First, an image in the processing area shown in FIG. 5A is set as an input image. In this input image, the lower body of the pedestrian 40 is hidden behind the stationary vehicle 30. Next, as shown in FIG. 5B, principal component analysis (PCA) is performed using a plurality of pedestrian sample images. That is, an image close to the input image is formed by linear combination of the base images. A plurality of pedestrian sample images are registered in advance in the complementation possibility determining means 17. Thereby, as shown in FIG.5 (c), the prediction image of the pedestrian who exists in a process area | region is produced | generated.

  Next, as shown in FIG. 5D, the input image and the predicted image are divided into small areas, and the corresponding small areas are compared with each other. Then, when the number of sub-regions that substantially match exceeds a predetermined value, it is determined that complementation is possible. That is, even if the pedestrian's whole body does not appear in the input image, it is determined that the hidden part of the pedestrian can be complemented by linear combination of the pedestrian sample images. In this case, the hidden part of the input image is complemented (S18). Specifically, as shown in FIG. 5E, a portion where the input image does not resemble the predicted image (substantially does not match) is brought closer to the predicted image (replaced with the predicted image). Then, the above process is repeated using the generated image as a new input image.

  On the other hand, there is no pedestrian in the input image shown in FIG. Even in this case, as shown in FIG. 6B, an image close to the input image is formed by linear combination of the base images. Thereby, the prediction image shown in FIG.6 (c) is produced | generated. Next, as shown in FIG. 6D, the input image and the predicted image are divided into small areas, and the corresponding small areas are compared with each other. In this case, since the number of small regions that substantially match does not exceed a predetermined value, it is determined that complementation is impossible.

  Next, the pedestrian identification unit 18 determines whether the complemented object in the image can be identified as a pedestrian (S20). As a specific method thereof, it is possible to employ a technique described in JP-A-2005-284799. Here, recognition processing such as feature amount calculation and shape discrimination using a pedestrian as a detection target is performed on the complemented image, and pedestrian candidates are extracted. In this recognition process, for example, when a circumscribed rectangle equivalent to a circumscribed rectangle for a pedestrian is set, the object is extracted as a pedestrian candidate.

  Furthermore, the relative distance from the own vehicle is calculated for a plurality of appropriate positions on the surface of the pedestrian. The relative distance from the own vehicle may be obtained by triangulation from the images of the stereo cameras 5R and 5L, or may be measured by mounting a radar on the own vehicle. Based on the relative distance calculated here, an uneven shape (for example, an uneven amount) on the surface of the pedestrian is detected. And it is determined whether the detected uneven | corrugated amount is larger than predetermined value. Thereby, a pedestrian with a relatively large unevenness on the surface and a wheel, a door, a window, a roadside tree, etc. with a relatively small unevenness on the surface are identified. That is, for pedestrians facing the front of the vehicle and pedestrians facing sideways, the relative distance to each part of the body such as the left and right arms, legs, and head changes relatively greatly. The amount of unevenness in the circumscribed rectangle set as described above becomes relatively large. On the other hand, in the circumscribed rectangle set for pedestrian candidates such as wheels, doors, windows, roadside trees, etc., the amount of unevenness is relatively small, and a flatter surface than the human body is detected. Become. Therefore, it is possible to determine whether or not the pedestrian can be identified by detecting the uneven shape on the surface of the pedestrian. And when it can identify as a pedestrian, a pedestrian candidate is detected in the said image frame.

If it is not identified as a pedestrian in S20, a new image is taken by the camera 5 in S10, and the above-described processing is repeated.
On the other hand, when a pedestrian is identified in S20, the pedestrian candidate is tracked (S22). In other words, the pedestrian candidate is tracked by determining whether the pedestrian candidates extracted in temporally continuous image frames are the same pedestrian candidates.

  By the way, in S16, when the pedestrian's whole body did not appear in the image in the processing region, it was determined whether or not the hidden part of the pedestrian could be complemented. When no pedestrians appear in the processing area (see FIG. 6), it is determined that complementation is impossible because there is no hidden part. Here, even when the pedestrian jumps out from the shadow of the stationary vehicle 30 and the pedestrian's whole body appears in the processing area, there is no hidden portion, and thus complementation is impossible. That is, when the pedestrian's whole body appears in the image within the processing region, it cannot be complemented as in the case where the pedestrian does not appear at all.

In this case, the process proceeds to S24, and finally it is determined whether or not the pedestrian can be identified from the whole body image that is not complemented. This determination can be made in the same manner as the pedestrian identification determination in S20. If no pedestrian appears in the image within the processing area, the determination in S24 is No, and the process returns to S10 and the above process is repeated.
On the other hand, if the pedestrian's whole body appears in the image within the processing area, the determination in S24 is Yes. In this case, the pedestrian candidate is finally detected as a pedestrian (S26).

  As described in detail above, the object detection device 1 according to the present embodiment includes the camera 5 that captures an image around the vehicle 2 within a predetermined imaging range, the image captured by the camera 5, and the speed of the vehicle 2. Based on the stationary object detecting means 12 for detecting the stationary vehicle 30 existing in the imaging range, and the processing area setting for setting the processing area 35 smaller than the imaging range at the end of the stationary vehicle 30 on the side close to the own vehicle 2 It is characterized by comprising means 14 and pedestrian detection means 16 for judging whether the object photographed in the processing area 35 is a pedestrian 40 or not.

  According to this configuration, since the processing area 35 is set at the end of the stationary vehicle 30 near the own vehicle 2, the pedestrian 40 who is about to jump out of the stationary vehicle 30 into the course of the own vehicle 2 is surely found. can do. Further, since the processing area 35 smaller than the imaging range is set, the pedestrian 40 who is about to jump out can be found at an early stage.

Further, the pedestrian detection means 16 includes a complementation possibility judgment means 17 that judges whether an object photographed in the processing area 35 can be complemented as a pedestrian.
According to this configuration, even when a part of the body of the pedestrian 40 is hidden behind the stationary vehicle 30, an object in the image can be complemented as a pedestrian. Thereby, it becomes possible to start tracking of a pedestrian candidate at an early stage, and a pedestrian who is about to jump out from behind a stationary vehicle can be discovered at an early stage.

The pedestrian detection means 16 is characterized by determining that a pedestrian exists when a pedestrian candidate is detected in a processing region 35 of a plurality of temporally continuous images.
By tracking pedestrian candidates in this way, pedestrian discovery accuracy can be improved.

Moreover, the pedestrian detection means 16 is characterized by determining that the pedestrian 40 exists when the whole body of the pedestrian 40 is detected in the latest image of the processing area 35.
Thus, the detection accuracy of the pedestrian 40 can be improved by determining that the pedestrian exists after the whole body of the pedestrian 40 is finally detected.

The present invention is not limited to the embodiment described above.
For example, in the embodiment, a pedestrian who tries to jump out from behind a stationary vehicle is detected, but a stationary object other than the stationary vehicle may be recognized, and a pedestrian trying to jump out from the shade may be detected. Further, as a stationary vehicle detection method, the template matching method is employed in the embodiment, but other methods can also be employed. In addition, the setting range of the processing area in the embodiment is an example, and a different range may be set as the processing area.

It is a schematic block diagram of the object detection apparatus which concerns on embodiment. It is a flowchart of an object detection method. It is an example of the image | photographed image. It is explanatory drawing of the setting method of the upper end coordinate of a process area. It is explanatory drawing of the complement method of the image in a process area. It is explanatory drawing of the complement method of the image in a process area.

Explanation of symbols

  DESCRIPTION OF SYMBOLS 1 ... Object detection apparatus 2 ... Vehicle 5 ... Camera (imaging means) 12 ... Stationary object detection means 14 ... Processing area setting means 16 ... Pedestrian detection means 30 ... Stationary vehicle (stationary object) 35 ... Processing area 40 ... Pedestrian

Claims (2)

Imaging means for capturing an image of the periphery of the vehicle within a predetermined imaging range;
A stationary object detection means for detecting a stationary object present in the imaging range based on an image captured by the imaging means and the speed of the host vehicle;
Processing area setting means for setting a processing area smaller than the imaging range at an end of the stationary object on the side close to the own vehicle;
Pedestrian detection means for determining whether an object photographed in the processing area is a pedestrian;
An object detection apparatus comprising:   The object detection apparatus according to claim 1, wherein the pedestrian detection unit includes a complementation possibility determination unit that determines whether an object photographed in the processing region can be complemented as a pedestrian.

Images