JP4127255B2 - Pop-up object detection device - Google Patents

Description

  The present invention relates to an apparatus for detecting an object that pops out ahead of a host vehicle.

  There is known a method of detecting an object such as a pedestrian that jumps out ahead of the host vehicle using stereo images captured by two cameras (Patent Document 1). This detects an object popping out ahead of the host vehicle based on the distance distribution data calculated from the stereo image and the movement of the object on the screen, and issues a warning to the driver.

JP 9-226490 A

  In the method disclosed in Patent Document 1, two cameras are required to acquire a stereo image, which is disadvantageous in terms of cost. Accordingly, there is a need for an apparatus that can detect an object that jumps forward of the host vehicle from an image captured by one camera.

The pop-out object detection device according to the present invention is based on an imaging unit that acquires a captured image of a destination of a host vehicle, a movement amount detection unit that detects a movement amount of the host vehicle, and a movement amount detected by the movement amount detection unit. The detection area setting means for setting the detection area virtually fixed in the real space in the captured image, and the optical flow indicating the velocity on the image of the object in the detection area set by the detection area setting means is calculated. An optical flow calculating means that detects and a pop-out object detecting means that detects a pop-out object that jumps out to the destination of the own vehicle based on the optical flow, and the detection area setting means is a boundary portion of the road on which the own vehicle is traveling A virtual set position in each real space is a point that is a predetermined distance away from the point and a point that is closer to the road by a predetermined lateral distance than that point. Two detection areas are set, and the optical flow calculation means includes a first optical flow indicating a speed on the image of the object in the first detection area set outside the road among the two detection areas; The pop-out object detection means calculates the second optical flow indicating the speed on the image of the object in the second detection area set inside the road from the first detection area. The difference between the first movement speed of the object calculated based on the flow in the real space and the second movement speed of the object calculated based on the second optical flow is less than or equal to a predetermined value. In some cases, the object is detected as a pop-out object.

  According to the present invention, a captured image of the destination of the host vehicle is acquired, a detection area virtually fixed in real space is set in the captured image, and an optical flow in the set detection area is calculated. Then, a pop-out object is detected based on the optical flow. That is, a point that is a predetermined distance away from the boundary portion of the road on which the vehicle is traveling and a point that is closer to the road by a predetermined lateral distance than the point are virtual set positions in the respective real spaces. A first optical flow indicating the speed on the image of the object in the first detection area set outside the road among the two detection areas, and the first detection area The second optical flow indicating the speed on the image of the object in the second detection region set on the inner side of the road, and the real space of the object calculated based on the first optical flow If the difference between the first moving speed above and the second moving speed of the object calculated based on the second optical flow in the real space is equal to or less than a predetermined value, the object is It was possible to detect Te. Since it did in this way, the object which jumps out ahead of the own vehicle can be detected with the picked-up image of one camera.

  FIG. 1 shows the configuration of an object detection apparatus according to an embodiment of the present invention. This object detection device is mounted on a vehicle and used to detect an object that jumps out ahead of the host vehicle based on a captured image captured by a single camera, and issues a warning to the driver. The vehicle detection device includes a camera 1, an image memory 2, a wheel speed sensor 3, a steering angle sensor 4, a microcomputer 5, and an alarm device 6.

  The camera 1 is installed at the front of the host vehicle so as to capture the front of the host vehicle, and outputs the captured image to the image memory 2 as an image signal. As the camera 1, for example, a CCD camera or the like is used. The image memory 2 is a frame memory for storing and holding an image signal input from the camera 1 as image data. The image data stored in the image memory 2 is output to the microcomputer 5 in units of image frames under the control of the microcomputer 5.

  The wheel speed sensor 3 is provided near each tire of the host vehicle, and measures the wheel speed by detecting the rotation of the tire of the host vehicle. The steering angle sensor 4 measures the steering angle by detecting the operation state of the steering of the host vehicle. Wheel speed data and steering angle data measured by these sensors are output to the microcomputer 5.

  The microcomputer 5 reads image data from the image memory 2 to read a captured image in front of the host vehicle captured by the camera 1. Further, the movement amount of the host vehicle is detected based on the wheel speed data and the steering angle data respectively measured by the wheel speed sensor 3 and the steering angle sensor 4. Then, based on the captured image and the amount of movement of the host vehicle, a process as described below is executed to detect an object such as a pedestrian jumping out in front of the host vehicle. The object detected in this way is hereinafter referred to as a pop-out object. When a pop-out object is detected in the microcomputer 5, a signal for operating the alarm device 6 is output from the microcomputer 5, and an alarm sound is output in the alarm device 6. In this way, the presence of the object popping out is notified to the driver of the host vehicle.

  Processing contents when the microcomputer 5 detects a pop-out object will be described. FIG. 2 shows a state when an intruder 200 entering the road from the side of the road on which the host vehicle is traveling is detected as a pop-out object from the captured image acquired by the camera 1. When a captured image is read by the microcomputer 5, a lane mark (white line) indicating a road boundary is first extracted from the captured image. The lane mark is extracted from the captured image by extracting a vertical edge from the captured image using an image filter called a Sobel filter and performing a process called Hough transform on the edge to obtain a linear shape. be able to. Note that such a lane mark extraction method is a well-known image processing method, and a detailed description thereof will be omitted here.

  When the lane mark is extracted from the captured image, a detection area for detecting an object from the captured image is set in the captured image. At this time, the setting position of each detection area is determined based on the position of the extracted lane mark, and the detection area is arranged at predetermined intervals along the lane mark so that it follows the road on which the host vehicle is traveling. The detection area is set at predetermined intervals. Further, at this time, two detection areas are arranged side by side at predetermined horizontal intervals. Thus, two rows of interdigital detection areas are set along the road, such as an outer detection area indicated by reference numerals 10 to 15 in FIG. 2 and an inner detection area indicated by reference numerals 20 to 25.

  As will be described later, when an object detected in the outer detection areas 10 to 15 moves toward the road, the object is further detected in the inner detection areas 20 to 25. Then, by comparing the moving speeds in the real space calculated in the outer and inner detection areas, it is determined whether or not the object is a pop-out object. Therefore, in the following description, the outer detection areas 10 to 15 are simply referred to as detection areas, and the inner detection areas 20 to 25 are referred to as determination areas.

  The positions of the detection areas 10 to 15 and the determination areas 20 to 25 set here are virtually fixed in real space. Therefore, when the host vehicle moves, the positions of the detection areas and the determination areas in the captured image change according to the movement amount. Note that the movement amount of the host vehicle is detected based on the measurement results of the wheel side data and the steering angle data as described above.

  Each of the detection area and the determination area is set so that the distance in the front-rear direction and the left-right direction is constant within a predetermined distance range ahead of the host vehicle in real space. When either the detection area or the determination area is outside the range of the captured image due to the movement of the host vehicle, the detection area or the determination area is deleted, and a new detection area or A determination area is set. In other words, when the host vehicle moves forward, the detection area and the determination area set at positions close to the own vehicle are sequentially moved off the screen and deleted, and new detection areas and determination areas are set one after another. Is done.

  A specific setting example of the detection areas 10 to 15 and the determination areas 20 to 25 will be described. For example, with reference to the position of the host vehicle in the real space, a point that is 5 m away from the vehicle in the direction of travel of the host vehicle and further 1 m away from the lane mark on the opposite side of the road is a virtual space in the real space of the detection area 15 Set position. A point closer to the road than the detection area 15 by a predetermined lateral interval (for example, 30 cm) is set as a virtual set position in the real space of the determination area 25. Then, points at fixed intervals (for example, 1 m) from the virtual setting positions of the detection area 15 and the determination area 25 toward the traveling direction of the host vehicle, respectively, are detected as virtual areas of the detection areas 10 to 14 and the determination areas 20 to 24. Set the correct position. Thus, the setting positions of the detection areas and the determination areas in the captured image are determined in correspondence with the virtual setting positions set in the real space.

  As described above, when the detection areas 10 to 15 and the determination areas 20 to 25 that are arranged at equal intervals along the road are set, the optical flow in each of the detection areas 10 to 15 is calculated next. . This optical flow indicates the speed on the image of the object in each detection region. As is well known, the optical flow can be calculated by extracting a portion showing the same object between image frames and obtaining a position change amount in the captured image of the portion.

  In this way, by calculating the optical flow in each of the detection regions 10 to 15, an object moving toward the road ahead of the host vehicle can be detected from the captured image. When the direction of the optical flow calculated in any of the detection areas 10 to 15 is directed to the road direction, that is, the inside of the captured image, the object in the detection area moves toward the road ahead of the host vehicle. Can be judged. In the example of FIG. 2, the optical flow of the intruder 200 is calculated as indicated by the arrow 201 in the detection area 13. Since the optical flow 201 is directed toward the road, it is determined that the intruder 200 is moving toward the road ahead of the host vehicle.

  However, the positions of the objects detected in the detection areas 10 to 15 in the real space do not always coincide with the virtual set positions of the detection areas in the real space, and will be described later with reference to FIG. Sometimes you are away from the road. For this reason, when all the objects moving in the direction of the road as described above are detected as pop-out objects, even objects at positions away from the road are erroneously detected as pop-out objects.

  Therefore, in the present invention, when an object moving in the direction of the road is detected in any of the detection areas 10 to 15, any of the determination areas 20 to 25 set corresponding to the detection area is also used. The same object is detected, and the optical flow in the determination region is calculated. Then, based on the optical flows calculated respectively in the detection area and the determination area, the moving speed of the object in the real space is calculated for each of the detection area and the determination area. The moving speeds are compared, and if the difference is less than or equal to a predetermined value, the detected object is detected as a pop-out object. A method for calculating the moving speed in the real space based on the optical flow will be described later.

  In the example of FIG. 2, the intruder 200 detected in the detection area 13 is detected again in the determination area 23 set side by side, and the optical flow of the intruder 200 in the determination area 23 is calculated. If the difference in moving speed of the intruder 200 in the real space calculated based on the optical flow and the optical flow 201 calculated in the detection area 13 is equal to or less than a predetermined value, the intruder 200 is regarded as a pop-up object. To detect. The pop-out object is detected by the processing as described above.

  Next, the relationship between the virtual setting position of the detection area in the real space and the presence position of the detection target object will be described. FIG. 3A is a diagram showing the positional relationship between the host vehicle and the intruder who is the detection target object in real space, and the host vehicle positions at time T1 and time T2 are denoted by reference numerals 101 and 102, respectively. The traveling speed of the host vehicle during this time is expressed as Vc. In addition, reference numerals 201 and 202 denote the positions of the intruders at time T1 and time T2, respectively. At this time, the moving speed of the intruder is assumed to be constant and is expressed as V0.

  As described with reference to FIG. 2, the detection area and the determination area are set in a state of being virtually fixed in real space along the road. At this time, assuming that the host vehicle is traveling on the center of the road, the lateral positions of the detection area and the determination area with respect to the center line of the road are represented as W1 and W2, respectively, as shown in FIG. The values of W1 and W2 are predetermined as setting conditions for the detection area and the determination area.

  (B) shows a captured image in front of the host vehicle captured at the host vehicle position 101 in (a) at time T1. In this captured image, an intruder present at the position 201 of (a) (this is represented as the intruder 201 for convenience) is captured, and this intruder 201 is detected in the detection region 11. At this time, in the real space, it is determined that the intruder 201 exists at the position where the detection area 11 is virtually set, that is, the detection position 201A shown in FIG. Therefore, in order to avoid erroneously detecting the intruder 201 as a jumping object in such a case, the optical flows calculated in the detection area and the determination area are compared as described below.

The optical flow of the intruder 201 calculated in the detection area 11 from the captured image of (b) is represented as F1. As shown in (a), when the moving speed of the intruder 201 when viewing the intruder 201 from the own vehicle position 101 is an angular flow _Fθ1 , this is expressed by the following equation (1) using the optical flow F1. Can be represented. It should be noted that the constant α in the equation (1) is determined in advance according to the imaging surface size and focal length of the camera 1.
F _θ1 = α · F1 (1)
Where α is a constant

Also, assuming that the distance from the own vehicle position 101 to the detected position 201A in the traveling direction of the own vehicle is the detected distance Di1, the moving speed Vi1 in the real space when the intruder 201 exists at the detected position 201A is as follows: It can be expressed by equation (2).
Vi1 = W1-Di1.tan {atan (W1 / Di1- _Fθ1 )} (2)

Here, since the value of the detection distance Di1 in the expression (2) is the distance from the own vehicle position 101 to the set position in the real space of the detection area 11, the detection area setting condition and the movement of the own vehicle before time T1 It can be determined by quantity. Further, the value of the lateral position W1 is determined in advance as described above, and the value of the angular flow _Fθ1 is obtained by the equation (1). Therefore, the value of the moving speed Vi1 in the real space of the intruder at the detection position 201A can be calculated from the equation (2).

  Next, a captured image in front of the host vehicle captured at the host vehicle position 102 in (a) at time T2 is shown in (c). In this captured image, an intruder present at the position 202 of (a) (for convenience, this is represented as the intruder 202) is captured. The intruder 202 is detected in the determination area 21 set inside the detection area 11. At this time, it is determined that the intruder 202 is present at the position where the determination area 21 is virtually set in the real space, that is, the determination position 202A shown in FIG.

The optical flow of the intruder 202 calculated in the determination area 21 from the captured image of (c) is represented as F2. As shown in (a), assuming that the moving speed of the intruder 202 when viewing the intruder 202 from the own vehicle position 102 is the angular flow _Fθ2 , this is the same as the expression (1) using the optical flow F2. Can be expressed by the following equation (3).
F _θ2 = α · F2 (3)
Where α is a constant

Assuming that the distance from the host vehicle position 102 to the determination position 202A in the traveling direction of the host vehicle is the determination distance Di2, the moving speed Vi2 in the real space when it is assumed that the intruder 202 exists at the determination position 202A is expressed by the equation (2). Similarly to the above, it can be expressed by the following formula (4).
Vi2 = W2-Di2 · tan {atan (W2 / Di2- _Fθ2 )} (4)

Here, the value of the determination distance Di2 in the equation (4) is a distance from the own vehicle position 102 to a set position in the real space of the determination area 21, and this is in the real space of the detection area 11 from the own vehicle position 102. It is equal to the distance to the set position, and can be obtained as Di2 = Di1-Vc · (T2-T1). Further, the value of the lateral position W2 is determined in advance as described above, and the value of the angular flow _Fθ2 is obtained by Expression (3). Therefore, the value of the moving speed Vi2 of the intruder in the real space at the determination position 202A can be calculated from the equation (4).

If the intruder actually exists at the detection position 201A at time T1 and exists at the determination position 202A at time T2, the values of the moving speeds Vi1 and Vi2 calculated by the equations (2) and (4) are equal. There must be. However, if the actual location is different as shown in (a), there will be a difference in the values of the moving velocities Vi1 and Vi2. From this, it is possible to determine whether or not the detected intruder is a pop-out object by determining whether or not the following expression (5) is satisfied.
| Vi1-Vi2 | ≦ Rth (5)
Where Rth is a constant

  When the above expression (5) is satisfied, since the position where the intruder is present is equal to the detection position and the determination position, the intruder is detected as a pop-out object. On the other hand, when the expression (5) is not satisfied, the position where the intruder is present is different from the detection position and the determination position, so that it is determined that the object is not a pop-out object. In this case, each time an intruder is detected in another detection area after that, the determination according to Expression (5) is performed. The constant Rth in equation (5) is determined in advance so that it can be determined that the moving velocities Vi1 and Vi2 are substantially equal in consideration of detection errors and the like.

  FIG. 4 shows a flowchart of processing executed in the microcomputer 5 when a pop-out object is detected as described above. This flowchart is executed for each image frame of the captured image for each frame rate. In step S10, detection area setting processing shown in the flowchart of FIG. 5 described later is performed, and a detection area and a determination area are set in the captured image. In step S20, a pop-out object determination process is performed by executing the flowchart of FIG. 6 described later using the detection area and the determination area set in step S10. After step S20 is executed, the flowchart of FIG. 4 ends.

  The contents of the detection area setting process executed in step S10 will be described with reference to the flowchart of FIG. In step S100, an image captured by the camera 1 is acquired by reading an image stored in the image memory 2. In step S101, a vertical edge is detected from the captured image acquired in step S100 using a Sobel filter, and in the next step S102, a Hough transform process is performed on the edge detected in step S101.

  In step S103, a white line portion that is a lane mark is detected from the captured image based on the result of the Hough transform process in step S102. In step S104, the vehicle behavior is calculated based on the measurement results of the wheel speed data and the steering angle data by the wheel speed sensor 3 and the steering angle sensor 4, and the movement amount of the host vehicle is obtained. In step S105, a detection region is set in the captured image based on the vehicle behavior calculation result in step S104. At this time, as described above, two rows of interdigital detection areas that are virtually fixed in real space are set along the lane mark. Thereby, the detection areas 10 to 15 and the determination areas 20 to 25 in FIG. 2 are set. If step S105 is performed, step S10 of FIG. 4 will be complete | finished. In this way, the detection area setting process is performed.

  Next, the pop-out object determination process executed in step S20 will be described with reference to the flowchart of FIG. In step S200, the optical flow in the detection areas 10 to 15 set in the captured image by the detection area setting process is calculated. In step S201, it is determined whether there is an optical flow of each detection area calculated in step S200 whose direction is toward the road. If there is an optical flow toward the road, the process proceeds to step S202. On the other hand, if there is no optical flow toward the road, the flowchart of FIG. 6 is terminated.

  In step S202, the optical flow for the same object is calculated in any of the determination areas 20 to 25 corresponding to the detection area for which the optical flow is determined to be heading for the road in step S201. That is, the optical flow for the intruder 200 is calculated in the determination area 23 corresponding to the detection area 13 where the optical flow 201 of FIG. 2 is detected. In steps S203 and S204, the moving speed of the intruder 200 in the detection area 13 and the determination area 23 in the real space is calculated using the equations (2) and (4), respectively.

  In step S205, the difference between the moving speed of the intruder 200 in the real space in the detection area 13 calculated in step S203 and the moving speed of the intruder 200 in the real space in the determination area 23 calculated in step S204 is obtained. It is determined whether or not it is equal to or less than a predetermined value. At this time, the determination is performed using Expression (5). Due to the determination in step S205, the intruder 200 is detected as a pop-out object.

  If the expression (5) is satisfied in step S205, the intruder 200 is detected as a jump-out object, and the process proceeds to step S206. In step S206, the alarm device 6 is turned on and an alarm sound is output. After step S206 is executed, the flowchart of FIG. 6 is terminated. On the other hand, if it is determined in step S205 that the expression (5) is not satisfied, the intruder 200 is not detected as a jump-out object, and the flowchart of FIG. 6 is terminated without executing step S206. As described above, the pop-up object is detected in the present apparatus.

According to the embodiment described above, the following operational effects can be obtained.
(1) A captured image of the destination of the host vehicle is acquired by the camera 1, and a detection region virtually fixed in real space is set in the captured image (step S105). Then, the optical flow in this detection area is calculated (step S200), and the pop-out object is detected based on the calculated optical flow (step S205). Since it did in this way, the object which jumps ahead of the own vehicle can be detected with the picked-up image of one camera.

(2) Since a plurality of detection areas are set at predetermined intervals along the road on which the host vehicle is traveling, it can be ensured that no matter where the jumping object jumps out on the road. Can be detected.

(3) Since the lane mark is extracted from the captured image (step S103) and the setting position of each detection area is determined based on the position of the extracted lane mark, the detection area is surely set along the road. be able to.

(4) The optical flow in the detection area set outside the road is calculated (step S200). If the optical flow is directed toward the road (step S201), the determination area set inside is determined. The optical flow of the same object at is calculated (step S202). Based on these optical flows, the moving speeds of the object in the detection area and the determination area in the real space are calculated (steps S203 and S204). Then, when the calculated difference in moving speed is equal to or less than the predetermined value, the object is detected as a pop-out object (step S205). Since it did in this way, it can prevent detecting even the object of the position away from the road as a jumping out object accidentally.

  In the above embodiment, each detection area is set along the road where the host vehicle travels by extracting the lane mark from the captured image and determining the setting position of the detection area based on the position of the lane mark. I was supposed to be. However, each detection area may be set along the road on which the host vehicle travels by determining the road shape using map information used in a navigation device or the like.

  In the above embodiment, the imaging means is realized by the camera 1, and the other respective means are realized by the processing of the microcomputer 5. However, this is only an example, and unless the characteristics of the present invention are impaired, The components are not limited to the above embodiment.

It is a figure which shows the structure of the object detection apparatus by one Embodiment of this invention. It is a figure which shows a mode when the pop-out object which approachs on the road from the side of the road where the own vehicle is moving is detected from a captured image. It is a figure for demonstrating the relationship between the virtual setting position of the detection area in real space, and the presence position of a detection target object, (a) is on the real space of the intruder who is the own vehicle and a detection target object. The positional relationship is shown, and (b) and (c) show the captured images at time T1 and time T2, respectively. It is a flowchart of the process performed when a pop-out object is detected. It is a flowchart which shows the content of a detection area setting process. It is a flowchart which shows the content of the pop-out object determination process.

Explanation of symbols

1: Camera 2: Image memory 3: Wheel speed sensor 4: Steering angle sensor 5: Microcomputer 6: Alarm 200: Intruder

Claims (4)

Imaging means for acquiring a captured image of the destination of the host vehicle;
A movement amount detecting means for detecting a movement amount of the host vehicle;
Detection area setting means for setting a detection area virtually fixed in real space in the captured image based on the movement amount detected by the movement amount detection means;
An optical flow calculating means for calculating an optical flow indicating a speed on the image of the object in the detection area set by the detection area setting means;
Based on the optical flow, provided with a protruding object detection means for detecting a protruding object that jumps out to the destination of the host vehicle ,
The detection area setting means includes a point that is a predetermined distance away from a boundary portion of the road on which the host vehicle is traveling and a point that is closer to the road by a predetermined lateral distance than the point in each real space. Set two detection areas as virtual setting positions in
The optical flow calculation means includes a first optical flow indicating a speed on the image of the object in a first detection area set outside the road among the two detection areas, and the first A second optical flow indicating a speed on the image of the object in the second detection region set inside the detection region relative to the road,
The pop-out object detection means includes a first movement speed of the object in the real space calculated based on the first optical flow, and a real space of the object calculated based on the second optical flow. A pop-out object detection device that detects the object as the pop-out object when the difference from the second moving speed is equal to or less than a predetermined value. In the pop-out object detection device according to claim 1,
The pop-up object detection apparatus, wherein the detection area setting means sets a plurality of detection areas at predetermined intervals along a road on which the host vehicle is traveling. In the pop-out object detection device according to claim 2,
Lane mark extraction means for extracting a lane mark indicating a boundary portion of the road from the captured image,
The pop-up object detection device, wherein the detection area setting means determines the setting positions of the plurality of detection areas based on the positions of the lane marks extracted by the lane mark extraction means. In the pop-out object detection device according to any one of claims 1 to 3,
The pop-out object detection device, wherein the optical flow calculation means further calculates the second optical flow when the direction of the first optical flow is in the direction of the road.

Images