JP2010093569A - Vehicle periphery monitoring device - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a vehicle periphery monitoring device capable of precisely discriminating an image portion of a vehicle traveling ahead or the like from an image captured by an imaging means mounted on a vehicle to exclude the image portion from objects to be monitored. <P>SOLUTION: The vehicle periphery monitoring device monitors the periphery of a vehicle, based on an image captured by infrared cameras 2R, 2L mounted on the vehicle. The vehicle periphery monitoring device includes: an object extraction means 20 for extracting the image portion of an object on a real space from the image captured by the infrared cameras 2R, 2L; a relative speed calculation means 21 for calculating the relative speed between the object on the real space corresponding to the image portion extracted by the object extraction means 20 and an own vehicle; and a non-monitored object exclusion means 25 for excluding the object on the real space corresponding to the image portion extracted by the object extraction means 20 from the object to be monitored when a state, where a rate of change per unit time of the relative speed calculated by the relative speed calculation means 21 is not more than a prescribed value, continues for not less than determination time. <P>COPYRIGHT: (C)2010,JPO&INPIT

Description

The present invention relates to a vehicle periphery monitoring device that monitors the periphery of the vehicle based on an image captured by an imaging unit mounted on the vehicle.

  Conventionally, a pedestrian existing in the vicinity of a vehicle is detected from an image around the vehicle captured by an in-vehicle camera, and the possibility of contact between the vehicle and the pedestrian is determined. When the possibility of contact is high, the driver There has been proposed a vehicle periphery monitoring device that alerts the vehicle.

  And when the image portion of the other vehicle is included in the image captured by the in-vehicle camera, for example, the image portion of the headlight of the other vehicle is erroneously recognized as the image portion of the pedestrian's head, etc. May interfere with detection. Therefore, a vehicle periphery monitoring device has been proposed in which an image portion of another vehicle included in the captured image is discriminated and this image portion is excluded from the monitoring target (see, for example, Patent Document 1).

  According to the vehicle periphery monitoring device described in Patent Literature 1, it is easy to extract a pedestrian's image portion by excluding an image portion of another vehicle from an image captured by an in-vehicle camera.

  However, in the vehicle periphery monitoring device described in Patent Document 1, an image portion having the same distance to a corresponding real space position around the high-luminance image portion assuming a headlamp of another vehicle is used. The luminance difference and the positional relationship between the two image portions are obtained, and it is determined whether or not it is an image portion of another vehicle.

Therefore, when the distance to the object in the real space becomes long and the detection accuracy of the distance to the object deteriorates, it becomes difficult to accurately determine whether the object is a vehicle. There is a risk of erroneously determining the image portion as a pedestrian.
JP 2003-230134 A

  The present invention has been made in view of the above-described background, and can accurately discriminate an image portion such as a preceding vehicle from an image captured by an imaging unit mounted on the vehicle and exclude it from a monitoring target. An object is to provide a peripheral monitoring device.

  The present invention has been made in order to achieve the above object, and relates to an improvement in a vehicle periphery monitoring device that monitors the periphery of the vehicle based on an image captured by an imaging unit mounted on the vehicle.

  And the 1st aspect of this invention is the object extraction means which extracts the image part of the target object in real space from the image imaged by the said imaging means, The image part extracted by the said object extraction means A state in which the relative speed calculation means for calculating the relative speed between the object in the real space corresponding to the vehicle and the vehicle, and the rate of change of the relative speed calculated by the relative speed calculation means is equal to or less than a predetermined value. And non-monitoring object exclusion means for excluding from the monitoring object the object in the real space corresponding to the image portion extracted by the object extraction means when the determination time continues for more than .

  According to the present invention, the rate of change in the relative speed between an object in real space (hereinafter simply referred to as an object) corresponding to the image portion extracted by the object extracting means and the vehicle is equal to or less than a predetermined value. In this case, it can be determined that the object and the vehicle are moving in the same direction at the same speed, or that the distance between the object and the vehicle is very far. In this case, the possibility that the object and the vehicle are in contact with each other is low.

  Therefore, by setting the condition that the rate of change of the relative speed between the object and the vehicle is equal to or less than a predetermined value as the determination condition, the non-monitoring object excluding means can Non-monitoring objects can be accurately determined and excluded from monitoring objects. Then, by excluding the non-monitoring object from the monitoring object, it is possible to reduce the amount of calculation required for determining the possibility of contact with a pedestrian or the like.

  Further, in the second aspect of the present invention, the object extraction unit that extracts an image portion of the object in real space from the image captured by the imaging unit and the image capturing unit are captured at intervals. For a plurality of time-series images, identity determining means for determining whether image portions extracted from each image by the object extracting means are image portions of the same object in real space; In the series image, between the image most recently captured by the imaging unit and an image before the image, the identity determining unit determines that the image part is the same object in real space, and When a state in which an image portion whose size change rate is equal to or smaller than a predetermined value continues for a predetermined determination time or longer, an object in the real space corresponding to the image portion is excluded from being monitored. With monitoring exclusion means It is characterized in.

  According to the present invention, in the time-series image, it is determined that the image portion of the same object in the real space is between the most recently captured image and the previous image by the identity determination unit. In addition, when there is an image part whose size change rate is equal to or less than a predetermined value, an object in real space corresponding to the image part (hereinafter simply referred to as an object) has the same speed as the vehicle. Thus, it can be determined that they are moving in the same direction or that the distance between the object and the vehicle is very far. In this case, the possibility that the object and the vehicle are in contact with each other is low.

  Therefore, between the most recently captured image and the previous image, it is determined by the identity determining means that the image portion is the same object in real space, and the rate of change in size is predetermined. By using the non-monitoring object exclusion means to accurately determine a non-monitoring object such as a preceding vehicle by setting a condition that an image portion that is less than or equal to the value continues for the determination time or longer. Can be excluded from monitoring. Then, by excluding the non-monitoring object from the monitoring object, it is possible to reduce the amount of calculation required for determining the possibility of contact with a pedestrian or the like.

  In the first aspect and the second aspect, there is provided determination time setting means for setting the determination time to a shorter time as the speed of the vehicle is higher.

  According to the present invention, the higher the speed of the vehicle, the change in the relative speed between the vehicle and the object in the first aspect per unit time and the size of the image portion in the second aspect. Changes are likely to occur. Therefore, by setting the determination time to be shorter as the vehicle speed is faster by the determination time setting means, it is possible to quickly determine an object to be excluded from the monitoring target, and to calculate the amount of calculation required for the determination. Can be reduced.

  Further, in the first aspect and the second aspect, the type determining means for determining the type of the object in the real space corresponding to the image portion extracted by the object extracting means, and the type determining means And a determination time setting means for setting the determination time according to the type of the determined object.

  In the present invention, for example, when the type of the object is far away from the vehicle such as the sun or a distant mountain, the object in the first aspect and the object in real space The change in the relative speed with the object and the change in the size of the image portion of the object in the second aspect are reduced. Therefore, in this case, by setting the determination time short by the determination time setting means, it is possible to quickly determine the object to be excluded from the monitoring target and reduce the amount of calculation required for the determination. be able to.

  On the other hand, for example, when the type of the object is another vehicle and traveling in front of the vehicle, the relative speed between the vehicle and the object in the first aspect and the object in the second aspect The size of the image portion of the object may change greatly instantaneously depending on the behavior of the other vehicle that is the object. Therefore, in this case, it is possible to stably determine whether or not to exclude other vehicles from the monitoring target by setting the determination time longer by the determination time setting means.

  Embodiments of the present invention will be described with reference to FIGS. FIG. 1 is a configuration diagram of a vehicle surroundings monitoring apparatus according to the present invention. The vehicle surroundings monitoring apparatus according to the present invention includes an image processing unit 1. The image processing unit 1 includes infrared cameras 2R and 2L (corresponding to the imaging means of the present invention) that can detect far infrared rays, a yaw rate sensor 3 that detects the yaw rate of the vehicle, a vehicle speed sensor 4 that detects the traveling speed of the vehicle, A brake sensor 5 for detecting the amount of brake operation by the driver, a speaker 6 for alerting by voice, and an image obtained by the infrared cameras 2R and 2L, and an object having a high possibility of contact Is connected to a display device 7 for performing display for the driver.

  Referring to FIG. 2, infrared cameras 2R and 2L are disposed at the front portion of vehicle 10 at a substantially target position with respect to the center portion of vehicle 10 in the vehicle width direction, and two infrared cameras 2R and 2L are arranged. The optical axes are parallel to each other, and the infrared cameras 2R and 2L are fixed at the same height from the road surface. The infrared cameras 2R and 2L have a characteristic that the output level increases (the luminance increases) as the temperature of the imaged object increases. Further, the display device 7 is provided so that the screen 7 a is displayed at a front position on the driver side of the front window of the vehicle 10.

  Referring to FIG. 1, an image processing unit 1 is an electronic unit composed of a microcomputer (not shown) and converts analog video signals output from infrared cameras 2R and 2L into digital data. The microcomputer has a function of performing various arithmetic processes by the microcomputer with respect to the image in front of the vehicle captured in the image memory (not shown).

  Then, by causing the microcomputer to execute a vehicle monitoring program, the microcomputer extracts the object portion of the object in real space from the image in front of the vehicle, the object extracting means 20, and the object extracting means 20. The relative speed calculation means 21 for calculating the relative speed between the object in the real space corresponding to the image part extracted by the vehicle 10 and the vehicle 10, and the extracted image part is the same object between the images taken at different time points. Identity determining means 22 for determining whether the object is a thing, type determining means 23 for determining the type of an object in the real space corresponding to the image portion, determination when executing a non-monitoring target process to be described later The determination time setting means 24 for setting the time and the image portion of the object (non-monitoring object) that is not the monitoring target among the image portions extracted from the image ahead of the vehicle is determined. Functioning as a non-monitored excluding means 25 for external.

  Next, the object extraction process, the non-monitoring object determination process, and the alert process by the image processing unit 1 will be described according to the flowchart shown in FIG.

  The image processing unit 1 first inputs an analog signal of an infrared image output from the infrared cameras 2R and 2L in STEP1, and then stores a grayscale image obtained by digitizing the analog signal by A / D conversion in an image memory in STEP2. To do.

  In STEP1 to STEP2, a grayscale image (hereinafter referred to as a right image) obtained by the infrared camera 2R and a grayscale image (hereinafter referred to as a left image) obtained by the infrared camera 2L are acquired. Since the right image and the left image have a shift (parallax) in the horizontal position of the image portion of the same object, the distance from the vehicle 10 to the object in real space can be calculated based on this parallax. it can.

  In subsequent STEP 3, the image processing unit 1 binarizes the right image as a reference image (pixels with luminance equal to or higher than a threshold value are set to “1 (white)”, and pixels smaller than the threshold value are set to “0 (black)”. To generate a binary image. The next STEP 4 to STEP 6 are processes by the object extraction means 20. In STEP 4, the object extraction means 20 converts the image portion of each white area included in the binary image into run length data (data of white pixel lines continuous in the x (horizontal) direction of the binary image). Further, the object extraction unit 20 labels a line having a portion overlapping in the y (vertical) direction of the binary image as one image part in STEP 5, and in STEP 6, the labeled image part is an image candidate of the monitoring object. Extract as

  In the next STEP 7, the image processing unit 1 calculates the gravity center G, area S, and circumscribing aspect ratio ASPECT of each image candidate. Note that a specific calculation method is described in detail in the above-mentioned Japanese Patent Application Laid-Open No. 2003-230134, and thus the description thereof is omitted here. Then, the image processing unit 1 executes the following STEP 8 to STEP 9 and STEP 20 to STEP 22 in parallel.

  STEP 8 is a process performed by the identity determination unit 22, and the identity determination unit 22 determines the identity of the image portion extracted from the binary image based on the image captured by the infrared cameras 2 R and 2 L at every predetermined sampling period. The time series data of the position (center of gravity position) of the image portion determined to be the same object image is stored in the memory (time tracking). In STEP 9, the image processing unit 1 reads the vehicle speed VCAR detected by the vehicle speed sensor 4 and the yaw rate YR detected by the yaw rate sensor 3, and calculates the turning angle θr of the vehicle 10 by time integration of the yaw rate YR. To do.

  On the other hand, in STEP 20, the image processing unit 1 selects one of the image candidates of the monitoring target tracked by the binary image of the reference image (right image), and searches from the grayscale image of the right image. R1 (an image of the entire region surrounded by a circumscribed rectangle of the selected candidate image) is extracted. In subsequent STEP 21, the image processing unit 1 sets a search area for searching for an image corresponding to the search image R1 (hereinafter referred to as the corresponding image R1 ′) from the grayscale image of the left image, and performs an inter-phase calculation with the search image R1. The corresponding image R1 ′ is extracted by executing. In STEP 22, the image processing unit 1 calculates the difference between the centroid position of the search image R 1 and the centroid position of the corresponding image R 1 ′ as a parallax amount Δd (number of pixels), and proceeds to STEP 10.

  In STEP 10, the image processing unit 1 calculates the distance z between the vehicle 10 and the object based on the parallax amount Δd, and calculates the coordinates (x, y) and the distance z of the search image as real space coordinates (X, Y, Z) to calculate the coordinates of the real space position corresponding to the search image. As shown in FIG. 2, the real space coordinates (X, Y, Z) are set such that the midpoint position of the attachment positions of the infrared cameras 2R, 2L is the origin 0, X is the vehicle width direction of the vehicle 10, and Y Is set in the vertical direction, and Z is set in the front direction of the vehicle 10. The image coordinates are set such that the center of the image is the origin, the horizontal direction is x, and the vertical direction is y.

  In subsequent STEP 12, the image processing unit 1 determines whether or not the object in the real space corresponding to the image portion extracted in STEP 6 is a non-monitoring target according to the flowchart shown in FIG. When it is determined that it is, the “non-monitoring target determination process” for excluding from the monitoring target is executed.

  STEP 50 in FIG. 4 is processing by the type discriminating means 23, and the type discriminating means 23 discriminates the type of the object in STEP 6. Specifically, the size of the image portion of the target object, the shape, the luminance distribution, and the like are compared with preset reference data for the type of the non-monitoring target object to determine the type of the target object. As the types of non-monitoring objects, for example, the sun, mountains, other vehicles, and the like are assumed, and reference data for discriminating these types is held in a memory (not shown).

  Subsequent STEP 51 to STEP 52 are processes by the determination time setting means 24. The determination time setting means 24 detects the vehicle speed of the vehicle 10 at STEP 51 by the vehicle speed sensor 4, and at STEP 52, the determination time based on the type of the object and the vehicle speed. Set. Here, as the vehicle speed of the vehicle 10 increases, it is easier to detect a change in the relative speed between the vehicle 10 and the object, and the type of the object is farther from the vehicle 10 such as the sun or a mountain. Sometimes, the relative speed between the vehicle 10 and the object hardly changes.

  Accordingly, the determination time setting unit 24 sets the determination time Td to be shorter as the vehicle speed of the vehicle 10 is faster, and sets the determination time to be shorter when it is recognized that the object exists far away from the type of the object.

  The subsequent STEP 53 is a process by the relative speed calculating means 21, and the relative speed calculating means 21 is determined to be an image portion of the same object by the identity determining means 22 extracted by tracking in STEP 3 of FIG. The relative speed between the host vehicle 10 and the object is calculated from the change in the position of the object in the real space corresponding to the image portion.

  The next STEP 54 to STEP 55 are processes performed by the non-monitoring object exclusion unit 25. The non-monitoring object exclusion means 25 calculates the change rate Rv of the relative speed between the host vehicle 10 and the object in STEP 54, and the change rate Rv is equal to or less than the determination threshold value Rv_th (corresponding to the predetermined value of the present invention). It is determined whether or not the state continues for a determination time Td or longer.

  Here, when the state in which the change rate Rv of the relative speed between the host vehicle 10 and the target object is equal to or less than the determination threshold value Rv_th continues for the determination time Td or longer, for example, the target object travels in front of the host vehicle 10 at the same speed. It may be determined that there is a low possibility that the host vehicle 10 and the target object are close to each other as in the case of another vehicle that travels or when the target object is the sun or a mountain and is far from the host vehicle 10. it can.

  Therefore, when the state in which the change rate Rv of the relative speed between the host vehicle 10 and the target object is equal to or less than the determination threshold value Rv_th continues in STEP54 for the determination time Td or longer, the process proceeds to STEP55, and the non-monitoring object exclusion means 25 It is determined that the object is a non-monitoring target. Then, the process proceeds to STEP 56 and the “non-monitoring object determination process” is ended. On the other hand, when the state in which the relative speed change rate Rv determination threshold value Rv_th is not longer than the determination time Td is not continued for more than the determination time Td in STEP54, the process branches to STEP56. It is not determined that

  In STEP 13 of FIG. 3, when the non-monitoring target exclusion unit 25 determines that the target object is a non-monitoring target by the “non-monitoring target determination process”, the process branches to STEP 1, thereby excluding the target object from the monitoring target. The On the other hand, when it is determined by the “non-monitoring object determination process” that the object is not a non-monitoring object, the process proceeds to STEP14.

  STEP14 to STEP16 and STEP30 are processes for determining whether or not an object is a target of alerting and alerting the driver. In STEP 14, the image processing unit 1 determines the target, the vehicle 10, and the time series data of the real space position of the same target after turning angle correction obtained from a plurality of images captured within a predetermined monitoring period. The relative movement vector of is calculated.

  The specific calculation method of the real space coordinates (X, Y, Z) and the movement vector of the object is described in detail in the above-mentioned Japanese Patent Application Laid-Open No. 2003-230134, and the description thereof is omitted here.

  Next, in STEP 15, the image processing unit 1 determines the possibility of contact between the vehicle 10 and the object, and executes “attention call determination processing” for determining whether or not it is necessary to call attention. When it is determined by the “attention call determination process” that it is necessary to call attention, the process branches to STEP 30 to output a warning sound by the buzzer 6 and display a warning on the display device 7. On the other hand, when it is determined by the “attention calling determination process” that it is not necessary to call attention, the process returns to STEP 1 and the image processing unit 1 does not call attention.

  In the “attention determination process”, the image processing unit 1 may check whether the target object is in contact with the host vehicle 10 within a margin time, whether the target object is within the approach determination area set around the host vehicle, The possibility that the object enters the approach determination area from outside the approach determination area and contacts the host vehicle 10, whether the object is a pedestrian, whether the object is an artificial structure, etc. Determine whether or not it is necessary to call attention.

  Note that the specific processing content of the “attention determination process” is described in detail as “alarm output determination process” in the above-described Japanese Patent Application Laid-Open No. 2003-230134, and thus the description thereof is omitted here.

  As described above, the image processing unit 1 is performed by performing the “non-monitoring target determination process” in STEP 12 to STEP 13 and excluding the object determined as the non-monitoring target from the target of the alerting process in STEP 14 to STEP 16. It is possible to efficiently monitor the surroundings of the host vehicle 10 by reducing the amount of computation at.

  Next, in the above-described embodiment, the object is a non-monitoring object based on the relative speed between the host vehicle 10 and the object by calculating the real space position of the object with the two infrared rays 2R and 2L. However, in a vehicle in which only one infrared camera is mounted, the non-monitoring object exclusion means 25 determines whether or not the object is a non-monitoring object by the following processing. be able to.

  FIG. 5A is an image in front of the host vehicle by an in-vehicle camera, and includes an image portion 31 of a preceding vehicle and an image 32 of a pedestrian. FIG. 5B is an explanatory diagram of the situation of FIG. 5A as viewed from above the road. The other vehicle 42 travels in front of the host vehicle 41 and walks on the sidewalk on the side of the host vehicle 41. Person 43 is walking.

  Here, since the moving speed Vd1 of the pedestrian 43 is extremely lower than the moving speed Vj of the host vehicle 41 (Vd1 << Vj), the relative speed between the pedestrian 43 and the host vehicle 41 can be regarded as Vj. If the distance between the host vehicle 41 and the pedestrian 43 is Z1, the pedestrian image portion 32 is not present in the image for T1 = Z1 / Vj or more.

  On the other hand, the host vehicle 41 and the other vehicle 42 normally travel at substantially the same speed (Vj≈Vd2), and therefore, as shown in FIG. 5A, in the image 30 captured by the camera, The state where the image portion 31 of the other vehicle 42 exists with substantially the same size continues. The same applies to objects that are distant from the vehicle, such as the sun and mountains.

  Therefore, the non-monitoring target exclusion means 25 is a time-series image captured by the camera, saying that “the same image between the latest image and the previous image is an image portion of the same object by the identity determination means 22. When the determination and the state in which the image portion whose size change rate is equal to or less than the predetermined value continues for the determination time set by the determination time setting means 24 ", the object is a non-monitoring target. It can be excluded from the monitoring target as being an object (previous vehicle, sun, mountain, etc.).

  In this case, since only one camera is provided and no distance measuring means such as a radar is provided, comparison is possible even with a vehicle in which the distance to the object cannot be calculated or a vehicle equipped with a camera with a low video resolution. Through simple processing, it is possible to discriminate between a monitoring target such as a pedestrian and a non-monitoring target such as a preceding vehicle.

  In the present embodiment, the configuration in which the front of the vehicle is imaged is shown. However, the possibility of contact with the object may be determined by imaging other directions such as the rear and side of the vehicle. .

  Further, in the present embodiment, the infrared cameras 2R and 2L are used as the imaging means of the present invention, but a visible camera that captures a visible image may be used.

  In the present embodiment, the distance between the host vehicle 10 and the object is calculated based on the parallax amount between the captured image of the infrared camera 2R and the captured image of the infrared camera 2L. You may make it detect directly the distance of the vehicle 10 and a target object.

The block diagram of the vehicle periphery monitoring apparatus of this invention. Explanatory drawing of the attachment aspect to the vehicle of the vehicle periphery monitoring apparatus shown in FIG. 3 is a flowchart showing a processing procedure in the image processing unit shown in FIG. 1. The flowchart of a non-monitoring object determination process. Explanatory drawing of discrimination | determination of a pedestrian and a preceding vehicle in a non-monitoring object determination process.

Explanation of symbols

  DESCRIPTION OF SYMBOLS 1 ... Image processing unit, 2R, 2L ... Infrared camera (imaging means), 3 ... Yaw rate sensor, 4 ... Vehicle speed sensor, 5 ... Brake sensor, 6 ... Speaker, 7 ... Display apparatus, 20 ... Object extraction means, 21 ... Relative speed calculation means, 22 ... identity determination means, 23 ... type determination means, 24 ... determination time setting means, 25 ... non-monitoring object exclusion means

Claims (4)

In a vehicle periphery monitoring device that monitors the periphery of the vehicle based on an image captured by an imaging unit mounted on the vehicle,
Object extraction means for extracting an image portion of the object in real space from the image captured by the imaging means;
A relative speed calculating means for calculating a relative speed between an object in real space corresponding to the image portion extracted by the object extracting means and the vehicle;
In a real space corresponding to the image portion extracted by the object extracting means when the rate of change of the relative speed calculated by the relative speed calculating means is not more than a predetermined value continues for a predetermined determination time or longer. A vehicle periphery monitoring apparatus comprising: a non-monitoring object exclusion unit that excludes the object from the monitoring object. In a vehicle periphery monitoring device that monitors the periphery of the vehicle based on an image captured by an imaging unit mounted on the vehicle,
Object extraction means for extracting an image portion of the object in real space from the image captured by the imaging means;
For a plurality of time-series images captured at intervals by the imaging unit, whether or not the image portions extracted by the object extraction unit from each image are image portions of the same object in real space Identity determining means for determining
In the time-series image, between the image most recently captured by the imaging unit and the image before the image, the identity determining unit determines that the image portion is the same object in real space. In addition, when a state in which an image portion whose size change rate is equal to or less than a predetermined value continues for a predetermined determination time or longer, an object in the real space corresponding to the image portion is excluded from the monitoring target. A vehicle periphery monitoring apparatus comprising: a non-monitoring object exclusion unit that performs In the vehicle periphery monitoring device according to claim 1 or claim 2,
A vehicle periphery monitoring device, comprising: a determination time setting means for setting the determination time to a shorter time as the speed of the vehicle is higher. In the vehicle periphery monitoring device according to claim 1 or claim 2,
Type discriminating means for discriminating the type of the object in the real space corresponding to the image portion extracted by the object extracting means;
A vehicle periphery monitoring device comprising: a determination time setting unit that sets the determination time according to the type of the object determined by the type determination unit.

Images