JP2008249517A - Obstacle detector - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an obstacle detector capable of detecting surely an obstacle having the risk of contacting with own vehicle when passing a narrow passage or the like at low speed, by simple computation. <P>SOLUTION: TV cameras 12 provided in side faces of the vehicle images an area including at least sides of the vehicle and diagonal front sides thereof, a feature extraction means M1 extracts feature parts from images thereof, an integration means M2 integrates a change amount in each prescribed time interval in each feature part to prepare an integrated image, a contraction/expansion means M3 erases the integrated image of a pattern on a road by executing contraction/expansion processing for the integrated image, so as to remain the integrated image of a solid object, an obstacle recognition means M4 recognizes the obstacle, based on the remained integrated image of the solid object, and the solid object of the obstacle is thereby detected surely by the simple computation of computing simply image data of the TV cameras 12. <P>COPYRIGHT: (C)2009,JPO&INPIT

Description

  The present invention relates to an obstacle detection device for avoiding contact with an obstacle when a vehicle moves in a narrow passage or the like at a low speed.

  An optical flow (motion vector) is calculated from two images that are time-sequentially imaged by an imaging means mounted on the vehicle, and the vehicle approaching or interrupting the vehicle is extracted by monitoring the optical flow. This is known from Patent Document 1 below.

Also, the optical flow of the distant view image captured by the image pickup means mounted on the vehicle is calculated, and when an optical flow that moves differently from the distant view optical flow exists in the image, it is caused by a three-dimensional object. What is determined to be present is known from Patent Document 2 below.
JP-A-6-314340 JP 2001-116527 A

  However, in the above-mentioned conventional apparatus, it is necessary to calculate an optical flow from an image captured by an imaging unit in order to detect a vehicle or a three-dimensional object around the own vehicle, and the amount of calculation becomes enormous, so There is a problem that the processing time is increased due to an increase in load.

  The present invention has been made in view of the above-described circumstances, and provides an obstacle detection device that can reliably detect an obstacle that may be contacted by the vehicle when passing through a narrow passage or the like at a low speed. The purpose is to provide.

  In order to achieve the above object, according to the invention described in claim 1, an image pickup unit that is provided on a side surface of the vehicle and picks up an area including at least an oblique front from the side of the vehicle, and an image pickup by the image pickup unit A feature extraction unit for extracting a feature from the obtained image, an integration unit for integrating the movement trajectory of the feature to create an integrated image, and integrating a figure on the road surface by contracting / expanding the integrated image An obstacle detection apparatus comprising: a contraction / expansion processing unit that erases an image and leaves an accumulated image of a three-dimensional object; and an obstacle recognition unit that recognizes an obstacle from the accumulated image of the remaining three-dimensional object. Proposed.

  According to the second aspect of the present invention, in addition to the configuration of the first aspect, a relative position calculation unit that calculates a relative position of the obstacle with respect to the vehicle, and a movement locus calculation unit that calculates the movement locus of the vehicle. And an obstacle position detecting device for tracing the relative position of the obstacle that changes in accordance with the movement locus of the vehicle calculated by the movement locus calculating means.

  According to a third aspect of the invention, in addition to the configuration of the first or second aspect, the relative position calculating means calculates a relative position of an obstacle closest to the vehicle among a plurality of obstacles. An obstacle detection device is proposed which is characterized by

  According to a fourth aspect of the present invention, in addition to the configuration of any one of the first to third aspects, the movement trajectory calculating means is configured to detect the movement trajectory of the vehicle based on the vehicle speed and the steering angle. An obstacle detection device is proposed which is characterized in that

  According to the invention described in claim 5, in addition to the configuration of any one of claims 1 to 4, a distance between the obstacle traced by the relative position tracing means and the vehicle is predetermined. Provided with at least one of a steering assist device that assists steering in a direction that avoids contact with the feature portion, a deceleration control device that automatically decelerates the vehicle, and an alarm device that issues a warning to the driver when the threshold value is below the threshold An obstacle detection apparatus characterized by the above is proposed.

  Note that the television camera 12 of the embodiment corresponds to the imaging means of the present invention.

  According to the configuration of the first aspect, the image pickup means provided on the side surface of the vehicle picks up an area including at least the diagonal front from the side of the vehicle, and the feature extraction means extracts the feature portion from the image picked up by the image pickup means. The integration means integrates the movement trajectory of the feature to create an integrated image, and the contraction / expansion processing means performs the contraction / expansion processing on the integrated image to erase the integrated image of the figure on the road surface. Since the three-dimensional integrated image is left and the obstacle recognition means recognizes the obstacle from the three-dimensional integrated image, it is possible to reliably detect the three-dimensional object as an obstacle by simply performing a simple calculation on the image data of the imaging means. can do. In addition, since it is sufficient to provide one image pickup means on each side of the vehicle body, not only is the cost low, but the influence on the design of the vehicle body side that is conspicuous in appearance can be minimized.

  According to the second aspect of the present invention, the relative position calculating means calculates the relative position of the obstacle to the vehicle, the moving locus calculating means calculates the moving locus of the vehicle, and the relative position tracing means responds to the moving locus of the vehicle. Therefore, the driver can sequentially recognize the position of the obstacle that the vehicle may come into contact with to avoid the contact.

  According to the configuration of the third aspect, the relative position calculating means calculates the relative position of the obstacle closest to the vehicle among the plurality of obstacles, so that the position of the obstacle that may contact the vehicle can be accurately determined. Can be calculated.

  According to the fourth aspect of the present invention, since the movement trajectory calculating means calculates the vehicle trajectory based on the vehicle speed and the steering angle, the vehicle movement trajectory can be accurately calculated by a simple calculation.

  According to the fifth aspect of the present invention, when the distance between the obstacle traced by the relative position tracing means and the vehicle is equal to or less than a predetermined threshold, the steering assist device steers in a direction to avoid contact with the obstacle. Assisting, the deceleration control device automatically decelerates the vehicle, or the alarm device issues an alarm to the driver, so that it is possible to reliably avoid the vehicle coming into contact with the obstacle.

  Hereinafter, embodiments of the present invention will be described with reference to the accompanying drawings.

  1 to 8 show an embodiment of the present invention, FIG. 1 is a view of a vehicle as viewed from above, FIG. 2 is a block diagram of an obstacle detection device, and FIG. 3 is an explanatory diagram of object detection by a television camera. FIG. 4 is a diagram for explaining changes in graphics of a figure and a three-dimensional object on the road surface, FIG. 5 is a diagram for explaining the operation of the feature extracting means, FIG. 6 is a diagram for explaining the action of the integrating means, and FIG. FIG. 8 is an explanatory view of the operation of the processing means, and FIG. 8 is an explanatory view of the operation of the relative position tracing means.

  As shown in FIG. 1, the vehicle V includes left and right television cameras 12, 12 provided on the left and right door mirrors 11, 11. Each television camera 12 captures a figure F such as a three-dimensional object O existing in a wide-angle region R from the side surface of the vehicle V to a diagonally forward direction or a white line on the road surface. Hereinafter, the three-dimensional object O and the figure F are referred to as objects. The television camera 12 can recognize the direction of the object from the position of the object on the captured image, but cannot recognize the distance of the object.

  As shown in FIG. 2, the electronic control unit U includes a feature extraction unit M1, an integration unit M2, a contraction / expansion processing unit M3, an obstacle recognition unit M4, a relative position calculation unit M5, and a movement trajectory calculation. Means M6 and relative position tracing means M7 are provided. Television cameras 12 and 12 are connected to the feature extraction means M1, and a vehicle speed sensor 15 that detects the traveling speed of the vehicle V and a steering that detects the steering angle of the steering wheel of the vehicle V are connected to the movement trajectory calculation means M6. An angle sensor 16 is connected. The relative position tracing means M7 includes a steering assist device 17 that assists the steering operation of the driver by applying a steering torque to the steering wheel by an actuator, and a deceleration control device 18 that automatically operates the brake to decelerate the vehicle V. And an alarm device 19 that issues an alarm to the driver with a monitor image, sound, buzzer, chime or the like.

  FIG. 3C shows an image on the right side of the vehicle body imaged by the television camera 12. Assume that an object is detected at a predetermined position A with respect to the vehicle body on the image of the television camera 12. However, since the image of the television camera 12 cannot detect the depth of the object (distance from the television camera 12), the image of the point A is the point A1 on the road surface as shown in FIGS. It is impossible to identify whether the image is an image of point A2 or an image of point A2 that is higher than the road surface.

  Therefore, even if an attempt is made to specify the coordinates of point A in the XY coordinate system fixed to the vehicle body, the coordinates when the object is at point A1 are (X1, Y1), and the coordinates when the object is at point A2 are ( X2, Y2), and the actual object position cannot be specified. Among objects, a figure such as a white line on the road surface does not need to be recognized as an obstacle because there is no possibility of contacting the vehicle V, but a three-dimensional object is recognized as an obstacle because there is a possibility of contacting the vehicle V. There is a need. Therefore, it is necessary to identify the figure F and the three-dimensional object on the road surface and recognize only the three-dimensional object as an obstacle.

  In the present embodiment, the XY coordinate system is provided separately for the object on the right side of the vehicle V and for the object on the left side of the vehicle V, and their origins are the left and right door mirrors 11, respectively. 11, the X-axis positive direction faces the front of the vehicle body, and the Y-axis positive direction faces the outside in the vehicle width direction. Of course, the setting of the XY coordinate system is not limited to that described above, and can be arbitrarily set.

  As shown in FIG. 1, a figure F such as a white line extending in a direction perpendicular to the traveling direction of the vehicle V exists on the road surface on the right side of the vehicle V, and a pole erected on the road surface on the right side of the vehicle V Consider a case where such a three-dimensional object O exists.

  FIG. 4A is an image of the figure F and the three-dimensional object O taken at the time t captured by the television camera 12, but it is possible to identify which is the figure F or the three-dimensional object O only with this image. Can not. FIG. 4B is the same image at time t + Δt, and the image of the figure F and the three-dimensional object O moves from the left side to the right side on the screen as the vehicle V moves forward during Δt. The moving distances of the figure F and the three-dimensional object O are not constant in each part, and the moving distance increases as the point is closer to the television camera 12. In an extreme example, a point that is closest to the television camera 12 passes from the left end to the right end of the screen, and conversely, a point that is present at infinity does not move on the screen.

  FIG. 4C shows an image at time t by a broken line, and an image at time t + Δt by a solid line. The horizontal shift of the broken line and the solid line differs depending on the distance from the TV camera 12 at each point. ing. Specifically, the upper end of the three-dimensional object O (pole) closest to the TV camera 12 has moved the most in the left-right direction, and the right end of the figure F farthest from the TV camera 12 has hardly moved in the left-right direction. Further, the left end of the figure F and the lower end of the three-dimensional object O are not substantially different from each other in the distance from the television camera 12, and therefore the movement distance in the left-right direction is substantially equal.

  Comparing the lower area “a” and the upper area “b” of the screen, there is no great difference in the distance between the figure F and the three-dimensional object O from the TV camera 12 in the lower area “a”, and therefore the movement distance in the left-right direction is large. Although no difference occurs, there is a large difference in the distance between the figure F and the three-dimensional object O from the television camera 12 in the upper region b, and thus a large difference occurs in the lateral movement distance. In the present embodiment, this phenomenon is used to identify a figure F that cannot be an obstacle in contact with the vehicle V and a three-dimensional object O that can be an obstacle in contact with the vehicle V.

  Hereinafter, the obstacle detection method will be described in detail based on the block diagram of FIG.

  First, image data captured by the television camera 12 is input to the feature extraction unit M1 of the electronic control unit U every time Δt. The feature extraction unit M1 receives the image data at time t as shown in FIG. Compared with the image data at time t + Δt, the change is extracted as a feature (edge extracted image), and this process is repeated every time Δt. Then, as shown in FIG. 6, the integrating means M2 calculates an integrated feature by integrating a plurality of features that are continuous over time Δt, that is, features at time t + Δt, time t + 2Δt, time t + 3Δt. The number of integrations is arbitrary.

  Subsequently, as shown in FIG. 7A, the contraction / expansion processing means M3 divides the screen into a plurality of regions by a plurality of horizontal parallel lines, and in each region, the contraction processing described later is performed on the image of the integrated feature portion. Perform inflation processing.

  The screen is composed of a large number of pixels arranged vertically and horizontally, and the image on the screen is composed of a set of pixels “1” having data and pixels “0” having no data. Shrinkage processing is a well-known technique in image processing, and four pixels (four neighboring pixels) adjacent to the predetermined pixel on the screen in the vertical and horizontal directions, or pixels adjacent to the predetermined pixel on the screen in the vertical and horizontal directions. When eight pixels (eight neighboring pixels) including pixels adjacent to the upper right, lower right, upper left, and lower left are defined as adjacent pixels, “1” is set to “0” for all the pixels on the screen. In this case, any one of the adjacent pixels is set to “0” when the pixel is “0”, and “1” is set to the other pixels. Each time this contraction process is performed, the contour of the image contracts by one pixel. On the other hand, in the expansion process, for all pixels on the screen, the pixel “1” is set to “1”, one of the adjacent pixels is set to “1”, and the other pixels are set to “0”. It is what. Each time this expansion process is performed, the contour of the image is expanded by one pixel.

  Since the image of the integrated feature portion of the figure F on the road surface far from the TV camera 12 has a narrow width in the left-right direction, the number of times of the contraction process required until the image disappears is reduced. Since the image of the integrated feature portion of the object O is wide in the left-right direction, the number of times of the contraction process required until the image disappears increases. Accordingly, by performing the contraction process a predetermined number of times for each area of the screen, the image of the integrated feature portion of the figure F on the narrow road surface is extinguished, and the image of the integrated feature portion of the wide three-dimensional object O is deleted. Can be left (see FIG. 7B).

  Subsequently, the contraction / expansion processing means M3 executes the expansion process for each region the same number of times as the number of executions of the contraction process. At this time, the image of the integrated feature portion of the figure F on the road surface once disappeared is not restored, but the image of the accumulated feature portion of the three-dimensional object O that has not completely disappeared is restored to the original state. Is done. In this way, it is possible to leave only the image of the integrated feature portion of the three-dimensional object O indicating the obstacle (see FIG. 7C).

  Subsequently, the obstacle recognizing means M4 recognizes the three-dimensional object O with which the vehicle V may come into contact as an obstacle based on the image of the integrated feature portion of the three-dimensional object O remaining on the screen. The integrated feature part of the three-dimensional object O corresponds to the edge of the three-dimensional part O (for example, the corner of a building or the corner of a column). This is a part that is highly likely to come into contact with the side of the vehicle body.

  In this way, when an obstacle is recognized based on the imaging data of the television camera 12, the relative position calculation means M5 calculates the position of the obstacle with respect to the vehicle V using XY coordinates. As described in FIG. 3, even when the image exists at the same position on the screen, the distance of the object at a high position from the road surface is actually close, and the distance of the object on the road surface is actually far, Therefore, the procedure for calculating the coordinates of the obstacle from the image becomes troublesome. Therefore, the relative position calculation means M5 calculates the XY coordinates of the lower end of the obstacle, that is, the portion where the obstacle is in contact with the road surface, as the position of the obstacle.

  The coordinates of the obstacle are specified by using a map search in order to shorten the calculation time. That is, a map storing obstacle coordinates corresponding to each detection direction of the television camera 12 is prepared, and the obstacle coordinates can be searched from this map. For example, when the pixel P at the lower end position where the obstacle (integrated feature) in the image of FIG. 7 is in contact with the road surface is determined, the XY coordinates of the obstacle corresponding to the position of the pixel P can be searched from the map. .

  When there are a plurality of obstacles, the one with the shortest distance (the one closest to the vehicle) is selected as the obstacle.

  The movement trajectory calculating means M6 calculates the movement trajectory of the host vehicle every predetermined time Δt from the vehicle speed detected by the vehicle speed sensor 15 and the steering angle detected by the steering angle sensor 16. The movement trajectory of the own vehicle includes the movement trajectory of the origin of the XY coordinate system and the directions of the X axis and the Y axis.

  The relative position tracing means M7 calculates the coordinates of the obstacle for each time Δt from the coordinates of the obstacle calculated by the relative position calculating means M5 and the movement trajectory of the own vehicle calculated by the movement locus calculating means M6. Trace. As a result, as shown in FIG. 8, it is possible to know how the obstacle moves relative to the vehicle every time Δt. If it is determined that the distance between the side of the vehicle body (its coordinates are known) and the obstacle is equal to or less than the threshold after time Δt, the steering assist device 17 generates a steering torque in a direction to avoid contact with the obstacle. The deceleration control device 18 activates the automatic brake, and the alarm device 19 notifies the driver that there is a possibility of contact, so that it is possible to reliably avoid the vehicle from coming into contact with the obstacle. .

  As described above, since it is sufficient to provide one television camera 12 on each side of the vehicle V, not only the cost is low, but also the influence on the design of the vehicle side that is conspicuous in appearance can be minimized. Moreover, since the figure F and the three-dimensional object O on the road surface can be identified by a simple calculation process, the calculation load on the computer can be reduced and the processing time can be shortened.

  The embodiments of the present invention have been described above, but various design changes can be made without departing from the scope of the present invention.

  For example, in the embodiment, the steering assist device 17, the deceleration control device 18, and the alarm device 19 are provided to avoid the vehicle V coming into contact with an obstacle, but at least one of them may be provided. Alternatively, the alarm device 19 may be activated first, the steering assist device 17 may be activated when the driver does not respond, and the deceleration control device 18 may be activated when there is still a possibility of contact.

View of vehicle from above Block diagram of obstacle detection device Illustration of object detection by TV camera The figure explaining the change of the figure on the road surface and the image of the three-dimensional object Explanatory diagram of the action of the feature extraction means Explanatory drawing of the action of the integration means Explanatory diagram of the action of the contraction / expansion processing means Explanatory drawing of operation of relative position tracing means

Explanation of symbols

12 TV camera (imaging means)
17 Steering support device 18 Deceleration control device 19 Alarm device M1 Feature extraction means M2 Accumulation means M3 Contraction / expansion processing means M4 Obstacle recognition means M5 Relative position calculation means M6 Movement trajectory calculation means M7 Relative position tracing means O Solid object F Figure R region V vehicle

Claims (5)

An imaging means (12) provided on a side surface of the vehicle (V) for imaging a region (R) including at least an oblique front from a side of the vehicle (V);
Feature extraction means (M1) for extracting a feature from the image captured by the imaging means (12);
Integration means (M2) for integrating the movement trajectory of the characteristic part to create an integrated image;
A contraction / expansion processing means (M3) for deleting the integrated image of the figure (F) on the road surface by contracting / expanding the integrated image and leaving the integrated image of the three-dimensional object (O);
Obstacle recognition means (M4) for recognizing obstacles from the accumulated image of the remaining three-dimensional object (O);
An obstacle detection device comprising:   Relative position calculation means (M5) for calculating the relative position of the obstacle to the vehicle (V), movement locus calculation means (M6) for calculating the movement locus of the vehicle (V), and movement locus calculation means (M6) The obstacle detection according to claim 1, further comprising relative position tracing means (M7) for tracing the relative position of the obstacle that changes in accordance with the movement trajectory of the vehicle (V) calculated in step (1). apparatus.   The obstacle detection according to claim 1 or 2, wherein the relative position calculation means (M5) calculates a relative position of an obstacle closest to the vehicle (V) among a plurality of obstacles. apparatus.   The said movement locus | trajectory calculation means (M6) calculates the movement locus | trajectory of a vehicle (V) based on the speed and steering angle of a vehicle (V), Any one of Claims 1-3 characterized by the above-mentioned. The obstacle detection device described.   Steering support for assisting steering in a direction to avoid contact with the obstacle when the distance between the obstacle and the vehicle (V) traced by the relative position tracing means (M7) is not more than a predetermined threshold. At least one of a device (17), a deceleration control device (18) that automatically decelerates the vehicle (V), and an alarm device (19) that issues a warning to the driver is provided. The obstacle detection device according to any one of 4.

Images