JP2008287692A - Obstacle recognition device - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To accurately recognize an obstacle by a single imaging means through easy arithmetic operation while removing the influence of reflection by wet road surface. <P>SOLUTION: An object image Q0 such as a pedestrian or the like in an image of the circumference of an own vehicle taken by a camera is vertically halved to an upper object image Q1 and a lower object image Q2, and the lower object image Q2 is reversed to form a reversed image Q2'. When a correlation value A between the upper object image Q1 and the reversed image Q2' is a threshold th1 or more, it is determined that the lower object image Q2 is taken by reflecting an object by a wet road surface, and only the upper object image Q1 is recognized as an obstacle. According to this, the obstacle can be accurately recognized while removing the influence of reflection by wet road surface only by providing one camera without needing two stereo-cameras and only by performing easy arithmetic processing to an image taken by the one camera. <P>COPYRIGHT: (C)2009,JPO&INPIT

Description

  The present invention relates to an obstacle recognizing device that recognizes an object image in an image around a host vehicle imaged by an imaging means as an obstacle.

  When an obstacle is identified from an image around the vehicle captured by the camera, if the road surface is wet and water has accumulated, the obstacle will be reflected on the wet road surface. There is a problem that an image and a reflected image obtained by inverting the image are captured together, and an obstacle having a shape and a position different from an actual obstacle is erroneously recognized.

Therefore, two images with different viewpoints are captured by two stereo cameras provided at positions separated in the left-right direction of the vehicle, and the object images are compared with each other by comparing the correlation values of the object images of the two images. Japanese Patent Application Laid-Open No. 2004-151867 discloses a technique for recognizing only a true obstacle by removing the influence of reflection on a wet road surface by determining whether or not the image includes a reflection image by reflection.
JP 2002-352225 A

  However, the above conventional system requires two stereo cameras, which increases the cost, and the computation for processing two images captured by the two stereo cameras is complicated, so that the computation of the computer is difficult. There was a problem that the load increased.

  The present invention has been made in view of the above-described circumstances, and an object of the present invention is to accurately recognize an obstacle by removing the influence of reflection on a wet road surface with a single imaging means and simple arithmetic processing.

  In order to achieve the above object, according to the first aspect of the present invention, in an obstacle recognition apparatus for recognizing an object image in an image around a host vehicle imaged by an imaging means as an obstacle, the object image Obstacle candidate extraction means for extracting an object image of an obstacle candidate from the inside, and the object image extracted by the obstacle candidate extraction means is divided into an upper object image and a lower object image in two parts, and Inverted image creating means for creating an inverted image obtained by inverting an object image up and down, similarity determining means for determining the similarity between the upper object image and the inverted image, and when the similarity is equal to or greater than a first threshold, An obstacle recognizing means for recognizing the upper object image as an obstacle, and for recognizing the object image before bisection as an obstacle when the similarity is less than a second threshold. A recognition device is proposed.

  According to the invention described in claim 2, in addition to the configuration of claim 1, the position where the similarity is highest when the similarity is greater than or equal to the second threshold and less than the first threshold. There is proposed an obstacle recognition apparatus comprising road surface position estimation means for estimating as a road surface position, wherein the obstacle recognition means recognizes an object image above the road surface position as an obstacle.

  According to the invention described in claim 3, in addition to the configuration of claim 2, the road surface position estimating means estimates the road surface position within a predetermined range from the center in the vertical direction of the object image before division. A featured obstacle recognition device is proposed.

  According to the invention described in claim 4, in addition to the configuration of claim 2 or claim 3, the road surface position estimation means estimates the road surface position below the center in the vertical direction of the object image before division. An obstacle recognizing device is proposed.

  According to the invention described in claim 5, in addition to the configuration of any one of claims 1 to 4, road surface condition determining means for determining a road surface condition based on the similarity is provided. An obstacle recognition device is proposed.

  Note that the camera C of the embodiment corresponds to the imaging means of the present invention.

  According to the configuration of the first aspect, the object image in the image around the own vehicle imaged by the imaging unit is divided into the upper object image and the lower object image in two parts, and the lower object image is inverted upside down. An inverted image is created, and when the similarity between the upper object image and the inverted image is equal to or higher than the first threshold, the upper object image is recognized as an obstacle, and when the similarity is lower than the second threshold, Since the object image before the division is recognized as an obstacle, only one image pickup means is provided without requiring two image pickup means, and simple arithmetic processing is performed on an image picked up by one image pickup means. By simply removing the influence of the reflection of the wet road surface, the obstacle can be recognized with high accuracy.

  According to the configuration of claim 2, when the similarity between the upper object image and the inverted image is equal to or higher than the second threshold and lower than the first threshold, the position having the highest similarity is estimated as the road surface position, and the road surface position Since the object image on the upper side is recognized as an obstacle, the obstacle can be recognized without hindrance even when a part of the object image on the lower side is missing due to a wet road surface condition.

  According to the third aspect of the present invention, since the road surface position is estimated within a predetermined range from the center in the vertical direction of the object image before division, the calculation load for searching for the road surface position can be minimized.

  According to the configuration of claim 4, the road surface position is estimated from the lower side of the center of the object image before the division, and therefore it is useless to search for the road surface position from the upper side of the vertical direction center of the image where the road surface position cannot exist. Can be avoided.

  Further, according to the configuration of the fifth aspect, it is possible to accurately determine whether or not the road surface is wet based on the similarity between the upper object image and the reverse image.

  Hereinafter, embodiments of the present invention will be described with reference to the accompanying drawings.

  1 to 6 show an embodiment of the present invention. FIG. 1 is a side view of an automobile equipped with a camera, FIG. 2 is a block diagram of an obstacle recognition device, and FIG. 3 is a view when the road surface is not wet. 4 is an explanatory diagram of the obstacle recognition process, FIG. 4 is an explanatory diagram of the obstacle recognition process when the road surface is wet over a wide area, and FIG. 5 is an explanatory diagram of the obstacle recognition process when the road surface is partially wet. 6 is a flowchart of obstacle recognition.

  As shown in FIG. 1, a single camera C is provided at the front end of the roof of the automobile. The camera C identifies a predetermined object including a road ahead of the own vehicle in order to identify an object that may be an obstacle of the own vehicle. Image the area.

  As shown in FIG. 2, the electronic control unit U includes an obstacle candidate extraction unit M1, a reverse image creation unit M2, a similarity determination unit M3, a road surface position estimation unit M4, an obstacle recognition unit M5, a road surface. A state determination unit M6 and a vehicle control unit M7 are provided, a camera C is connected to the obstacle candidate extraction unit M1, and a warning is issued to the vehicle control unit M7 by a buzzer, chime, voice, etc. The device W is connected to a braking device B that automatically activates the wheel brake.

  Obstacle candidate extraction means M1 extracts an object image from an image in front of the host vehicle captured by the camera C by edge detection processing or the like, and applies a technique such as pattern matching to the extracted object image, thereby An object image of an object that can be an obstacle is extracted. Characters and graphics drawn on the road surface are excluded from this object image. The reason is that characters and figures drawn on the road surface cannot be an obstacle of the vehicle. The object image is excluded from the vertically symmetrical shape. The reason is that, in the method of the present embodiment, it is impossible to identify whether or not an object image having a vertically symmetric shape includes a reflection image of a road surface.

  An object that can be an obstacle of the own vehicle that is actually extracted by the obstacle candidate extracting means M1 is, for example, a pedestrian or a vehicle. Hereinafter, a case where a pedestrian is recognized as an obstacle will be described as an example.

  3 is a pedestrian object image extracted by the obstacle candidate extracting means M1 when the road surface is dry, and the reference symbol Q0 in FIG. 4 is when a water pool is formed over a wide area of the road surface. The object image of the pedestrian extracted by the obstacle candidate extraction unit M1, and the symbol R0 in FIG. 5 indicates the object image of the pedestrian extracted by the obstacle candidate extraction unit M1 when there is a puddle on a part of the road surface. It is. When the road surface is wet and a large puddle is formed, the pedestrian's figure is reflected on the road surface, so the true object image of the pedestrian and the reflected image of the pedestrian that is vertically symmetrical with respect to the road surface The integrated object is extracted as an object image by the obstacle candidate extracting means M1 (see Q0 in FIG. 4). In such a case, if the true object image of a pedestrian and the reflection image of a pedestrian that is symmetrical with respect to the pedestrian are mistakenly recognized as an object image of a single pedestrian, Since the position of the pedestrian is recognized based on the lower end (the position corresponding to the head), the distance from the vehicle to the pedestrian is mistakenly recognized as being closer than the actual distance, and a useless warning or automatic braking is performed. there is a possibility.

  In order to avoid such an inconvenience, according to the present embodiment, as shown in FIGS. 3 to 5, the reverse image creating means M2 converts the object images P0, Q0, R0 into the upper object images P1, Q1, R1 and the lower object images P2, Q2, and R2 are divided into upper and lower parts, and the lower object images P2, Q2, and R2 are turned upside down to create inverted images P2 ′, Q2 ′, and R2 ′. As a result, the object image P0 in FIG. 3 that does not include the reflected image is divided into an upper object image P1 corresponding to the upper half of the pedestrian and an inverted image P2 ′ corresponding to the lower half of the pedestrian. On the other hand, the object image Q0 of FIG. 4 including the reflection image is divided into an upper object image Q1 corresponding to the whole body of the pedestrian and a reverse image Q2 ′ corresponding to the whole body of the pedestrian. Furthermore, the object image R0 in FIG. 5 in which the head of the lower object image R2 is missing due to a puddle on a part of the road surface is divided into two parts in the upper and lower directions, and the foot part of the upper object image R1 is missing. Therefore, the foot portion is added to the reverse image R2 ′ side where the head is missing.

  The similarity determination means M3 includes a correlation value between the upper object image P1 and the inverted image P2 ′ in FIG. 3, a correlation value between the upper object image Q1 and the inverted image Q2 ′ in FIG. 4, and an upper value in FIG. A correlation value between the object image R1 and the reverse image R2 ′ is calculated.

In the case of FIG. 3, since the upper object image P1 corresponds to the upper body of the pedestrian and the inverted image P2 ′ corresponds to the lower body of the pedestrian that is inverted vertically, the upper object image P1 and the inverted image P2 ′ The correlation value A is less than the second threshold th2. in this way,
A <th2
Is established, the similarity determination unit M3 determines that the correlation value A between the upper object image P1 and the inverted image P2 ′ is low, and the obstacle recognition unit M5 determines the object image before being divided into two vertically (FIG. 3). P0) is recognized as an object image of an obstacle.

On the other hand, in the case of FIG. 4, since the upper object image Q1 corresponds to the whole body of the pedestrian and the inverted image Q2 ′ also corresponds to the whole body of the pedestrian, the correlation between the upper object image Q1 and the inverted image Q2 ′. The value A is equal to or greater than the first threshold th1 that is greater than the second threshold th2. in this way,
A ≧ th1
Is established, the similarity determination unit M3 determines that the correlation value A between the upper object image Q1 and the inverted image Q2 'is high, and the obstacle recognition unit M5 determines the upper object image (upper and lower divided) ( 4 is recognized as an object image of an obstacle.

Further, in the case of FIG. 5, the upper object image R1 lacks the foot part of the pedestrian and the inverted image R2 ′ lacks the head and the foot part is added, so the upper object image R1 and the inverted image R2 are added. The correlation value A with ′ is medium, and is greater than or equal to the second threshold th2 and less than the first threshold th1. in this way,
th2 ≦ A <th1
Is established, the similarity determination means M3 determines that the correlation value A between the upper object image R1 and the lower inverted image R2 'is medium, and the road surface position estimation means M4 determines the road surface position in the object image R0. (See FIG. 5).

  Specifically, in FIG. 5, the vertical dividing line is moved in the vertical direction with reference to the center position in the vertical direction of the object image R0, and the correlation value A between the object image R1 and the reverse image R2 ′ sandwiching the moved vertical dividing line. And a final upper and lower dividing line is set at a position where the calculated correlation value A is maximized. When the correlation value A is maximized, the position of the upper and lower dividing lines is the boundary between the foot of the object image R1 and the foot of the reverse image R2 ′, that is, the road surface position. This is because when the object image R1 and the inverted image R2 ′ are divided at the road surface position, the correlation value A is maximized. The obstacle recognition means M5 recognizes the object image above the road surface position (R1 ′ in FIG. 5) as an obstacle object image.

  When estimating the road surface position (see FIG. 5), the calculation load for estimating the road surface position is increased by moving the vertical dividing line in the vicinity of the vertical center position and below the vertical center position. Can be minimized. This is because the lower inverted image R2 is partially missing, and the road surface position should be slightly shifted below the center position in the vertical direction of the object image R0.

  The road surface state determination means M6 determines that the road surface is divided when the object image (see P0 in FIG. 3) before being divided into two parts is recognized as an obstacle object image, that is, when there is no reflection on the road surface. Is determined to be dry. On the other hand, the road surface state determination means M6 determines that the upper object image (see Q1 in FIG. 4) after being divided into two parts is recognized as an obstacle object image, that is, when reflection on the road surface is occurring. It is determined that the road surface is wet over a wide area. Further, when the road surface condition judging means M6 recognizes that a part of the reverse image (see R2 'in FIG. 5) after being divided into two parts is missing, that is, reflection on the road surface is partially generated. If it is, it is determined that the road surface is partially wet.

  Thus, the vehicle control means M7 compares the distance and position of the obstacle recognized by the obstacle recognition means M5 with the driving state (vehicle speed and steering angle) of the own vehicle at that time, and the own vehicle becomes an obstacle. When it is determined that there is a possibility of contact, the alarm device W is activated to urge the driver to brake, or the brake device B is automatically activated to avoid contact with an obstacle. At this time, when the road surface state determining means M6 determines that the road surface state is wet, the timing for operating the alarm device W or the timing for operating the braking device B is advanced according to the wetness. Thus, contact with an obstacle can be avoided more reliably.

  The above operation will be described again based on the flowchart of FIG.

  First, in step S1, an image in front of the host vehicle is captured by the camera C, and if an image of an object that becomes an obstacle candidate such as a pedestrian or a vehicle is extracted from the image in step S2, an obstacle candidate in step S3. Is divided into an upper object image and a reversal image obtained by vertically inverting the lower object image. In step S4, a correlation value A between the upper object image and the inverted image is calculated. If the correlation value A is less than the second threshold th2 in step S5, the object image before the two-division is directly obstructed in step S6. It is determined that the road surface is not wet in step S7. On the other hand, if the correlation value A is not less than the second threshold th2 in step S5 and the correlation value A is greater than or equal to the first threshold th1 in step S8, the inverted image is discarded in step S9 and only the upper object image is obstructed. It recognizes as an image of a thing, and it determines with the road surface getting wet in a wide range at Step S10.

  If the correlation value A is greater than or equal to the second threshold th2 and less than the first position th1 in steps S5 and S8, the road surface position in the object image is estimated in step S11, and the object image above the road surface position is determined in step S12. It is recognized as an image of an obstacle, and it is determined in step S13 that the road surface is partially wet.

  In step S14, vehicle control corresponding to the road surface condition is executed.

  As described above, according to the present embodiment, only the single camera C is used without the need for two stereo cameras, and the influence of the reflection of a wet road surface is eliminated, and the pedestrian or vehicle The shape and position of such an obstacle can be identified with high accuracy. In addition, since it is not necessary to process two images captured by two stereo cameras, it is only necessary to process one image, so that the processing load on the electronic control unit U is reduced and quick processing is possible. be able to.

  The embodiments of the present invention have been described above, but various design changes can be made without departing from the scope of the present invention.

  For example, any known method can be used for the process of extracting the object image from the image captured by the camera C.

  Any known method can be used for the process of calculating the correlation value A between the upper object images P1, Q1, R1 and the inverted images P2 ′, Q2 ′, R2 ′.

Side view of a car equipped with a camera Block diagram of obstacle recognition device Illustration of obstacle recognition process when the road surface is not wet Illustration of obstacle recognition process when the road surface is wet over a wide area Illustration of obstacle recognition process when the road surface is partially wet Obstacle recognition flowchart

Explanation of symbols

C camera (imaging means)
P0, Q0, R0 Object images P1, Q1, R1, R1 'Upper object images P2, Q2, R2 Lower object images P2', Q2 ', R2' Reverse image M1 Obstacle candidate extraction means M2 Reverse image creation means M3 Similarity determination means M4 Road surface position estimation means M5 Obstacle recognition means M6 Road surface condition determination means

Claims (5)

In the obstacle recognition apparatus for recognizing an object image in an image around the host vehicle imaged by the imaging means (C) as an obstacle,
Obstacle candidate extraction means (M1) for extracting obstacle candidate object images (P0, Q0, R0) from the object images;
The object image (P0, Q0, R0) extracted by the obstacle candidate extraction means (M1) is divided into upper and lower object images (P1, Q1, R1) and lower object images (P2, Q2, R2). And a reverse image creating means (M2) for creating a reverse image (P2 ′, Q2 ′, R2 ′) obtained by vertically inverting the lower object image (P2, Q2, R2);
Similarity determination means (M3) for determining the similarity between the upper object image (P1, Q1, R1) and the inverted image (P2 ′, Q2 ′, R2 ′);
When the similarity is greater than or equal to the first threshold (th1), the upper object image (Q1) is recognized as an obstacle, and when the similarity is less than the second threshold (th2), the object before the bisection Obstacle recognition means (M5) for recognizing the image (P0) as an obstacle;
An obstacle recognition device comprising:   Road surface position estimation means (M4) for estimating a position where the similarity is the highest as a road surface position when the similarity is greater than or equal to the second threshold (th2) and less than the first threshold (th1), The obstacle recognition device according to claim 1, wherein the obstacle recognition means (M5) recognizes the object image (R1 ') above the road surface position as an obstacle.   The obstacle recognition apparatus according to claim 2, wherein the road surface position estimating means (M4) estimates the road surface position within a predetermined range from the center in the vertical direction of the object image (R0) before division.   The obstacle recognition according to claim 2 or 3, wherein the road surface position estimating means (M4) estimates a road surface position from the center in the vertical direction of the object image (R0) before division. apparatus.   The obstacle recognition apparatus according to any one of claims 1 to 4, further comprising road surface state determination means (M6) for determining a road surface state based on the similarity.

Images