JP2009025932A - Obstacle recognition device - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To reliably and stably recognize an obstacle by an easy method through simple processing of photographic images. <P>SOLUTION: The photographic images for nearly the same road surface areas photographed by a photographing means 3 of one's own traveling vehicle 1 from a plurality of different directions are acquired by an image acquisition means, a prescribed monitoring range included in the photographic image from each the direction is extracted and identified by an identification means of an image processing means 4, and the obstacle perpendicular to a road surface in front of or in the rear of the own vehicle 1 is stably and certainly recognized by a recognition means of the image processing means 4 by simple difference detection of the images without performing complicated image processing or the like based on an identification result. <P>COPYRIGHT: (C)2009,JPO&INPIT

Description

  The present invention relates to an obstacle recognition device that recognizes an obstacle on a road on which a vehicle travels by processing a captured image.

  Conventionally, when realizing pre-crash safety (reduction of damage, collision avoidance, etc.) of a vehicle, it is important how to recognize obstacles in front of and behind the traveling vehicle.

As a device for recognizing this kind of obstacle, conventionally, one direction (for example, the front of the own vehicle) is photographed at regular intervals by a camera mounted on the own vehicle, and the captured images at time ta and time tb are projected. The difference is taken after the conversion, and the feature point where the time lag has occurred is detected from the difference, the optical flow of the motion vector is detected from these feature points, and the obstacle such as the vehicle in front of the host vehicle (the road surface vertical object) ) Has been proposed (see, for example, Patent Document 1).
Japanese Patent No. 3463858 (for example, claim 1, paragraphs [0014], [0041]-[0045], [0118], FIGS. 1 to 3)

  In the case of the conventional apparatus described in Patent Document 1, by taking a projective transformation of a captured image whose time is around and taking the difference, the difference image is a road marking character or figure drawn on the road surface (paint on the road surface). ) Is removed, and the motion vector is detected from the difference image, and an obstacle on the road can be detected without erroneously recognizing the road marking. There is a problem that requires extremely complicated image processing to detect the optical flow using, and it is easy to recognize the obstacle reliably and stably because the obstacle is recognized from a small motion vector on the image. There is no problem.

  In the image of the obstacle, the lower end portion close to the road surface becomes dark due to the influence of the shadow of the obstacle itself. In addition, when the projective transformation is performed, the shape of the lower end portion close to the road surface does not change much with respect to a change in the distance between the vehicle and the obstacle due to the characteristics of the projective transformation. Therefore, the image of the difference is in a state in which the outline of the lower end portion of the obstacle near the road surface has almost disappeared. Therefore, it is difficult to grasp the distance between the obstacle and the vehicle necessary for avoiding or predicting the collision based on the image distance from the lower end of the obstacle in the difference image.

  Further, in the case of the conventional device, if the obstacle is a preceding vehicle or the like ahead of the own vehicle traveling at the same speed as the own vehicle, there is no change in the distance between the obstacle and the own vehicle, and the difference hardly occurs. Therefore, there is a problem that obstacles such as preceding cars cannot be recognized.

  That is, the conventional obstacle recognition device of this type uses a photographed image viewed from one direction such as the front of the host vehicle, and changes the distance or shape of the vehicle between the obstacle such as the preceding vehicle of the image and the host vehicle. Since it is a configuration that recognizes an obstacle from a slight difference, it not only requires complicated image processing to detect an optical flow, but also it is not easy to recognize the obstacle, and there is a possibility that the obstacle cannot be recognized depending on the situation. There is no stable and reliable recognition of obstacles.

  An object of the present invention is to reliably and stably recognize an obstacle by an easy method by simple processing of a captured image.

  In order to achieve the above-described object, the obstacle recognition device of the present invention includes a photographing unit capable of photographing a road surface viewed from a plurality of directions of a traveling vehicle and a plurality of directions of the photographing unit for substantially the same road surface area. The image acquisition means for acquiring the captured image, the identification means for extracting and identifying a predetermined monitoring range included in the captured image from a plurality of directions acquired by the image acquisition means, and the identification result of the identification means, Recognizing means for recognizing an image portion that differs between captured images in the monitoring range as an obstacle perpendicular to the road surface is provided.

  The identifying means preferably performs projective transformation in addition to extraction of the monitoring range to identify the monitoring range of each captured image.

  The obstacle recognition device according to the present invention further comprises road surface information acquisition means for acquiring road surface information such as a road surface shape of the traveling road of the own vehicle in the obstacle recognition device having the configuration according to claim 1, Is characterized in that, in addition to the extraction of the monitoring range, image processing based on the road surface information is performed to identify the monitoring range of each captured image (Claim 3).

  In the case of the invention of claim 1, if a plurality of directions are assumed to be the two directions before and after the own vehicle for the sake of simplicity of explanation, the image acquisition means is, for example, a certain front part of the own vehicle taken by the photographing means at time tm. A captured image (front image) of the road surface area and a captured image (rear image) of substantially the same road surface area behind the own vehicle captured by the imaging means at time tn when the vehicle passed through the road surface area are acquired.

  Then, the identification unit extracts the substantially same monitoring range of the front image and the rear image acquired by the image acquisition unit, and identifies from the difference between the images.

  At this time, if the monitoring range is set, for example, to a range where obstacles ahead and behind the vehicle may exist, if there is an obstacle ahead of the vehicle, there is an obstacle in the monitoring range of the front image. However, there is no obstacle in the rear image monitoring range. On the contrary, if there is an obstacle behind the host vehicle, there is no obstacle in the monitoring range of the front image, but there is an obstacle in the monitoring range of the rear image.

  That is, when an obstacle exists in front of or behind the host vehicle, the obstacle is included only in one of the monitoring range of the front image and the rear image, and the contents of the monitoring range of both images are greatly different.

  Therefore, it is possible to easily and reliably recognize an obstacle in front of or behind the host vehicle by simply detecting the difference in the image without performing complicated image processing for detecting an optical flow. In this case, of course, even if the obstacle is a preceding or following vehicle that travels at the same speed as the own vehicle, the obstacle can be reliably recognized.

  In addition, when there are no obstacles in both the front and rear of the vehicle, the monitoring range of either the front image or the rear image will be an image of only a similar road surface that does not include an obstacle. It is possible to reliably recognize that there are no obstacles in front of and behind the vehicle by matching the images. In addition, when there are obstacles in both the front and rear of the vehicle, the obstacles are included in the monitoring range of both the front and rear images, but the obstacles in both images are separate. In addition, even in the same type of vehicle, the shape on the front side and the shape on the rear side often differ greatly, and thus the shape, size, position, and the like do not match at all. Therefore, also in this case, since both images do not coincide with each other, it can be surely recognized that an obstacle exists in front of and behind the host vehicle.

  And, of course, when the plurality of directions are two directions of the front, rear, left and right of the own vehicle, and when the plurality of directions are three or more directions of the front, rear, left and right of the own vehicle. In addition, based on the presence or absence of obstacles in the monitored range of each captured image, obstacles can be easily and reliably recognized by the same image processing for image comparison as described above.

  According to the second aspect of the present invention, an image of a plane (bird's-eye view plane) obtained by bird's-eye view (bird's-eye view) of the monitoring range of each captured image is obtained by projective transformation of the identification means.

  At this time, in the image of each overhead view plane, the obstacle perpendicular to the road surface of the monitoring range changes in shape due to the difference in shooting time, but the shape of the road marking painted on the road surface of the monitoring range hardly changes. .

  Therefore, the identification means can remove the road marking from the image of each overhead view plane after the projective transformation and identify the obstacle more accurately.

  According to the invention of claim 3, instead of performing the projective transformation, the identification means performs image processing based on the road surface information of the road surface information acquisition means to extract obstacles and road markings in the monitored range of each captured image. The obstacle can be easily and accurately identified by removing the road marking from the difference between the images.

  Next, in order to describe the present invention in more detail, an embodiment corresponding to claims 1 and 2 will be described in detail with reference to FIGS.

  FIG. 1 shows a block configuration of an obstacle recognition device 2 mounted on the host vehicle 1, FIG. 2 shows a configuration example of an imaging means of the obstacle recognition device 2, and FIG. 3 is for explaining the operation of the obstacle recognition device 2. FIG. 4 shows a processing example of the photographed image, and FIG. 5 shows the identification result.

  FIG. 6 is an explanatory diagram of projective transformation, FIG. 7 is an explanatory diagram of a photographing example when photographing and recognizing only the front of the vehicle 1, and FIG. 8 is an explanatory diagram of projective transformation of the photographing result of FIG. 9 is an explanatory diagram of a processing example of a captured image when there is no obstacle in FIG. 7, and FIG. 10 is an explanatory diagram of a processing example of the captured image when there is an obstacle in FIG.

(Constitution)
As shown in FIG. 1, an obstacle recognition device 2 mounted on a vehicle 1 has a photographing unit 3, an image processing unit 4 having a microcomputer configuration for processing and identifying the photographed image, and a photographed image and the like can be rewritten. The data storage means 5 stores the data.

  The photographing means 3 is composed of a monochrome or color monocular camera capable of photographing a road surface viewed from a plurality of directions of the traveling vehicle 1 and outputs a frame image (or field image) taken every moment.

  The plural directions are directions around the vehicle 1 as viewed from the own vehicle 1, and in practical terms, all or a part of the front, rear, left, and right of the own vehicle 1 (partially two or more directions). is there.

  And in this embodiment, in order to demonstrate easily, let the imaging | photography direction of the imaging | photography means 3 be two directions of the front of the own vehicle 1, and back.

  In the case of the present embodiment, the photographing means 3 is formed by providing a monocular camera for each photographing direction, for example, so that the plural directions are two directions forward and backward of the host vehicle 1 in order to perform the road surface photographing in the plural directions. As shown in FIG. 2, monocular cameras 3a and 3b are provided at upper and lower positions in the vehicle 1 so as to aim at an obliquely lower road surface from above at an appropriate angle from above. Is formed.

  The monocular camera 3a is a camera mounted on the vehicle 1 as a so-called pre-crash sensor, and the monocular camera 3b is a camera that is also used as a back sensor, for example.

  Further, an image of the road surface obliquely below the entire circumference of the own vehicle 1 may be taken by the photographing means 3, and in this case, for example, a conical mirror is provided in the upper part of the interior of the own vehicle 1, and this mirror A photographed image viewed from the entire circumference of the traveling vehicle 1 is projected on the vehicle, and the photographed image is photographed by one or a plurality of monocular cameras.

  Next, the image processing means 4 forms the image acquisition means, the identification means, and the recognition means of the present invention, and repeatedly executes the recognition processing program in steps A1 to A6 in FIG.

  The image acquisition unit is a unit that acquires captured images from a plurality of different directions of the imaging unit 3 for substantially the same road surface area. In the present embodiment, the image acquisition unit operates in steps A1 and A2 in FIG. For example, a photographed image (hereinafter referred to as a forward image) P (t1) of the road surface area in front of the host vehicle 1 of FIG. 4 taken by the camera 3a is captured and held in the data storage means 5, and a vehicle speed sensor (not shown) A captured image (hereinafter referred to as a rear image) P (t2) of substantially the same road area behind the vehicle 1 of FIG. 4 taken by the monocular camera 3b at time t2 delayed by a time determined from the vehicle speed or the like is captured. The data is stored in the data storage means 5.

  The identification unit is a unit that extracts and identifies a predetermined monitoring range included in the captured images from a plurality of directions acquired by the image acquisition unit. In this embodiment, the identification unit operates in steps A3 and A4 in FIG. Then, projective transformation described later is performed on at least a road surface portion of the front image P (t1) and the rear image P (t2), and a predetermined monitoring range including an obstacle of both the images P (t1) and P (t2) is extracted. Identify.

  The recognizing unit operates in step A5 in FIG. 3, and recognizes an image portion different between the images P (t1) and P (t2) in the monitoring range as an obstacle perpendicular to the road surface based on the identification result of the identifying unit. The recognition result is sent to a collision prediction processing means (not shown) for reducing damage of the own vehicle 1 or avoiding collision.

  Next, the projective transformation and the like of the image acquisition unit will be specifically described.

  First, the projective transformation is performed to remove unnecessary road markings in the images P (t1) and P (t2). Generally, as shown in FIG. This is a transformation in which a point P (x, y) in the y coordinate system is projected as a point P ′ (u, v) in the uv coordinate system of another plane L ′ with respect to the projection center O. By this conversion, the points P, A, B, C, and D on the plane L are projected on the plane L ′ as points P ′, A ′, B ′, C ′, and D ′.

  The projective transformation is expressed by the following equation (1) and equation (2) using the parameters a11, a12, a13, a21, a22, a23, a31, a32, and a33. .

  However, the condition of the following expression (3) is imposed on each parameter a11,..., A33.

  Next, the removal of road marking by this projective transformation will be described.

  First, as shown in FIG. 7, only the monocular camera 3aa that is the same as the monocular camera 3a is mounted on the own vehicle 1, and a certain road surface area in front of the own vehicle is inclined obliquely from above by the monocular camera 3aa while the own vehicle 1 is traveling. Shooting and obtaining time-series captured images P (t) in front of the vehicle, and in these captured images P (t), the plane L is an imaging plane and the plane L ′ is an overhead plane after projective transformation. For example, as shown in FIG. 8, projective conversion is performed and converted to an image P (t) ′ of a bird's-eye view plane from above the captured image P (t).

  In this case, in the image P (t) ′ of the overhead view plane after the projective transformation, an obstacle (such as a preceding vehicle) α perpendicular to the road surface in the photographed image P (t) and road markings β, β * painted on the road surface And, a difference arises in the time change of a shape. The road marking β is a white line on the left and right of the traveling lane, and the road marking β * is an arrow or a number in the traveling lane.

  That is, a captured image (front image) P (t11) at time t = t11 of the monocular camera 3aa, and a captured image (front image) P (t22) at time t = t22 delayed by Δt seconds from the image P on the overhead view plane, respectively. When projective transformation is performed to (t11) ′ and P (t22) ′, the road markings β and β * on the road plane hardly change in shape after the projective transformation depending on the shooting times t11 and t22. The obstacle α having a difference in shape after the projective transformation changes depending on the shooting times t11 and t22.

  Therefore, when the difference in brightness between the pixels of the images P (t11) ′ and P (t22) ′ is obtained and identified, the road markings β and β * are substantially removed.

  Then, by removing the road markings β and β *, if there is no obstacle α, the identification result is “no image”, and if the obstacle α exists, the identification result is the shape of the obstacle α. “Difference image exists” based on the difference, and the obstacle α can be recognized by the difference in the identification result.

  Explaining with an actual photographed image example, when there is no preceding vehicle that becomes an obstacle α in the monitoring range in front of the host vehicle 1, it is obtained by photographing a certain road surface area in front of the host vehicle with the monocular camera 3aa. In the image P (tx1) at time tx1 in FIG. 9 and the image P (ty1) at time ty1 delayed by Δt seconds, for example, only the road markings β and β * exist in the monitoring ranges x and y in front of the own vehicle. In the projected images P * (tx1) and P * (ty1) after the projective transformation, only road markings β and β * exist in substantially the same shape (including dimensions).

  On the other hand, when there is a preceding vehicle that becomes an obstacle α in the monitoring range in front of the host vehicle 1, the image at time tx2 in FIG. 10 obtained by photographing a certain road surface area in front of the host vehicle with the monocular camera 3aa. An image P (ty2) at time ty2 delayed by P (tx2) and then Δt seconds includes, for example, an obstacle α and a road marking β in the monitoring range x, y ahead of the host vehicle, and an image P after projective transformation thereof. * (Tx1) and image P * (ty1) have obstacles α in different shapes and road markings β in substantially the same shape.

  Then, when the difference in brightness of each pixel of the image P * (tx1) and the image P * (ty1) in FIG. 9 is obtained, the identification image P1xy in the same figure is obtained. Similarly, when the difference in brightness of each pixel in the image P * (tx2) and the image P * (ty2) in FIG. 10 is obtained, the identification image P2xy in the same figure is obtained.

  At this time, in the identification image P1xy, road signs β and β * having substantially the same shape in the monitoring ranges x and y cancel each other out, and the identification image P2xy disappears substantially in the same shape in the monitoring ranges x and y. * Cancels and almost disappears, but the obstacle α having a different shape includes a contour image of the difference.

  Since the above-described difference occurs between the identification image P1xy and the identification image P2xy, in principle, the obstacle α in front of the host vehicle 1 is recognized also by the captured images taken before and after the same direction with the monocular camera 3aa. obtain. In addition, in principle, the vehicle 1 is equipped with a monocular camera similar to the monocular camera 3b, and the same projective transformation is applied to the captured image of the rear of the vehicle taken by the camera before and after the time. Obstacles behind 1 can also be recognized.

  However, for example, when there is an obstacle α, the captured images P (tx2) and P (ty2) taken before and after include the target obstacle α as apparent from FIG. Since the difference between the shapes of the obstacles α in the images P * (tx2) and P * (ty2) is slight, the identification result image P2xy is not clear because the area of the obstacle α is small.

  On the other hand, when there is no obstacle α, as is clear from FIG. 9, images P * (tx1) and P * (P * (tx1) and P (ty1) obtained by projective transformation of the captured images P (tx1) and P (ty1) taken before and after. In ty1), the shapes and positions of the road markings β and β * do not completely match, and actually, an image based on these mismatches remains in the image P1xy of the identification result.

  For this reason, the method of projecting and recognizing captured images taken before and after the same direction from the same direction is likely to cause erroneous recognition of the obstacle α depending on the driving environment and the like, and the obstacle α is recognized reliably and stably. Is difficult.

  Moreover, as is clear from the image P2xy of the identification result, the lower end portion of the obstacle α has a little change in shape, and the outline is almost lost. Therefore, it is not easy to obtain the distance between the obstacle α and the vehicle 1 by obtaining the image distance from the lower end of the obstacle α in the image p2xy as the identification result.

  Therefore, in the present invention, as described above, the same road surface area is photographed from a plurality of different directions of the own vehicle 1 traveling by the photographing means 3.

  That is, in the case of the present embodiment, the monocular camera 3a, 3b in FIG. 2 takes several seconds determined by the monocular camera 3a taking a fixed road surface area as the front image P (t1) at t1, for example, based on the traveling speed of the vehicle 1 and the like. The monocular camera 3b captures the same road surface area as a rear image P (t2) at time t2 after a certain amount of time has elapsed, and the front image P (t1) and time at the time t1 of the preceding and following monocular camera 3a by the image acquisition means. The rear image P (t2) of t2 is captured.

  At this time, if there is a preceding vehicle (automobile) as an obstacle α in front of the vehicle 1, the front image P (t1) includes the obstacle α as shown in FIG. The image P (t2) does not include the obstacle α. Further, if there is a following vehicle (two-wheeled vehicle) as another obstacle α ′ behind the host vehicle 1, as shown in FIG. 4, the rear image P (t2) includes the obstacle α ′. However, the front image P (t1) does not include the obstacle α ′.

  Then, in the present embodiment, the identification unit performs projective conversion of the images P (t1) and P (t2) as they are into an overhead plane image. This projective transformation detects the white line of the road marking β when the images P (t1) and P (t2) of the photographing planes of the monocular cameras 3a and 3b are detected by lane width recognition using a well-known Hough transform. From the viewpoint of image comparison and the like, it is preferable to set the parameters a11 to a33 so that the detected white line is a vertical line of the images P (t1) and P (t2). Of course, the white line may be either a continuous line or a broken line. Further, when traveling on a road without a white line, for example, projective transformation may be performed with a set standard parameter.

  Furthermore, the identification means uses the central portion of the road surface portion below the horizontal line of the lane portion where the vehicle 1 travels as a monitoring range for predicting the presence of an obstacle from the projected transformation images of the images P (t1) and P (t2). For example, the prediction images P * (t1) and P * (t2) in the front and rear of FIG. 4 are formed, and the lane widths and the like are aligned for both the images P * (t1) and P * (t2). The brightness is compared for each pixel by overlapping in a state, and the difference is detected and identified.

  At this time, the image of the identification result obtained by superimposing the images P * (t1) and P * (t2) has a substantially forward image P as shown in FIG. * If the image of the part of the obstacle α in (t1) is present and the obstacle α ′ exists only in the rear, the obstacle α in the substantially rear image P * (t2) as shown in FIG. It becomes the image of 'part. If there are no obstacles α and α ′ in front and rear, the image of the identification result is a so-called blank image. If obstacles α and α ′ exist in the front and rear, The image is, for example, an image obtained by superimposing the images shown in FIGS.

  Even when driving at night or in tunnels, the headlights and taillights of the vehicle 1 and the obstacle α ensure the brightness necessary for the identification of the roadway, so that it can be identified without problems as in daytime. Can be done.

  Then, the recognizing means, for the portions where the brightness between the predicted images P * (t1) and P * (t2) is different, for example, the predicted images P * (t1) and P * (t2) having different brightness from the surroundings. Grasping and recognizing that there are obstacles α, α 'on the side that are perpendicular to the road surface, such as a preceding vehicle and a following vehicle, and processing the recognition result to reduce damage to the vehicle 1 and avoid collision (not shown) Send to etc.

  When there are no obstacles α and α ′ in front and rear, the predicted images P * (t1) and P * (t2) are images of only similar road surfaces in which no obstacles α and α ′ exist. Since there is no portion with different brightness between the predicted images P * (t1) and P * (t2), it is recognized that there are no obstacles α and α ′ in front and rear. When obstacles α and α ′ are present in the front and rear, the image of the front identification result is, for example, an image obtained by superimposing the images in FIGS. 5A and 5B. Obstacles α and α ′ in the images P * (t1) and P * (t2) are separate, and the shape of the front side and the shape of the rear side are greatly different even in the same type of vehicle. Because there are many, the shape, size, position, etc. do not match at all, and since images of a plurality of obstacles α, α ′ are obtained as a recognition result, the obstacles α, α ′ are present forward and backward. Recognize that it exists.

  Next, the recognized obstacles α and α ′ have clear outlines including the lower end portion on the image of the identification result. Therefore, the recognizing means accurately obtains the distance (inter-vehicle distance) from the lower end of the obstacles α, α ′ to the own vehicle 1 on the image so that the vehicle 1 and the obstacles α, α ′ The distance can be detected, and the distance of the detection result is also sent to the above-described damage reduction and collision avoidance processing means.

  Therefore, in the case of the present embodiment, the front or rear of the host vehicle 1 or both of the front and rear of the host vehicle 1 can be detected by simple difference detection of the captured images before and after the host vehicle 1 without performing complicated image processing or the like for detecting the optical flow. Obstacles α and α ′ can be recognized stably and reliably.

  In this case, of course, even if the obstacles α and α ′ are the preceding vehicle and the following vehicle that travel at the same speed as the own vehicle 1, the obstacle α can be surely recognized.

  In addition, since the captured images P (t1) and P (t2) are subjected to projective transformation to convert them into images P * (t1) and P * (t2) of the overhead view plane, the vehicle 1 is traveling. The obstacle α can be recognized without any problem even when the road is undulated or the traveling path of the host vehicle 1 is an uphill or downhill.

  The obstacle α is not limited to a moving body such as a vehicle or a person, and can be recognized similarly even if it is a stationary object such as a utility pole or a signboard.

  For example, the obstacles α and α ′ of the images P (t1) and P (t2) can be recognized by detecting horizontal and vertical edges by a so-called edge histogram method. α and α ′ are difficult to recognize in the case of a person with a difficult horizontal edge, a thin vertical object such as a motorcycle or a utility pole. On the other hand, in the case of the present invention, horizontal and vertical edge detection is unnecessary, and even if the obstacles α and α ′ are thin vertical objects, they can be reliably recognized.

  Next, since the images of the obstacles α and α ′ obtained as a result of the identification are clear and the entire image including the lower end portion can be clearly recognized, from the lower ends of the obstacles α and α ′ measured on the image From the length to the own vehicle 1, it is possible to accurately detect and grasp the distances from the own vehicle 1 to the obstacles α and α ′ that change every moment that is important for collision prediction.

  The present invention is not limited to the above-described embodiment, and various modifications other than those described above can be made without departing from the spirit of the present invention.

  For example, in the above embodiment, the identification unit first performs projective transformation on a captured image from a different direction, and then extracts the monitoring range to obstruct the obstacle. The monitoring range may be extracted from the captured image, and then the projection range may be subjected to projective transformation to recognize the obstacle.

  Next, in the embodiment, the front and rear obstacles α and α ′ are recognized based on the front image P (t1) and the rear image P (t2) obtained by photographing the road surface from the front and rear of the host vehicle 1. However, the combination of the shooting directions is not limited to the front and rear, for example, the shooting direction is two directions of the front or rear and left or right side, or the front, Afterward, it may be in three or more directions, either left or right.

  Specifically, for example, when the rear and the side are combined and applied to prevent the collision of an obstacle such as a motorcycle at the time of turning right or left, the above-described vehicle 1 is used as a photographing unit as shown in FIG. The monocular camera 3b is mounted, and the monocular cameras 3c and 3d are attached to one or both of the left and right side mirrors of the vehicle 1 or the like.

  Then, the case where the vehicle 1 is applied to the collision prevention of the obstacle involving at the time of the left turn effective when the vehicle 1 is the right steering wheel will be described. The road surface viewed from the left side at the time t1 is photographed by the monocular camera 3c of the left side mirror. Then, at the time t2 several seconds after that, the same road surface region viewed from the rear is photographed by the monocular camera 3b.

  At this time, the left side of the rear image P (t2) of the monocular camera 3b is substantially the same range (monitoring range) as the image (hereinafter referred to as the left image) P (t1) taken by the monocular camera 3c at time t1 from different directions. This is a photographed image. If there is an obstacle on the side, the obstacle is included only in the left image P (t1), and if there is an obstacle behind, the obstacle is included only in the rear image P (t1). .

  Therefore, the left image P (t1) and the rear image P (t2) are processed by the image acquisition unit, the identification unit, and the recognition unit of the image processing unit 4 in the same manner as in the above embodiment, and the left image P (t1), An obstacle is recognized by identifying the monitoring range of the image of the overhead view of the rear image P (t2).

  In addition, when applying to the collision prevention of the obstacle involving at the time of the right turn, the road surface is photographed from the right side at time t1 by the monocular camera 3d of the right side mirror, and from the rear by the monocular camera 3b at time t2 several seconds later. Recognize obstacles by photographing the same road surface area.

  In addition, when an obstacle is recognized based on captured images from three or more directions of front, rear, left, and right, a common range of substantially the same road surface area captured from different directions before and after them is displayed. It is extracted as a monitoring range, and obstacles can be recognized by the image acquisition means, identification means, and recognition means of the image processing means 4 in the same manner as in the above embodiment.

  Next, in the above-described embodiment, the obstacle is recognized by performing projective transformation on the captured images from different directions by the identification unit. However, the road surface of the traveling path of the own vehicle 1 such as a car navigation device is used as the own vehicle 1. Road surface information acquisition means for acquiring road surface information (map data) such as a shape, and the projection conversion processing is omitted from the identification means of the image processing means 4; in addition to the monitoring range extraction by the identification means, the road surface Image processing based on information may be performed to identify the monitoring range of each captured image and recognize an obstacle (corresponding to claim 3).

  Specifically, for example, with respect to the road surface ranges of the photographed front image P (t1) and rear photographed image P (t2), the identification unit first performs image processing based on the road surface information of the road surface information acquisition unit. Are taken from the front and rear (hereinafter referred to as a map image). These map images do not include road markings painted on the road surface.

  Then, the difference between the front image P (t1), the rear shot image P (t2), and the map image is taken, and the front image P (t1) and the rear shot image P (t2) are processed into substantially obstacle and road marking images. To do.

  Further, the front image P (t1) and the rear photographed image P (t2) processed into the image of the obstacle α and the road marking are taken as they are, or the necessary coordinate conversion is performed to obtain a difference in brightness. Easily and accurately identify obstacles by removing road markings from dissimilarities.

  Also in this case, the photographing direction may be two directions other than the front-rear direction or three or more directions.

  Next, the configuration and operation (processing procedure) of the photographing unit 3, the image processing unit 4 and the like are not limited to those of the above-described embodiment.

  The present invention can be applied to obstacle recognition of various vehicles.

It is a block diagram of one embodiment of the present invention. It is explanatory drawing of the structural example of the imaging means of FIG. It is a flowchart for operation | movement description of FIG. It is explanatory drawing of the example of a process of the picked-up image of FIG. It is explanatory drawing of the identification result of FIG. It is explanatory drawing of projective transformation. It is explanatory drawing of the imaging | photography example in the case of image | photographing only the front and recognizing. It is explanatory drawing of the projection conversion of the imaging | photography result of FIG. It is explanatory drawing of the example of a process of a picked-up image when there is no obstacle of FIG. It is explanatory drawing of the example of a process of a picked-up image when there exists an obstruction of FIG. It is explanatory drawing of the structural example of the other imaging means of this invention.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Own vehicle 2 Obstacle recognition apparatus 3 Imaging | photography means 4 Image processing means alpha, alpha 'Obstacle

Claims (3)

Photographing means capable of photographing the road surface from a plurality of directions of the traveling vehicle;
Image acquisition means for acquiring captured images from a plurality of different directions of the imaging means for substantially the same road surface area;
An identification unit that extracts and identifies a predetermined monitoring range included in a captured image from a plurality of directions acquired by the image acquisition unit;
An obstacle recognizing apparatus comprising: a recognizing unit that recognizes an image portion that differs between captured images in the monitoring range as an obstruction perpendicular to a road surface based on an identification result of the identifying unit. The obstacle recognition apparatus according to claim 1,
The obstacle recognizing apparatus characterized in that the identifying means performs projective transformation in addition to extraction of the monitoring range to identify the monitoring range of each captured image. The obstacle recognition apparatus according to claim 1,
It further comprises road surface information acquisition means for acquiring road surface information such as the road surface shape of the traveling road of the own vehicle,
The obstacle recognizing apparatus characterized in that the identification means identifies the monitoring range of each captured image by performing image processing based on the road surface information in addition to extraction of the monitoring range.