JP2010283722A - Obstacle detector and obstacle detection method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide "an obstacle detector and an obstacle detection method" capable of detecting a wall spreading around a vehicle as an obstacle only by image recognition processing without using an ultrasonic sensor or the like. <P>SOLUTION: Even in the case that an obstacle is not detected from the vehicle peripheral image of a certain photographing area as a result of performing obstacle detection processing by an obstacle detection part 25 for the vehicle peripheral images of the photographing areas in four directions, when it is determined by an overlapping area determination part 26 that the obstacle is detected in the overlapping area of the adjacent photographing areas for the vehicle peripheral image of the photographing area adjacent to the certain photographing area, it is considered that the obstacle is present in the photographing area of the vehicle peripheral image where the obstacle is not detected. Thus, even in the case that the wall projected in the vehicle peripheral image of the certain photographing area cannot be detected as an obstacle, the wall is detected as the obstacle by subsidiarily using an obstacle detection result in the overlapping area of the other vehicle peripheral images. <P>COPYRIGHT: (C)2011,JPO&INPIT

Description

  The present invention relates to an obstacle detection apparatus and an obstacle detection method, and in particular, compares an image of a current frame with an image of a previous frame, detects a motion vector from the difference, and is located around the vehicle based on the motion vector. It is suitable for use in an apparatus for detecting an obstacle.

  2. Description of the Related Art Conventionally, there has been provided a technique for automatically detecting obstacles around a vehicle and issuing a warning to a driver or automatically controlling braking according to the situation. There are many techniques for detecting obstacles using an image obtained from a vehicle-mounted camera. Among them, an image difference method (optical flow) that compares an image of the current frame with an image of the previous frame, detects a motion vector from the difference, and detects obstacles around the vehicle based on the motion vector. There is a device that uses the method.

  In addition, a method of detecting an obstacle (whether the object on the road is a plane object or a three-dimensional object) without using the optical flow method that increases the processing amount has been proposed (see, for example, Patent Document 1). Further, as another technique for detecting an obstacle, there is a technique that uses another sensor such as an ultrasonic sensor or a millimeter wave radar instead of using an image obtained from an in-vehicle camera (see, for example, Patent Document 2). .

JP-A-10-222679 Japanese Patent No. 3300334

  However, in the conventional technique for detecting an obstacle by image recognition processing using an image difference method or the like, when a wall is reflected on the entire surface of an image taken by an in-vehicle camera, an edge cannot be detected from the image, There was a problem that the wall could not be detected as an obstacle. Note that when a fisheye lens is used for the in-vehicle camera, the ground is also reflected in the image, so that the edge between the ground and the wall is detected. However, since the edge is at the same height as the ground (plane), it is recognized as a line drawn on the plane and cannot be recognized as an obstacle (solid object).

  On the other hand, if an ultrasonic sensor or the like is used, even a wall extending around the vehicle can be detected as an obstacle. However, providing a separate sensor while mounting a vehicle-mounted camera as in Patent Document 2 has a problem of increasing costs. In recent years, an increasing number of vehicles are equipped with an in-vehicle camera. Therefore, it is desired to detect an obstacle using an image captured by the in-vehicle camera in order to suppress an increase in cost.

  The present invention has been made to solve such a problem, and an object of the present invention is to enable detection of a wall extending around a vehicle as an obstacle only by a recognition process for an image photographed by an in-vehicle camera. And

  In order to solve the above-described problem, in the present invention, an obstacle detection process is performed on each of the vehicle surrounding images captured by a plurality of imaging devices installed so that a part of the imaging region has an overlapping region. I do. Here, even when an obstacle is not detected from a vehicle surrounding image shot by a certain imaging device, another vehicle surrounding image shot by another imaging device having a shooting area adjacent to the shooting area When an obstacle is detected in the overlapping area, it is assumed that an obstacle is also present in the shooting area of the vehicle surrounding image where no obstacle is detected.

  According to the present invention configured as described above, a wall is reflected on the entire surface of one vehicle surrounding image taken by a certain imaging device, and an edge cannot be detected from within the image, so that the wall is detected as an obstacle. Even if this is not possible, a part of the wall is shown in the overlapping area of other vehicle surrounding images taken by another imaging device having an adjacent shooting area. The edge is detected and can be detected as an obstacle. In the present invention, if an obstacle is detected in the overlap area for another vehicle surrounding image, it is considered that an obstacle is also present in the shooting area of the vehicle surrounding image where the obstacle could not be detected. A wall can be detected as an obstacle. Thereby, the wall spreading around the vehicle can be detected as an obstacle only by the recognition process for the image taken by the imaging device without using another sensor such as an ultrasonic sensor.

It is a figure which shows the structural example of the image processing system to which the obstacle detection apparatus by this embodiment is applied. It is a figure which shows the example of arrangement | positioning and imaging | photography area | region of the vehicle-mounted camera by this embodiment. It is a figure which shows an example of the specific scene where a wall exists around a vehicle. It is a flowchart which shows the operation example of the image processing system to which the obstacle detection apparatus by this embodiment is applied.

  Hereinafter, an embodiment of the present invention will be described with reference to the drawings. FIG. 1 is a diagram illustrating a configuration example of an image processing system to which the obstacle detection apparatus according to the present embodiment is applied. As shown in FIG. 1, the image processing system of the present embodiment includes an in-vehicle camera (imaging device) 1, an image processing unit 2, a display device 3, a control unit 4, and a notification unit 5. The in-vehicle camera 1 is a front camera 1a for photographing the front side of the vehicle, a left side camera 1b for photographing the left side of the vehicle, a right side camera 1c for photographing the right side of the vehicle, and a rear side of the vehicle. It is comprised by the rear camera 1d for doing.

  FIG. 2 is a diagram illustrating an example of the arrangement of the in-vehicle cameras 1a to 1d and the imaging region. In FIG. 2, the front camera 1a is installed in front of the vehicle 100, and images the front imaging region H1. The left side camera 1b is installed on the left side of the vehicle 100 and photographs the left photographing area H2. The right side camera 1c is installed on the right side of the vehicle 100 and shoots the right shooting area H3. The rear camera 1d is installed at the rear of the vehicle 100 and photographs the rear photographing area H4.

  The front shooting area H1 and the left shooting area H2 overlap in a partial area OA1. The front shooting area H1 and the right shooting area H3 overlap in a partial area OA2. The rear imaging area H4 and the left imaging area H2 overlap in a partial area OA3. In addition, the rear imaging area H4 and the right imaging area H3 overlap in a partial area OA4.

  The image processing unit 2 includes an image input unit 21, a viewpoint conversion vehicle surrounding image generation unit 22, a mapping table storage unit 23, a vehicle image storage unit 24, an obstacle detection unit 25, and an overlapping area determination unit 26 as its functional configuration. ing. The image input unit 21 inputs vehicle surrounding images captured by the plurality of in-vehicle cameras 1a to 1d. The viewpoint conversion vehicle surrounding image generation unit 22 performs viewpoint conversion processing on the vehicle surrounding images input from the plurality of in-vehicle cameras 1 a to 1 d by the image input unit 21, so that the vehicle surrounding image viewed from the virtual viewpoint above the vehicle (hereinafter, referred to as the vehicle surrounding image). , Referred to as a viewpoint conversion vehicle surrounding image).

  Specifically, the viewpoint conversion vehicle surrounding image generation unit 22 first generates a background image when the periphery of the vehicle 100 is viewed from above according to the conversion information stored in the mapping table storage unit 23. Here, the background image includes a front vehicle surrounding image generated by converting the image captured by the front camera 1a and a left side vehicle periphery generated by converting the image captured by the left side camera 1b. The image, the right side vehicle surrounding image generated by converting the viewpoint of the image captured by the right side camera 1c, and the rear vehicle surrounding image generated by converting the viewpoint of the image captured by the rear camera 1d are combined. To generate.

  The mapping table storage unit 23 is a table that describes the correspondence between the pixel data of the vehicle surrounding image captured by each of the in-vehicle cameras 1a to 1d and the pixel data of the background image when the periphery of the vehicle 100 is viewed from the upper virtual viewpoint. Yes, conversion information indicating which pixel in the background image is associated with a certain pixel in the photographed vehicle surrounding image is described. The vehicle image storage unit 24 stores a vehicle image indicating a shape when the vehicle 100 is viewed from above, and is read by the viewpoint conversion vehicle surrounding image generation unit 22. The viewpoint conversion vehicle surrounding image generation unit 22 generates a viewpoint conversion vehicle surrounding image by synthesizing the vehicle image at the approximate center of the background image described above. The display device 3 displays the viewpoint conversion vehicle surrounding image generated by the viewpoint conversion vehicle surrounding image generation unit 22.

  The obstacle detection unit 25 calculates, for each of the plurality of vehicle surrounding images input by the image input unit 21, for example, a motion vector obtained by comparing the current frame with the previous frame according to a known image difference method (optical flow method). Detect obstacles based on. In the present embodiment, four vehicle surrounding images (front vehicle surrounding image, left side vehicle surroundings) generated by the viewpoint conversion vehicle surrounding image generation unit 22 using the four vehicle surrounding images input by the image input unit 21 are used. Obstacles are detected for images, right side vehicle surrounding images, and rear vehicle surrounding images).

  When the obstacle detection unit 25 detects an obstacle in any of the four vehicle surrounding images, the obstacle detection unit 25 notifies the control unit 4 of information indicating in which imaging region the obstacle has been detected. Upon receiving this notification, the control unit 4 sets information indicating that there is an obstacle in the direction of the imaging region indicated by the notification in an internal memory (not shown). In addition, when there is a vehicle surrounding image in which no obstacle is detected, the obstacle detection unit 25 transmits information indicating which shooting area of the four obstacles is not detected to the overlapping area determination unit 26. Notice.

  Upon receiving this notification, the overlapping area determination unit 26 detects an obstacle for another vehicle surrounding image having a shooting area adjacent to the shooting area of the vehicle surrounding image in which no obstacle is detected by the obstacle detection unit 25. It is determined whether or not an obstacle is detected by the obstacle detection unit 25 in an overlapping area with the imaging area of the vehicle surrounding image that has not been present. For example, when no obstacle is detected in the right-side vehicle surrounding image, the overlapping area determination unit 26 captures the front and surrounding images of the front and rear shooting areas H1 and H4 adjacent to the right shooting area H3. Whether the obstacle is detected by the obstacle detection unit 25 in the overlapping areas OA2 and OA4 with the imaging area H3 of the right-side vehicle surrounding image is determined for two of the rear vehicle surrounding image. Then, the control unit 4 is notified of the determination result.

  It should be noted that no obstacle is detected in the right side vehicle surrounding image obtained by photographing the right imaging region H3 when there is no obstacle in the right imaging region H3 or there is a wall in the right imaging region H3. One of the cases where the edge cannot be detected from the right side vehicle surrounding image although it exists. In the former case, no obstacle is detected in both the overlapping area OA2 of the front vehicle surrounding image and the overlapping area OA4 of the rear vehicle surrounding image. On the other hand, in the latter case, an obstacle is detected in the overlapping area OA2 of the front vehicle surrounding image and the overlapping area OA4 of the rear vehicle surrounding image, and this is notified from the overlapping area determination unit 26 to the control unit 4.

  When the overlapping area determination unit 26 determines that an obstacle is detected in the overlapping area, the control unit 4 regards that the obstacle exists in the shooting area of the vehicle surrounding image in which the obstacle is not detected. Information indicating that there is an obstacle in the direction of the imaging region is set in an internal memory (not shown). In the above-described example, when the overlapping area determination unit 26 determines that an obstacle is detected in at least one of the overlapping areas OA2 and OA4, the control unit 4 determines that the obstacle is not detected around the right side vehicle. It is assumed that an obstacle exists in the image capturing area H3.

  That is, an obstacle is detected in the overlapping area OA2 of the front vehicle surrounding image and the overlapping area OA4 of the rear vehicle surrounding image although no obstacle is detected from the right side vehicle surrounding image. This is because a wall exists in the area, and a part of the wall is reflected in the overlapping areas OA2 and OA4 of the front / rear vehicle surrounding image. Accordingly, in this case, the control unit 4 regards that a wall is present in the right imaging region H3, and sets information indicating that there is an obstacle on the right side in an internal memory (not shown).

  In the present embodiment, when an obstacle is detected in at least one of two overlapping areas on both sides of the imaging area where no obstacle is detected, there is a wall in the imaging area where no obstacle is detected. I think that. As illustrated in FIG. 3, the end position of the wall 200 and the end position of the vehicle 100 (installation position of the rear camera 1 d) are aligned, and the vehicle 100 is quite close to the right wall 200. In this case, the wall 200 is shown in the overlap area OA2 between the right shooting area H3 and the front shooting area H1, but the wall 200 is shown in the overlap area OA4 between the right shooting area H3 and the rear shooting area H4. Because is not reflected. Even in such a case, in order to be able to consider that the wall 200 is on the right side, at least without the condition that an obstacle is detected in both of the two overlapping regions OA2 and OA4, On the other hand, on the condition that an obstacle is detected, it is considered that a wall exists in the imaging region where the obstacle is not detected.

  The notification unit 5 notifies the user in which direction there is an obstacle according to the information set in the internal memory of the control unit 4. For example, the alerting | reporting part 5 outputs with a sound from a speaker which direction an obstruction exists. Alternatively, a warning mark may be displayed superimposed on a vehicle surrounding image in the direction of an obstacle, or a simulated image of a wall may be displayed superimposed.

  Next, the operation of the image processing system according to the present embodiment configured as described above will be described. FIG. 4 is a flowchart illustrating an operation example of the image processing system according to the present embodiment. The flowchart shown in FIG. 4 starts when the power of the image processing system is turned on, for example. Or you may make it start when it detects that the gear of the vehicle was put into the back gear.

  In FIG. 4, first, the image input unit 21 inputs the vehicle surrounding images of the vehicle surrounding areas in the four directions taken by the plurality of in-vehicle cameras 1a to 1d (step S1). Next, the viewpoint conversion vehicle surrounding image generation unit 22 performs viewpoint conversion processing on the vehicle surrounding images input from the plurality of in-vehicle cameras 1 a to 1 d by the image input unit 21, so that the vehicle is displayed at the center of the four-direction vehicle surrounding image. A viewpoint conversion vehicle surrounding image in which the image is arranged is generated (step S2).

  Subsequently, the obstacle detection unit 25 executes an obstacle detection process based on the motion vector for each of the four vehicle surrounding images generated by the viewpoint conversion vehicle surrounding image generation unit 22 (step S3). Here, the obstacle detection unit 25 extracts one of the four vehicle surrounding images (step S4), and determines whether or not an obstacle has been detected for the extracted vehicle surrounding images (step S5).

  When the obstacle detection unit 25 determines that an obstacle has been detected, the obstacle detection unit 25 notifies the control unit 4 of information indicating in which imaging region the obstacle has been detected. Upon receiving this notification, the control unit 4 sets information indicating that there is an obstacle in the direction of the imaging area indicated by the notification in the internal memory (step S6). Thereafter, the obstacle detection unit 25 determines whether or not all four vehicle surrounding images have been extracted and obstacle detection has been performed (step S7). Here, when the undetermined vehicle surrounding image remains, the process returns to step S4, and the next vehicle surrounding image is extracted.

  In step S5, when it is determined that no obstacle is detected from the vehicle surrounding image extracted in step S4, the obstacle detection unit 25 duplicates information indicating in which imaging region the obstacle is not detected. The area determination unit 26 is notified. Upon receiving this notification, the overlapping area determination unit 26 detects an obstacle from another vehicle surrounding image in an overlapping area of the shooting area of the vehicle surrounding image in which no obstacle has been detected and the shooting area of another vehicle surrounding image adjacent thereto. Is detected (step S8).

  When the overlapping area determination unit 26 determines that an obstacle is detected in the overlapping area, the control unit 4 receives a notification to that effect from the overlapping area determination unit 26 and the vehicle in which no obstacle is detected. It is assumed that an obstacle exists in the shooting area of the surrounding image, and information indicating that there is an obstacle in the direction of the shooting area is set in the internal memory (step S6). On the other hand, if the overlap area determination unit 26 determines that no obstacle is detected in the overlap area, the process proceeds to step S7. In this step S7, when the obstacle detection unit 25 determines that all four vehicle surrounding images are extracted and the obstacle detection determination is performed, the notification unit 5 sets the information set in the internal memory of the control unit 4 The user is informed in which direction there is an obstacle (step S9).

  As described above in detail, in the present embodiment, even when no obstacle is detected from the vehicle surrounding image in a certain shooting area, in the vehicle surrounding image in the shooting area adjacent to the shooting area, When an obstacle is detected, it is assumed that an obstacle is present in the shooting area of the vehicle surrounding image where no obstacle is detected. For this reason, even if a wall is reflected on the entire surface of the vehicle surrounding image in a certain shooting area, and an edge cannot be detected from within the image, the adjacent shooting area cannot be detected as an obstacle. By using the obstacle detection result in the overlapping area of the other vehicle surrounding images taken as an auxiliary, a wall reflected on the entire surface of the vehicle surrounding image in a certain imaging area can be detected as an obstacle. As a result, a wall extending around the vehicle can be detected as an obstacle only by image recognition processing without using another sensor such as an ultrasonic sensor.

  In the above-described embodiment, an example in which the obstacle detection device of the present invention is applied to an image processing system that generates a viewpoint-converted vehicle surrounding image has been described, but the present invention is not limited to this. For example, the viewpoint conversion vehicle surrounding image generation unit 22, the mapping table 23, and the vehicle image storage unit 24 are omitted, and the obstacle detection unit 25 and the duplication are performed for each of the plurality of vehicle surrounding images input by the image input unit 21. Image processing by the region determination unit 26 may be performed.

  In addition, each of the above-described embodiments is merely an example of actualization in carrying out the present invention, and the technical scope of the present invention should not be construed in a limited manner. In other words, the present invention can be implemented in various forms without departing from the spirit or main features thereof.

1 In-vehicle camera (imaging device)
4 Control Unit 21 Image Input Unit 25 Obstacle Detection Unit 26 Overlapping Area Determination Unit

Claims (3)

A plurality of imaging devices that are installed at different locations on a vehicle and each capture a predetermined imaging region, where a part of the imaging region of one imaging device overlaps a part of the imaging region of another imaging device An image input unit that inputs vehicle surrounding images captured by a plurality of imaging devices installed to
For each of the plurality of vehicle surrounding images input by the image input unit, an obstacle detection unit that detects an obstacle based on a motion vector obtained by comparing the current frame and the previous frame;
The vehicle surrounding image in the shooting area adjacent to the shooting area of the vehicle surrounding image in which the obstacle has not been detected by the obstacle detection unit is overlapped with the shooting area of the vehicle surrounding image in which the obstacle is not detected. An overlapping area determination unit that determines whether the obstacle is detected by the obstacle detection unit;
When it is determined by the overlapping area determination unit that the obstacle is detected in the overlapping area, the obstacle is considered to exist even in the shooting area of the vehicle surrounding image where the obstacle is not detected. An obstacle detection apparatus comprising: a control unit that sets information indicating presence. The overlapping area determination unit detects the obstacle for each of the two vehicle surrounding images in the shooting area adjacent to both sides of the shooting area of the vehicle surrounding image in which the obstacle is not detected by the obstacle detection unit. Determining whether or not the obstacle is detected by the obstacle detection unit in the overlapping area with the imaging area of the vehicle surrounding image that has not been,
The control unit sets information representing the presence of the obstacle when the overlap region determination unit determines that the obstacle is detected in the overlap region for at least one of the two vehicle surrounding images. The obstacle detection device according to claim 1, wherein: A plurality of imaging devices that are installed at different locations on a vehicle and each capture a predetermined imaging region, where a part of the imaging region of one imaging device overlaps a part of the imaging region of another imaging device A first step of inputting vehicle surrounding images respectively captured by a plurality of imaging devices installed so as to perform;
A second step of detecting an obstacle based on a motion vector obtained by comparing the current frame and the previous frame for each of the plurality of vehicle surrounding images input in the first step;
In the overlapping area with the shooting area of the vehicle surrounding image where the obstacle is not detected, the vehicle surrounding image of the shooting area adjacent to the shooting area of the vehicle surrounding image where the obstacle is not detected in the second step A third step of determining whether the obstacle is detected;
When it is determined in the third step that the obstacle is detected in the overlapping area, the obstacle is considered to exist in the shooting area of the vehicle surrounding image where the obstacle is not detected. An obstacle detection method comprising: a fourth step of setting information indicating presence.