WO2022009537A1

WO2022009537A1 - Image processing device

Info

Publication number: WO2022009537A1
Application number: PCT/JP2021/019335
Authority: WO
Inventors: 琢馬大里; フェリペゴメズカバレロ; 雅幸竹村; 健永崎
Original assignee: 日立Astemo株式会社
Priority date: 2020-07-07
Filing date: 2021-05-21
Publication date: 2022-01-13
Also published as: JP7404173B2; DE112021002598T5; JP2022014673A

Abstract

Provided is an image processing device which, even when a parameter for associating two images with each other changes from one region to another, can calculate an appropriate response for each region and correctly detect and recognize an obstacle (object). This image processing device comprises: an image conversion unit 241 that converts two images obtained at two different time points into bird's-eye-view images; an obstacle candidate region extraction unit 242 that uses the two bird's-eye-view images or the two images to extract, as obstacle candidate regions, regions at a prescribed height or less from a road surface from among regions indicating obstacles; and an obstacle detection unit 243 that aligns the obstacle candidate regions using image features in the obstacle candidate regions and determines a region where a difference appears in the alignment to be a region where an obstacle is present.

Description

Image processing equipment

The present invention relates to an image processing device.

As a background technology in this technical field, there is Japanese Patent Application Laid-Open No. 2007-235642 (Patent Document 1). The publication describes as a problem "eliminating false detection of obstacles caused by detection errors of vehicle speed sensors and steering angle sensors, and performing three-dimensional object detection with a monocular camera with higher accuracy" as a solution. "A top view of the first image including the road surface captured by the camera mounted on the vehicle and a top view of the second image captured at a timing different from the first image are created, and the above two The top views of the two sheets are made to correspond based on the characteristic shape on the road surface, and the region where the difference occurs in the overlapping portion of the two top views is recognized as an obstacle. "

Japanese Unexamined Patent Publication No. 2007-235642

However, in an image including a road surface having a change in the road surface gradient, a step or a height difference, the correspondence based on the characteristic shape on the road surface changes for each region of the image, and therefore, for example, described in Patent Document 1. When the entire screen is dealt with as in the above, there is a problem that the recognition accuracy of obstacles changes depending on the area.

An object of the present invention is to calculate an appropriate correspondence for each region even when the parameter corresponding to the two images changes for each region, and to correctly detect and recognize an obstacle (hereinafter, also referred to as an object). It is to provide an image processing apparatus capable of.

In order to achieve the above object, the present invention uses an image conversion unit that converts two images at two different times into a bird's-eye view image, and an obstacle using the two bird's-eye view images or the two images. The obstacle candidate area extraction unit that extracts the area within a predetermined height from the road surface among the areas showing the above and the obstacle candidate area are aligned by using the image features in the obstacle candidate area. It is provided with an obstacle detection unit that determines that the area where the difference is generated is an area where an obstacle exists.

According to the present invention, an image processing device capable of correctly detecting and recognizing an obstacle (object) by calculating an appropriate correspondence for each region even when the parameter corresponding to the two images changes for each region. Can be provided.

Issues, configurations and effects other than those described above will be clarified by the explanation of the following embodiments.

An explanatory diagram showing a schematic configuration of an in-vehicle camera system according to the first embodiment. The explanatory view which shows the structure of the object detection apparatus and the image processing unit in Example 1. FIG. Explanatory diagram of the object detection method by the bird's-eye view difference method. An explanatory diagram of the problem of the bird's-eye view difference method in a scene with a road surface gradient. Explanatory diagram of the distance estimation method to the detection obstacle assuming the road surface. An example of a scene where an error occurs in a uniform projective transformation. Explanatory drawing of a concrete example using the bird's-eye view image of the obstacle candidate area extraction. A flowchart of a specific example using a bird's-eye view image of obstacle candidate area extraction. Explanatory drawing of a specific example using a classifier for extracting an obstacle candidate area. A flowchart of a specific example using an obstacle candidate area extraction classifier. Flow chart of obstacle detection unit. Explanatory drawing of the alignment means of the obstacle candidate area. The flowchart of obstacle candidate area extraction using map information and GPS information in Example 2. An explanatory diagram of obstacle candidate area extraction using map information and GPS information in Example 2. The flowchart of obstacle candidate area extraction for pop-out correspondence in Example 3. The explanatory view of the obstacle candidate area extraction for pop-out correspondence in Example 3. FIG.

Hereinafter, embodiments of the present invention will be described with reference to the drawings.

[Example 1]
An outline of the vehicle-mounted camera system equipped with the object detection device according to the first embodiment will be described with reference to FIG. The camera 101 is mounted on the vehicle 100. The camera 101 is for acquiring an image of the surroundings of the vehicle 100, and may be a monocular camera or a stereo camera composed of two or more cameras as long as a plurality of images different in time can be acquired. An object detection device 102 is mounted on the camera 101, and for example, the distance to the object in front and the relative speed are measured and transmitted to the vehicle control unit 103. The vehicle control unit 103 controls the brake accelerator 105 and the steering 104 from the distance and the relative speed received from the object detection device 102.

The camera 101 includes the object detection device 102 shown in FIG. The object detection device 102 includes an image pickup device 201, a memory 202, a CPU 203, an image processing unit (image processing device) 204, an external output unit 205, and the like. Each component constituting the object detection device 102 is communicably connected via the communication line 206. The CPU 203 executes the arithmetic processing described below according to the instructions of the program stored in the memory 202.

The image taken by the image sensor 201 is transmitted to the image processing unit 204, and an obstacle reflected in the image is detected. Specific means for detection will be described later. The detection result is transmitted from the external output unit 205 to the outside of the object detection device 102, and is used in the vehicle control unit 103 for determining vehicle control such as the brake accelerator 105 and the steering 104 in the above-mentioned vehicle-mounted camera system.

Hereinafter, the components of the image processing unit 204 will be described.

The image processing unit 204 detects and tracks an object using an image. As a means for detecting an object using an image, for example, a method using a difference in a bird's-eye view image (also called a bird's-eye view difference method) is used. In this method, as shown in FIG. 3, two images 301 and 302 (image 301 taken at the past time T-1 and image 302 taken at the current time T) of the object are used. Perform detection. Of the two

images

301 and 302, the previously captured image 301 is converted into a bird's-eye view image, and at the same time, the change in appearance due to the movement of the vehicle is calculated from information such as the vehicle speed, and it is predicted that the image will be captured in the current frame. Image 304 is generated. Homography is generally used for conversion to a bird's-eye view image. The projective transformation is a transformation in which a certain plane is projected onto another plane, and the road plane in the image 301 and the road plane in the image 302 can be converted into a bird's-eye view image looking down from directly above. The difference image 306 is created by comparing the predicted image 304 with the image 305 obtained by converting the image 302 actually captured in the current frame into a bird's-eye view image. The difference image 306 has a difference value in each pixel, and a region without a difference is represented by black and a region with a difference is represented by white. If there is no error in the projective transformation, the road plane becomes the same image due to the projective transformation and no difference occurs, but a difference occurs in a region other than the road plane, that is, a region where the obstacle 303 exists. By detecting this difference, the object detection result 307 can be obtained.

Since this method detects an object other than the plane to be converted by the projective transformation, it is important to select the ground plane of the obstacle as the conversion target. As shown in FIG. 4, in a scene where there is a change in the slope of the road surface, there are a plurality of different road planes in one image. Therefore, for example, when image conversion is performed on a road surface plane in a region where the vehicle is in contact with the ground, a different plane may be projected and converted. At this time, in the bird's-eye view image 404 and the bird's-eye view image 405 generated from the two

images

401 and 402 captured in time series, an unintended difference occurs in the texture of the road plane, in this example, the pedestrian crossing portion, which causes an obstacle. A difference image 406 with a difference in an area other than the object 303 is obtained, and the object detection result 407 detected as a difference erroneously detects the pedestrian crossing as an obstacle, erroneously detects the area of the obstacle to be detected, and the like. Problem occurs. Therefore, in order to correctly detect obstacles, it is necessary to perform appropriate projective transformation for each area in a scene where there are multiple road planes so that differences between images do not occur on the road planes. There is.

Further, as one of the methods for calculating the distance to the detected obstacle, a method for calculating by assuming that the foot portion of the detected obstacle region is in contact with the road surface is known. FIG. 5 is an explanatory diagram when an error is included in the generation of the bird's-eye view image and the road surface region near the foot of the obstacle is mistaken as an obstacle. Based on the result 501 of false detection including the road surface area as an obstacle area, the lower end position of the detection frame (here, the rectangular detection frame) on the image is projected onto the road surface to reach the obstacle 303. Estimate the distance of. At this time, if the lower end position of the obstacle 303 is erroneously estimated, erroneous estimation occurs if there is an obstacle at the position of the erroneous estimation result 502. Since erroneous estimation of the distance to the detection obstacle leads to erroneous control of the vehicle, it is desirable to estimate the lower end position of the detection frame as accurately as possible.

The details of the image processing unit 204 will be described below. As shown in FIG. 2, the image processing unit 204 of this embodiment has an image conversion unit 241, an obstacle candidate region extraction unit 242, and an obstacle detection unit 243.

(Image conversion unit 241)
The image conversion unit 241 converts two images at two different times into a bird's-eye view image by a first parameter that projects and transforms (also called a bird's-eye view conversion) a plane that is dominant in each image. The first parameter is the same parameter for each screen as a whole. For example, in the case of the above-mentioned in-vehicle camera system, the dominant plane can be a road surface when the vehicle is converted on the assumption that the vehicle is on a flat road surface. Further, when a distance sensor such as LiDAR or a stereo camera exists, the result of estimating the road surface using the distance measurement result may be used. At this time, as shown in FIG. 6, in the scene where the road surface gradient changes, the entire road surface is represented by one (same) plane 602 based on the posture of the own vehicle 601. May occur.

(Obstacle candidate area extraction unit 242)
In the obstacle candidate area extraction unit 242, the area should be aligned with high accuracy from the two images captured at two different times, and the area of the obstacle should be calculated accurately (hereinafter, the obstacle candidate). (Also called a region) is extracted. Since there are multiple planes in the entire scene, one (uniform) projective transformation cannot be fully expressed (see FIG. 6). Therefore, by extracting a small area that can be expressed by one projective transformation and performing image conversion and superimposition by projective transformation again in the subsequent processing, the difference in the road surface part is eliminated and only the difference in the obstacle part is used. To detect obstacles. Here, since it is the lower end portion that needs to accurately estimate the position of the detection frame in obstacle detection, the region close to the ground contact position (that is, the region indicating the obstacle within a predetermined height from the road surface). ) Is selected.

As an example of the obstacle candidate area extraction means, a flowchart of a specific example using a bird's-eye view image is shown in FIG. 8, and an actual area is shown in FIG. As shown in FIG. 8, in step S801, two images converted from a bird's-eye view by the image conversion unit 241 are given as inputs, and the images (that is, the two bird's-eye view images) are superimposed with the first parameter that is the same for the entire image. Object detection is performed using the area (difference area) where the difference (image difference) is generated. At this time, since the bird's-eye view conversion includes an error, a detection frame 701 that includes a large area under the feet when detecting an obstacle in the difference image 406 and an erroneous detection frame 702 in an area having only a road surface can be obtained. From here, in step S802, a region close to the road surface (a region within a predetermined height from the road surface) 703 of the detection frame (that is, an obstacle detection region indicating an obstacle) is extracted, and two images 401 captured. , 704 and 705 corresponding to the regions 703 extracted in 402 are extracted as obstacle candidate regions. By this processing, only the vicinity of the feet (that is, the area within a predetermined height from the road surface) is extracted from the areas (obstacle detection areas) where there is a possibility of obstacles in the two

images

401 and 402 captured. Can be done. By extracting the area individually for each object, a processing area for removing the error of the bird's-eye view conversion due to the difference in the plane described above is obtained.

As another example of the obstacle candidate area extraction means, a flowchart of a specific example using a classifier is shown in FIG. 10, and an actual area is shown in FIG. The classifier here has a pedestrian leg detection function, and the peripheral area of the pedestrian leg candidate (the pedestrian leg candidate area where the pedestrian leg is likely to be imaged) by the function. Is detected. As shown in FIG. 10, two captured images are given as inputs, and in step S1001, the region where the pedestrian leg is likely to be imaged is obtained while gradually shifting the processing window 901 of the classifier from the image region. Explore. The discriminator is an evaluation function designed so that the score is high when the pedestrian leg is imaged in the target area, and the pedestrian leg is searched for the area where the score is equal to or higher than the threshold value. The region 902 that is likely to be imaged can be extracted as an obstacle candidate region. By this processing, only the vicinity of the feet (that is, the area within a predetermined height from the road surface) is extracted from the areas (obstacle detection areas) where there is a possibility of obstacles in the two

images

401 and 402 captured. Can be done. However, since it is not possible to accurately estimate the boundary portion such as the lower end, the obstacle area and the road surface area are accurately separated by aligning the area extracted by this process in the subsequent process.

(Obstacle detection unit 243)
The obstacle detection unit 243 calculates a second parameter for projecting and transforming a dominant plane by performing alignment in the obstacle candidate area extracted by the obstacle candidate area extraction unit 242, and using the second parameter. The converted images are superimposed (in other words, alignment is performed), and the area where the difference occurs is detected as an area where an obstacle exists, that is, an obstacle. The flowchart of the obstacle detection unit 243 is shown in FIG. As shown in FIG. 11, the above-mentioned obstacle candidate area (of two images) is given as an input. In step S1101, a feature point is detected from a given region. Known examples of the means for detecting feature points include FAST (Features from Accelerated Segment Test) and Harris's corner detection. These are for detecting points that are corners in an image (see FIG. 12), and are known as a method for detecting points that are characteristic as compared with the surroundings. Next, in step S1102, the feature amount is calculated from the detected feature points. As a means for calculating the feature amount, for example, SIFT (Scale-Invariant Feature Transform) is known. SIFT is a method known as a robust feature quantity that does not change the feature quantity even when image changes such as lighting changes, rotation, and scaling occur, and the appearance changes depending on the position change of the camera and the passage of time. Even in this case, the same points can be determined to be the same. Next, in step S1103, matching of feature points at two times is performed (see FIG. 12). The feature points (of the obstacle candidate area) at the past time T-1 and the feature points (of the obstacle candidate area) at the current time T are compared, and the points having similar feature quantities calculated in step S1102 are matched with each other. , As shown in FIG. 12, a feature point pair (1201, 1202) pointing to the same location is obtained at 2 hours. In step S1104, the projection transformation matrix is estimated using the information of the matched points. A projective transformation matrix with the smallest amount of point shift is obtained when all points are transformed by the same projective transformation matrix and superposed. For the small area (obstacle candidate area) extracted by the obstacle candidate area extraction unit 242, which is composed of the lower end of the obstacle and the road surface at the feet, the second parameter according to the road surface at the feet. Find the projective transformation matrix by. In step S1105, the images converted using the projective transformation matrix according to the second parameter (that is, the bird's-eye view image of the obstacle candidate region portion) are superimposed, and the difference in the luminance values of the images is calculated. In step S1106, an obstacle detection result is obtained assuming that the region (difference region) in which the image difference is generated by superposition (alignment) is an obstacle (region in which the obstacle exists). Since the obstacle candidate area is a small area that includes only the obstacle and its foot road surface, this process changes the road surface gradient in the entire scene, or one projective transformation due to a step or height difference. Even in a scene that cannot be represented by a matrix, an appropriate projective transformation matrix (projection transformation matrix with the second parameter according to the foot road surface) can be estimated for each obstacle, and unnecessary differences are obtained from the imaged area of the road surface. Can be prevented from occurring.

As described above, the image processing unit (image processing device) 204 of the present embodiment includes an image conversion unit 241 that converts two images at two different times into a bird's-eye view image, and the two bird's-eye view images or the above. Using two images, the obstacle candidate area extraction unit 242 that extracts the area within a predetermined height from the road surface among the areas showing obstacles as the obstacle candidate area, and the image features in the obstacle candidate area are It is provided with an obstacle detection unit 243 that aligns the obstacle candidate region and determines that the region where the difference is generated is the region where the obstacle exists.

Further, as an example, the obstacle candidate region extraction unit 242 extracts an region in which an image difference occurs when the two bird's-eye view images are superimposed with the same first parameter in the entire image, thereby extracting the obstacle. The object candidate region is extracted, and the obstacle detection unit 243 aligns the obstacle candidate region according to the second parameter using the image feature in the obstacle candidate region, and the obstacle determines the region where the difference is generated. Judge as an existing area.

Further, the obstacle candidate region extraction unit 242 extracts a region where an image difference occurs when the two bird's-eye views images are superimposed with the same first parameter in the entire image, and the road surface among the extracted regions. A region within a predetermined height is extracted from the above, and the regions of the two images corresponding to the extracted regions are extracted as the obstacle candidate regions.

As another example, the obstacle candidate region extraction unit 242 sets the peripheral region of the pedestrian leg candidate detected by the pedestrian leg detection function (discriminator) in the two images as the obstacle candidate. Extract as an area.

When object detection is performed using the difference in the bird's-eye view image according to the configuration of this embodiment, it can be generally considered that there is no change in the gradient even in a scene where the difference occurs on the road surface and the gradient changes leading to false detection. Since the projective conversion is performed on the area (obstacle candidate area), it is possible to prevent the occurrence of false detection. Since the projective transformation matrix is estimated for each obstacle, it is possible to detect obstacles with high accuracy by minimizing the influence of changes due to vibration and tilt of the own vehicle, not limited to gradient changes.

This provides an image processing device that can correctly detect and recognize obstacles (objects) by calculating an appropriate correspondence for each area even when the parameters corresponding to the two images change for each area. be able to.

[Example 2: Method of linking with map information and GPS information]
The second embodiment is a modification of the first embodiment, and shows an example in which a region having a high possibility of false detection is extracted by using the map information and the vehicle GPS information in the obstacle candidate region extraction unit 242. According to this embodiment, a scene with a texture on the road surface, which is likely to cause false detection due to an error in projective transformation, is extracted as an obstacle candidate area without being missed, and is aligned with high accuracy. It is possible to prevent false detection of the texture on the road surface as an obstacle.

FIG. 13 shows a flowchart of the obstacle candidate area extraction unit 242 in this embodiment. As shown in FIG. 13, the image 13A, the road surface paint position 13B included in the map information, and the GPS information 13C including the own vehicle position information are received as inputs. Here, the road surface paint is a texture drawn on the road surface such as a pedestrian crossing or a U-turn prohibition, and the position of this kind of texture is stored in a navigation system or the like. Also, if the texture of the road surface is used, there is a possibility of false detection as well, so if you can obtain information on the texture that occurs on the entire road surface such as low-height curbs, gravel roads, and fallen leaves, not limited to road surface paint. You may receive that information as input. In step S1301, the road surface paint imaging region is calculated from the image. By comparing the GPS information with the map information of the road surface paint position, it is determined whether or not the road surface paint is included in the image area currently captured. When the road surface paint is included, the area of the image in which the image is captured is calculated, and the image area (that is, the road surface paint imaging area) is extracted. FIG. 14 shows an explanatory diagram. It is known from the map information and GPS information that the area 1402 is an area without road surface paint with respect to the front of the own vehicle 1401. If there is no road surface texture, even if there is an error in the projective transformation, no difference will occur in the road surface area, so high-precision alignment is not necessary. On the other hand, it is known that the pedestrian crossing is painted on the road surface in the area 1403. If the projective transformation is incorrect, the textures of the pedestrian crossing will not be properly superimposed and a difference will occur, and there is a high possibility that they will be falsely detected as obstacles. Therefore, the obstacle candidate region extraction unit 242 extracts the region 1404 corresponding to the region (road surface paint imaging region) 1403 on the image as the obstacle candidate region, and the obstacle detection unit 243 accurately aligns the small region. By implementing, false detection is prevented.

As described above, in the image processing unit (image processing device) 204 of the present embodiment, the obstacle candidate area extraction unit 242 acquires map information and position information of the own vehicle, and in the two images. The area where the road surface paint is imaged is extracted as the obstacle candidate area.

According to this embodiment, it is possible to calculate the projective transformation matrix without missing the area where the road surface paint has a high risk of false detection (imaging), and prevent the difference occurring in the road surface area.

[Example 3: Extraction of an image area having a high risk of popping out]
Example 3 is a modification of Example 1, and is not only an area where there is a possibility of obstacles at present in the obstacle candidate area extraction unit 242, but also obstacles (pedestrians of adults and children, bicycles, motorcycles) in the future. , Automobile, etc.) An example is shown in the case of extracting an area where popping out may occur. According to this embodiment, the distance to an obstacle can be estimated without erroneous estimation even when a sudden jump occurs.

The flowchart of the obstacle candidate area extraction unit 242 in this embodiment is shown in FIG. 15, and the explanatory diagram is shown in FIG. As shown in FIG. 15, the object detection S801 using the difference region is performed by a projective transformation including an error as in the first embodiment. At this time, an object detection result including an error in the depth direction is obtained for the object 1601 as in the detection result 1602. Next, in step S1501, of the left and right ends of the detection result 1602, a region having a predetermined size on the own vehicle side is extracted as a pop-out possibility region 1603. The region 1603 is a region where the object 1601 may pop out from behind, and if the pop-out occurs and the distance is incorrect, erroneous braking may occur. Therefore, in the obstacle candidate area extraction unit 242, based on the area where there is a possibility of an obstacle, the area where there is a risk of popping out (probably popping out area) 1603 adjacent to the left and right ends thereof is extracted as an obstacle candidate area. Then, the obstacle detection unit 243 preferentially performs highly accurate image alignment to estimate the obstacle position with high accuracy.

In this embodiment, the left and right edges of the object detection result including false detection are targeted for extraction, but the extraction method does not matter as long as the area has a high possibility of popping out. For example, the shadow of a building or the like may be extracted from the map information, or it may be fused with the object detection result by another sensor.

As described above, in the image processing unit (image processing apparatus) 204 of the present embodiment, the obstacle candidate region extraction unit 242 has the left and right ends of the region where there is a possibility of obstacles in the two images. A region having a risk of popping out (a region having a possibility of popping out) adjacent to the above is extracted as the obstacle candidate region.

According to this embodiment, it is possible to preferentially extract an area where an obstacle has a high possibility of popping out, and prevent erroneous detection and erroneous estimation of distance when popping out occurs.

The present invention is not limited to the above-described embodiment, but includes various modifications. For example, the above-described embodiment has been described in detail in order to explain the present invention in an easy-to-understand manner, and is not necessarily limited to the one including all the described configurations. Further, it is possible to replace a part of the configuration of one embodiment with the configuration of another embodiment, and it is also possible to add the configuration of another embodiment to the configuration of one embodiment. Further, it is possible to add / delete / replace a part of the configuration of each embodiment with another configuration.

Further, each of the above configurations may be configured in whole or in part by hardware, or may be configured to be realized by executing a program on a processor. In addition, the control lines and information lines indicate those that are considered necessary for explanation, and do not necessarily indicate all the control lines and information lines on the product. In practice, it can be considered that almost all configurations are interconnected.

100 vehicle, 101 camera, 102 object detection device, 103 vehicle control unit, 201 image pickup element, 202 memory, 203 CPU, 204 image processing unit (image processing device), 205 external output unit, 206 communication line, 241 image conversion unit, 242 Obstacle candidate area extraction unit 243 Obstacle detection unit

Claims

An image conversion unit that converts two images at two different times into a bird's-eye view image,
An obstacle candidate area extraction unit that extracts an area within a predetermined height from the road surface as an obstacle candidate area among the areas showing obstacles by using the two bird's-eye view images or the two images.
An image characterized by comprising an obstacle detection unit that aligns the obstacle candidate region using the image feature in the obstacle candidate region and determines that the region where the difference occurs is the region where the obstacle exists. Processing device.
The image processing apparatus according to claim 1.
The obstacle candidate region extraction unit extracts the obstacle candidate region by extracting the region where the image difference occurs when the two bird's-eye view images are superimposed with the same first parameter in the entire image. ,
The obstacle detection unit uses the image features in the obstacle candidate region to perform alignment according to the second parameter of the obstacle candidate region, and determines that the region where the difference is generated is the region where the obstacle exists. An image processing device that features it.
The image processing apparatus according to claim 2.
The obstacle candidate region extraction unit extracts a region where an image difference occurs when the two bird's-eye view images are superimposed with the same first parameter in the entire image, and the predetermined height from the road surface among the extracted regions. An image processing apparatus comprising extracting an area within the range and extracting an area of the two images corresponding to the extracted area as an obstacle candidate area.
The image processing apparatus according to claim 1.
The image processing featured in that the obstacle candidate region extraction unit extracts the peripheral region of the pedestrian leg candidate detected by the pedestrian leg detection function in the two images as the obstacle candidate region. Device.
The image processing apparatus according to claim 1.
The obstacle candidate area extraction unit acquires map information and position information of the own vehicle, and extracts an area in which the road surface paint is imaged in the two images as the obstacle candidate area. Device.
The image processing apparatus according to claim 1.
The obstacle candidate region extraction unit is characterized in that, in the two images, regions having a risk of popping out adjacent to the left and right ends of the potential obstacle region are extracted as the obstacle candidate region. Image processing device.
An image conversion unit that converts two images at two different times into a bird's-eye view image,
An obstacle candidate area extracted from the two images using the two bird's-eye view images or the two images as an obstacle candidate area within a predetermined height from the road surface among the areas showing obstacles. Extractor and
A bird's-eye view image of the obstacle candidate region is generated using the image features in the obstacle candidate region, the bird's-eye view image of the obstacle candidate region is aligned, and the region where the difference occurs is determined to be the region where the obstacle exists. An image processing device including an obstacle detection unit.