JP7318377B2 - Object detection device - Google Patents

Description

The present invention relates to an object detection device that is mounted on a vehicle and configured to detect an object existing around the vehicle.

As a device of this type, for example, one described in Patent Document 1 is known. The device described in Patent Literature 1 includes a height calculation section and a determination section. The height calculator detects an obstacle between the car and the parking position and calculates the height of the obstacle. Specifically, the height calculator detects an obstacle by referring to the comparison result between the distance image and the ground distance image stored in advance. The determination unit refers to the calculated height of the obstacle and determines whether to control either the acceleration device or the braking device of the vehicle. That is, the determination unit determines that the braking device should be controlled to stop the vehicle when the calculated height of the obstacle is greater than a predetermined threshold. On the other hand, when the height is smaller than the threshold value, the determination unit determines that the acceleration device should be controlled so that the vehicle accelerates or the vehicle speed is maintained.

JP 2013-244852 A

In a vehicle equipped with this type of device, the photographing posture of an imaging device such as a camera changes depending on the passenger's boarding condition, luggage loading condition, vehicle height adjustment condition, and the like. Therefore, in the device described in Patent Document 1, the photographing posture may differ between the ground distance image stored in advance and the distance image corresponding to the detected obstacle. Therefore, an error may occur in height information and distance information of an obstacle due to such a difference in photographing attitude.

Specifically, in the device described in Patent Literature 1, for example, a low level difference that is not originally determined as an obstacle and can be run over is an obstacle having a height that makes it difficult to run over in a low vehicle height state. may be erroneously determined to be Conversely, an obstacle having a height that would otherwise make it difficult for the vehicle to travel over may be erroneously determined to be a low step that allows the vehicle to travel over it in the high vehicle height state.

The present invention has been made in view of the circumstances exemplified above. That is, the present invention provides a configuration capable of detecting, for example, the height of an object existing around the own vehicle with higher accuracy.

An object detection device (20) according to claim 1 is configured to detect an object (B) existing around the vehicle (10) by being mounted on the vehicle (10).
This object detection device
a road surface feature point extracting unit (274) for extracting road surface feature points, which are feature points belonging to a road surface, from a photographed image of the surroundings of the own vehicle including the object;
a parallax image acquisition unit (272) for acquiring parallax images based on the plurality of captured images from different viewpoints;
a reference height setting unit (275) for setting, as reference height information, height information corresponding to the road surface characteristic points based on the parallax image and the road surface characteristic points extracted by the road surface characteristic point extraction unit; ,
a coordinate information calculation unit (276) for calculating three-dimensional coordinate information of the object based on the reference height information set by the reference height setting unit;
with
The road surface characteristic point extracting unit sets the reference height information based on the road surface characteristic points at which parallelism between the edge direction and the epipolar line is equal to or less than a predetermined value.

In addition, in each column of the application documents, each element may be given a reference sign with parentheses. In this case, the reference numerals simply indicate an example of the correspondence relationship between the same element and the specific configuration described in the embodiment described later. Therefore, the present invention is not limited in any way by the description of the reference numerals.

BRIEF DESCRIPTION OF THE DRAWINGS It is a top view which shows schematic structure of the vehicle which mounts the object detection apparatus which concerns on embodiment. 2 is a block diagram showing a schematic functional configuration in one embodiment of the object detection device shown in FIG. 1; FIG. 3 is a flow chart showing an operation example of the object detection device shown in FIG. 2; FIG. 4 is a flow chart showing a modification of the operation example shown in FIG. 3; FIG. 2 is a block diagram showing a schematic functional configuration in a modified example of the object detection device shown in FIG. 1; FIG. 6 is a flow chart showing an operation example of the object detection device shown in FIG. 5;

(embodiment)
BEST MODE FOR CARRYING OUT THE INVENTION Hereinafter, embodiments of the present invention will be described based on the drawings. It should be noted that if various modifications applicable to one embodiment are inserted in the middle of a series of explanations related to the embodiment, there is a risk that the understanding of the embodiment will be hindered. Therefore, the modified examples will be collectively described after the series of descriptions regarding the embodiment, not in the middle of the description.

(Overall vehicle configuration)
Referring to FIG. 1, a vehicle 10 is a so-called four-wheel vehicle and includes a substantially rectangular vehicle body 11 in a plan view. Hereinafter, an imaginary straight line passing through the center of the vehicle 10 in the vehicle width direction and parallel to the vehicle full length direction of the vehicle 10 will be referred to as a vehicle center axis X. As shown in FIG. In FIG. 1, the vehicle width direction is the horizontal direction in the drawing. The vehicle length direction is a direction orthogonal to the vehicle width direction and orthogonal to the vehicle height direction. The vehicle height direction is a direction that defines the vehicle height of the vehicle 10, and is parallel to the direction of gravity when the vehicle 10 is placed on a horizontal plane. Furthermore, an arbitrary direction orthogonal to the vehicle height direction in which the vehicle 10 can move by running is sometimes referred to as the “translational direction” of the vehicle 10 .

For convenience of explanation, "front", "rear", "left", and "right" of the vehicle 10 are defined as indicated by arrows in FIG. That is, the vehicle length direction is synonymous with the front-rear direction. Further, the vehicle width direction is synonymous with the left-right direction. Note that the vehicle height direction may not be parallel to the gravity action direction depending on the mounting conditions or running conditions of the vehicle 10 . However, in many cases, the vehicle height direction is along the direction of gravity action, so the "translational direction" orthogonal to the vehicle height direction is "horizontal direction", "in-plane direction", "approach direction", and " It may also be referred to as "heading" or "tracking".

A front bumper 13 is attached to a front portion 12 that is a front end portion of the vehicle body 11 . A rear bumper 15 is attached to a rear surface portion 14 that is a rear end portion of the vehicle body 11 . A door panel 17 is attached to a side portion 16 of the vehicle body 11 . In the specific example shown in FIG. 1, a total of four door panels 17 are provided, two on each side. A door mirror 18 is attached to each of the pair of left and right door panels 17 on the front side.

An object detection device 20 is mounted on the vehicle 10 . The object detection device 20 is mounted on the vehicle 10 so as to detect an object B existing outside and around the vehicle 10 . Hereinafter, the vehicle 10 equipped with the object detection device 20 may be abbreviated as "own vehicle".

Specifically, the object detection device 20 includes an imaging unit 21, a sonar sensor 22, a radar sensor 23, a vehicle speed sensor 24, a shift position sensor 25, a steering angle sensor 26, an object detection ECU 27, and a display unit 28. and an audio output unit 29 . ECU is an abbreviation for Electronic Control Unit. The details of each part constituting the object detection device 20 will be described below. For the sake of simplification of illustration, FIG.

The imaging unit 21 is provided to capture an image of the surroundings of the own vehicle and generate and acquire image information corresponding to the captured image. In this embodiment, the imaging unit 21 is a digital camera device, and includes an image sensor such as a CCD or CMOS. CCD is an abbreviation for Charge Coupled Device. CMOS is an abbreviation for Complementary MOS.

In this embodiment, the vehicle 10 is equipped with a plurality of imaging units 21, that is, a front camera CF, a rear camera CB, a left camera CL, and a right camera CR. When not specifying any of the front camera CF, the rear camera CB, the left camera CL, and the right camera CR, hereinafter, the singular expression “imaging unit 21” or “plural imaging units 21” is sometimes used.

The front camera CF is provided to acquire image information corresponding to an image in front of the vehicle. Specifically, in the present embodiment, the front camera CF is attached to a rearview mirror (not shown) arranged in the vehicle interior of the vehicle 10 .

The rear camera CB is attached to the rear surface portion 14 of the vehicle body 11 so as to obtain image information corresponding to an image behind the own vehicle. The left camera CL is attached to the left side door mirror 18 so as to acquire image information corresponding to the left side image of the vehicle. The right camera CR is attached to the right side door mirror 18 so as to obtain image information corresponding to the right image of the vehicle.

Each of the imaging units 21 is electrically connected to the object detection ECU 27 . That is, each of the plurality of imaging units 21 acquires image information under the control of the object detection ECU 27 and outputs the acquired image information so that the object detection ECU 27 can receive the image information.

The sonar sensor 22 is a ranging sensor that detects the distance to the object B and is attached to the vehicle body 11 . In the present embodiment, the sonar sensor 22 is a so-called ultrasonic sensor, and is configured to be able to transmit search waves, which are ultrasonic waves, toward the outside of the vehicle and receive received waves including ultrasonic waves. . That is, the sonar sensor 22 is provided so as to output distance measurement information, which is the result of detection of the distance to the distance measurement point on the object B, by receiving the received waves including the reflected waves of the search waves from the object B. there is A “range-finding point” is a point on the surface of the object B that is estimated to have reflected the search wave transmitted from the sonar sensor 22 , and corresponds to the “reflection point” on the radar sensor 23 .

The object detection device 20 has at least one sonar sensor 22 . Specifically, in this embodiment, a plurality of sonar sensors 22 are provided. Each of the plurality of sonar sensors 22 is shifted from the vehicle center axis X to one side in the vehicle width direction. Moreover, at least some of the plurality of sonar sensors 22 are provided so as to emit survey waves along the direction intersecting with the vehicle center axis X. As shown in FIG.

Specifically, the front bumper 13 is equipped with a first front sonar SF1, a second front sonar SF2, a third front sonar SF3, and a fourth front sonar SF4 as sonar sensors 22. FIG. Similarly, the rear bumper 15 is equipped with a first rear sonar SR1, a second rear sonar SR2, a third rear sonar SR3, and a fourth rear sonar SR4 as sonar sensors 22. As shown in FIG. A first side sonar SS1, a second side sonar SS2, a third side sonar SS3, and a fourth side sonar SS4 are mounted as sonar sensors 22 on the side portion 16 of the vehicle body 11 .

1st front sonar SF1, 2nd front sonar SF2, 3rd front sonar SF3, 4th front sonar SF4, 1st rear sonar SR1, 2nd rear sonar SR2, 3rd rear sonar SR3, 4th rear sonar SR4, 1st side sonar SS1, Hereinafter, when not specifying any of the second side sonar SS2, the third side sonar SS3, and the fourth side sonar SS4, the singular expression "sonar sensor 22" or "plurality of sonar sensors 22” is sometimes used.

One sonar sensor 22 is called a "first sonar sensor" and another sonar sensor 22 is called a "second sonar sensor", and "direct waves" and "indirect waves" are defined as follows. A received wave that is received by the first sonar sensor and is caused by a reflected wave from the object B of the search wave emitted from the first sonar sensor is referred to as a "direct wave". That is, the direct wave is the received wave when the sonar sensor 22 that transmitted the search wave and the sonar sensor 22 that detected the reflected wave of the search wave from the object B as the received wave are the same. On the other hand, a received wave that is received by the second sonar sensor and is caused by a reflected wave from the object B of the search wave transmitted from the first sonar sensor is referred to as an "indirect wave." That is, the indirect wave is a received wave when the sonar sensor 22 that transmitted the search wave is different from the sonar sensor 22 that detects the reflected wave of the search wave from the object B as the received wave.

The first front sonar SF1 is provided at the left end of the front surface of the front bumper 13 so as to emit a survey wave to the left front of the vehicle. The second front sonar SF2 is provided at the right end of the front side surface of the front bumper 13 so as to transmit a survey wave to the right front of the vehicle. The first front sonar SF1 and the second front sonar SF2 are arranged symmetrically with the vehicle center axis X interposed therebetween.

The third front sonar SF3 and the fourth front sonar SF4 are arranged in the width direction of the vehicle at positions near the center of the front surface of the front bumper 13 . The third front sonar SF3 is arranged between the first front sonar SF1 and the vehicle center axis X in the vehicle width direction so as to emit survey waves substantially forward of the vehicle. The fourth front sonar SF4 is arranged between the second front sonar SF2 and the vehicle center axis X in the vehicle width direction so as to emit survey waves substantially forward of the vehicle. The third front sonar SF3 and the fourth front sonar SF4 are arranged symmetrically with the vehicle center axis X interposed therebetween.

As described above, the first front sonar SF1 and the third front sonar SF3 mounted on the left side of the vehicle body 11 are arranged at different positions in plan view. In addition, the first front sonar SF1 and the third front sonar SF3, which are adjacent to each other in the vehicle width direction, are in a positional relationship such that the reflected waves of the survey waves emitted by one of them by the object B can be received as received waves by the other. is provided.

That is, the first front sonar SF1 is arranged so as to be able to receive both the direct wave corresponding to the search wave emitted by itself and the indirect wave corresponding to the search wave emitted by the third front sonar SF3. Similarly, the third front sonar SF3 is arranged so as to be able to receive both the direct wave corresponding to the search wave emitted by itself and the indirect wave corresponding to the search wave emitted by the first front sonar SF1.

Similarly, the third front sonar SF3 and the fourth front sonar SF4 mounted near the center of the vehicle body 11 in the vehicle width direction are arranged at different positions in plan view. Further, the third front sonar SF3 and the fourth front sonar SF4, which are adjacent to each other in the vehicle width direction, are in a positional relationship such that the reflected wave of the survey wave transmitted by one of them by the object B can be received by the other as a received wave. is provided.

Similarly, the second front sonar SF2 and the fourth front sonar SF4 mounted on the right side of the vehicle body 11 are arranged at positions different from each other in plan view. Further, the second front sonar SF2 and the fourth front sonar SF4, which are adjacent to each other in the vehicle width direction, are in a positional relationship such that the reflected wave of the survey wave transmitted by one of them by the object B can be received by the other as a received wave. is provided.

The first rear sonar SR1 is provided at the left end of the rear surface of the rear bumper 15 so as to transmit a survey wave to the left rear of the vehicle. The second rear sonar SR2 is provided at the right end portion of the rear surface of the rear bumper 15 so as to transmit a survey wave to the right rear of the vehicle. The first rear sonar SR1 and the second rear sonar SR2 are arranged symmetrically with the vehicle center axis X interposed therebetween.

The third rear sonar SR3 and the fourth rear sonar SR4 are arranged in the vehicle width direction at positions near the center of the rear surface of the rear bumper 15 . The third rear sonar SR3 is arranged between the first rear sonar SR1 and the vehicle center axis X in the vehicle width direction so as to transmit a survey wave substantially behind the own vehicle. The fourth rear sonar SR4 is arranged between the second rear sonar SR2 and the vehicle center axis X in the vehicle width direction so as to emit survey waves substantially behind the vehicle. The third rear sonar SR3 and the fourth rear sonar SR4 are arranged symmetrically with the vehicle center axis X interposed therebetween.

As described above, the first rear sonar SR1 and the third rear sonar SR3 mounted on the left side of the vehicle body 11 are arranged at different positions in plan view. Further, the first rear sonar SR1 and the third rear sonar SR3, which are adjacent to each other in the vehicle width direction, are provided in a positional relationship such that the reflected waves of the survey waves emitted by one of them and reflected by the object B can be received as received waves by the other. ing.

That is, the first rear sonar SR1 is arranged so as to be able to receive both the direct wave corresponding to the search wave emitted by itself and the indirect wave corresponding to the search wave emitted by the third rear sonar SR3. Similarly, the third rear sonar SR3 is arranged so as to be able to receive both the direct wave corresponding to the search wave emitted by itself and the indirect wave corresponding to the search wave emitted by the first rear sonar SR1.

Similarly, the third rear sonar SR3 and the fourth rear sonar SR4, which are mounted near the center of the vehicle body 11 in the vehicle width direction, are arranged at different positions in plan view. Further, the third rear sonar SR3 and the fourth rear sonar SR4, which are adjacent to each other in the vehicle width direction, are provided in a positional relationship such that the reflected waves of the survey waves emitted by one of them and reflected by the object B can be received by the other as received waves. ing.

Similarly, the second rear sonar SR2 and the fourth rear sonar SR4 mounted on the right side of the vehicle body 11 are arranged at different positions in plan view. Further, the second rear sonar SR2 and the fourth rear sonar SR4, which are adjacent to each other in the vehicle width direction, are provided in a positional relationship such that the reflected waves of the survey waves emitted by one of them and reflected by the object B can be received as received waves by the other. ing.

The first side sonar SS1, the second side sonar SS2, the third side sonar SS3, and the fourth side sonar SS4 are designed to transmit survey waves to the side of the vehicle from the side of the vehicle, which is the outer surface of the side portion 16. is provided. The first side sonar SS1, the second side sonar SS2, the third side sonar SS3, and the fourth side sonar SS4 are each provided so as to receive only direct waves.

The first side sonar SS1 is arranged between the left side door mirror 18 and the first front sonar SF1 in the front-rear direction so as to emit a search wave to the left of the vehicle. The second side sonar SS2 is arranged between the right side door mirror 18 and the second front sonar SF2 in the front-rear direction so as to transmit a search wave to the right of the vehicle. The first side sonar SS1 and the second side sonar SS2 are provided symmetrically with the vehicle center axis X interposed therebetween.

The third side sonar SS3 is arranged between the left rear door panel 17 and the first rear sonar SR1 in the front-rear direction so as to emit a search wave to the left of the vehicle. The fourth side sonar SS4 is arranged between the right rear door panel 17 and the second rear sonar SR2 in the front-rear direction so as to emit a search wave to the right of the vehicle. The third side sonar SS3 and the fourth side sonar SS4 are provided symmetrically with the vehicle center axis X interposed therebetween.

Each of the multiple sonar sensors 22 is electrically connected to the object detection ECU 27 . Each of the plurality of sonar sensors 22 transmits a survey wave under the control of the object detection ECU 27, generates a signal corresponding to the reception result of the received wave, and outputs it so as to be receivable by the object detection ECU 27. . Information included in the signal corresponding to the reception result of the received wave is hereinafter referred to as "ranging information". The ranging information includes information related to the reception intensity of received waves and distance information. “Distance information” is information related to the distance between each of the plurality of sonar sensors 22 and the object B. FIG. Specifically, for example, the distance information includes information related to the time difference between the transmission of the search wave and the reception of the received wave.

The radar sensor 23 is a laser radar sensor or millimeter wave radar sensor that transmits and receives radar waves, and is attached to the front portion 12 of the vehicle body 11 . The radar sensor 23 is configured to generate a signal corresponding to the position and relative velocity of the reflection point and output the signal so that it can be received by the object detection ECU 27 . A “reflection point” is a point on the surface of the object B that is presumed to have reflected the radar wave. "Relative velocity" is the relative velocity of the reflection point, that is, the object B that reflected the radar wave, with respect to the own vehicle.

Vehicle speed sensor 24 , shift position sensor 25 and steering angle sensor 26 are electrically connected to object detection ECU 27 . The vehicle speed sensor 24 is provided so as to generate a signal corresponding to the running speed of the own vehicle and output the signal so as to be received by the object detection ECU 27 . The running speed of the own vehicle is hereinafter simply referred to as "vehicle speed". The shift position sensor 25 is provided so as to generate a signal corresponding to the shift position of the host vehicle and output the signal so as to be receivable by the object detection ECU 27 . The steering angle sensor 26 is provided so as to generate a signal corresponding to the steering angle of the host vehicle and output the signal so that it can be received by the object detection ECU 27 .

The object detection ECU 27 is arranged inside the vehicle body 11 . The object detection ECU 27 is a so-called in-vehicle microcomputer, and includes a CPU, ROM, RAM, non-volatile rewritable memory, etc. (not shown). Nonvolatile rewritable memory is, for example, EEPROM, flash ROM, and the like. EEPROM is an abbreviation for Electronically Erasable and Programmable Read Only Memory. The CPU, ROM, RAM and non-volatile rewritable memory of the object detection ECU 27 are hereinafter simply referred to as "CPU", "ROM", "RAM" and "non-volatile RAM".

The object detection ECU 27 is configured so that various control operations can be realized by the CPU reading and executing programs from the ROM or the nonvolatile RAM. This program includes those corresponding to the routines described below. Also, the RAM and the non-volatile RAM are configured to be able to temporarily store processing data when the CPU executes the program. Further, the ROM and/or the non-volatile RAM pre-store various data used for executing the program. Various data include, for example, initial values, lookup tables, maps, and the like.

The object detection ECU 27 is configured to execute an object detection operation based on signals and information received from each of the plurality of imaging units 21, the vehicle speed sensor 24, the shift position sensor 25, the steering angle sensor 26, and the like. . In addition, the object detection ECU 27 controls the operations of the display unit 28 and the audio output unit 29 to perform notification operations associated with the object detection state.

In this embodiment, the object detection ECU 27 is configured to have a parking assistance function. That is, the object detection ECU 27 detects a parking space using the imaging unit 21, and assists the movement of the own vehicle to the detected parking space based on the obstacle detection result by the sonar sensor 22 or the like. . "Obstacle" refers to an object B existing around the own vehicle that is difficult or impossible to travel over because its height is higher than a predetermined threshold.

The display unit 28 and the audio output unit 29 are arranged in the vehicle interior of the vehicle 10 . Also, the display unit 28 and the audio output unit 29 are electrically connected to the object detection ECU 27 . The display unit 28 is configured to perform a notification operation associated with object detection and parking assistance using a display screen or an indicator. The audio output unit 29 is configured to output a notification operation associated with object detection and parking assistance by audio output using a speaker.

(Functional configuration of object detection ECU)
Referring to FIG. 2, the object detection ECU 27 has the following functional configuration implemented on a microcomputer. That is, the object detection ECU 27 includes an image information acquisition unit 270, a feature point extraction unit 271, a parallax image acquisition unit 272, a parking space acquisition unit 273, a road surface feature point extraction unit 274, a reference height setting unit 275, It has a coordinate information calculation unit 276 , a distance measurement information acquisition unit 277 and an obstacle detection unit 278 . Details of the functional configuration of the object detection ECU 27 in this embodiment will be described below.

The image information acquisition unit 270 is provided so as to acquire image information corresponding to a photographed image of the surroundings of the own vehicle. Specifically, the image information acquisition section 270 receives image information generated by the imaging section 21 from the imaging section 21 and stores the received image information in time series in the non-volatile RAM.

The feature point extraction section 271 is provided to extract feature points in the captured image based on the image information acquired by the image information acquisition section 270 . A feature point is a point that characterizes the shape of the object B in the captured image. In other words, the feature point is a characteristic point, ie, a pixel within the angle of view of the captured image. Specifically, for example, a feature point is a pixel that has a large luminance change with an adjacent pixel. Note that feature points and their extraction method are well known at the time of filing of the present application. Specifically, known methods (eg, Sobel filter, Laplacian filter, Canny method, etc.) can be used as feature point detection methods. Therefore, in this specification, the detailed description of the feature point extraction method by the feature point extraction unit 271 will be omitted. Note that "extraction" of feature points can also be expressed as "detection".

The parallax image acquisition unit 272 is provided to acquire parallax images based on a plurality of captured images from different viewpoints. Specifically, the parallax image acquisition unit 272 calculates parallax images by performing parallax calculation using a moving stereo or monocular moving stereo method based on a plurality of captured images captured by the same imaging unit 21 at different times. It is calculated. Motion stereo is also referred to as SFM. SFM is an abbreviation for Structure from Motion. It should be noted that moving stereo or SFM has already been publicly known or known at the time of filing of the present application. Therefore, the details of the moving stereo are omitted in this specification.

The parking space acquisition unit 273 is provided to acquire the detection result of at least one parking space that can be a target parking space candidate for use in parking assistance. Specifically, in this embodiment, the parking space acquisition section 273 detects a parking space based on the image information acquired by the image information acquisition section 270 . A method of detecting a parking space based on image information has already been publicly known or known at the time of filing of this application. Therefore, in this specification, the detailed description of the parking space detection method based on the image information will be omitted.

The road surface characteristic point extraction unit 274 is provided to extract road surface characteristic points, which are characteristic points belonging to the road surface, from the photographed image of the surroundings of the vehicle including the object B. Specifically, the road surface characteristic point extraction unit 274 extracts road surface characteristic points from among the plurality of characteristic points extracted by the characteristic point extraction unit 271 based on the obtained parking space. That is, the road surface characteristic point extracting section 274 extracts the characteristic points included in the parking space and its neighboring area as the road surface characteristic points. For example, when the detected parking space is sandwiched or surrounded by white lines, the "neighboring area" of the parking space is the white line and a predetermined width (for example, 0.5 to 0.5 of the width of the white line). double width).

Based on the parallax image acquired by the parallax image acquiring unit 272 and the road surface characteristic points extracted by the road surface characteristic point extracting unit 274, the reference height setting unit 275 sets height information corresponding to the road surface characteristic points to a reference height. Information is provided to be set. Specifically, the reference height setting unit 275 sets height information corresponding to the road surface characteristic point as the reference height information when parallax exists in the vicinity of the road surface characteristic point. "Reference height information" is height information corresponding to the road surface height, that is, information indicating that the height from the road surface is 0 m. In this embodiment, the reference height setting unit 275 sets the reference height information based on the minimum height information among the feature points included in the parking space and its neighboring area. .

The coordinate information calculation section 276 is provided to calculate three-dimensional coordinate information of the object B based on the reference height information set by the reference height setting section 275 . Specifically, the coordinate information calculator 276 determines the three-dimensional positions of feature points other than the road surface feature points based on the reference height information corresponding to the road surface feature points and the parallax image. That is, the coordinate information calculation unit 276 calculates the three-dimensional positions of feature points other than the road surface feature points using the moving stereo or monocular moving stereo technique, while using the height of 0 m from the road surface at the road surface feature points as a reference for height. is designed to determine

The ranging information acquisition unit 277 is provided to acquire ranging information, which is the result of distance detection by the sonar sensor 22 . Further, the ranging information acquisition unit 277 is provided so as to acquire relative position information of the object B with respect to the own vehicle by triangulation based on ranging information acquired using a plurality of sonar sensors 22 . Furthermore, the distance measurement information acquisition unit 277 stores the acquisition results of the distance measurement information and the relative position information in time series in the non-volatile RAM.

The obstacle detection unit 278 detects an obstacle based on the calculation result of the three-dimensional coordinate information by the coordinate information calculation unit 276 and the distance measurement information and the relative position information acquired by the distance measurement information acquisition unit 277. is provided in That is, in this embodiment, the obstacle detection unit 278 detects an obstacle using a so-called "sensor fusion" technique that fuses the image recognition result and the distance measurement result.

(action/effect)
An overview of the operation of the object detection device 20, that is, the object detection ECU 27, according to the present embodiment will be described below together with the effects achieved by the configuration of the present embodiment.

Each of the plurality of imaging units 21, that is, the front camera CF, the rear camera CB, the left camera CL, and the right camera CR captures an image of the surroundings of the own vehicle, and generates and acquires image information corresponding to the captured image. do. Further, each of the plurality of imaging units 21 outputs the acquired image information so that the object detection ECU 27 can receive it.

Each of the plurality of sonar sensors 22 measures the distance to a point on an object B existing around the vehicle by receiving received waves including reflected waves of the search waves transmitted toward the outside of the vehicle. do. Further, each of the plurality of sonar sensors 22 outputs the acquired distance measurement information so that the object detection ECU 27 can receive it.

The radar sensor 23 generates a signal corresponding to the position and relative speed of the reflection point on the object B, and outputs the signal so that it can be received by the object detection ECU 27 . The vehicle speed sensor 24 generates a signal corresponding to the vehicle speed and outputs the signal so that it can be received by the object detection ECU 27 . The shift position sensor 25 generates a signal corresponding to the shift position of the host vehicle and outputs the signal so that the object detection ECU 27 can receive the signal. The steering angle sensor 26 generates a signal corresponding to the steering angle of the host vehicle and outputs the signal so that it can be received by the object detection ECU 27 .

The object detection ECU 27 receives image information from each of the plurality of imaging units 21 . The object detection ECU 27 also receives ranging information from each of the plurality of sonar sensors 22 . The object detection ECU 27 also receives output signals from the radar sensor 23 , vehicle speed sensor 24 , shift position sensor 25 and steering angle sensor 26 . The object detection ECU 27 performs object detection operation and parking based on signals and information received from each of the plurality of sonar sensors 22, each of the plurality of imaging units 21, the vehicle speed sensor 24, the shift position sensor 25, the steering angle sensor 26, and the like. Perform assistive actions.

Specifically, the image information acquisition unit 270 acquires image information corresponding to the captured images around the vehicle from each of the plurality of imaging units 21 . In addition, the image information acquisition unit 270 stores the acquired image information in time series in the non-volatile RAM. The feature point extraction section 271 extracts feature points in the captured image based on the image information acquired by the image information acquisition section 270 .

The parallax image acquisition unit 272 acquires parallax images based on a plurality of captured images from different viewpoints. Specifically, the parallax image acquisition unit 272 performs parallax calculation by moving stereo or monocular moving stereo based on a plurality of captured images captured by the same imaging unit 21 at different times. By doing so, a parallax image is calculated.

The parking space acquisition unit 273 acquires a parking space. Specifically, parking space acquisition section 273 detects a parking space based on the image information acquired by image information acquisition section 270 . The road surface characteristic point extraction unit 274 extracts road surface characteristic points, which are characteristic points belonging to the road surface, from the photographed image of the surroundings of the vehicle including the object B. Specifically, the road surface characteristic point extraction unit 274 extracts the characteristic points included in the parking space and its neighboring area as the road surface characteristic points.

Based on the parallax image acquired by the parallax image acquiring unit 272 and the road surface characteristic points extracted by the road surface characteristic point extracting unit 274, the reference height setting unit 275 sets height information corresponding to the road surface characteristic points to a reference height. Set to information. Specifically, when parallax exists in the vicinity of a road surface characteristic point, the reference height setting unit 275 sets height information corresponding to the road surface characteristic point as the reference height information.

The coordinate information calculation section 276 calculates three-dimensional coordinate information of the object B based on the reference height information set by the reference height setting section 275 . Specifically, the coordinate information calculator 276 determines the three-dimensional positions of feature points other than the road surface feature points based on the reference height information corresponding to the road surface feature points and the parallax image. More specifically, the coordinate information calculation unit 276 uses a moving stereo or a monocular moving stereo technique while setting the height of the road surface characteristic point to 0 m from the road surface as a reference for the height of the road surface characteristic point, and calculates the height of the characteristic points other than the road surface characteristic point. Determine the three-dimensional position.

The ranging information acquisition unit 277 acquires ranging information corresponding to the object B from each of the multiple sonar sensors 22 . Also, the ranging information acquisition unit 277 acquires relative position information of the object B with respect to the own vehicle based on the ranging information. The distance measurement information acquisition unit 277 stores the acquisition results of the distance measurement information and the relative position information in time series in the non-volatile RAM. The obstacle detection unit 278 detects obstacles based on the calculation result of the three-dimensional coordinate information by the coordinate information calculation unit 276 and the distance measurement information and relative position information acquired by the distance measurement information acquisition unit 277 .

The photographing posture of the image pickup unit 21 changes according to the passenger's boarding state, luggage loading state, vehicle height adjustment state, and the like in the own vehicle. For example, there may be a case where the own vehicle has a height adjustable configuration and the vehicle height setting state has been changed from the standard state to the low vehicle height state or the high vehicle height state. Then, the relative height between the imaging unit 21 and the object B projecting from the road surface changes in the vehicle height direction.

Specifically, when the vehicle height setting state is changed to the low vehicle height state, the viewpoint in the imaging unit 21 is lowered and approaches the road surface. Then, the height of the object B seen from the imaging unit 21 apparently rises. The same applies to the case where the rear portion of the vehicle body 11 sinks due to a heavy object being loaded in the luggage compartment at the rear portion of the vehicle body 11 of the own vehicle. On the other hand, when the vehicle height setting state is changed to the high vehicle height state, the viewpoint in the imaging unit 21 is raised and moved away from the road surface. Then, the height of the object B seen from the imaging unit 21 is apparently lowered. In this way, when the viewpoint in the imaging unit 21 changes in the vehicle height direction, the absolute height information at each feature point in the captured image, that is, the vehicle height direction component in the three-dimensional coordinate information changes.

However, the inventors of the present invention found that even if the viewpoint in the imaging unit 21 changes in the vehicle height direction, the relative relationship of the detected heights between a plurality of detection points, that is, feature points in the captured image hardly changes. I found out. That is, for example, when a white line provided on the road surface and an object B projecting upward from the road surface exist within the same angle of view, the relative height relationship is almost unchanged regardless of viewpoint changes. Also, the relative height relationship between the plurality of feature points on the object B is almost unchanged regardless of the viewpoint change.

Therefore, the object detection ECU 27 extracts road surface characteristic points, which are characteristic points belonging to the road surface, from the photographed image of the surroundings of the own vehicle including the object B. FIG. Further, the object detection ECU 27 sets the height information corresponding to the road surface characteristic points to the reference height information corresponding to a height of 0 m from the road surface based on the parallax image and the extracted road surface characteristic points. Then, the object detection ECU 27 calculates three-dimensional coordinate information of the object B based on the reference height information.

According to such a configuration, the calculation error of the three-dimensional coordinate information of the object B due to the viewpoint change in the imaging unit 21 can be suppressed as much as possible. Therefore, erroneous determination of the height of the object B around the own vehicle can be satisfactorily avoided, and the height of the object B can be detected with higher accuracy.

Road feature points are easy to extract in parking spaces and their neighboring areas. Specifically, for example, road surface feature points are extracted from white lines on the road surface for demarcating parking spaces, road surface markings such as space numbers in parking spaces, bases of wheel chocks installed in parking spaces, and the like. easy to be These road surface feature points are used in detecting parking spaces.

Therefore, the object detection ECU 27 extracts feature points included in the parking space and its vicinity area as road feature points for setting reference height information corresponding to a height of 0 m from the road surface. As a result, the process of extracting the road characteristic points is performed satisfactorily, and the calculation accuracy of the three-dimensional coordinate information of the object B is improved. In addition, the search range for the road surface characteristic points can be easily defined, and the calculation load can be reduced.

For example, if a wheel chock is installed in the parking space, feature points can be extracted at both the base and top of the chock. In this way, a distribution of heights may occur at a plurality of feature points extracted from the parking space and its neighboring area.

Therefore, the object detection ECU 27 sets the reference height information based on the minimum height information among the feature points included in the parking space and its neighboring area. This improves the accuracy of extracting the road surface characteristic points.

(Operation example)
A specific operation example corresponding to the above outline of operation according to the configuration of this embodiment will be described below with reference to the flowchart shown in FIG. In the drawings, "step" is simply abbreviated as "S". The CPU of the object detection ECU 27 repeatedly activates the routine shown in FIG. 3 at predetermined time intervals while a predetermined activation condition is satisfied.

When this routine is activated, first, in step 311, the CPU acquires image information. The image information acquired here is, for example, image information acquired by the rear camera CB while the host vehicle is moving backward. That is, when the host vehicle is moving backward, the CPU acquires image information from the rear camera CB.

Next, at step 312, the CPU extracts feature points based on the acquired image information. Subsequently, at step 313, the CPU obtains or calculates a parallax image by parallax calculation. Additionally, at step 314, the CPU acquires or detects parking spaces. After that, the CPU advances the process to step 321 .

At step 321, the CPU extracts, from the feature points extracted at step 312, the feature points included in the parking space and its neighboring area as road surface feature points, and one of them is set as the road surface feature point Pn. select. n is a natural number equal to or smaller than N; N is the number of feature points extracted in step 312 that are included in the parking space and its neighboring area.

Next, at step 322, the CPU determines whether or not parallax exists in the vicinity of the road surface feature point Pn selected this time. If parallax exists in the vicinity of the road surface characteristic point Pn (that is, step 322 =YES), the CPU advances the process to step 323 . At step 323, the CPU calculates the height information of the road characteristic point Pn selected this time based on the parallax, and sets the calculation result as the reference height information. On the other hand, if there is no parallax in the vicinity of the road surface feature point Pn (that is, step 322 =NO), the CPU advances the process to step 324 . At step 324, the CPU excludes the height information of the road characteristic point Pn selected this time from the reference height information.

After executing the process of step 323 or step 324 , the CPU advances the process to step 325 . At step 325, the CPU determines whether or not the selection of the road surface characteristic point Pn has ended, that is, whether or not the current selection of the road surface characteristic point Pn is the Nth time.

When the selection of the road surface characteristic point Pn is completed (that is, step 325=YES), the CPU advances the process to steps 330-332. On the other hand, if the selection of road surface characteristic points Pn has not ended (that is, step 325 =NO), the CPU returns the process to step 321 .

The CPU ends this routine after sequentially executing the processing of steps 330 to 332 . At step 330, the CPU extracts the lowest high feature point. The lowest height feature point is the road feature point Pn set in the reference height information that has the lowest height information. At step 331, the CPU sets a reference height map based on the extracted minimum height feature point. The reference height map is a height map corresponding to the road surface height in the parallax image. At step 332, the CPU calculates three-dimensional coordinate information corresponding to each feature point other than the road surface feature point based on the reference height map.

(Modification)
The present invention is not limited to the above embodiments. Therefore, the above embodiment can be modified as appropriate. A representative modified example will be described below. In the following description of the modified example, differences from the above embodiment will be mainly described. Moreover, in the above-described embodiment and modifications, the same or equivalent portions are denoted by the same reference numerals. Therefore, in the description of the modification below, the description in the above embodiment can be used as appropriate for components having the same reference numerals as those in the above embodiment, unless there is a technical contradiction or special additional description.

The present invention is not limited to the specific device configurations shown in the above embodiments. That is, for example, the vehicle 10 equipped with the object detection device 20 is not limited to a four-wheeled vehicle. Specifically, the vehicle 10 may be a three-wheeled vehicle, or a six-wheeled or eight-wheeled vehicle such as a freight truck. The type of vehicle 10 may be an automobile equipped only with an internal combustion engine, an electric vehicle or a fuel cell vehicle without an internal combustion engine, or a so-called hybrid vehicle. The shape and structure of the vehicle body 11 are also not limited to a box shape, that is, a substantially rectangular shape in plan view. The number of door panels 17 is also not particularly limited.

There is also no particular limitation on the application target of the object detection device 20 . That is, for example, the object detection device 20 is not limited to a parking assistance device. Specifically, for example, the object detection device 20 can be suitably applied to semi-automatic driving or automatic driving corresponding to levels 2 to 5 in the definition of automatic driving.

The arrangement and number of imaging units 21 are not limited to the above example. That is, for example, the front camera CF can be arranged outside the vehicle compartment. Specifically, for example, the front camera CF can be attached to the front portion 12 of the vehicle body 11 . The number of front cameras CF may be one or two. That is, the object detection device 20 may have a compound-eye stereo camera configuration. For example, the left camera CL and the right camera CR may be arranged at positions different from the door mirror 18 . Alternatively, the left camera CL and right camera CR may be omitted.

The arrangement and number of sonar sensors 22 are not limited to the above specific examples. That is, for example, referring to FIG. 1, when the third front sonar SF3 is arranged at the center position in the vehicle width direction, the fourth front sonar SF4 is omitted. Similarly, when the third rear sonar SR3 is arranged at the central position in the vehicle width direction, the fourth rear sonar SR4 is omitted. The third side sonar SS3 and fourth side sonar SS4 may be omitted.

Various sensors used in the object detection device 20 are not limited to the vehicle speed sensor 24, the shift position sensor 25, the steering angle sensor 26, and the like. That is, for example, at least one of the vehicle speed sensor 24, the shift position sensor 25, and the steering angle sensor 26 may be omitted or replaced with another sensor.

The object detection ECU 27 constitutes a main part of the object detection device 20 in the above embodiment. Therefore, the imaging unit 21, the sonar sensor 22, the radar sensor 23, the vehicle speed sensor 24, the shift position sensor 25, the steering angle sensor 26, the display unit 28, and the audio output unit 29 are not components of the object detection device 20, but rather It may be understood to be an ancillary element of sensing device 20 . Alternatively, the imaging unit 21 and the sonar sensor 22 may be grasped as constituent elements of the object detection device 20 as constituting the image information acquisition unit 270 and the ranging information acquisition unit 277, respectively.

In the above-described embodiment, the object detection ECU 27 is configured such that the CPU reads a program from the ROM or the like and starts it. However, the invention is not limited to such a configuration. That is, for example, the object detection ECU 27 may have a configuration including a digital circuit, such as an ASIC or FPGA, configured to enable the above operations. ASIC is an abbreviation for Application Specific Integrated Circuit. FPGA stands for Field Programmable Gate Array.

The present invention is not limited to the specific functional configurations and operation examples shown in the above embodiments. For example, the parking space acquisition unit 273 may detect the parking space using the sonar sensor 22 or the like.

As described above, the object detection ECU 27 is not limited to the configuration that provides the parking assistance function. That is, for example, the object detection ECU 27 may be configured to have an automatic driving function. In this case, the parking space acquisition unit 273 may acquire the parking space detected by a parking assistance ECU different from the object detection ECU 27 from the parking assistance ECU.

With respect to road surface feature points, it is difficult to obtain parallax when edges of the same pattern are continuous along the epipolar line. Therefore, the road surface characteristic point extracting section 274 may set the reference height information based on the road surface characteristic points whose degree of parallelism with the epipolar line in the edge direction is less than or equal to a predetermined value. FIG. 4 shows part of a flow chart corresponding to such a modification. That is, steps 421 to 427 shown in FIG. 4 are replaced with steps 321 to 325 shown in FIG.

Specifically, after executing the process of step 314 in FIG. 3, the CPU advances the process to step 421 . The processing contents of step 421 are the same as the processing contents of step 321 in FIG. That is, at step 421, the CPU selects a road surface characteristic point Pn.

Next, at step 422, the CPU detects the edge direction at the road characteristic point Pn selected this time. A method for detecting the edge direction is already publicly known or known at the time of filing of the present application. Therefore, in this specification, the detailed description of the edge direction detection method is omitted.

Subsequently, at step 423, the CPU determines whether or not the parallelism of the edge direction detected at step 422 with the epipolar line is equal to or less than a predetermined value. If the parallelism is equal to or less than the predetermined value (that is, step 423 =YES), the CPU advances the process to step 424 . At step 424, the CPU determines whether or not parallax exists in the vicinity of the road surface characteristic point Pn selected this time. The CPU advances the process to step 427 after executing the process of step 425 or step 426 according to the determination result of step 424 . That is, the processing contents of steps 424 to 427 are the same as the processing contents of steps 322 to 325 in FIG.

Specifically, when parallax exists in the vicinity of the road surface characteristic point Pn (that is, step 424 =YES), the CPU advances the process to step 425 . At step 425, the CPU calculates the height information of the road characteristic point Pn selected this time based on the parallax, and sets the calculation result as the reference height information. On the other hand, if there is no parallax in the vicinity of the road surface feature point Pn (that is, step 424 =NO), the CPU advances the process to step 426 . At step 426, the CPU excludes the height information of the road characteristic point Pn selected this time from the reference height information.

If the degree of parallelism is greater than or equal to the predetermined value (that is, step 423 =NO), the CPU advances the process to step 426 . At step 426, the CPU excludes the height information of the road characteristic point Pn selected this time from the reference height information.

At step 427, the CPU determines whether or not the selection of the road surface characteristic points Pn has been completed. When the selection of road surface characteristic points Pn is completed (ie, step 427=YES), the CPU advances the process to steps 330 to 332 shown in FIG. On the other hand, if the selection of road surface characteristic points Pn has not ended (that is, step 427 =NO), the CPU returns the process to step 421 .

The viewpoint change in the imaging unit 21 may be caused by a cause other than the change in the vehicle height setting, that is, by the boarding state of the passenger and/or the load state of the luggage. Specifically, for example, the rear portion of the vehicle body 11 sinks when a heavy object is loaded in the luggage compartment at the rear portion of the vehicle body 11 of the host vehicle.

In such a case, the viewpoint change in the imaging unit 21 is accompanied not only by vertical movement of the viewpoint but also by a change in the bird's-eye view angle, that is, the photographing angle. The overhead angle is the angle at which the imaging unit 21 looks down on the object B, and is the same as the elevation angle at which the object B looks up at the imaging unit 21 . Then, the variation of the three-dimensional coordinate information at the detection point on the object B due to the variation of the viewpoint occurs not only in the vehicle height direction but also in the translational direction.

Therefore, as shown in FIG. 5 , the object detection ECU 27 further includes a coordinate information correction section 279 that corrects the three-dimensional coordinate information based on the distance measurement information that is the detection result of the sonar sensor 22 . Specifically, the coordinate information correction unit 279 calculates the distance between the imaging unit 21 and the object B estimated based on the three-dimensional coordinate information calculated based on the image information, and the sonar sensor 22 acquired by the sonar sensor 22. If the difference from the distance to the object B is within a predetermined value, the distance information is used to correct the three-dimensional coordinate information. This makes it possible to detect the height and position of the obstacle with high accuracy.

FIG. 6 shows part of a flow chart corresponding to such a modification. That is, steps 631 to 635 shown in FIG. 6 are replaced with steps 331 to 332 shown in FIG.

Specifically, after executing the process of step 330 in FIG. 3, the CPU advances the process to steps 631 and 632 . The processing contents of steps 631 and 632 are the same as the processing contents of steps 331 and 332 in FIG.

That is, at step 631, the CPU sets a reference height map based on the extracted minimum height feature point. Also, at step 632, the CPU calculates three-dimensional coordinate information corresponding to each feature point other than the road surface feature point based on the reference height map. After that, the CPU executes the processing of steps 633 and 634 .

At step 633, the CPU acquires distance measurement information. At step 634, the CPU determines whether or not the three-dimensional coordinates calculated based on the image information exist in the vicinity of the survey point whose relative position information was obtained by triangulation based on the distance measurement information.

If the three-dimensional coordinates exist near the survey point (that is, step 634=YES), the CPU executes the process of step 635 and then terminates this routine. At step 635, the CPU corrects the three-dimensional coordinate information based on the relative position information corresponding to the survey point, that is, the distance measurement information. On the other hand, if the three-dimensional coordinates do not exist near the survey point (that is, step 634=NO), the CPU skips the processing of step 635 and ends this routine.

The road surface feature point extraction section 274 may be provided as one function of the feature point extraction section 271 . That is, in the functional block configurations shown in FIGS. 2 and 5 , the road surface feature point extraction section 274 can be integrated with the feature point extraction section 271 .

The road surface characteristic point extraction unit 274 may extract road surface characteristic points based on predetermined extraction conditions other than the parking space. As the predetermined extraction condition, for example, position information in the angle of view, texture information, parallax information, calculated height information, edge direction, and the like can be used. In this case, parking space acquisition unit 273 may be omitted.

The processing content in the coordinate information calculation unit 276 is not limited to monocular movement stereo. Specifically, for example, compound eye stereo processing or integrated processing of SFM and compound eye stereo may be used. The compound-eye stereo processing or the integration processing of SFM and compound-eye stereo has already been publicly known or known at the time of filing of this application. Therefore, in this specification, detailed descriptions of the compound-eye stereo processing and the integration processing of SFM and compound-eye stereo are omitted.

The ranging information acquisition unit 277 may acquire ranging information based on the output of the radar sensor 23 instead of or together with the output of the sonar sensor 22 . That is, ultrasonic waves or electromagnetic waves can be used as probe waves. Also, a reflection point acquired by the radar sensor 23 can be used as the ranging point. In this case, the relative position information about the reflection points can be used as a substitute for the ranging points acquired by the sonar sensor 22 . Alternatively, relative position information about the reflection points can be used as a correction factor for ranging points acquired by sonar sensors 22 .

In the above specific example, the operation of detecting the object B using the image captured by the rear camera CB has been described. However, the invention is not limited to such aspects. That is, for example, the present invention can be suitably applied to the operation of detecting an object B using an image captured by the front camera CF. Similarly, the present invention can also be suitably applied to the operation of detecting an object B using images captured by the left camera CL and the right camera CR.

The expression "acquire" and similar expressions such as "estimation", "detection", "detection", and "calculation" can be appropriately replaced within a technically consistent range. Both "detection" and "extraction" can be interchanged as long as they are not technically inconsistent. The inequality sign in each determination process may or may not have an equal sign. That is, for example, "less than a predetermined value" and "not more than a predetermined value" can be interchanged within a technically consistent range.

Needless to say, the elements constituting the above-described embodiments are not necessarily essential, unless explicitly stated as essential or clearly considered essential in principle. In addition, when numerical values such as the number, numerical value, amount, range, etc. of a constituent element are mentioned, unless it is explicitly stated that it is particularly essential or when it is clearly limited to a specific number in principle, The present invention is not limited to the number of . Similarly, when the shape, direction, positional relationship, etc. of the constituent elements, etc. are mentioned, unless it is explicitly stated that it is particularly essential, or when it is limited to a specific shape, direction, positional relationship, etc. in principle , the shape, direction, positional relationship, etc., of which the present invention is not limited.

Each functional arrangement and method described above may be implemented by a dedicated computer provided by configuring a processor and memory programmed to perform one or more functions embodied by a computer program. . Alternatively, each functional configuration and method described above may be implemented by a dedicated computer provided by configuring a processor with one or more dedicated hardware logic circuits. Alternatively, each of the functional configurations and methods described above is one configured by a combination of a processor and memory programmed to perform one or more functions and a processor configured by one or more hardware logic circuits. It may be implemented by one or more dedicated computers. The computer program may also be stored in a computer-readable non-transitional tangible storage medium as instructions executed by a computer. That is, the device or method according to the present invention can be expressed as a computer program including procedures for realizing each function or method described above, or as a non-transitional material storage medium storing the program.

Modifications are also not limited to the above examples. Also, multiple variants can be combined with each other. Furthermore, all or part of the above embodiments and all or part of the modifications may be combined with each other.

10 vehicle 20 object detection device 22 sonar sensor 272 parallax image acquisition unit 273 parking space acquisition unit 274 road surface characteristic point extraction unit 275 reference height setting unit 276 coordinate information calculation unit 278 ranging information acquisition unit 279 coordinate information correction unit

Claims (4)

An object detection device (20) configured to detect an object (B) existing around the own vehicle (10) by being mounted on the own vehicle (10),
a road surface feature point extracting unit (274) for extracting road surface feature points, which are feature points belonging to a road surface, from a photographed image of the surroundings of the own vehicle including the object;
a parallax image acquisition unit (272) for acquiring parallax images based on the plurality of captured images from different viewpoints;
a reference height setting unit (275) for setting, as reference height information, height information corresponding to the road surface characteristic points based on the parallax image and the road surface characteristic points extracted by the road surface characteristic point extraction unit; ,
a coordinate information calculation unit (276) for calculating three-dimensional coordinate information of the object based on the reference height information set by the reference height setting unit;
with
The road surface feature point extraction unit sets the reference height information based on the road surface feature points at which parallelism between the edge direction and the epipolar line is a predetermined value or less.
Object detection device. a ranging information acquisition unit (277) that acquires ranging information that is a result of detection by a ranging sensor (22) that detects the distance to the object;
a coordinate information correction unit (279) that corrects the three-dimensional coordinate information based on the ranging information;
further comprising
The object detection device according to claim 1 . further comprising a parking space acquisition unit (273) that acquires the detection result of the parking space used for parking assistance,
The road surface feature point extraction unit extracts the road surface feature points based on the parking space.
The object detection device according to claim 1 or 2 . The reference height setting unit sets the reference height information based on the characteristic point having the minimum height information among the feature points included in the parking space and its neighboring area.
The object detection device according to claim 3 .

Images