JP6834914B2 - Object recognition device - Google Patents

Description

The present invention relates to an object recognition device.

Japanese Unexamined Patent Publication No. 2017-102861 discloses an object recognition device that acquires an image of the surroundings of a vehicle using an in-vehicle camera and calculates the position of an object included in the image on a map. Since the positional relationship between the object and the landmark that is established on the captured image is also established on the map, this object recognition device is the positional relationship between the object and the landmark that is specified on the captured image, and known information. Based on the position information of the landmark on the map, the position of the object specified on the captured image on the map is calculated.

Japanese Unexamined Patent Publication No. 2017-102861

However, since the accuracy of the front-rear distance is not high in the in-vehicle camera, the positional relationship between the object specified on the captured image and the landmark is not always accurate. Millimeter-wave radar can be used instead of the in-vehicle camera. However, millimeter-wave radar does not have high lateral position accuracy, so it cannot be said that it is the most suitable substitute for in-vehicle cameras.

In this respect, lidar (LIDAR: Laser Imaging Detection and Ranging) can correctly identify the positional relationship between an object and a landmark. However, if the rider's optical axis deviates from the desired direction, this particular accuracy will be reduced. Therefore, it is desirable that the correction of the optical axis of the rider is performed regularly. However, the correction of the optical axis is generally performed by using the certainty based on the relative position and relative speed of the vehicle and the object as an index. Therefore, the number of samples of the object required for the correction increases, and it takes time to complete the correction. Therefore, it is desired to develop a technique for completing the correction of the optical axis in a short time.

The present invention has been made in view of the above-mentioned problems, and an object of the present invention is to provide a technique capable of completing correction of the optical axis of a rider used for object recognition in a short time.

The first invention is an object recognition device for solving the above-mentioned problems, and has the following features.
The object recognition device includes a camera, a rider, a map database, a vehicle position estimation unit, an azimuth angle estimation unit, a fixed object position identification unit, and an optical axis reference correction unit.
The camera captures the external situation of the own vehicle.
The rider detects a three-dimensional object around the own vehicle.
The map database stores at least map information and location information and style information of fixed objects on the map.
The vehicle position estimation unit performs a process of estimating the position of the own vehicle on the map.
The azimuth estimation unit performs a process of estimating an angle formed by the front-rear axis of the own vehicle as a reference azimuth by using the image pickup information from the camera and the position information.
The fixed object position specifying unit performs a process of identifying the position of the fixed object detected by the rider on the map by using the fixed object information detected by the rider, the style information, and the position information. ..
The optical axis reference correction unit is a laser beam for a position on the map of the own vehicle, an azimuth angle, a position on the map of the fixed object, and an optical axis reference when the rider detects the fixed object. The optical axis reference is corrected by using the delivery angle of.

The second invention has the following features in the first invention.
The estimation process by the vehicle position estimation unit, the estimation process by the azimuth angle estimation unit, and the identification process by the fixed object position identification unit are performed at least while the own vehicle is stopped.

The third invention has the following features in the first or second invention.
The object recognition device further includes an optical axis abnormality determination unit.
When the angle difference between the optical axis reference after the correction process by the optical axis reference correction unit and the initial angle of the optical axis reference is equal to or greater than a predetermined angle, the optical axis abnormality determination unit has an abnormality in the optical axis reference. Judge that it has occurred.

According to the first invention, the position information of the fixed object on the map is used in the estimation process by the azimuth angle estimation unit. In addition, the position information and style information of the fixed object on the map are used in the identification process by the fixed object position specifying unit. Therefore, it is possible to shorten the time required for the estimation process by the azimuth angle estimation unit and the identification process by the fixed object position identification unit. Therefore, it is possible to complete the correction of the optical axis reference of the rider in a shorter time than when the optical axis correction is performed based on the certainty based on the relative position and the relative speed of the own vehicle and the object.

According to the second invention, the estimation process by the vehicle position estimation unit, the estimation process by the azimuth angle estimation unit, and the identification process by the fixed object position identification unit are performed at least while the own vehicle is stopped. When the own vehicle is stopped, it is possible to improve the estimation accuracy of the position and azimuth of the own vehicle on the map and the accuracy of specifying the position on the map of the fixed object as compared with the running of the own vehicle. .. Therefore, it is possible to improve the accuracy of the correction of the optical axis reference of the rider.

According to the third invention, the determination process by the optical axis abnormality determination unit becomes possible. Therefore, it is possible to call attention to the driver of the own vehicle.

It is a block diagram which shows the structure of the object recognition apparatus which concerns on Embodiment 1 of this invention. It is a figure explaining the outline of the detection method of a three-dimensional object by a rider. It is a figure explaining the calculation method of the deviation angle θgap by the optical axis reference correction part. It is a block diagram which shows the structure of the object recognition apparatus which concerns on Embodiment 2 of this invention.

Hereinafter, embodiments of the present invention will be described with reference to the drawings. However, when the number, quantity, quantity, range, etc. of each element is referred to in the embodiment shown below, the reference is made unless otherwise specified or clearly specified by the number in principle. The invention is not limited to these numbers. In addition, the structures, steps, and the like described in the embodiments shown below are not necessarily essential to the present invention, except when explicitly stated or clearly specified in principle.

Embodiment 1.
1. 1. Configuration of Object Recognition Device FIG. 1 is a block diagram showing a configuration of an object recognition device according to the first embodiment of the present invention. The object recognition device 1 shown in FIG. 1 is mounted on a vehicle and recognizes three-dimensional objects existing in front, side, and rear of the vehicle. The three-dimensional object to be recognized is, for example, a moving object such as a pedestrian, a bicycle, or an automobile, and a fixed object such as a tree, a utility pole, a building, or a road structure. Road structures include, for example, guardrails, road signs, road signs. In the present specification, the vehicle equipped with the object recognition device 1 is also referred to as "own vehicle A".

As shown in FIG. 1, the object recognition device 1 includes a camera 10, a rider 11, a GPS (Global Positioning System) device 12, a speed sensor 13, a gyro sensor 14, a map database 15, and an ECU (Electric Control Unit) 20. ing.

The camera 10 is a device that captures an external situation of the own vehicle A. The camera 10 is provided, for example, on the back side of the windshield of the own vehicle A. The camera 10 may be a monocular camera or a stereo camera. A stereo camera has, for example, two imaging units arranged to reproduce binocular parallax. The imaging information of the stereo camera also includes information in the depth direction. The camera 10 transmits the imaging information regarding the external situation of the own vehicle A to the ECU 20.

The rider 11 is a device that detects a three-dimensional object around the own vehicle A. The rider 11 irradiates the surroundings of the own vehicle A with a laser that emits a pulse of light, and measures the distance to the reflection point by receiving the reflected laser light from the three-dimensional object, thereby measuring the three-dimensional shape around the own vehicle A. Detect an object. The lidar 11 includes, for example, a laser controller, a laser oscillator, an optical system path, and a light receiving sensor. When the laser controller operates the laser oscillator, laser light is transmitted from the optical system path. The light receiving sensor signals the reflected laser light received via the optical system path. The rider 11 is the distance from the position of the own vehicle A (more specifically, the mounting position of the rider 11; the same applies hereinafter) to the position of the three-dimensional object, and the orientation of the three-dimensional object based on the same position (that is, the relative orientation). , And the outer shape (for example, height, width) of the three-dimensional object is output as the three-dimensional object information. The rider 11 transmits the detected three-dimensional object information to the ECU 20.

FIG. 2 is a diagram illustrating an outline of a method for detecting a three-dimensional object by the rider 11. In FIG. 2, the own vehicle A on which the rider 11 is mounted in front, the road sign B, and the road sign C are drawn. The laser beam emitted from the rider 11 at a wide angle of 90 ° or more is reflected by the road sign B or the road sign C. By measuring the time from the transmission of the laser light to the reception of the reflected laser light, the distance dAB from the position of the own vehicle A to the position of the road signboard B and the position of the own vehicle A to the position of the road sign C Distance dAC is calculated. Further, the relative orientation of the road sign B is calculated by measuring the transmission angle θABlider of the laser light with respect to the optical axis reference (the center line of the irradiation range of the laser light; the same applies hereinafter) when the reflected laser light is received. Will be done. That is, the relative direction of the road signboard B is calculated by measuring the transmission angle θABlider of the laser beam when the road signboard B is detected. Similar to the relative direction of the road sign B, the relative direction of the road sign C is calculated by measuring the transmission angle θAC lider of the laser light with respect to the optical axis reference when the reflected laser light is received.

The GPS device 12 measures the current position of the own vehicle A (for example, the latitude and longitude of the own vehicle A) by receiving signals from three or more GPS satellites. The GPS device 12 transmits the measured current position information of the own vehicle A to the ECU 20.

The speed sensor 13 is a device that detects the traveling speed of the own vehicle A. As the speed sensor 13, for example, a wheel speed sensor provided on a wheel of the own vehicle A, a drive shaft that rotates integrally with the wheel, or the like and detecting the rotation speed of the wheel is used. The speed sensor 13 transmits the traveling speed information of the own vehicle A to the ECU 20.

The gyro sensor 14 is a two-axis gyro sensor for detecting the angular velocity (specifically, the yaw rate) accompanying the change of direction of the own vehicle A. The gyro sensor 14 transmits the detected yaw rate information to the ECU 20. The yaw rate information is used for measuring the current position of the own vehicle A when the GPS device 12 cannot receive the radio waves from the GPS satellites. The current position measurement of the own vehicle A when the GPS device 12 cannot receive the radio waves from the GPS satellites is performed based on the yaw rate information and the traveling speed information from the speed sensor 13.

The map database 15 is a database provided with high-precision map information. The map database 15 is formed in, for example, an HDD (Hard Disk Drive) mounted on the own vehicle A. High-precision map information includes, for example, road position information, road shape information (for example, curve, straight line type, curve curvature, etc.), and position information of intersections and branch points. High-precision map information also includes information regarding location information (eg, latitude and longitude) of fixed objects such as buildings and road structures. High-precision map information also includes marking information for roads and road structures (eg, speed limit, destination, stop), update date of this marking information, style information for roads and road structures (eg, color, high). The width) is also included.

The ECU 20 is a control unit having a CPU (Central Processing Unit), a ROM (Read Only Memory), a RAM (Random Access Memory), a CAN (Controller Area Network) communication circuit, and the like. In the ECU 20, for example, various functions are realized by loading the program stored in the ROM into the RAM and executing the program loaded in the RAM in the CPU. The ECU 20 may be composed of a plurality of electronic control units.

2. 2. Configuration of ECU 20 Next, the functional configuration of ECU 20 will be described. As shown in FIG. 1, the ECU 20 includes a vehicle position estimation unit 21, an azimuth angle θazimuth estimation unit 22, a fixed object position identification unit 23, and an optical axis reference correction unit 24.

The vehicle position estimation unit 21 is based on the current position information of the own vehicle A measured by the GPS device 12 and the high-precision map information from the map database 15, and the current position of the own vehicle A on the high-precision map ( xAmap, yAmap) is estimated. When the GPS device 12 cannot receive the radio waves from the GPS satellites, the vehicle position estimation unit 21 determines the yaw rate information acquired from the gyro sensor 14, the traveling speed information acquired from the speed sensor 13, and a high-precision map from the map database 15. The current position (xAmap, yAmap) is estimated based on the information.

The azimuth θazimuth estimation unit 22 estimates the azimuth θazimuth. The estimation process of the azimuth angle θazimuth is performed, for example, as follows. First, the image pickup information from the camera 10 is analyzed, and the characteristics of an arbitrary fixed object in the captured image are recognized. Subsequently, matching processing is performed between the recognized feature of the fixed object and the high-precision map information of the map database 15. In this matching process, the position information of the recognized fixed object is searched from the high-precision map information. Then, when the position information exists in the high-precision map information, the direction (east, west, south) toward which the front part of the own vehicle A is facing is based on this position information and the current position (xAmap, yAmap). , And north) are set as the reference orientation. The azimuth θazimuth is calculated as the angle formed by the reference azimuth set in this way and the front-rear axis of the own vehicle A.

The fixed object position specifying unit 23 includes the position (xDlider, yDlider) of the fixed object (hereinafter, also referred to as “detection fixed object D”) detected by the rider 11 and the position (xDlider, yDlider) of the detected fixed object D on the high-precision map. xDmap, yDmap) and. The position (xDlider, yDlider) is specified based on the current position (xAmap, yAmap) and the three-dimensional object information from the rider 11. The process of specifying the position (xDmap, yDmap) is performed, for example, as follows. First, matching processing is performed between the style information of the road structure included in the high-precision map information and the detected fixed object D. In this matching process, the style information of the road structure and the outer shape of the detected fixed object D are collated, and the candidate E of the road structure is extracted. Candidate E is, for example, a road structure closest to the outer shape of the detection fixed object D. When the candidate E is extracted, the distance dDE between the position (xEmap, yEmap) of the candidate E on the high-precision map and the position (xDlider, yDlider) is calculated. When the distance dDE is equal to or less than the predetermined distance dTH, it is determined that the candidate E and the detected fixed object D are the same, and the position (xEmap, yEmap) is specified as the position (xDmap, yDmap).

When the distance dDE is larger than the predetermined distance dTH, it is based on the distance dDF between the position (xFmap, yFmap) of another candidate F of the road structure on the high-precision map and the position (xDlider, yDlider). Judgment is made. The other candidate F is, for example, a road structure second closest to the outer shape of the detection fixed object D. When the distance dDF is larger than the predetermined distance dTH, the matching process described above is performed on another fixed object detected by the rider 11. Even when the other candidate F is not extracted, or when the candidate E of the road structure is not extracted in the first place, the matching process described above is performed on another fixed object detected by the rider 11.

In the position (xDmap, yDmap) specifying process, the matching process of the plurality of detected fixed objects D and the plurality of road structures included in the high-precision map information may be performed in parallel. In this matching process, first, the one corresponding to the road structure is extracted from the plurality of detected fixed objects D. Subsequently, the relative positional relationship between the extracted plurality of detected fixed objects D (detected fixed objects D1, D2, ...) And the road structure corresponding to the detected fixed objects D1, D2, ... Is collated with the relative positional relationship on the high-precision map of. Then, when the relative positional relationships match, the detected fixed objects D1, D2, ... The positions (xD1map, yD1map), the positions (xD2map, yD2map), ... Of the corresponding road structures on the high-precision map.・ ・ Is specified.

In the case of the matching process based on the single detected fixed object D, there is an advantage that the result of the same object determination described above can be obtained in a short time, but there are the following disadvantages. That is, when the optical axis reference of the rider 11 deviates significantly from the desired direction, the above-mentioned distance dDE may become large and the determination based on the predetermined distance dTH may not be completed. In this regard, according to the matching process based on the plurality of detected and fixed objects D, the positions and positions of the detected and fixed objects D1, D2, ... On the high-precision map even when the deviation of the optical axis reference is large ( xD1map, yD1map), position (xD2map, yD2map), ... Can be specified. However, if the total number of samples of the detected fixed objects D1, D2, ... To obtain the relative positional relationship is increased, it takes time to complete the matching process, so the total number of samples is small (specifically, 2 or 2 or It is desirable to set it to 3).

In the position (xDmap, yDmap) identification process, even if the matching process based on the predetermined distance dTH described above is performed only for the detection fixed object D whose distance from the position of the own vehicle A is short to medium distance. Good. The shorter the distance dAD from the position of the own vehicle A to the detected fixed object D, the longer the above-mentioned distance dDE. Therefore, when the distance dAD is short, the fact that the distance (that is, the distance dDE) between the candidate E closest to the outer shape of the detected fixed object D and the detected fixed object D is equal to or less than the predetermined distance dTH means that the candidate E and the detection are detected. It means that the fixed object D is very likely to be the same object. Further, such a limiting condition may be combined with the matching process based on the plurality of detected fixed objects D1, D2, ...

The optical axis reference correction unit 24 starts from a desired direction of the optical axis reference of the rider 11 based on the current position (xAmap, yAmap), the azimuth angle θazimuth, the position (xDmap, yDmap), and the transmission angle θADlider. The deviation angle θgap is calculated and the optical axis reference is corrected. FIG. 3 is a diagram illustrating a method of calculating the deviation angle θgap by the optical axis reference correction unit 24. In the example shown in FIG. 3, the current position (xAmap, yAmap) is set at the origin of the xy coordinate plane, and the reference direction is aligned with the y axis of the xy coordinate plane. The delivery angle θADlider is an angle formed by the line segment VSVDlider passing through the current position (xAmap, yAmap) and the position (xDlider, yDlider) and the optical axis reference. The angle θmap formed by the line segment VSVDmap passing through the current position (xAmap, yAmap) and the position (xDmap, yDmap) and the y-axis (that is, the reference orientation) is the azimuth angle of the detected fixed object D on the high-precision map. I can say. The angle θmap can be calculated using, for example, the definition of the inner product of vectors.

As can be seen from FIG. 3, the relationship of the equation (1) is established between the azimuth angle θazimuth, the delivery angle θADlider, the angle θmap, and the deviation angle θgap.
θmap + θazimuth ＝ θADlider ＋ θgap ・ ・ ・ (1)
By transforming the equation (1), the deviation angle θgap can be expressed by the equation (2).
θgap ＝ θmap + θazimuth−θADlider ・ ・ ・ (2)

In FIG. 3, the optical axis reference is located on the left side of the reference direction, and the position (xDmap, yDmap) is located on the right side of the position (xDlider, yDlider). The deviation angle θgap can be expressed by the equation (3) when the cases are classified in consideration of these positional relationships.
θgap = θmap + θazimuth−θADlider (θazimuth <0, xDmap> xDlider)
θgap = θADlider− (θazimuth + θmap) (θazimuth <0, xDlider> xDmap)
θgap = θmap− (θazimuth + θADlider) (θazimuth> 0, xDmap> xDlider)
θgap ＝ θADlider ＋ θazimuth−θmap (θazimuth> 0, xDlider> xDmap)
... (3)

The optical axis reference correction unit 24 corrects the optical axis reference by using the deviation angle θgap calculated in this way. The optical axis reference before and after the correction is represented by the equation (4).
Optical axis reference after correction = Optical axis reference before correction-Deviation angle θgap ・ ・ ・ (4)

Processing other than the processing by the optical axis reference correction unit 24, that is, the estimation processing of the current position (xAmap, yAmap) by the vehicle position estimation unit 21, the estimation processing of the azimuth angle θazimuth by the azimuth angle θazimuth estimation unit 22, and the fixed object position. It is preferable that the position (xDmap, yDmap) identification process by the identification unit 23 is performed while the own vehicle A is stopped. In the present specification, "the own vehicle A is stopped" means, strictly speaking, while the traveling speed of the own vehicle A is zero, but it travels at an extremely low speed (for example, the traveling speed is 5 km / h or less). It is also included in "the own vehicle A is stopped" while it is being used.

In the estimation process of the azimuth angle θazimuth and the identification process of the position (xDmap, yDmap), it is necessary to perform the matching process. In these matching processes, the accuracy of the image pickup information from the camera 10 and the three-dimensional object information from the rider 11 is important. Therefore, by performing these matching processes while the own vehicle A is stopped and the estimation process of the current position (xAmap, yAmap) performed in parallel with these matching processes while the own vehicle A is stopped, the azimuth It is possible to improve the estimation accuracy of the angle θazimuth and the identification accuracy of the position (xDmap, yDmap), and thus the accuracy of the correction of the optical axis reference. Whether or not the own vehicle A is stopped is determined based on the traveling speed information from the speed sensor 13.

3. 3. Effect of the object recognition device according to the first embodiment According to the object recognition device according to the first embodiment described above, the matching process performed when estimating the azimuth angle θazimuth is the position of the fixed object included in the high-precision map information. It is done using information. Further, the matching process performed when specifying the position (xDmap, yDmap) is performed by using the style information and the position information of the road structure included in the high-precision map information. Therefore, the time required for these matching processes can be remarkably shortened. Therefore, it is possible to complete the correction of the optical axis reference of the rider 11 in a shorter time than when the optical axis correction is performed based on the certainty based on the relative position and the relative speed of the own vehicle A and the detected fixed object D. Become.

Embodiment 2.
1. 1. Features of the second embodiment The object recognition device according to the second embodiment of the present invention is different from the object recognition device according to the first embodiment in the functional configuration of the ECU. Therefore, in the following, this difference will be mainly described.

2. 2. Configuration of ECU 20 The functional configuration of the ECU 20 included in the object recognition device 2 according to the second embodiment will be described with reference to FIG. The ECU 20 shown in FIG. 4 includes a vehicle position estimation unit 21, an azimuth angle θazimuth estimation unit 22, a fixed object position identification unit 23, an optical axis reference correction unit 24, and an optical axis abnormality determination unit 25. .. The configuration other than the optical axis abnormality determination unit 25 is as described in the first embodiment.

The optical axis abnormality determination unit 25 is used when the angle difference between the corrected optical axis reference calculated based on the above equation (4) and the initial angle of the optical axis reference is a predetermined angle (for example, 10 °) or more. , It is determined that an abnormality has occurred in the optical axis reference of the rider 11. When it is determined that an abnormality has occurred in the optical axis reference, the optical axis abnormality determination unit 25 transmits an abnormality occurrence signal to an alarm ECU (not shown).

The alarm ECU is connected to an HMI (Human Machine Interface) (for example, an audio output means such as a buzzer or a speaker, a HUD (Head Up Display), a display of a navigation system, a display means such as a combination meter). The alarm ECU outputs a warning voice from the voice output means according to the abnormality occurrence signal, or displays a warning message, a warning lamp, or the like on the display means to notify the driver of the own vehicle A of the operation status of the support control.

3. 3. Effect of Object Recognition Device According to Embodiment 2 According to the object recognition device according to Embodiment 2 described above, the corrected optical axis reference calculated based on the above equation (4) and the optical axis reference When the angle difference from the initial angle is a predetermined angle (for example, 10 °) or more, an abnormality occurrence signal is transmitted to the alarm ECU. Therefore, the driver of the own vehicle A can be alerted.

1, 2, Object recognition device 10 Camera 11 Rider 15 Map database 20 ECU
21 Vehicle position estimation unit 22 Azimuth angle θazimuth estimation unit 23 Fixed object position identification unit 24 Optical axis reference correction unit 25 Optical axis abnormality judgment unit A Own vehicle B Road sign C Road sign D Detection fixed object VSVDlider, VSVDmap Line segment θABlider, θAClider ， ΘAD lider delivery angle θazimuth azimuth θgap deviation angle

Claims (3)

A camera that captures the external situation of your vehicle,
The rider who detects the three-dimensional object around the own vehicle and
A map database that stores at least map information and location and style information of fixed objects on the map,
A vehicle position estimation unit that performs processing for estimating the position of the own vehicle on the map, and
An azimuth angle estimation unit that performs a process of estimating an angle formed by the front-rear axis of the own vehicle as a reference direction using the image pickup information from the camera and the position information.
A fixed object position specifying unit that performs a process of specifying a position on a map of a fixed object detected by the rider by using the fixed object information detected by the rider, the style information, and the position information.
Using the position on the map of the own vehicle, the azimuth, the position on the map of the fixed object, and the transmission angle of the laser beam with respect to the optical axis reference when the rider detects the fixed object. , An optical axis reference correction unit that performs processing to correct the optical axis reference,
An object recognition device characterized by comprising. The first aspect of the present invention is that the estimation process by the vehicle position estimation unit, the estimation process by the azimuth angle estimation unit, and the identification process by the fixed object position identification unit are performed at least while the own vehicle is stopped. The object recognition device described. Light that determines that an abnormality has occurred in the optical axis reference when the angle difference between the optical axis reference after the correction process by the optical axis reference correction unit and the initial angle of the optical axis reference is equal to or greater than a predetermined angle. The object recognition device according to claim 1 or 2, further comprising an axis abnormality determination unit.

Images