JP6905390B2 - Own vehicle position estimation environment map generator, own vehicle position estimation device, own vehicle position estimation environment map generation program, and own vehicle position estimation program - Google Patents

Description

The present invention relates to an environment map generator for estimating the position of the own vehicle, an environment map generator for estimating the position of the own vehicle, an environment map generation program for estimating the position of the own vehicle, and an environment map generator for estimating the position of the own vehicle, which generates a map for estimating the position of the own vehicle. ..

A technique for estimating the position of a moving body in a three-dimensional space has been proposed based on a captured image taken by a photographing unit such as a camera mounted on the moving body.

For example, in Patent Document 1, an acquisition unit that acquires information including first data relating to a first image and second data relating to a second image including a difference between the first image due to movement of a moving body and the like. A direction estimation process for estimating the direction of rotation accompanying the movement of the moving body based on the above information, a plurality of first feature points in the first region in the first image based on the direction of rotation, and a first image. A plurality of second feature points in the second feature point are detected, and the first density of the plurality of first feature points in the first region is the first density of the plurality of second feature points in the second region. Detection processing higher than 2 densities and a plurality of first correspondence points in a third region in the second image, each of the plurality of first correspondence points is at least one of a plurality of first feature points. A position estimation process for estimating a change in the position of a moving body based on each of a plurality of first feature points and each of a plurality of first corresponding points by obtaining a plurality of first corresponding points corresponding to the above. A moving body position estimation device including a processing unit for carrying out the above has been proposed.

Further, in Patent Document 1, in addition to acquiring and estimating the rotation direction accompanying the movement of the moving body from the information, it is possible to estimate using a speed sensor, an acceleration sensor, an angular velocity sensor, and an angular acceleration sensor. Has been described.

Japanese Unexamined Patent Publication No. 2016-61604

However, as in the technique of Patent Document 1, when the rotation direction of the photographing unit mounted on the moving body is estimated based on the movement of the moving body based on the two captured images, two images are taken in order to detect the rotation direction. There is room for improvement because the processing load such as comparing the feature points of images becomes high. Further, when estimating the rotation direction of the photographing unit accompanying the movement of the moving body based on the detection results of the speed sensor, the acceleration sensor, the angular velocity sensor, and the angular acceleration sensor, the acceleration sensor, the angular velocity sensor, and the angular acceleration sensor are used. Since the noise is large, there is room for improvement in stably estimating the rotation direction of the photographing unit due to the movement of the moving object. In particular, in the low speed range, the rotation direction of the photographing unit due to the movement of the moving body cannot be stably estimated.

Further, when generating an environmental map, if the rotation direction of the photographing unit due to the movement of the moving object cannot be estimated stably, the image area for extracting the feature points cannot be set appropriately, so that the map accuracy is lowered and the moving object is generated. The estimation accuracy of the position of is also reduced.

The present invention has been made in consideration of the above facts, and is an environment map generation device for estimating the position of the own vehicle, a position estimation device for the own vehicle, and an estimation of the position of the own vehicle that can stably estimate the position and posture of the photographing unit. It is an object of the present invention to provide an environmental map generation program and a vehicle position estimation program.

In order to achieve the above object, the vehicle position estimation environment map generator according to the present invention has an imaging unit that captures the moving direction of the vehicle and an acquisition unit that acquires the detection results of the detection unit that detects the steering angle of the vehicle. Then, using the detection result acquired by the acquisition unit, feature points are extracted from an area other than a predetermined range corresponding to the moving direction of the vehicle in the photographed image captured by the photographing unit, or Using the detection result acquired by the acquisition unit, feature points are extracted from the entire region of the captured image, and other regions are weighted than the weighting of regions in a predetermined range corresponding to the moving direction of the vehicle in the captured image. The position and orientation of the vehicle are estimated based on the correspondence between the extraction unit for increasing the size, the feature points extracted by the extraction unit, and the feature points of other captured images taken at different positions, and based on the estimation result. It includes a generation unit that obtains the three-dimensional position of the feature point and generates a local environmental map.

The generation unit may generate an environmental map of a local space having restrictions on the traveling route of the vehicle as a local environmental map.

Further, in order to achieve the above object, the own vehicle position estimation device according to the present invention includes a photographing unit that photographs the moving direction of the vehicle, an acquisition unit that acquires the detection result of the detecting unit that detects the steering angle of the vehicle, and the acquisition unit. Using the detection result acquired by the acquisition unit, feature points are extracted from a region other than a predetermined range corresponding to the moving direction of the vehicle in the captured image captured by the imaging unit, or the acquisition is performed. Using the detection result acquired by the unit, feature points are extracted from the entire region of the captured image, and the weighting of other regions is larger than the weighting of the region in the predetermined range corresponding to the moving direction of the vehicle in the captured image. The position and orientation of the vehicle are estimated based on the extraction unit, the feature points extracted by the extraction unit, and the corresponding feature points on the environmental map including the three-dimensional coordinates and the feature amount generated in advance. It includes an estimation unit that estimates the position of the own vehicle based on the estimation result.

Further, in order to achieve the above object, the own vehicle position estimation environment map generation program according to the present invention acquires the detection result of the detection unit that detects the steering angle of the vehicle, and uses the detection result to move the vehicle. Feature points are extracted from a region other than a predetermined range corresponding to the moving direction of the vehicle in the captured image captured by the photographing unit that captures the direction, or the entire area of the captured image is used by using the detection result. increasing the weighting of the other region than the weighting of the region of a predetermined range corresponding to the moving direction of the vehicle at both the captured image and extracts feature points from the extracted feature points has, captured another taken at different positions The position and attitude of the vehicle are estimated based on the correspondence with the feature points of the image, the three-dimensional position of the feature points is obtained based on the estimation result, and the process of generating a local environmental map is executed on the computer. Let me.

Further, in order to achieve the above object, the own vehicle position estimation program according to the present invention acquires the detection result of the detection unit that detects the steering angle of the vehicle, and uses the detection result to photograph the moving direction of the vehicle. Feature points are extracted from a region other than a predetermined range corresponding to the moving direction of the vehicle in the captured image captured by the photographing unit, or the feature points are extracted from the entire region of the captured image using the detection result. In addition to extracting, the weighting of other areas is made larger than the weighting of the area in the predetermined range corresponding to the moving direction of the vehicle in the captured image, and the extracted feature points, the three-dimensional coordinates and the feature amount generated in advance are included. It was on the environmental map corresponding based on the feature points to estimate the position and orientation of the vehicle, based on the estimation result, thereby executing the process of estimating the position of the vehicle to the computer.

As described above, according to the present invention, there is an effect that the position and orientation of the photographing unit can be stably estimated.

It is a figure which shows the vehicle which carries the own vehicle position estimation device which concerns on this embodiment. It is a block diagram which shows the schematic structure of the own vehicle position estimation device which concerns on this embodiment. It is a figure which shows the photographed image at 2 time and the region in the traveling direction of the photographed image. (A) is a diagram showing an example of a captured image at the current time, and (B) is a diagram showing an example of a captured image at the previous time. It is a flowchart which shows an example of the flow of the process performed by executing the environment map generation program in the own vehicle position estimation apparatus which concerns on this embodiment. It is a flowchart which shows an example of the flow of the process performed by executing the own vehicle position estimation program in the own vehicle position estimation apparatus which concerns on this embodiment.

Hereinafter, an example of the embodiment of the present invention will be described in detail with reference to the drawings. FIG. 1 is a diagram showing a vehicle 20 equipped with the own vehicle position estimation device according to the present embodiment, and FIG. 2 is a block diagram showing a schematic configuration of the own vehicle position estimation device according to the present embodiment.

The own vehicle position estimation device 10 includes a camera 14 as a photographing unit. The camera 14 is provided in a trunk or the like at the rear of the vehicle, and photographs the rear of the vehicle. For example, as shown in FIG. 1, the camera 14 is provided near a substantially central portion in the vehicle width direction, and is arranged so that the optical axis of the camera 14 faces slightly downward from the horizontal direction.

Further, the own vehicle position estimation device 10 includes a control unit 12 that performs calculations and the like for estimating the own vehicle position. As shown in FIG. 2, the control unit 12 includes, for example, a computer in which the CPU 12A, ROM 12B, RAM 12C, and I / O (input / output interface) 12D are connected to the bus 12E.

The ROM 12B stores various programs and data including an environment map generation program for generating an environment map and a vehicle position estimation program for estimating the vehicle position. By expanding the program stored in the ROM 12B into the RAM 12C and executing it by the CPU 12A, the environment map is generated and the position of the own vehicle is estimated.

A camera 14 is connected to the I / O 12D, and a steering angle sensor 16 that detects the steering angle of the vehicle 20 is connected to the I / O 12D. Then, the captured image of the camera 14 and the steering angle (hereinafter referred to as the steering angle) detected by the steering angle sensor 16 are input to the control unit 12.

Further, a storage unit 18 such as an HDD (Hard Disk Drive), an SSD (Solid State Drive), and a flash memory is connected to the I / O 12D. The storage unit 18 may be provided in the control unit 12 or may be externally connectable. Further, although the storage unit 18 stores the environment map generated by the control of the control unit 12, the environment map generation program and the own vehicle position estimation program may be stored.

By executing the environment map generation program, the control unit 12 creates an environment map of a three-dimensional point cloud based on the captured image taken by the camera 14 and the steering angle detected by the steering angle sensor 16. It is generated and stored in the storage unit 18. Further, the control unit 12 estimates the position of the own vehicle based on the environment map stored in the storage unit 18 by executing the own vehicle position estimation program. In the present embodiment, the control unit 12 generates a local environmental map as the generated environmental map, but more specifically, a local space (for example, a parking space or the like) in which the traveling route of the vehicle 20 is restricted. ) Generate an environmental map.

Here, a method of creating an environmental map by the own vehicle position estimation device 10 according to the present embodiment will be described.

The control unit 12 of the own vehicle position estimation device 10 according to the present embodiment extracts feature points from each photographed image taken by the camera 14 at two different positions, and associates the corresponding feature points with each other. Then, the position and orientation of the camera 14 are estimated based on the corresponding feature points of the two captured images, the three-dimensional coordinates of the feature points are calculated, and an environmental map is generated.

Specifically, the control unit 12 acquires the captured image at the current time and generates a pair with the captured image at the previous time. Then, based on the detection result of the steering angle sensor 16, as shown in FIG. 3, a region in the traveling direction of the captured image at the current time is specified. Then, the control unit 12 extracts feature points other than the region in the traveling direction of the captured image at the current time (the region indicated by the dotted line circle in FIG. 3). Similarly, it is assumed that the feature points other than the region in the traveling direction have already been extracted from the captured image at the previous time.

Here, in the present embodiment, the region other than the region in the traveling direction is specified by deriving the region in the traveling direction of the captured image based on the steering angle detected by the steering angle sensor 16. That is, since the traveling direction can be determined if the steering angle of the vehicle 20 is known, the region of the traveling direction of the captured image can be specified from the steering angle detected by the steering angle sensor 16. As a result, the processing load when detecting the region in the traveling direction of the vehicle 20 can be reduced as compared with the case where the region in the traveling direction is determined using the two captured images. In addition, more stable detection of the traveling direction is possible than when the region of the vehicle 20 in the traveling direction is detected by using the detection results of the acceleration sensor, the angular velocity sensor, and the angular acceleration sensor. In particular, when the vehicle 20 is moving at a low speed, it is difficult to detect the traveling direction of the vehicle 20 from the acceleration sensor, the angular velocity sensor, and the angular acceleration sensor, but the detection result of the steering angle sensor 16 is used. This enables stable detection of the traveling direction. The region in the traveling direction specifies a region in a predetermined range including the traveling direction detected from the steering angle. For example, the region in the traveling direction is specified by dividing the captured image into a plurality of regions and associating the captured images with the regions divided into a plurality of captured images based on the steering angle detected by the steering angle sensor 16.

As a method for extracting feature points, for example, ORB-SLAM can be applied. ORB-SLAM detects a corner as a feature point and uses FAST (Features from Accelerated Segment Test) as a detection method. In addition, ORB-SLAM uses ORB (Oriented FAST and Rotated BRIEF) to describe the features. The ORB is based on BRIEF (Binary Robust Independent Elementary Features) and has been developed to have scale invariance and rotation invariance. Since ORB-SLAM is disclosed in Reference 1 below, detailed description thereof will be omitted.
[Reference 1] Raul Mur-Artal, JMMMontiel and Juan D.Tardos. ORB-SLAM: A Versatile and Accurate Monocular SLAM System. IEEE Transactions on Robotics, vol.31, no.5, pp.1147-1163, 2015.

For example, an example of the captured image at the current time is shown in FIG. 4 (A), and an example of the captured image at the previous time is shown in FIG. 4 (B). In the case of this example, the region in each traveling direction specified from the detection result of the rudder angle sensor 16 is defined as the region indicated by the dotted line circle in the figure, and the result of extracting the feature points of the region other than this region is the solid line. It is indicated by a circle.

Further, when extracting the feature points, the moving amount of the feature points is smaller in the region corresponding to the traveling direction of the vehicle 20 in the captured image than in the region other than the region in the traveling direction, so that the traveling direction is predetermined. The feature points in the area other than the above area are extracted. As a result, the processing load for extracting the feature points can be reduced as compared with the case where the feature points on the entire surface of the captured image are extracted. In addition, by extracting the feature points from the region where the amount of movement of the feature points is large, the deviation of the corresponding feature points between the captured images corresponding to the parallax when estimating the three-dimensional position of the feature points in the subsequent processing is eliminated. As the size increases, more accurate three-dimensional position estimation becomes possible. In particular, in a local space such as a parking space where the traveling path of the vehicle 20 is restricted, it is possible to accurately estimate the three-dimensional position by extracting feature points in a region other than a predetermined region in the traveling direction. Become.

Further, in the association of the feature points between the captured images of the current time and the previous time, for example, the feature points of the current time and the feature points of the previous time are associated with each other by using the feature amount such as ORB described above.

Further, in the estimation of the position and orientation of the camera 14 and the calculation of the three-dimensional coordinates of the feature points, in the initialization stage, for example, from the pair of the associated plurality of feature points, the 5-point method and the 7-point method, etc. Is used to estimate the basic matrix based on the captured image at the previous time. The elementary matrix may be estimated based on the captured image at the current time.

In addition, the translation vector and the rotation matrix are calculated from the basic matrix obtained by estimation by singular value decomposition or the like. By calculating the translation vector and the rotation matrix, the position and orientation of the camera 14 at the time of shooting the captured image at the current time is estimated based on the position and orientation of the camera 14 at the time of capturing the captured image at the previous time. The three-dimensional map coordinate system is set with the position of the camera 14 at the time of shooting the captured image at the previous time as the origin and the posture of the camera 14 at the time of shooting the captured image at the previous time as a reference.

In addition, the position in the captured image at the previous time and the position in the captured image at the current time of the pair of associated feature points, and the estimated position and orientation of the camera 14 at the time of capturing the captured image at the previous time. Using the position and orientation of the camera 14 at the time of shooting the captured image at the time, the three-dimensional coordinates of the feature points with respect to the position and orientation of the camera 14 at the time of capturing the captured image at the previous time are calculated by the principle of triangular surveying. do. Then, the three-dimensional coordinates are converted into the three-dimensional coordinates of the map coordinate system.

Then, the environment map information including the three-dimensional coordinates of the map coordinate system obtained for each feature point and the feature amount is generated. Further, the position and orientation of the camera 14 at the time of shooting the captured image at the previous time in the map coordinate system and the position and orientation of the camera 14 at the time of capturing the captured image at the current time are obtained and included in the image information and the image. It is stored in the environment map information together with the feature point information.

On the other hand, when it is not in the initialization stage, each feature point of the environmental map information is collated with the feature point of the captured image at the current time, and the position of the camera 14 at the time of capturing the captured image at the current time in the map coordinate system and Ask for a posture.

In addition, when there is a new feature point that is not registered in the environmental map information among the associated feature points, the position of the new feature point in the captured image at the current time and the position in the other image are used. , Triangular survey using the position and orientation of the camera 14 at the time of shooting the captured image at the previous time and the position and orientation of the camera 14 at the time of capturing the captured image at the obtained current time obtained from the environmental map information. According to the principle of, the three-dimensional coordinates of the feature points with respect to the position and orientation of the camera 14 at the time of shooting the captured image at the previous time are calculated. Then, the three-dimensional coordinates are converted into the three-dimensional coordinates of the map coordinate system.

Then, the three-dimensional coordinates and feature quantities of the map coordinate system obtained for the new feature points are stored in the environment map information. Further, the position and orientation of the camera 14 at the time of shooting the obtained captured image at the current time are stored in the environment map information together with the image information and the feature point information included in the image.

Further, the control unit 12 of the own vehicle position estimation device 10 according to the present embodiment extracts feature points from the captured image taken by the camera 14, collates them with the feature points included in the environmental map information, and collates the feature points with the map coordinate system. The position and orientation of the camera 14 at the time of taking the captured image in the above are obtained, and the self-position and orientation of the vehicle 20 are used. At this time, similarly to creating the environment map, the region in the traveling direction of the captured image at the current time is specified based on the detection result of the rudder angle sensor 16, and the region other than the region in the traveling direction of the captured image at the current time is specified. Extract feature points.

Subsequently, the process of generating the environmental map performed by the own vehicle position estimation device 10 according to the present embodiment configured as described above will be described. FIG. 5 is a flowchart showing an example of the flow of processing performed by executing the environment map generation program in the own vehicle position estimation device 10 according to the present embodiment, and in the processing, captured images at each time are acquired. It is repeated every time.

In step 100, the CPU 12A acquires the detection result of the steering angle sensor 16 and proceeds to step 102.

In step 102, the CPU 12A calculates the traveling direction of the vehicle 20 (camera 14) based on the acquired detection result of the steering angle sensor 16 and proceeds to step 104.

In step 104, the CPU 12A determines the feature point extraction area and proceeds to step 106. The feature point extraction region is determined as a feature point extraction region in the image captured by the camera 14 other than the region in the predetermined range corresponding to the traveling direction of the vehicle 20 calculated in step 102.

In step 106, the CPU 12A extracts feature points from the captured image at the current time and proceeds to step 108. That is, feature points in a region other than the predetermined range corresponding to the traveling direction of the vehicle 20 determined in step 104 are extracted. For example, a feature point is extracted from a region other than the moving direction of the vehicle 20 (camera 14) by using FAST or the like.

In step 108, the CPU 12A uses the feature points extracted in step 106 of the previous time and the feature points extracted in step 106 of the previous time to feature between the captured image at the previous time and the captured image at the current time. The points are associated with each other and the process proceeds to step 110. That is, the feature points between the captured images at different times are associated with each other. For example, a feature amount such as ORB is used to associate a feature point at the current time with a feature point at the previous time. If there is no captured image at the previous time, the process ends without performing the processes after step 106.

In step 110, the CPU 12A determines whether or not it is in the initialization stage. The determination determines whether or not the feature points are not registered on the map. If the determination is affirmed, the process proceeds to step 112, and if the determination is denied, the process proceeds to step 116.

In step 112, the CPU 12A estimates the position and orientation of the camera 14 from the associated feature points and proceeds to step 114. For example, an elementary matrix based on the captured image at the previous time is estimated. In addition, the translation vector and rotation matrix are calculated from the basic matrix estimated by singular value decomposition, etc., and the captured image at the current time is based on the position and orientation of the camera 14 at the time of capturing the captured image at the previous time. The position and orientation of the camera 14 at the time of shooting are estimated.

In step 114, the CPU 12A estimates the three-dimensional position of the feature point based on the position and orientation of the camera 14 and registers it in the map. Specifically, the position in the image captured image at the previous time and the position in the captured image at the current time of the pair of associated feature points, and the estimated position of the camera 14 at the time of capturing the captured image at the previous time. Using the position and orientation of the camera 14 at the time of shooting the captured image at the current time with respect to the posture, the third order of the feature points with respect to the position and orientation of the camera 14 at the time of capturing the captured image at the current time by the principle of triangulation. Calculate the original coordinates. Then, the three-dimensional coordinates are converted into the three-dimensional coordinates of the map coordinate system to generate an environment map, and the generated environment map is stored in the storage unit 18.

On the other hand, in step 116, the CPU 12A estimates the position and orientation of the camera 14 at the time of shooting the captured image at the current time from the feature points already registered on the map and the feature points on the captured image, and proceeds to step 118. Transition.

In step 118, the CPU 12A generates an environmental map by estimating the three-dimensional position of the new feature point and registering it in the map based on the position and orientation of the camera 14 at the time of shooting the captured image at the current time. That is, the position of the new feature point in the captured image at the current time, the position in the captured image at the previous time, the position and orientation of the camera 14 at the time of capturing the captured image at the previous time registered on the map, and step 116. Using the position and orientation of the camera 14 at the time of shooting the captured image at the current time obtained in the above, the third order of the feature points with respect to the position and orientation of the camera 14 at the time of capturing the captured image at the previous time according to the principle of triangulation. Calculate the original coordinates. Then, the three-dimensional coordinates are converted into the three-dimensional coordinates of the map coordinate system to generate an environment map, and the generated environment map is stored in the storage unit 18.

In addition, steps 100 to 106 in the process of FIG. 5 correspond to the extraction unit of the own vehicle position estimation environment map generation device and the own vehicle position estimation environment map generation program, and steps 108 to 118 correspond to the generation unit.

Next, the process of estimating the own vehicle position performed by the own vehicle position estimation device 10 according to the present embodiment will be described. FIG. 6 is a flowchart showing an example of the flow of processing performed by executing the own vehicle position estimation program in the own vehicle position estimation device 10 according to the present embodiment.

In step 200, the CPU 12A acquires the detection result of the steering angle sensor 16 and proceeds to step 202.

In step 202, the CPU 12A calculates the traveling direction of the vehicle 20 (camera 14) based on the acquired detection result of the steering angle sensor 16 and proceeds to step 204.

In step 204, the CPU 12A determines the feature point extraction area and proceeds to step 106. The feature point extraction region is determined as a feature point extraction region in the image captured by the camera 14 other than the region in the predetermined range corresponding to the traveling direction of the vehicle 20 calculated in step 202.

In step 206, the CPU 12A extracts feature points from the captured image at the current time and proceeds to step 208. That is, feature points in a region other than the predetermined range corresponding to the traveling direction of the vehicle 20 determined in step 204 are extracted. For example, a feature point is extracted from a region other than the moving direction of the vehicle 20 (camera 14) by using FAST or the like.

In step 208, the CPU 12A collates the feature points included in the registered environment map with the feature points extracted in step 206, and determines the position and orientation of the camera 14 at the time of shooting the captured image in the map coordinate system. Estimate and move to step 210.

In step 210, the CPU 12A estimates the position of the own vehicle from the estimated position and orientation of the camera 14, and ends the process. That is, since the position of the camera 14 with respect to the vehicle 20 is known, the position of the own vehicle is estimated from the position and posture of the camera 14.

Steps 200 to 206 in the process of FIG. 6 correspond to the extraction unit of the own vehicle position estimation device and the own vehicle position estimation program, and steps 208 to 210 correspond to the estimation unit.

As described above, in the present embodiment, when extracting feature points in a region other than the region in the traveling direction from the captured image taken by the camera 14 mounted on the vehicle 20, it is based on the detection result of the steering angle sensor 16. To specify the traveling direction of the captured image. As a result, the processing load when detecting the region in the traveling direction of the vehicle 20 can be reduced as compared with the case where the region in the traveling direction is determined using the two captured images. In addition, more stable detection of the traveling direction is possible than when the region of the vehicle 20 in the traveling direction is detected by using the detection results of the acceleration sensor, the angular velocity sensor, and the angular acceleration sensor. In particular, when the vehicle 20 is moving at a low speed, it is difficult to detect the traveling direction of the vehicle 20 from the acceleration sensor, the angular velocity sensor, and the angular acceleration sensor, but the detection result of the steering angle sensor 16 is used. This enables stable detection of the traveling direction.

Further, since the region other than the region in the traveling direction is extracted when the feature points are extracted, the processing load can be reduced as compared with the extraction of the feature points from the entire captured image.

Further, since the feature points are extracted from the region other than the traveling direction in which the movement amount of the feature points becomes large, the deviation of the feature points between the captured images corresponding to the parallax when estimating the three-dimensional position of the feature points becomes large. , It is possible to generate an environmental map with high positional accuracy of the three-dimensional positions of the feature points. In particular, it is possible to improve the map accuracy when generating an environmental map in a local space such as a parking space where the traveling route of the vehicle 20 is restricted.

When the environmental map generated in this way is applied to an automatic parking system or the like, the vehicle 20 can be guided by using the high-precision environmental map, so that the vehicle 20 can be parked in a narrow place.

In the above embodiment, feature points in a region other than the region corresponding to the traveling direction of the vehicle 20 in the captured image are extracted to obtain the position and posture of the camera 14, but the present invention is not limited to this. For example, the position and orientation of the camera 14 may be estimated by extracting feature points in the entire region of the captured image and increasing the weighting of the region other than the region corresponding to the traveling direction of the vehicle 20. For example, the position of the camera 14 at which this evaluation function is minimized is set as the evaluation function by using the error between the position of the feature point at the current time estimated from the position of the feature point in the captured image at the previous time and the position of the feature point at the current time as an evaluation function. And when estimating the posture, the evaluation function is weighted to estimate the position and posture of the camera 14.

Further, although the processes of FIGS. 5 and 6 performed by the control unit 12 in the above embodiment have been described as software processes performed by executing a program by a computer, they may be processes performed by hardware. Alternatively, the processing may be a combination of both software and hardware. Further, the program to be processed by software may be stored in various storage media and distributed.

Further, in the above embodiment, the mode in which the camera 14 is provided at the rear of the vehicle 20 to photograph the rear of the vehicle has been described, but the present invention is not limited to this. For example, it may be provided in the front part of the vehicle 20 to photograph the front of the vehicle, or may be provided in the side of the vehicle 20 to photograph at least one of the front side of the vehicle and the rear side of the vehicle.

Further, in the above embodiment, the environment map generation program and the own vehicle position estimation program have been described as being stored in the ROM 12B of the control unit 12 in advance, but the present invention is not limited to this. For example, the environment map generation program and the vehicle position estimation program can be used in recording media such as a CD-ROM (Compact Disk Read Only Memory), a DVD-ROM (Digital Versatile Disk Read Only Memory), and a USB (Universal Serial Bus) memory. It may be provided in recorded form. Further, the environment map generation program and the own vehicle position estimation program may be downloaded from an external device via a network.

Further, the present invention is not limited to the above, and it goes without saying that the present invention can be variously modified and implemented within a range not deviating from the gist thereof.

10 Own vehicle position estimation device 12 Control unit 14 Camera 16 Steering angle sensor 18 Storage unit 20 Vehicle

Claims (5)

A shooting unit that captures the direction of movement of the vehicle,
An acquisition unit that acquires the detection result of the detection unit that detects the steering angle of the vehicle,
Using the detection result acquired by the acquisition unit, feature points are extracted from a region other than a predetermined range corresponding to the moving direction of the vehicle in the captured image captured by the imaging unit, or the acquisition is performed. Using the detection result acquired by the unit, feature points are extracted from the entire area of the captured image, and the weighting of other regions is larger than the weighting of the region in the predetermined range corresponding to the moving direction of the vehicle in the captured image. Extraction part and
The position and posture of the vehicle are estimated based on the correspondence between the feature points extracted by the extraction unit and the feature points of other captured images taken at different positions, and the three-dimensional feature points are estimated based on the estimation result. A generator that finds the position and generates a local environmental map,
Environmental map generator for vehicle position estimation including. The environment map generation device for estimating the position of the own vehicle according to claim 1, wherein the generation unit generates an environment map of a parking space in which the traveling route of the vehicle is restricted as a local environment map. A shooting unit that captures the direction of movement of the vehicle,
An acquisition unit that acquires the detection result of the detection unit that detects the steering angle of the vehicle,
Using the detection result acquired by the acquisition unit, feature points are extracted from a region other than a predetermined range corresponding to the moving direction of the vehicle in the captured image captured by the imaging unit, or the acquisition is performed. Using the detection result acquired by the unit, feature points are extracted from the entire area of the captured image, and the weighting of other regions is larger than the weighting of the region in the predetermined range corresponding to the moving direction of the vehicle in the captured image. Extraction part and
The position and orientation of the vehicle are estimated based on the feature points extracted by the extraction unit and the corresponding feature points on the environmental map including the three-dimensional coordinates and the feature amount generated in advance, and based on the estimation result. , The estimation unit that estimates the position of the own vehicle,
Own vehicle position estimation device including. Acquire the detection result of the detection unit that detects the steering angle of the vehicle, and use the detection result to capture the moving direction of the vehicle. A predetermined range corresponding to the moving direction of the vehicle in the captured image taken by the photographing unit. The feature points are extracted from the region other than the region of, or the feature points are extracted from the entire region of the captured image by using the detection result, and the region of a predetermined range corresponding to the moving direction of the vehicle in the captured image. Increase the weighting of other areas than the weighting of
The position and orientation of the vehicle are estimated based on the correspondence between the extracted feature points and the feature points of other captured images taken at different positions.
An environment map generation program for estimating the position of the own vehicle for causing a computer to execute a process of generating a local environment map by obtaining a three-dimensional position of a feature point based on the estimation result. Acquire the detection result of the detection unit that detects the steering angle of the vehicle, and use the detection result to capture the moving direction of the vehicle. A predetermined range corresponding to the moving direction of the vehicle in the captured image taken by the photographing unit. The feature points are extracted from the region other than the region of the above, or the feature points are extracted from the entire region of the captured image by using the detection result, and the region of a predetermined range corresponding to the moving direction of the vehicle in the captured image. Increase the weighting of other areas than the weighting of
The position and orientation of the vehicle are estimated based on the extracted feature points and the corresponding feature points on the environmental map including the three-dimensional coordinates and feature quantities generated in advance.
A vehicle position estimation program for causing a computer to execute a process of estimating the position of the vehicle based on the estimation result.

Images