JP7393987B2 - Map creation device, position estimation device, vehicle control system, map creation method, computer program, and location estimation method - Google Patents

Description

The present invention relates to a map creation device, a position estimation device, a vehicle control system, a map creation method, a computer program, and a position estimation method.

2. Description of the Related Art An automatic driving control device is known that automatically drives a vehicle using feature points in images taken by an on-vehicle camera mounted on the vehicle (for example, see Patent Document 1). The automatic driving control device described in Patent Document 1 uses feature points photographed by an on-vehicle camera to park a vehicle in a parking space without a parking frame line.

Patent No. 6564713

The device described in Patent Document 1 detects feature points from images taken during manual driving by a driver, and creates a map of parking spaces using the detected feature points. When detecting feature points, if the vehicle speed is high, the feature points necessary for map creation may not be detected, and there is a risk that automatic driving may not be realized. On the other hand, if the vehicle speed when detecting feature points is made too slow, there is a risk that it will take more time than necessary to create a map. Therefore, there was a problem in realizing automatic driving control efficiently and stably.

The present invention has been made to solve the above-mentioned problems, and an object of the present invention is to provide a technique for efficiently and stably realizing automatic driving control.

The present invention has been made to solve the above-mentioned problems, and can be realized as the following forms.

(1) According to one embodiment of the present invention, a map creation device is provided. This map creation device is mounted on a vehicle and includes an on-vehicle camera that photographs the surrounding environment of the vehicle, a detection unit that detects feature points from images taken by the on-vehicle camera, and a position and orientation of the on-vehicle camera. a calculation unit that calculates the three-dimensional position of the same feature point included in a plurality of captured images using the above, the position and orientation of the in-vehicle camera, and the position of the feature point calculated by the calculation unit; a vehicle speed control unit that controls an upper limit vehicle speed of the vehicle so that the same feature point is included in the plurality of captured images using the azimuth angle of the plurality of captured images; A map creation unit that creates a map including information on each of the three-dimensional positions using the three-dimensional positions of the different feature points.

According to this configuration, by setting the upper limit vehicle speed, the same feature point photographed at different points is included in the plurality of photographed images under minimum deceleration. As a result, based on the principle of triangulation, sufficient information on the three-dimensional positions of feature points necessary to create a map can be obtained, and a map containing the position information of multiple feature points can be stably created. . In this way, by stably creating a map that includes position information of multiple feature points under minimal deceleration, it is possible to suppress the lack of information in the map during automatic driving control, and to Automatic driving control will be realized efficiently and stably.

(2) In the map creation device of the above aspect, the vehicle speed control unit detects the same feature point in a preset number or more of the captured images using the frame rate and angle of view of the in-vehicle camera. The upper limit vehicle speed may be set so as to be equal to or lower than the vehicle speed.
According to this configuration, the frame rate and angle of view of the vehicle-mounted camera are used to keep the upper limit vehicle speed at the minimum vehicle speed necessary for creating a map. As a result, a map including position information of a plurality of feature points can be created more efficiently with minimal deceleration.

(3) In the map creation device of the above aspect, the vehicle speed control unit may control the upper limit vehicle speed only in the section of the route from the departure point to the destination where the map is created by the map creation unit. good.
According to this configuration, since the upper limit vehicle speed of the vehicle is not set in the section where no map is created, the map is stably created in the section where the map is not created, and the driving of the vehicle is not unnecessarily restricted.

(4) According to another aspect of the present invention, a position estimation device is provided. This position estimating device includes an on-vehicle camera that is mounted on a vehicle and photographs the environment around the vehicle, a map storage unit that stores a map including information on three-dimensional positions of a plurality of feature points, and the on-vehicle camera. A detection unit that detects a feature point from a photographed image taken by the camera, the position and orientation of the vehicle-mounted camera, and an azimuth angle to the position of the feature point detected by the detection unit are used to detect the same feature. a vehicle speed control unit that controls an upper limit vehicle speed of the vehicle, and the vehicle speed detected by the detection unit in a state where the upper limit vehicle speed is controlled by the vehicle speed control unit so that the point is included in the plurality of photographed images. A position estimating unit that estimates the position of the vehicle by comparing the feature points with the feature points in the map stored in the map storage unit.
According to this configuration, when a feature point is detected from an image captured by an on-vehicle camera, the vehicle speed is set to be equal to or lower than the upper limit vehicle speed. Therefore, since the same feature points can be tracked between successive captured images while the vehicle is running, the current position of the vehicle can be stably estimated. As a result, automatic driving control is stably performed in the vehicle.

(5) According to another aspect of the present invention, a vehicle control system is provided. This vehicle control system includes the position estimating device described above, a trajectory storage section that stores a past trajectory of the vehicle, and a trajectory storage section in which a route traveled by the vehicle is stored in the trajectory storage section. and a tracking control unit that controls the vehicle so that the position of the vehicle estimated by the position estimating unit follows the travel trajectory when the route is the same as that of the vehicle.
According to this configuration, it is possible to realize automatic driving of a vehicle that follows a past travel trajectory.

Note that the present invention can be realized in various aspects, and includes, for example, a map creation device, a position estimation device, a vehicle control system, a vehicle equipped with these devices, a map creation method, a position estimation method, and It can be realized in the form of a computer program for executing the apparatus or method, a server device for distributing this computer program, a non-temporary storage medium storing the computer program, etc.

1 is a schematic block diagram of a map creation system as a first embodiment of the present invention. FIG. 2 is an explanatory diagram of the amount of change in the position and posture of the vehicle-mounted camera. FIG. 2 is an explanatory diagram of the amount of change in the position and posture of the vehicle-mounted camera. FIG. 2 is an explanatory diagram of the relationship between feature points detected from a photographed image and an upper limit vehicle speed. FIG. 2 is an explanatory diagram of the relationship between feature points detected from a photographed image and an upper limit vehicle speed. FIG. 2 is an explanatory diagram of the relationship between feature points detected from a photographed image and an upper limit vehicle speed. It is an explanatory diagram about a map created by a map creation part. It is an explanatory diagram about a map created by a map creation part. It is a flowchart of a map creation method. It is a flowchart of the subflow of a calculation process. FIG. 2 is a schematic block diagram of a vehicle control system according to a second embodiment. 3 is a flowchart of a vehicle control method. It is a flowchart of the subflow of a position estimation process.

<First embodiment>
FIG. 1 is a schematic block diagram of a map creation system 100 as a first embodiment of the present invention. A map creation system (map creation device) 100 creates a map including information on a three-dimensional structure such as a garage, using images captured by an on-vehicle camera 11 mounted on a vehicle 10. As shown in FIG. 1, the map creation system 100 includes an on-vehicle camera 11 that photographs the environment around the rear of the vehicle 10, and a control device 20 mounted on the vehicle 10.

The vehicle-mounted camera 11 is a monocular camera. The angle of view of the vehicle-mounted camera 11 of this embodiment is 100° horizontally, 55° vertically, and 125° diagonally. Further, the frame rate of the vehicle-mounted camera 11 is 30.0 fps. The control device 20 performs various controls on the vehicle 10 using captured images captured by the on-vehicle camera 11.

As shown in FIG. 1, the control device 20 includes an input section 60 that receives various user operations, an output section 70 that includes a monitor that displays various images, a speaker that outputs audio, and a CPU (Central Processing Unit). ) 30, a ROM (Read Only Memory) 21, a RAM (Random Access Memory) 22, and a storage section 40 for storing various information.

The input unit 60 of this embodiment includes a keyboard and a microphone that accepts voice input. The CPU 30 functions as a detection section 31, a calculation section 32, a vehicle speed control section 33, and a map creation section 34 by loading a computer program stored in the ROM 21 into the RAM 22 and executing it.

The storage unit 40 is configured with a hard disk drive (HDD) or the like. The storage unit 40 includes a map database (map DB) 42 that stores map data used for route guidance. The map DB 42 also stores map data including three-dimensional information created by a map creation unit 34, which will be described later.

The detection unit 31 detects feature points from each of the plurality of images taken by the in-vehicle camera 11 using a FAST (Features from Accelerated Segment Test) algorithm or the like. Here, the term "feature point" refers to a point representing a feature of a photographed image (for example, a corner of a building in a photographed image). The calculation unit 32 calculates the three-dimensional position of the feature point detected by the detection unit 31 using the position and orientation of the in-vehicle camera 11. The calculation unit 32 of this embodiment uses ORB (Oriented FAST and Rotated BRIEF) or the like to associate the same feature points included in a plurality of captured images.

2 and 3 are explanatory diagrams of the amount of change in the position and posture of the vehicle-mounted camera 11. When the vehicle-mounted camera 11 is activated, it is first necessary to specify the position and orientation of the vehicle-mounted camera 11 in the world coordinate system. Therefore, first, the detection unit 31 detects the feature point image FP1 by FAST or the like, using the (n-1)th captured image IM _n-1 shown in FIG. 2 as the first image. The calculation unit 32 calculates a feature amount using OBR or the like for the detected feature point image FP1.

Next, the detection unit 31 detects a feature point image FP2 using the n-th captured image IM _n as the second image. The calculation unit 32 calculates the feature amount of the detected feature point image FP2. The calculation unit 32 uses the feature amount of the feature point image FP1 calculated from the captured image IM _n-1 and the feature amount of the feature point image FP2 calculated from the captured image IM _n to calculate each captured image IM _n-1 , The feature point images FP1 and FP2 in IM _n are associated. The calculation unit 32 calculates a fundamental matrix F from the associated feature point images FP1 and FP2 by using an 8-point algorithm and RANSAC (Random Sampling consensus).

The calculation unit 32 calculates the position and orientation of the vehicle-mounted camera 11 using the calculated basic matrix F and the internal parameters of the vehicle-mounted camera 11. Note that the unit (scale) of the position of the vehicle-mounted camera 11 is unknown from only the two captured images IM _n-1 and IM _n shown in FIG. 2 . Therefore, the calculation unit 32 of this embodiment calculates the moving distance of the vehicle-mounted camera 11 using a wheel rotation angle sensor or the like. The calculation unit 32 determines the unit of the position of the vehicle-mounted camera 11 using the moving distance of the vehicle-mounted camera 11. The calculation unit 32 uses the determined position and orientation of the vehicle-mounted camera 11 to calculate the three-dimensional position of the feature point P1 based on the principle of triangulation. Note that in this embodiment, the "position" is exemplified by coordinates representing the current location, and the "posture" is exemplified by elements (for example, rotation angles of roll, pitch, and yaw) that are different from the position (coordinate values). . Furthermore, hereinafter, a feature point image included in a captured image corresponding to a feature point will also be simply referred to as a "feature point."

FIG. 3 shows feature point images FP3 to FP8 of feature points P1 to P3 included in the m-th (≧3) captured images IM _m-1 and IM _m . The detection unit 31 detects the feature point images FP3 to FP8 of the captured images IM _m-1 and IM _m in the same way as the processing for the feature point P1 in FIG. The calculation unit 32 calculates the feature amounts of the feature point images FP3 to FP8 in the same way as the processing for the feature point P1 in FIG. The calculation unit 32 uses the calculated feature amounts of the feature point images FP3 to FP8 to associate the feature point images FP3 to FP8 in each photographed image IM _m-1 and IM _m .

The calculation unit 32 uses the feature point images FP3 and FP6 of the feature point P1 whose three-dimensional position is known among the associated feature point images FP3 to FP8 to calculate the in-vehicle camera 11 that minimizes the sum of reprojection errors. Calculate the position and orientation of. The reprojection error is the difference represented by the arrow between the provisional feature point image FP01 in which the feature point P1 is reprojected onto the captured image IM _m and the feature point image FP6. Note that in the third and subsequent photographed images IM _m , the unit of the three-dimensional position of the known feature point P1 is determined, so the unit of the position of the vehicle-mounted camera 11 is determined. Therefore, the calculation unit 32 can calculate the three-dimensional positions of the feature points P2 and P3 without using the movement distance of the in-vehicle camera 11 using a wheel rotation angle sensor or the like. The calculation unit 32 calculates the three-dimensional positions of the associated feature points P2 and P3 whose three-dimensional positions are unknown, using the position and orientation of the vehicle-mounted camera 11, based on the principle of triangulation.

The vehicle speed control unit 33 uses the position and orientation of the in-vehicle camera 11 specified by the calculation unit 32 and the azimuth from the in-vehicle camera 11 to the three-dimensional position of the feature point, and uses the same feature point in multiple captured images. The upper limit vehicle speed of the vehicle 10 is controlled so as to be included in the above. Specifically, the vehicle speed control unit 33 uses the frame rate and angle of view of the in-vehicle camera 11 to set the upper limit vehicle speed to a value below the vehicle speed at which the same feature point is detected in a preset number or more of captured images. .

4 to 6 are explanatory diagrams of the relationship between feature points detected from photographed images and the upper limit vehicle speed. FIG. 4 shows the k-th captured image IM _k that includes a feature point FP9 corresponding to the feature point P5. FIG. 5 shows the (k+1)th captured image IM _k+1 that includes the feature point FP10 corresponding to the feature point P5. When the position and orientation of the in-vehicle camera 11 change due to the movement of the vehicle 10, as shown in FIGS. 4 and 5, the same feature point P5 changes to the feature point FP9, FP9, in the corresponding photographed images _IMk , IMk ₊₁ . The position of FP10 changes. If the amount of change from the position and orientation of the vehicle-mounted camera 11 that photographed the k-th photographed image IM _k to the position and orientation of the vehicle-mounted camera 11 that photographed the (k+1)-th photographed image IM _k+1 exceeds a predetermined value. , the captured image IM _k+1 may not include the feature point FP10 corresponding to the feature point P5. Therefore, by setting the upper limit vehicle speed of the vehicle 10, the vehicle speed control unit 33 causes the plurality of captured images to include the same feature point.

FIG. 6 shows the amount of change in the position of the vehicle-mounted camera 11 and the positional relationship with the feature point P5. The vehicle speed control unit 33 calculates the upper limit vehicle speed V _max to be set using the following formula (1). Note that FIG. 6 shows, as an example, an example in which the attitude of the vehicle-mounted camera 11 does not change.

ΔＴ：撮影時間間隔（フレームレートの逆数）
α：方位角
θ：車載カメラ１１の画角

ΔT: Shooting time interval (reciprocal of frame rate)
α: Azimuth angle θ: Angle of view of the in-vehicle camera 11

The distance D to the feature point P5 in the above equation (1) is an assumed value of the lateral distance with the optical axis of the vehicle-mounted camera 11 as a reference. The vehicle speed control unit 33 of this embodiment uses 0.5 times the width of the vehicle 10 as the distance D as an assumed value. By using this assumed value, the in-vehicle camera 11 can continuously photograph the feature point P5 that exists outside the side surface of the vehicle 10.

The azimuth angle α is the angle formed by the optical axis of the vehicle-mounted camera 11 when the k-th photographed image IM _k is photographed and the feature point P5. As shown in FIG. 4, when the x-coordinate of the feature point P5 in the image coordinate system is xp, the x-coordinate of the image center OC is xc, the focal length of the in-vehicle camera 11 is f, and the pixel pitch is p, the azimuth angle α is Calculated using formula (2).

The vehicle speed control unit 33 sets the upper limit vehicle speed V _max calculated from the above equation (1), so that the same feature point P5 is included in the captured images of a preset number or more. The calculation unit 32 calculates the three-dimensional position of the same feature point included in a plurality of captured images based on the principle of triangulation.

The map creation unit 34 creates a map including the calculated three-dimensional positions and feature amounts of the plurality of feature points. Examples of maps that include three-dimensional locations include parking spaces such as garages. The feature amount of each feature point included in the map is the median value of the feature amounts in a plurality of captured images. The map creation unit 34 stores the created map in the map DB 42.

7 and 8 are explanatory diagrams of maps created by the map creation section 34. FIG. 7 shows an image IM _i taken by the in-vehicle camera 11 when the vehicle 10 is parked in a parking space of a garage. FIG. 8 shows a plan view of the state shown in FIG. 7 from above. In the state shown in FIGS. 7 and 8, the driver of the vehicle 10 manually parks the vehicle in a parking space along the route RT. At this time, the vehicle speed control unit 33 sets the upper limit vehicle speed V _max so that the feature point images FPn corresponding to the plurality of feature points Pn detected from the photographed image IM _i are included in the plurality of photographed images. Therefore, under the control of the vehicle speed control unit 33, the vehicle speed of the vehicle 10 during map creation becomes equal to or lower than the upper limit vehicle speed _Vmax , regardless of the amount by which the driver depresses the accelerator.

FIG. 9 is a flowchart of the map creation method. As shown in FIG. 9, in the map creation flow, first, the in-vehicle camera 11 mounted on the vehicle 10 starts a photographing process of photographing the surrounding environment of the vehicle 10 (step S1). Next, the detection unit 31 performs a detection step of detecting feature points from the image taken by the vehicle-mounted camera 11 using the FAST algorithm or the like (step S2).

The calculation unit 32 uses the feature points detected by the detection unit 31 to perform a calculation step of calculating the three-dimensional positions of the feature points included in the plurality of captured images (step S3). FIG. 10 is a flowchart of a subflow of the calculation process. As shown in FIG. 10, in the calculation step, the calculation unit 32 first determines whether the captured image in which the feature point has been detected by the detection unit 31 is the third or subsequent captured image captured by the in-vehicle camera 11. (Step S31). If the calculation unit 32 determines that the captured image is not the third image (step S31: NO), the calculation unit 32 uses an 8-point algorithm or the like to calculate the three-dimensional position of the feature point P1 in FIG. , the position and orientation of the vehicle-mounted camera 11 are calculated (step S32). Specifically, the calculation unit 32 calculates the feature amount of the feature point detected by the detection unit 31, and uses the calculated feature amount to associate the same feature points included in the first and second captured images. . The calculation unit 32 calculates the fundamental matrix F from the same matched feature points by using the 8-point algorithm and RANSAC. The calculation unit 32 calculates the position and orientation of the vehicle-mounted camera 11 by using the basic matrix F and the movement distance of the vehicle-mounted camera 11.

If it is determined in the process of step S31 that the photographed image is the third or subsequent image (step S31: YES), the calculation unit 32 calculates the three-dimensional positions of the feature points P2 and P3 in FIG. Then, the position and orientation of the vehicle-mounted camera 11 that minimizes the sum of reprojection errors are calculated (step S33). Specifically, the calculation unit 32 calculates the feature amount of the feature point detected by the detection unit 31, and associates the same feature point included in a plurality of captured images. The calculation unit 32 calculates the position and orientation of the vehicle-mounted camera 11 that minimizes the sum of reprojection errors, using feature points whose three-dimensional positions are known among the matched feature points. Thereafter, the calculation unit 32 uses the calculated position and orientation of the in-vehicle camera 11 to calculate the three-dimensional position of each associated feature point according to the principle of triangulation shown in FIG. 6 (step S34), and calculates Finish the process.

After the process of step S3 in FIG. 9 is performed, the vehicle speed control unit 33 performs a vehicle speed control step to control the upper limit vehicle speed V _max of the vehicle 10 (step S4). As shown in FIG. 6, the vehicle speed control unit 33 calculates the upper limit vehicle speed V _max by solving the above equation (1) using the azimuth angle α from the in-vehicle camera 11 to the feature point P5. The vehicle speed control unit 33 controls the speed of the vehicle to be equal to or lower than the upper limit vehicle speed _Vmax . Next, the map creation unit 34 uses the three-dimensional positions of the plurality of different feature points calculated by the calculation unit 32 to create a map including three-dimensional position information as shown in FIG. (Step S5).

The map creation unit 34 determines whether the creation of the map has been completed (step S6). If the map creation unit 34 determines that the creation of the map has not been completed yet (step S6: NO), the map creation unit 34 repeats the processing from step S1 onwards. When the map creation unit 34 determines that the map creation has been completed (step S6: YES), the map creation flow ends.

As explained above, in the map creation system 100 of the first embodiment, the calculation unit 32 uses the position and orientation of the in-vehicle camera 11 to calculate the three-dimensional position of the same feature point included in a plurality of captured images. do. The vehicle speed control unit 33 controls the upper limit vehicle speed V _max of the vehicle 10 using the azimuth angle α from the in-vehicle camera 11 to the three-dimensional position of the feature point so that the same feature point is included in a plurality of captured images. . The map creation unit 34 uses the calculated three-dimensional positions of the plurality of feature points to create a map including three-dimensional position information. Therefore, when the map creation system 100 of the first embodiment creates a map that includes information on the three-dimensional positions of a plurality of feature points, the same feature points are included in a plurality of captured images captured by the in-vehicle camera 11. The upper limit vehicle speed V _max is set so that That is, by setting the upper limit vehicle speed _Vmax , the same feature point photographed at different points is included in a plurality of photographed images under minimum deceleration. As a result, based on the principle of triangulation, sufficient information on the three-dimensional positions of feature points necessary to create a map can be obtained, and a map containing the position information of multiple feature points can be stably created. . In this way, by stably creating a map that includes position information of multiple feature points under minimal deceleration, it is possible to suppress the lack of information in the map during automatic driving control, and to 10 automatic driving controls are realized efficiently and stably.

Further, the vehicle speed control unit 33 of the first embodiment uses the frame rate and angle of view of the on-vehicle camera 11 to set the upper limit vehicle speed V below the vehicle speed at which the same feature point is detected in a preset number or more of captured images. Set _max . That is, the map creation system 100 of the first embodiment uses the frame rate and angle of view of the vehicle-mounted camera 11 to determine the upper limit vehicle speed V _max as the minimum vehicle speed necessary to create a map. As a result, a map including position information of a plurality of feature points can be created more efficiently with minimal deceleration.

<Modified example of the first embodiment>
In the first embodiment described above, as shown in FIGS. 7 and 8, map creation when the vehicle 10 is parked in a parking space has been described. Of these, a map for a predetermined section may be created. For example, when the vehicle 10 moves from a garage to a predetermined destination for which a map containing information on the three-dimensional positions of feature points has not been created, a map is created only for the section until the vehicle 10 leaves the garage. Good too. In this case, the vehicle speed control section 33 sets the upper limit vehicle speed V _max of the vehicle only in the section of the route from the departure point to the destination until leaving the garage, which is the section where the map is created by the map creation section 34. Set. When the vehicle speed control section 33 passes the created section, the vehicle speed control section 33 cancels the set upper limit vehicle speed V _max of the vehicle 10 .

As explained above, the vehicle speed control unit 33 may set the upper limit vehicle speed V _max of the vehicle only in the creation section where the map is created by the map creation unit 34 . In the map creation system 100 of this modified example, the upper limit vehicle speed V _max of the vehicle is not set in sections where no map is to be created. is not unduly restricted.

<Second embodiment>
FIG. 11 is a schematic block diagram of a vehicle control system 200 according to the second embodiment. The vehicle control system 200 is a system that automatically drives the vehicle 10 using a map that includes information on three-dimensional positions of a plurality of feature points. In the vehicle control system 200 of the second embodiment, compared to the map creation system 100 of the first embodiment, the CPU 30a functions as the destination setting section 35, the position estimation section 36, and the tracking control section 37, and the storage section 40a differs in that it includes a trajectory database (trajectory DB) 43 and that it does not function as a calculation unit 32. Therefore, in the second embodiment, configurations and the like that are different from the first embodiment will be described, and descriptions of the same configurations and the like as the first embodiment will be omitted.

The destination setting unit 35 receives radio waves transmitted from artificial satellites that constitute a GPS (Global Positioning System). The destination setting unit 35 uses the current position of the vehicle 10 specified by the position estimation unit 36, the destination accepted by the input unit 60, and the map data stored in the map DB 42 to determine the destination from the current position. Set the route to. The destination setting section 35 displays the plurality of set route candidates on the monitor of the output section 70. When the input unit 60 receives an operation to select one route from a plurality of route candidates, the destination setting unit 35 guides the route to the destination using the image displayed on the monitor and the audio output from the speaker. do.

The trajectory DB 43 shown in FIG. 11 stores the travel trajectory that the vehicle 10 has traveled in the past in association with the map stored in the map DB. The map DB 42 stores a map created by, for example, the map creation system 100 of the first embodiment, and includes information on the three-dimensional positions and feature amounts of a plurality of feature points.

The position estimation unit 36 estimates the current position of the vehicle 10 using the feature points detected by the detection unit 31 while the upper limit vehicle speed V _max of the vehicle 10 is controlled by the vehicle speed control unit 33 . Specifically, the position estimation unit 36 associates the same feature points included in a plurality of captured images and detected by the detection unit 31. When the number of pairs of associated feature points exceeds a predetermined value (for example, 30 pairs), the position estimation unit 36 calculates the sum of reprojection errors performed by the calculation unit 32 of the first embodiment. The position and orientation of the in-vehicle camera 11 that minimizes are calculated. Note that if a predetermined value or more of feature points cannot be associated from multiple captured images, and if a feature point is detected from the first captured image, the position estimating unit 36 determines whether the feature points are detected by the detecting unit 31 or not. The feature points stored in the map are associated with feature points in the map stored in the map DB 42. The position estimating unit 36 calculates the position and orientation of the vehicle-mounted camera 11 that minimizes the sum of reprojection errors for the associated feature points. The position estimation unit 36 estimates the current position of the vehicle 10 using the calculated position and orientation of the vehicle-mounted camera 11.

The tracking control unit 37 controls the steering angle of the vehicle 10 using the current position of the vehicle 10 estimated by the position estimation unit 36 and the past travel trajectory of the vehicle 10 stored in the trajectory DB 43. Specifically, the tracking control unit 37 controls the steering angle so that the estimated current position of the vehicle 10 follows the past travel trajectory. For example, the tracking control unit 37 automatically controls the vehicle 10 to follow the same trajectory as the past parking trajectory for a parking space for which a map including three-dimensional position information has already been completed. do.

FIG. 12 is a flowchart of the vehicle control method. In the vehicle control flow shown in FIG. 12, first, the destination setting unit 35 sets the destination of the vehicle 10 according to the operation received by the input unit 60 (step S11). In the second embodiment, a case will be described in which the vehicle is automatically parked in a garage in which a map including a plurality of feature points whose three-dimensional positions are known in advance is registered in the map DB 42. Once the parking position as the destination is set, steps S12 to S14 are performed. Note that each process in steps S12 to S14 is the same as each process in steps S1, S2, and S4 of the first embodiment (FIG. 9), so the processes after step S15 in FIG. 12 will be described.

After the process of step S14 in FIG. 12 is performed, the position estimating unit 36 performs a position estimating step of estimating the current position of the vehicle 10 (step S15). FIG. 13 is a flowchart of a subflow of the position estimation process. As shown in FIG. 13, the position estimation unit 36 first determines whether the photographed image in which the detection unit 31 detected the feature point is the first image (step S151). If the position estimation unit 36 determines that the captured image in which the feature point has been detected is not the first captured image (step S151: NO), the position estimation unit 36 determines whether the same feature point is detected between the captured image and the previous captured image. A correspondence is established (step S152). The position estimation unit 36 determines whether the matched pair of feature points is a combination of a predetermined value or more (step S153). If the combination is equal to or greater than the predetermined value (step S153: YES), the position estimation unit 36 uses the associated feature points to perform the process of step S155, which will be described later.

In the process of step S153, if the matched pair of feature points is a combination less than the predetermined value (step S153: NO), the process of step S154 described later is performed. In the process of step S151, if the position estimation unit 36 determines that the captured image in which the feature point was detected by the detection unit is the first one (step S151: YES), the position estimation unit 36 stores the detected feature point and the map DB 42. The stored feature points in the map are associated with each other (step S154). The position estimation unit 36 calculates the position and orientation of the in-vehicle camera 11 that minimizes the sum of reprojection errors for the feature points associated in the process of step S153 or step S154 (step S155), and performs position estimation. Finish the process. Note that the position estimating unit 36 can calculate the position and orientation of the in-vehicle camera 11 in a shorter time by using the feature points associated in step S152 than by using the feature points associated in step S154. When the position and orientation of the vehicle-mounted camera 11 are calculated in a short time, the tracking accuracy of the tracking control unit 37 is improved.

When the process of step S15 in FIG. 12 is performed, the tracking control unit 37 performs a tracking process of causing the vehicle 10 to follow the past travel trajectory from the position estimated by the position estimation unit 36 to the destination (step S16). . The follow-up control unit 37 acquires the travel trajectory when the vehicle is parked in the destination parking space from the trajectory DB 43. The tracking control unit 37 controls the steering angle of the vehicle 10 so that the vehicle 10 follows the acquired travel trajectory from the current position. Note that at this time, the vehicle speed control unit 33 controls the speed of the vehicle 10 to be equal to or lower than the upper limit vehicle speed V _max .

The follow-up control unit 37 determines whether the follow-up control for moving the vehicle 10 to the set destination has been completed (step S17). If it is determined that the follow-up control has not ended (step S17: NO), the processes from step S12 onwards are repeated. When the follow-up control unit 37 determines that the follow-up control has ended (step S17: YES), it ends the vehicle control flow.

As explained above, the position estimating unit 36 of the second embodiment transfers the feature points detected by the detecting unit 31 to the map DB 42 while the upper limit vehicle speed V _max of the vehicle 10 is controlled by the vehicle speed control unit 33. The current position of the vehicle 10 is estimated by comparing it with the feature points in the map stored in the map (step S154 in FIG. 13). At this time, the vehicle speed is set below the upper limit vehicle speed _Vmax . Therefore, since the same feature points can be tracked between successive captured images while the vehicle is running, the current position of the vehicle 10 can be stably estimated. Thereby, automatic driving control is stably performed in the vehicle 10.

Furthermore, in the vehicle control system 200 of the second embodiment, the trajectory DB 43 stores the travel trajectory in which the vehicle 10 traveled in the past. The tracking control unit 37 controls the vehicle 10 so that the current position of the vehicle 10 estimated by the position estimating unit 36 follows the past travel trajectory stored in the trajectory DB 43. Therefore, the vehicle control system 200 of the second embodiment can realize automatic driving of the vehicle 10 that follows the past travel trajectory.

<Other variations>
In the above embodiments, a part of the configuration that is realized by hardware may be replaced with software, or conversely, a part of the configuration that is realized by software may be replaced by hardware. good. The present invention is not limited to the above-described embodiments, and can be implemented in various forms without departing from the spirit thereof. For example, the following modifications are also possible.

The configurations of the map creation system 100 of the first embodiment and the vehicle control system 200 of the second embodiment are merely examples, and can be modified in various ways. For example, the map creation system 100 only needs to include the in-vehicle camera 11 and function as the detection unit 31, the calculation unit 32, the vehicle speed control unit 33, and the map creation unit 34, and does not include the input unit 60 and the output unit 70. It's okay. The vehicle control system 200 only needs to include the in-vehicle camera 11 and function as the detection section 31, the calculation section 32, the vehicle speed control section 33, and the position estimation section 36, and does not include the input section 60 and the output section 70. It does not have to function as the destination setting section 35.

The vehicle control system 200 may function as a position estimating device that estimates the current position of the vehicle 10 without functioning as the tracking control unit 37. The vehicle control flow (FIG. 12) executed by the vehicle control system 200 may not include the destination setting step (step S11) and the following step (step S16).

The frame rate and angle of view of the vehicle-mounted camera 11 in the first embodiment are merely examples, and may be different numerical values. Moreover, the position of the vehicle-mounted camera 11 mounted on the vehicle 10 does not have to be at the rear of the vehicle 10, but may be at the side or the front. Further, a plurality of on-vehicle cameras 11 may be mounted on the vehicle 10. The frame rates and angles of view of the plurality of in-vehicle cameras 11 may be different. Depending on the frame rate and angle of view of the vehicle-mounted camera 11, the number of captured images to be captured in order to identify feature points may be set.

The method by which the detection unit 31 detects feature points from a photographed image is not limited to the FAST algorithm, 8-point algorithm, and RANSAC of the first embodiment, and any known technique can be applied. The method by which the calculation unit 32 associates the same feature points included in a plurality of photographed images is not limited to the ORB of the first embodiment, and any known technique can be applied.

The method of setting the upper limit vehicle speed V _max of the vehicle speed control section 33 can be modified in various ways. The upper limit vehicle speed V _max may be set by a known method other than that shown in equation (1) above. Further, regarding the distance D in the above formula (1), in the first embodiment, 0.5 times the width of the vehicle 10 is used as an assumed value, but it may be larger than 0.5 times, or another value may be used. A value may be set. In the first embodiment described above, as shown in FIG. Vehicle speed V _max can be calculated.

In the first embodiment and the second embodiment, one control device 20, 20a functions as the detection section 31, the calculation section 32, the vehicle speed control section 33, etc., but these functions are performed by multiple control devices. It may be performed in a distributed manner. In this case, various data may be exchanged by wireless communication via a communication unit included in the control device of the modification.

The input section 60 and output section 70 of the first embodiment and the second embodiment described above are merely examples, and can be modified in various ways. For example, the input unit 60 may be a touch panel displayed on the output unit 70, or may be a push button arranged around the monitor 70. The output unit 70 may output images and audio to a portable terminal via the communication unit.

Although the present aspect has been described above based on the embodiments and modified examples, the embodiments of the above-described aspect are for facilitating understanding of the present aspect, and do not limit the present aspect. This aspect may be modified and improved without departing from the spirit and scope of the claims, and this aspect includes equivalents thereof. Furthermore, if the technical feature is not described as essential in this specification, it can be deleted as appropriate.

10...Vehicle 11...Vehicle camera 20, 20a...Control device 21...ROM
22...RAM
30, 30a...CPU
31...Detection section 32...Calculation section 33...Vehicle speed control section 34...Map creation section 35...Destination setting section 36...Position estimation section 37...Following control section 40, 40a...Storage section 42...Map DB
43...Trajectory DB
60...Input section 70...Output section 100...Map creation system (map creation device)
200...Vehicle control system (position estimation device)
D...Distance FP1 to FP10, FPn...Feature point image (feature point)
FP01... Temporary feature point image IM _i , IM _k , IM _k+1 , IM _m , IM _n-1 , IM _n ... Captured image OC... Image center P1 to P3, P5, Pn... Feature point RT... Route V _max ... Upper limit vehicle speed θ…Angle of view of the onboard camera α…Azimuth angle ΔT…Shooting time interval

Claims (8)

A map creation device,
an on-vehicle camera that is mounted on a vehicle and photographs the environment around the vehicle;
a detection unit that detects feature points from an image taken by the in-vehicle camera;
a calculation unit that calculates a three-dimensional position of the same feature point included in the plurality of captured images using the position and orientation of the in-vehicle camera;
Using the position and orientation of the vehicle-mounted camera and the azimuth to the position of the feature point calculated by the calculation unit, the vehicle is configured to include the same feature point in the plurality of captured images. a vehicle speed control section that controls an upper limit vehicle speed;
a map creation unit that creates a map including information on each of the three-dimensional positions using the three-dimensional positions of the plurality of different feature points calculated from the plurality of captured images;
A map creation device comprising: The map creation device according to claim 1,
The vehicle speed controller uses the frame rate and angle of view of the vehicle-mounted camera to adjust the upper limit vehicle speed so that the vehicle speed is equal to or lower than the vehicle speed at which the same feature point is detected in a preset number or more of the captured images. A map creation device that sets. The map creation device according to claim 1 or claim 2,
The vehicle speed control unit is a map creation device that controls the upper limit vehicle speed only in a section of the route from a departure point to a destination where the map is created by the map creation unit. A position estimation device,
an on-vehicle camera that is mounted on a vehicle and photographs the environment around the vehicle;
a map storage unit storing a map including information on three-dimensional positions of a plurality of feature points;
a detection unit that detects feature points from an image taken by the in-vehicle camera;
Using the position and orientation of the vehicle-mounted camera and the azimuth to the position of the feature point detected by the detection unit, the vehicle is configured to include the same feature point in the plurality of captured images. a vehicle speed control section that controls an upper limit vehicle speed;
By comparing the feature points detected by the detection unit with the feature points in the map stored in the map storage unit while the upper limit vehicle speed is controlled by the vehicle speed control unit, a position estimation unit that estimates the position of the vehicle;
A position estimation device comprising: A vehicle control system,
The position estimation device according to claim 4,
a trajectory storage unit that stores a past travel trajectory of the vehicle;
When the route traveled by the vehicle is the same as the route corresponding to the travel trajectory stored in the trajectory storage unit, the position of the vehicle estimated by the position estimation unit is caused to follow the travel trajectory. a follow-up control section that controls the vehicle in such a manner;
A vehicle control system equipped with A map creation method, the information processing device comprising:
a photographing step of photographing the environment around the vehicle with an on-vehicle camera mounted on the vehicle;
a detection step of detecting feature points from an image taken by the in-vehicle camera;
a calculation step of calculating the three-dimensional position of the same feature point included in the plurality of captured images using the position and orientation of the in-vehicle camera with respect to the vehicle;
Using the position and orientation of the vehicle-mounted camera and the calculated azimuth to the position of the feature point, the upper limit vehicle speed of the vehicle is controlled so that the same feature point is included in the plurality of captured images. A vehicle speed control process,
a map creation step of creating a map including information on each of the three-dimensional positions using the three-dimensional positions of the plurality of different feature points calculated from the plurality of captured images;
A map creation method comprising: A computer program,
a photographing function for photographing the environment around the vehicle using an on-vehicle camera mounted on the vehicle;
a detection function that detects feature points from images taken by the in-vehicle camera;
a calculation function that calculates a three-dimensional position of the same feature point included in a plurality of captured images using the position and orientation of the in-vehicle camera with respect to the vehicle;
Using the position and orientation of the vehicle-mounted camera and the calculated azimuth to the position of the feature point, the upper limit vehicle speed of the vehicle is controlled so that the same feature point is included in the plurality of captured images. vehicle speed control function,
a map creation function that creates a map including information on each of the three-dimensional positions using the three-dimensional positions of the plurality of different feature points calculated from the plurality of captured images;
A computer program that allows an information processing device to realize the following. A position estimation method, the information processing device comprising:
a photographing step of photographing the environment around the vehicle with an on-vehicle camera mounted on the vehicle;
a detection step of detecting feature points from an image taken by the in-vehicle camera;
Controlling the upper limit vehicle speed of the vehicle so that the same feature point is included in the plurality of captured images using the position and orientation of the in-vehicle camera and the azimuth to the detected position of the feature point. A vehicle speed control process,
a position estimation step of estimating the position of the vehicle by comparing the detected feature point with the feature point in a map containing three-dimensional position information of a plurality of feature points stored in a map storage unit; ,
A position estimation method comprising:

Images