JP2021149396A - Map generation apparatus, position estimation apparatus, vehicle control system, map generation method, computer program, and position estimation method - Google Patents

Abstract

To efficiently and stably implement automatic drive control of a vehicle.SOLUTION: A map generation apparatus includes: an in-vehicle camera which is mounted on a vehicle to image an environment surrounding the vehicle; a detection unit which detects feature points from images captured by the in-vehicle camera; a calculation unit which calculates a three-dimensional position of the same feature point included in a plurality of captured images by using position and attitude of the in-vehicle camera; a vehicle speed control unit which controls a maximum speed of the vehicle so that the same feature point may be included in a plurality of captured images, by using the position and attitude of the in-vehicle camera and an azimuth to the position of the feature point calculated by the calculation unit; and a map generation unit which generates a map including information on three-dimensional positions by using the three-dimensional positions of different feature points calculated from each of the captured images.SELECTED DRAWING: Figure 1

Description

The present invention relates to a cartographic device, a position estimation device, a vehicle control system, a cartographic method, a computer program, and a position estimation method.

An automatic driving control device that automatically drives a vehicle by using feature points in an image captured by an in-vehicle camera mounted on the vehicle is known (see, for example, Patent Document 1). The automatic driving control device described in Patent Document 1 parks a vehicle in a parking space without a parking border by using feature points photographed by an in-vehicle camera.

Japanese Patent No. 6564713

The device described in Patent Document 1 detects feature points in a captured image taken during manual driving by a driver, and creates a map of a parking space using the detected feature points. If the vehicle speed is high when detecting feature points, the feature points required for map creation cannot be detected, and automatic driving may not be realized. On the other hand, if the vehicle speed at the time of detecting the feature point is too slow, it may take longer than necessary to create the map. Therefore, there is a problem that it is desired to realize automatic operation control efficiently and stably.

The present invention has been made to solve the above-mentioned problems, and an object of the present invention is to provide a technique for efficiently and stably realizing automatic driving control.

The present invention has been made to solve the above-mentioned problems, and can be realized as the following forms.

(1) According to one embodiment of the present invention, a cartographic device is provided. This map-creating device is mounted on a vehicle and has an in-vehicle camera that captures the environment around the vehicle, a detection unit that detects feature points from an image captured by the in-vehicle camera, and a position and orientation of the in-vehicle camera. To the calculation unit that calculates the three-dimensional position of the same feature point included in the plurality of captured images, the position and orientation of the in-vehicle camera, and the position of the feature point calculated by the calculation unit. A vehicle speed control unit that controls the upper limit vehicle speed of the vehicle so that the same feature point is included in the plurality of captured images by using the azimuth angle of the vehicle, and a plurality of images calculated from the plurality of captured images. A map creation unit that creates a map including information on each of the three-dimensional positions by using three-dimensional positions of the different feature points is provided.

According to this configuration, by setting the upper limit vehicle speed, the same feature points taken at different points are included in a plurality of captured images under the minimum deceleration. As a result, based on the principle of triangulation, sufficient information on the three-dimensional positions of the feature points necessary for creating a map can be obtained, and a map containing the position information of a plurality of feature points can be stably created. .. In this way, by stably creating a map containing the position information of a plurality of feature points under the minimum deceleration, it is possible to suppress the loss of information in the map in the automatic driving control, and the vehicle. Automatic operation control is realized efficiently and stably.

(2) In the cartography apparatus of the above aspect, the vehicle speed control unit detects the same feature points in the captured images of a preset number or more by using the frame rate and the angle of view of the vehicle-mounted camera. The upper limit vehicle speed may be set so as to be equal to or lower than the vehicle speed.
According to this configuration, the upper limit vehicle speed is kept at the minimum vehicle speed necessary for creating a map by using the frame rate and the angle of view of the vehicle-mounted camera. As a result, a map containing position information of a plurality of feature points is created more efficiently with a minimum deceleration.

(3) In the map creation device of the above aspect, the vehicle speed control unit may control the upper limit vehicle speed only in the map creation section of the route from the departure point to the destination. good.
According to this configuration, since the upper limit vehicle speed of the vehicle is not set in the section where the map is not created, the map is stably created in the created section, and the driving of the vehicle is not restricted more than necessary.

(4) According to another aspect of the present invention, a position estimation device is provided. This position estimation device includes an in-vehicle camera that is mounted on the vehicle and captures the environment around the vehicle, a map storage unit that stores a map including information on three-dimensional positions of a plurality of feature points, and the in-vehicle camera. The same feature using the detection unit that detects the feature point from the captured image captured by the camera, the position and orientation of the in-vehicle camera, and the azimuth angle to the position of the feature point detected by the detection unit. The vehicle speed control unit that controls the upper limit vehicle speed of the vehicle and the detection unit that detects the upper limit vehicle speed while the upper limit vehicle speed is controlled by the vehicle speed control unit so that the points are included in the plurality of captured images. It includes a position estimation unit that estimates the position of the vehicle by collating the feature points with the feature points in the map stored in the map storage unit.
According to this configuration, the vehicle speed is set to be equal to or lower than the upper limit vehicle speed when the feature point is detected from the image taken by the vehicle-mounted camera. Therefore, the same feature point can be tracked between consecutive captured images during traveling, so that the current position of the vehicle can be estimated stably. As a result, automatic driving control is stably performed in the vehicle.

(5) According to another aspect of the present invention, a vehicle control system is provided. This vehicle control system corresponds to the position estimation device described above, a locus storage unit that stores the past travel locus of the vehicle, and the travel locus in which the route on which the vehicle travels is stored in the locus storage unit. The vehicle is provided with a tracking control unit that controls the vehicle so that the position of the vehicle estimated by the position estimation unit follows the traveling locus when the route is the same as the route to be performed.
According to this configuration, it is possible to realize automatic driving of the vehicle that follows the past traveling locus.

The present invention can be realized in various aspects, for example, a map-making device, a position estimation device, a vehicle control system, a vehicle provided with these devices, a map-making method, a position estimation method, and the like. It can be realized in the form of a computer program for executing the device or method, a server device for distributing the computer program, a non-temporary storage medium for storing the computer program, or the like.

It is a schematic block diagram of the cartography system as the 1st Embodiment of this invention. It is explanatory drawing about the change amount of the position and posture of an in-vehicle camera. It is explanatory drawing about the change amount of the position and posture of an in-vehicle camera. It is explanatory drawing about the relationship between the feature point detected from a photographed image, and the upper limit vehicle speed. It is explanatory drawing about the relationship between the feature point detected from a photographed image, and the upper limit vehicle speed. It is explanatory drawing about the relationship between the feature point detected from a photographed image, and the upper limit vehicle speed. It is explanatory drawing about the map created by the map making part. It is explanatory drawing about the map created by the map making part. It is a flowchart of a map making method. It is a flowchart of the subflow of the calculation process. It is a schematic block diagram of the vehicle control system of the 2nd Embodiment. It is a flowchart of a vehicle control method. It is a flowchart of the subflow of the position estimation process.

<First Embodiment>
FIG. 1 is a schematic block diagram of a cartography system 100 as a first embodiment of the present invention. The map creation system (map creation device) 100 creates a map including information on a three-dimensional structure such as a garage by using images taken by an in-vehicle camera 11 mounted on the vehicle 10. As shown in FIG. 1, the cartography system 100 includes an in-vehicle camera 11 that captures the environment around the rear of the vehicle 10 and a control device 20 mounted on the vehicle 10.

The in-vehicle camera 11 is a monocular camera. The angle of view of the vehicle-mounted camera 11 of the present embodiment is horizontal 100 °, vertical 55 °, and diagonal 125 °. The frame rate of the vehicle-mounted camera 11 is 30.0 fps. The control device 20 performs various controls on the vehicle 10 by using the captured image captured by the vehicle-mounted camera 11.

As shown in FIG. 1, the control device 20 includes an input unit 60 that receives various operations of the user, an output unit 70 that includes a monitor that displays various images and a speaker that outputs audio, and a CPU (Central Processing Unit). ) 30, a ROM (Read Only Memory) 21, a RAM (Random Access Memory) 22, and a storage unit 40 for storing various information.

The input unit 60 of the present embodiment includes a keyboard and a microphone that receives voice input. The CPU 30 functions as a detection unit 31, a calculation unit 32, a vehicle speed control unit 33, and a map creation unit 34 by expanding and executing a computer program stored in the ROM 21 in the RAM 22.

The storage unit 40 is composed of a hard disk drive (HDD: Hard Disk Drive) or the like. The storage unit 40 includes a map database (map DB) 42 that stores map data used for route guidance. The map DB 42 also stores map data including three-dimensional information created by the map creation unit 34, which will be described later.

The detection unit 31 detects feature points from each of the plurality of captured images captured by the vehicle-mounted camera 11 by using a FAST (Features from Accelerated Segment Test) algorithm or the like. Here, the "feature point" means a point representing the feature of the photographed image (for example, a corner of a building in the photographed image). The calculation unit 32 calculates the three-dimensional position of the feature point detected by the detection unit 31 using the position and orientation of the vehicle-mounted camera 11. The calculation unit 32 of the present embodiment associates the same feature points included in a plurality of captured images by using an ORB (Oriented FAST and Rotated BRIEF) or the like.

2 and 3 are explanatory views of the amount of change in the position and posture of the vehicle-mounted camera 11. When the vehicle-mounted camera 11 is activated, it is first necessary to specify the position and orientation of the vehicle-mounted camera 11 in the world coordinate system. Therefore, first, the detection unit 31 detects the feature point image FP1 by FAST or the like, using the _{(n-1) th captured image IM n-1 shown in FIG. 2 as the first image.} The calculation unit 32 calculates the feature amount of the detected feature point image FP1 by OBR or the like.

Next, the detection unit 31 detects the feature point image FP2 with the _{nth captured image IM n as the second image.} The calculation unit 32 calculates the feature amount of the detected feature point image FP2. The calculation unit 32 uses the feature amount of the feature point image FP1 calculated from _{the captured image IM n-1} and the feature amount of the feature point image FP2 calculated from _{the captured image IM n to use each captured image IM n-1} , The feature point images FP1 and FP2 in IM _{n are associated with each other.} The calculation unit 32 calculates the basic matrix F from the associated feature point images FP1 and FP2 by using an 8-point algorithm and RANSAC (Random Sampling consensus).

The calculation unit 32 calculates the position and orientation of the vehicle-mounted camera 11 by using the calculated basic matrix F and the internal parameters of the vehicle-mounted camera 11. It should be noted that the unit (scale) of the position of the vehicle-mounted camera 11 is unknown only with the _{two captured images IM n-1} and IM _{n shown in FIG.} Therefore, the calculation unit 32 of the present embodiment calculates the moving distance of the vehicle-mounted camera 11 by using the wheel rotation angle sensor or the like. The calculation unit 32 determines the unit of the position of the vehicle-mounted camera 11 by using the moving distance of the vehicle-mounted camera 11. The calculation unit 32 calculates the three-dimensional position of the feature point P1 by the principle of triangulation using the determined position and orientation of the vehicle-mounted camera 11. In the present embodiment, the "position" is an example of the coordinates representing the current position, and the "posture" is an example of an element having a direction different from the position (coordinate value) (for example, the rotation angle of the roll pitch yaw). .. Further, hereinafter, the feature point image included in the captured image corresponding to the feature point is also simply referred to as a “feature point”.

FIG. 3 shows the feature point images FP3 to FP8 of the feature points P1 to P3 included _{in the captured images IM m-1} and IM _m taken at the m (≧ 3) th position. _{The detection unit 31 detects the captured images IM m-1} and the feature point images FP3 to FP8 of the _{IM m in} the same manner as the processing for the feature point P1 in FIG. The calculation unit 32 calculates the feature amounts of the feature point images FP3 to FP8 in the same manner as the processing for the feature point P1 in FIG. The calculation unit 32 associates the feature point images FP3 to FP8 in each of the captured images IM _m-1 and IM _{m by using the calculated feature quantities of the feature point images FP3 to FP8.}

The calculation unit 32 uses the feature point images FP3 and FP6 of the feature point P1 whose three-dimensional positions are known among the associated feature point images FP3 to FP8, and the vehicle-mounted camera 11 that minimizes the sum of the reprojection errors. Calculate the position and posture of. The reprojection error is the difference represented by the arrow between the temporary feature point image FP01 in which the feature point P1 is _{reprojected on the captured image IM m and the feature point image FP6.} Since the unit of the three-dimensional position of the known feature point P1 is fixed in the captured image IM _{m taken after the third, the unit of the position of the vehicle-mounted camera 11 is fixed.} Therefore, the calculation unit 32 can calculate the three-dimensional positions of the feature points P2 and P3 without using the moving distance of the vehicle-mounted camera 11 using the wheel rotation angle sensor or the like. The calculation unit 32 calculates the three-dimensional position of the feature points P2 and P3 whose associated three-dimensional position is unknown by using the position and orientation of the vehicle-mounted camera 11 according to the principle of triangulation.

The vehicle speed control unit 33 uses the position and orientation of the vehicle-mounted camera 11 specified by the calculation unit 32 and the azimuth angle from the vehicle-mounted camera 11 to the three-dimensional position of the feature points, and the vehicle speed control unit 33 uses a plurality of captured images having the same feature points. The upper limit vehicle speed of the vehicle 10 is controlled so as to be included in. Specifically, the vehicle speed control unit 33 uses the frame rate and angle of view of the vehicle-mounted camera 11 to set the upper limit vehicle speed to be equal to or lower than the vehicle speed at which the same feature points are detected in the number of captured images or more set in advance. ..

4 to 6 are explanatory views on the relationship between the feature points detected from the captured image and the upper limit vehicle speed. FIG. 4 shows the k-th captured image IM _k including the feature point FP9 corresponding to the feature point P5. FIG. 5 shows the (k + 1) th captured image IM _{k + 1} including the feature point FP10 corresponding to the feature point P5. When the position and orientation of the vehicle-mounted camera 11 change due to the movement of the vehicle 10, as shown in FIGS. 4 and 5, the feature points FP9 in _{the captured images IM k} and IM _{k + 1 corresponding to the same feature point P5,} The position of FP10 changes. When the amount of change from the position and orientation of the vehicle-mounted camera 11 that captured the k-th captured image IM _k to the position and orientation of the vehicle-mounted camera 11 that captured the (k + 1) th captured image IM _{k + 1} exceeds a predetermined value. , The captured image IM _{k + 1} may not include the feature point FP10 corresponding to the feature point P5. Therefore, the vehicle speed control unit 33 sets the upper limit vehicle speed of the vehicle 10 to include the same feature points in a plurality of captured images.

FIG. 6 shows the positional relationship between the amount of change in the position of the vehicle-mounted camera 11 and the feature point P5. The vehicle speed control unit 33 _{calculates the upper limit vehicle speed V max} to be set by using the following formula (1). Note that FIG. 6 shows, as an example, an example in which the posture of the vehicle-mounted camera 11 has not changed.

ΔＴ：撮影時間間隔（フレームレートの逆数）
α：方位角
θ：車載カメラ１１の画角

ΔT: Shooting time interval (reciprocal of frame rate)
α: Azimuth θ: Angle of view of the vehicle-mounted camera 11

The distance D to the feature point P5 in the above formula (1) is an assumed value of the lateral distance with respect to the optical axis of the vehicle-mounted camera 11. The vehicle speed control unit 33 of the present embodiment uses 0.5 times the width of the vehicle 10 as a hypothetical value as the distance D. By using this assumed value, the vehicle-mounted camera 11 can continuously photograph the feature points P5 existing outside the vehicle body side surface of the vehicle 10.

The azimuth angle α is an angle formed by the optical axis of the vehicle-mounted camera 11 and the feature point P5 when the _{k-th captured image IM k is captured.} As shown in FIG. 4, when the x-coordinate of the feature point P5 in the image coordinate system is xp, the x-coordinate of the image center OC is xc, the focal length of the in-vehicle camera 11 is f, and the pixel pitch is p, the azimuth angle α is set. Calculate with equation (2).

_{By setting the upper limit vehicle speed V max} calculated from the above equation (1), the vehicle speed control unit 33 includes the same feature point P5 in the captured image in the number of captured images or more set in advance. The calculation unit 32 calculates the three-dimensional position of the feature points for the same feature points included in the plurality of captured images based on the principle of triangulation.

The map creation unit 34 creates a map including the calculated three-dimensional positions of the plurality of feature points and the feature amount. Examples of the map including the three-dimensional position include a parking space such as a garage. The feature amount of each feature point included in the map is the median value of the feature amount in a plurality of captured images. The map creation unit 34 stores the created map in the map DB 42.

7 and 8 are explanatory views of a map created by the map creation unit 34. FIG. 7 shows an _{image IM i} taken by the vehicle-mounted camera 11 when the vehicle 10 is parked in the parking space of the garage. FIG. 8 shows a plan view of the state of FIG. 7 as viewed from above. In the state shown in FIGS. 7 and 8, the driver of the vehicle 10 manually parks in the parking space along the route RT. At that time, the vehicle speed control unit 33 sets the _{upper limit vehicle speed V max} so that the feature point image FPn corresponding to the plurality of feature point Pn detected from _{the captured image IM i is included in the plurality of captured images.} Therefore, under the control of the vehicle speed control unit 33, the vehicle speed of the vehicle 10 during map creation becomes equal to or less than the _{upper limit vehicle speed V max regardless of the amount of depression of the accelerator by the driver.}

FIG. 9 is a flowchart of a map creation method. As shown in FIG. 9, in the map creation flow, first, the vehicle-mounted camera 11 mounted on the vehicle 10 starts a photographing process of photographing the surrounding environment of the vehicle 10 (step S1). Next, the detection unit 31 performs a detection step of detecting feature points from the captured image of the vehicle-mounted camera 11 by using a FAST algorithm or the like (step S2).

The calculation unit 32 performs a calculation step of calculating the three-dimensional positions of the feature points included in the plurality of captured images by using the feature points detected by the detection unit 31 (step S3). FIG. 10 is a flowchart of a sub-flow of the calculation process. As shown in FIG. 10, in the calculation step, the calculation unit 32 first determines whether or not the captured image in which the feature point is detected by the detection unit 31 is the third or subsequent captured image captured by the vehicle-mounted camera 11. (Step S31). When the calculation unit 32 determines that the captured image is not the third image (step S31: NO), the calculation unit 32 uses an 8-point algorithm or the like to calculate the three-dimensional position of the feature point P1 in FIG. , The position and orientation of the vehicle-mounted camera 11 are calculated (step S32). Specifically, the calculation unit 32 calculates the feature amount of the feature point detected by the detection unit 31, and associates the same feature points included in the first and second captured images with the calculated feature amount. .. The calculation unit 32 calculates the basic matrix F from the same feature points associated with each other by using the 8-point algorithm and RANSAC. The calculation unit 32 calculates the position and orientation of the vehicle-mounted camera 11 by using the basic matrix F and the moving distance of the vehicle-mounted camera 11.

When it is determined in the process of step S31 that the captured image is the third and subsequent images (step S31: YES), the calculation unit 32 calculates the three-dimensional positions of the feature points P2 and P3 in FIG. In addition, the position and orientation of the vehicle-mounted camera 11 that minimizes the sum of the reprojection errors are calculated (step S33). Specifically, the calculation unit 32 calculates the feature amount of the feature points detected by the detection unit 31, and associates the same feature points included in the plurality of captured images. The calculation unit 32 calculates the position and orientation of the vehicle-mounted camera 11 that minimizes the sum of the reprojection errors by using the feature points whose three-dimensional positions are known among the associated feature points. After that, the calculation unit 32 calculates the three-dimensional position of each of the associated feature points by using the calculated position and orientation of the vehicle-mounted camera 11 according to the principle of triangulation shown in FIG. 6 (step S34). End the process.

When the process of step S3 in FIG. 9 is performed, the vehicle speed control unit 33 performs vehicle speed control step of controlling the upper limit vehicle speed V _max of the vehicle 10 (step S4). As shown in FIG. 6, the vehicle speed control unit 33 calculates the _{upper limit vehicle speed V max} by solving the above equation (1) using the azimuth angle α from the vehicle-mounted camera 11 to the feature point P5. The vehicle speed control unit 33 controls the speed of the vehicle so that it is equal to or less than the _{upper limit vehicle speed V max.} Next, the map creation unit 34 uses the three-dimensional positions of a plurality of different feature points calculated by the calculation unit 32 to create a map including information on the three-dimensional positions as shown in FIG. 7. (Step S5).

The map creation unit 34 determines whether or not the map creation is completed (step S6). When the map creation unit 34 determines that the map creation has not been completed yet (step S6: NO), the map creation unit 34 repeats the processes after step S1. When the map creation unit 34 determines that the map creation is completed (step S6: YES), the map creation unit 34 ends the map creation flow.

As described above, in the map creation system 100 of the first embodiment, the calculation unit 32 calculates the three-dimensional position of the same feature point included in the plurality of captured images by using the position and orientation of the in-vehicle camera 11. do. _{The vehicle speed control unit 33 controls the upper limit vehicle speed V max} of the vehicle 10 so that the same feature point is included in a plurality of captured images by using the azimuth angle α from the vehicle-mounted camera 11 to the three-dimensional position of the feature point. .. The map creation unit 34 creates a map including information on the three-dimensional positions by using the calculated three-dimensional positions of the plurality of feature points. Therefore, when the map creation system 100 of the first embodiment creates a map including information on the three-dimensional positions of a plurality of feature points, the plurality of captured images captured by the in-vehicle camera 11 include the same feature points. The upper limit vehicle speed V _max is set so as to be achieved. That is, by setting the upper limit vehicle speed V _max , the same feature points taken at different points are included in a plurality of captured images under the minimum deceleration. As a result, based on the principle of triangulation, sufficient information on the three-dimensional positions of the feature points necessary for creating a map can be obtained, and a map containing the position information of a plurality of feature points can be stably created. .. In this way, by stably creating a map containing the position information of a plurality of feature points under the minimum deceleration, it is possible to suppress the loss of information in the map in the automatic driving control, and the vehicle. 10 automatic operation controls are realized efficiently and stably.

Further, the vehicle speed control unit 33 of the first embodiment uses the frame rate and angle of view of the vehicle-mounted camera 11 to detect the same feature points in a preset number of captured images or more, and the upper limit vehicle speed V is equal to or lower than the vehicle speed. Set _max. That is, the map creation system 100 of the first embodiment uses the frame rate and the angle of view of the vehicle-mounted camera 11 to obtain the _{upper limit vehicle speed V max at the minimum vehicle speed necessary for creating a map.} As a result, a map containing position information of a plurality of feature points is created more efficiently with a minimum deceleration.

<Modified example of the first embodiment>
In the first embodiment, as shown in FIGS. 7 and 8, the map creation when the vehicle 10 is parked in the parking space has been described, but the route from the departure point to the set destination is described. Of these, a map for a predetermined section may be created. For example, when the vehicle 10 moves from the garage where the map including the three-dimensional position information of the feature points is not created to a predetermined destination, the map is created only in the section until the vehicle 10 leaves the garage. May be good. In this case, the vehicle speed control unit 33 of the route from the departure point to the destination, only the section until it exits the garage is created section by mapping unit 34 maps are created, the upper limit speed V _max of the vehicle To set. When the created section is passed, the vehicle speed control unit 33 releases the _{set upper limit vehicle speed V max of the vehicle 10.}

As described above, the vehicle speed control unit 33 may set _{the upper limit vehicle speed V max of the} vehicle only in the creation section in which the map is created by the map creation unit 34. In the map creation system 100 of this modified example, the upper limit vehicle speed V _{max of the} vehicle is not set in the section where the map is not created. Therefore, the map is stably created in the section where the map creation is required, and then the vehicle 10 is operated. Is not restricted more than necessary.

<Second Embodiment>
FIG. 11 is a schematic block diagram of the vehicle control system 200 of the second embodiment. The vehicle control system 200 is a system that automatically drives the vehicle 10 by using a map including information on three-dimensional positions of a plurality of feature points. In the vehicle control system 200 of the second embodiment, the CPU 30a functions as a destination setting unit 35, a position estimation unit 36, and a follow-up control unit 37, and a storage unit 40a, as compared with the map creation system 100 of the first embodiment. Is different in that it has a locus database (trajectory DB) 43 and that it does not function as a calculation unit 32. Therefore, in the second embodiment, the configuration and the like different from those of the first embodiment will be described, and the description of the same configuration and the like as the first embodiment will be omitted.

The destination setting unit 35 receives radio waves transmitted from artificial satellites constituting a GPS (Global Positioning System). The destination setting unit 35 uses the current position of the vehicle 10 specified by the position estimation unit 36, the destination received by the input unit 60, and the map data stored in the map DB 42 from the current position to the destination. Set the route to. The destination setting unit 35 displays a plurality of set route candidates on the monitor of the output unit 70. When the input unit 60 receives an operation of selecting one route from a plurality of route candidates, the destination setting unit 35 guides the route to the destination by using the image displayed on the monitor and the voice output from the speaker. do.

The locus DB 43 shown in FIG. 11 stores the travel locus that the vehicle 10 has traveled in the past in association with the map stored in the map DB. The map DB 42 stores a map including information on three-dimensional positions of a plurality of feature points and feature quantities, which is created by, for example, the map creation system 100 of the first embodiment.

The position estimation unit 36 estimates the current position of the vehicle 10 by using the feature points detected by the detection unit 31 in a state where _{the upper limit vehicle speed V max} of the vehicle 10 is controlled by the vehicle speed control unit 33. Specifically, the position estimation unit 36 is included in a plurality of captured images and associates the same feature points detected by the detection unit 31. When the pair of the associated feature points is a combination of a predetermined value (for example, 30 pairs) or more set in advance, the position estimation unit 36 is the sum of the reprojection errors performed by the calculation unit 32 of the first embodiment. Is calculated as the minimum position and orientation of the vehicle-mounted camera 11. When feature points of a predetermined value or more cannot be associated with a plurality of captured images, or when feature points are detected from the first captured image, the position estimation unit 36 is detected by the detection unit 31. The feature points in the map stored in the map DB 42 are associated with the feature points. The position estimation unit 36 calculates the position and orientation of the vehicle-mounted camera 11 that minimizes the sum of the reprojection errors with respect to the associated feature points. The position estimation unit 36 estimates the current position of the vehicle 10 by using the calculated position and orientation of the vehicle-mounted camera 11.

The follow-up control unit 37 controls the steering angle of the vehicle 10 by using the current position of the vehicle 10 estimated by the position estimation unit 36 and the past travel locus of the vehicle 10 stored in the locus DB 43. Specifically, the follow-up control unit 37 controls the steering angle so that the estimated current position of the vehicle 10 follows the past travel locus. For example, the follow-up control unit 37 controls the vehicle 10 by automatic driving so as to follow the same locus as the running locus at the time of parking in the past with respect to the parking space for which the map including the information of the three-dimensional position has already been completed. do.

FIG. 12 is a flowchart of the vehicle control method. In the vehicle control flow shown in FIG. 12, first, the destination setting unit 35 sets the destination of the vehicle 10 according to the operation received by the input unit 60 (step S11). In the second embodiment, a case will be described in which the map DB 42 automatically parks in a garage in which a map including a plurality of feature points whose three-dimensional positions are known in advance is registered. When the position after parking as the destination is set, the processes of steps S12 to S14 are performed. Since the processes of steps S12 to S14 are the same as the processes of steps S1, S2, and S4 of the first embodiment (FIG. 9), the processes after step S15 of FIG. 12 will be described.

When the process of step S14 of FIG. 12 is performed, the position estimation unit 36 performs a position estimation step of estimating the current position of the vehicle 10 (step S15). FIG. 13 is a flowchart of a sub-flow of the position estimation process. As shown in FIG. 13, the position estimation unit 36 first determines whether or not the captured image in which the detection unit 31 has detected the feature point is the first image (step S151). When the position estimation unit 36 determines that the captured image in which the feature point is detected is not the first (step S151: NO), the position estimation unit 36 determines that the captured image and the previous captured image have the same feature point. Correspondence (step S152). The position estimation unit 36 determines whether or not the pair of the associated feature points is a combination of a predetermined value or more (step S153). In the case of a combination of a predetermined value or more (step S153: YES), the position estimation unit 36 performs the process of step S155, which will be described later, using the associated feature points.

In the process of step S153, if the pair of the associated feature points is a combination of less than a predetermined value (step S153: NO), the process of step S154 described later is performed. In the process of step S151, when the position estimation unit 36 determines that the captured image in which the feature point is detected by the detection unit is the first (step S151: YES), the detected feature point and the map DB 42 are displayed. The feature points in the stored map are associated with each other (step S154). The position estimation unit 36 calculates the position and orientation of the vehicle-mounted camera 11 that minimizes the sum of the reprojection errors with respect to the feature points associated in the process of step S153 or step S154 (step S155), and position estimation is performed. End the process. By using the feature points associated in step S152, the position estimation unit 36 can calculate the position and orientation of the vehicle-mounted camera 11 in a shorter time than using the feature points associated in step S154. When the position and orientation of the vehicle-mounted camera 11 are calculated in a short time, the tracking system of the tracking control unit 37 is improved.

When the process of step S15 of FIG. 12 is performed, the follow-up control unit 37 performs a follow-up step of making the vehicle 10 follow the past travel locus from the position estimated by the position estimation unit 36 to the destination (step S16). .. The follow-up control unit 37 acquires a travel locus when parked in a parking space, which is a destination, from the locus DB 43. The follow-up control unit 37 controls the steering angle of the vehicle 10 so that the vehicle 10 follows the acquired travel locus from the current position and travels. At this time, the vehicle speed control unit 33 controls the speed of the vehicle 10 to be equal to or less than the _{upper limit vehicle speed V max.}

The follow-up control unit 37 determines whether or not the follow-up control for moving the vehicle 10 to the set destination has been completed (step S17). If it is determined that the follow-up control has not been completed (step S17: NO), the processes after step S12 are repeated. When the follow-up control unit 37 determines that the follow-up control has been completed (step S17: YES), the follow-up control unit 37 ends the vehicle control flow.

As described above, the position estimation unit 36 of the second embodiment displays the feature points detected by the detection unit 31 in the state where _{the upper limit vehicle speed V max of the vehicle 10 is controlled by the vehicle speed control unit 33 in the map DB 42.} The current position of the vehicle 10 is estimated by collating with the feature points in the map stored in (step S154 in FIG. 13). At this time, the vehicle speed is set to the _{upper limit vehicle speed V max or less.} Therefore, the same feature point can be tracked between consecutive captured images during traveling, so that the current position of the vehicle 10 can be estimated stably. As a result, automatic driving control is stably performed in the vehicle 10.

Further, in the vehicle control system 200 of the second embodiment, the locus DB 43 stores the travel locus in which the vehicle 10 has traveled in the past. The follow-up control unit 37 controls the vehicle 10 so that the current position of the vehicle 10 estimated by the position estimation unit 36 follows the past travel locus stored in the locus DB 43. Therefore, the vehicle control system 200 of the second embodiment can realize automatic driving of the vehicle 10 that follows the past traveling locus.

<Other variants>
In the above embodiment, a part of the configuration realized by the hardware may be replaced with software, and conversely, a part of the configuration realized by the software may be replaced with the hardware. good. The present invention is not limited to the above-described embodiment, and can be implemented in various aspects without departing from the gist thereof. For example, the following modifications are also possible.

The configurations of the cartography system 100 of the first embodiment and the vehicle control system 200 of the second embodiment are examples and can be variously modified. For example, the cartography system 100 may include an in-vehicle camera 11, function as a detection unit 31, a calculation unit 32, a vehicle speed control unit 33, and a map creation unit 34, and does not include an input unit 60 and an output unit 70. You may. The vehicle control system 200 may include an in-vehicle camera 11, function as a detection unit 31, a calculation unit 32, a vehicle speed control unit 33, and a position estimation unit 36, and does not include an input unit 60 and an output unit 70. It does not have to function as the destination setting unit 35.

The vehicle control system 200 may function as a position estimation device that estimates the current position of the vehicle 10 without functioning as the follow-up control unit 37. The vehicle control flow (FIG. 12) executed by the vehicle control system 200 may not include a step of setting a destination (step S11) and a follow-up step (step S16).

The frame rate and the angle of view of the vehicle-mounted camera 11 of the first embodiment are examples, and may be different numerical values. Further, the position of the vehicle-mounted camera 11 mounted on the vehicle 10 does not have to be the rear of the vehicle 10, and may be the side or the front. Further, a plurality of in-vehicle cameras 11 may be mounted on the vehicle 10. The frame rates and angles of view of the plurality of vehicle-mounted cameras 11 may be different. The number of captured images to be captured in order to identify the feature points may be set according to the frame rate and the angle of view of the vehicle-mounted camera 11.

The method for the detection unit 31 to detect the feature points from the captured image is not limited to the FAST algorithm, the 8-point algorithm, and the RANSAC of the first embodiment, and well-known techniques can be applied. The method in which the calculation unit 32 associates the same feature points included in the plurality of captured images is not limited to the ORB of the first embodiment, and a well-known technique can be applied.

The method of setting the upper limit vehicle speed V _max of the vehicle speed control unit 33 can be variously modified. _{The upper limit vehicle speed V max} may be set by a well-known method other than that shown in the above formula (1). Further, regarding the distance D in the above formula (1), 0.5 times the width of the vehicle 10 is used as a hypothetical value in the above first embodiment, but it may be larger than 0.5 times, or another A value may be set. In the first embodiment, as shown in FIG. 6, an example in which the posture of the vehicle-mounted camera 11 does not change has been described, but even if the posture of the vehicle-mounted camera 11 changes, the vehicle speed control unit 33 has an upper limit. The vehicle speed V _max can be calculated.

In the first embodiment and the second embodiment, one control device 20, 20a functions as a detection unit 31, a calculation unit 32, a vehicle speed control unit 33, etc., but these functions can be applied to a plurality of control devices. It may be distributed. In this case, various data may be exchanged by wireless communication via the communication unit included in the control device of the modified example.

The input unit 60 and the output unit 70 of the first embodiment and the second embodiment are examples and can be deformed in various ways. For example, the input unit 60 may be a touch panel displayed on the output unit 70, or may be a push-type button arranged around the monitor 70. The output unit 70 may output an image or sound to a portable terminal via the communication unit.

Although the present embodiment has been described above based on the embodiments and modifications, the embodiments of the above-described embodiments are for facilitating the understanding of the present embodiment and do not limit the present embodiment. This aspect may be modified or improved without departing from its spirit and claims, and this aspect includes its equivalents. In addition, if the technical feature is not described as essential in the present specification, it may be deleted as appropriate.

10 ... Vehicle 11 ... In-vehicle camera 20, 20a ... Control device 21 ... ROM
22 ... RAM
30, 30a ... CPU
31 ... Detection unit 32 ... Calculation unit 33 ... Vehicle speed control unit 34 ... Map creation unit 35 ... Destination setting unit 36 ... Position estimation unit 37 ... Follow-up control unit 40, 40a ... Storage unit 42 ... Map DB
43 ... Trajectory DB
60 ... Input unit 70 ... Output unit 100 ... Map creation system (map creation device)
200 ... Vehicle control system (position estimation device)
D ... Distance FP1 to FP10, FPn ... Feature point image (feature point)
FP01 ... Temporary feature point image IM _i , IM _k , IM _{k + 1} , IM _m , IM _n-1 , IM _n ... Photographed image OC ... Image center P1-P3, P5, Pn ... Feature point RT ... Path V _max ... Upper limit vehicle speed θ… Angle of view of in-vehicle camera α… Azimuth angle ΔT… Shooting time interval

Claims (8)

It is a cartography device
An on-board camera mounted on the vehicle that captures the environment around the vehicle,
A detection unit that detects feature points from the captured image taken by the in-vehicle camera,
A calculation unit that calculates the three-dimensional position of the same feature point included in the plurality of captured images by using the position and orientation of the vehicle-mounted camera.
Using the position and orientation of the vehicle-mounted camera and the azimuth angle to the position of the feature point calculated by the calculation unit, the vehicle can include the same feature point in the plurality of captured images. A vehicle speed control unit that controls the upper limit vehicle speed,
A map creation unit that creates a map including information on each of the three-dimensional positions using the three-dimensional positions of the plurality of different feature points calculated from the plurality of captured images.
A cartography device equipped with. The cartographic device according to claim 1.
The vehicle speed control unit uses the frame rate and angle of view of the vehicle-mounted camera so that the upper limit vehicle speed is equal to or lower than the vehicle speed at which the same feature points are detected in the preset number of captured images or more. A cartography device to set up. The cartographic device according to claim 1 or 2.
The vehicle speed control unit is a map creation device that controls the upper limit vehicle speed only in the map creation section of the route from the departure point to the destination. It is a position estimation device
An on-board camera mounted on the vehicle that captures the environment around the vehicle,
A map storage unit that stores a map containing information on the three-dimensional positions of multiple feature points,
A detection unit that detects feature points from the captured image taken by the in-vehicle camera,
Using the position and orientation of the vehicle-mounted camera and the azimuth angle to the position of the feature point detected by the detection unit, the vehicle can include the same feature point in the plurality of captured images. A vehicle speed control unit that controls the upper limit vehicle speed,
In a state where the upper limit vehicle speed is controlled by the vehicle speed control unit, the feature points detected by the detection unit are compared with the feature points in the map stored in the map storage unit. A position estimation unit that estimates the position of the vehicle and
A position estimation device. It ’s a vehicle control system.
The position estimation device according to claim 4 and
A locus storage unit that stores the past travel locus of the vehicle and
When the route on which the vehicle travels is the same as the route corresponding to the travel locus stored in the locus storage unit, the position of the vehicle estimated by the position estimation unit is made to follow the travel locus. A follow-up control unit that controls the vehicle and
A vehicle control system. It is a map creation method, and the information processing device
The shooting process of shooting the surrounding environment of the vehicle with the in-vehicle camera mounted on the vehicle,
A detection process for detecting feature points from an image taken by the in-vehicle camera, and
A calculation step of calculating the three-dimensional position of the same feature point included in the plurality of captured images by using the position and orientation of the vehicle-mounted camera with respect to the vehicle.
Using the position and orientation of the in-vehicle camera and the calculated azimuth angle to the position of the feature point, the upper limit vehicle speed of the vehicle is controlled so that the same feature point is included in the plurality of captured images. Vehicle speed control process and
A map creation step of creating a map including information on each of the three-dimensional positions using the three-dimensional positions of the plurality of different feature points calculated from the plurality of captured images.
How to make a map. It ’s a computer program,
A shooting function that captures the environment around the vehicle with an in-vehicle camera mounted on the vehicle,
A detection function that detects feature points from the captured image taken by the in-vehicle camera, and
A calculation function for calculating the three-dimensional position of the same feature point included in the plurality of captured images by using the position and orientation of the vehicle-mounted camera with respect to the vehicle.
Using the position and orientation of the vehicle-mounted camera and the calculated azimuth angle to the position of the feature point, the upper limit vehicle speed of the vehicle is controlled so that the same feature point is included in the plurality of captured images. Vehicle speed control function and
A map creation function that creates a map including information on each of the three-dimensional positions by using the three-dimensional positions of a plurality of different feature points calculated from the plurality of captured images.
A computer program that realizes information processing equipment. It is a position estimation method, and the information processing device
The shooting process of shooting the surrounding environment of the vehicle with the in-vehicle camera mounted on the vehicle,
A detection process for detecting feature points from an image taken by the in-vehicle camera, and
Using the position and orientation of the in-vehicle camera and the azimuth angle to the position of the detected feature point, the upper limit vehicle speed of the vehicle is controlled so that the same feature point is included in the plurality of captured images. Vehicle speed control process and
A position estimation step of estimating the position of the vehicle by collating the detected feature points with the feature points in the map including information on the three-dimensional positions of the plurality of feature points stored in the map storage unit. ,
A position estimation method.

Images