JP2020201210A - Position estimating device and computer program - Google Patents

Abstract

To provide a position estimating device with which it is possible to estimate the correct position of a moving vehicle.SOLUTION: A position estimating device comprises: a storage unit 22 for storing a map in which ground features are shown; a detection unit 31 for detecting a plurality of candidate points representing ground features from the area data showing ground features in a surrounding area of a moving vehicle 10 having been acquired using a sensor 2 of the moving vehicle 10; a candidate point count unit 32 for counting the number of candidate points in a first verification range candidate within the area data from among the plurality of candidate points and counting the number of candidate points in a second verification range candidate within a data area that includes the first verification range candidate and is larger than the first verification range candidate; a verification range setting unit 33 for setting a verification range to be verified with the positions of corresponding ground features on the map on the basis of the number of candidate points in the first verification range candidate or the number of candidate points in the second verification range candidate; and a position estimation unit 34 for verifying the candidate points included in the verification range from among the plurality of candidate points with the corresponding ground features on the map and thereby estimating the position of the moving vehicle 10.SELECTED DRAWING: Figure 3

Description

The present invention relates to a position estimation device and a computer program.

Estimating the position of a vehicle has been studied for use in vehicle driving assistance or autonomous driving control.

For example, a camera mounted on a vehicle is used to detect a lane marking in an image of the front of the vehicle, and the vehicle on the road is based on the relative positional relationship between the lane marking on the road and the vehicle. It has been proposed to estimate the position of.

The vehicle travel path detection device described in Patent Document 1 sets a search range for detecting a white line in a road image of a road in front of a vehicle, and a situation in which a white line is not detected within the search range is predetermined. If the period continues, the search range is widened and white lines are detected.

Japanese Unexamined Patent Publication No. 9-35198

However, the vehicle travel path detection device described in Patent Document 1 may erroneously detect a feature other than the white line such as a road shoulder as a white line if the width of the search range is simply widened.

If the position of the vehicle is estimated based on the detection result of such an erroneous white line, the vehicle travel path detection device may not be able to estimate the travel route or detect the correct travel route.

Therefore, an object of the present invention is to provide a position estimation device capable of estimating the correct position of a moving object so that a correct traveling path can be detected.

According to one embodiment, a position estimation device is provided. This position estimation device is a storage unit that stores a map showing a feature, and area data representing a feature in a predetermined area around the moving object acquired by using a sensor attached to the moving object. From the detection unit that detects a plurality of candidate points representing a feature, and from among the plurality of candidate points, the number of candidate points in the first collation range candidate set for the area data is counted, and Candidate point count that counts the number of candidate points in the second collation range candidate set for the data area that includes the first collation range candidate and is larger than the first collation range candidate from among a plurality of candidate points. Collation range to match the position of the corresponding feature on the map based on the unit and at least one of the number of candidate points in the first collation range candidate or the number of candidate points in the second collation range candidate. By collating the collation range setting unit that sets the data area with the candidate points included in the collation range from a plurality of candidate points and the corresponding feature on the map, the position of the moving object can be determined. It has a position estimation unit for estimating.

In this position estimation device, the collation range setting unit determines that the number of candidate points in the first collation range candidate is larger than the first threshold value, or the number of candidate points in the first collation range candidate is large. When it is equal to or less than the first threshold value and the ratio of the number of candidate points in the first collation range candidate to the number of candidate points in the second collation range candidate is larger than the second threshold value, the first One collation range candidate is determined as a collation range, the number of candidate points in the first collation range candidate is equal to or less than the first threshold value, and the first with respect to the number of candidate points in the second collation range candidate. When the ratio of the number of candidate points in the collation range candidate is equal to or less than the second threshold value, it is preferable to determine the second collation range candidate as the collation range.

Further, in this position estimation device, it is preferable that the collation range setting unit sets the collation range so that the width of the collation range increases as the distance from the moving object increases.

Further, in this position estimation device, the feature is a lane marking line, and the collation range setting unit is the curvature of the lane marking line in the data area from the curve formed by a plurality of candidate points detected in the data area. It is preferable to set the collation range so that the width of the collation range becomes wider as the curvature becomes larger.

Alternatively, in this position estimation device, the feature is a lane marking line, the map shows the position of the lane marking line, and the collation range setting unit is the current position and traveling direction of the moving object estimated by the position estimation unit. It is preferable to obtain the curvature of the lane marking line in the data area based on the position of the lane marking line shown on the map, and set the matching range so that the larger the curvature, the wider the width of the matching range.

In particular, in this position estimation device, it is preferable that the collation range setting unit excludes the region in the region data including the lane marking line having a curvature equal to or higher than the third threshold value from the collation range.

Further, in this position estimation device, the area data is divided into a plurality of areas along the traveling direction of the moving object, and the detection unit is used for each of a plurality of areas from the plurality of candidate points. The candidate point counting unit detects the candidate points, counts the number of candidate points in the first collation range candidate set for each of a plurality of areas, and sets the second collation range candidate for each of the plurality of areas. The number of candidate points is counted, and the collation range setting unit counts the number of candidate points in the first collation range candidate or the number of candidate points in the second collation range candidate for each of a plurality of areas. It is preferable to set the collation range based on at least one.

Also, according to other embodiments, a computer program is provided. This computer program detects a plurality of candidate points representing a feature from area data representing a feature in a predetermined area around the moving object acquired by using a sensor attached to the moving object. , Count the number of candidate points in the first collation range candidate set for the area data from a plurality of candidate points, and include the first collation range candidate from the plurality of candidate points. Counting the number of candidate points in the second collation range candidate set for the data area, which is larger than the first collation range candidate, and the number of candidate points in the first collation range candidate or the second collation range. Setting a collation range for the data area to match the position of the corresponding feature on the map on which the feature is represented, based on at least one of the number of candidate points in the candidate, and multiple The processor is made to estimate the position of the moving object by collating the candidate points included in the collation range from the candidate points with the corresponding features on the map.

The position estimation device according to the present invention has the effect of being able to estimate the correct position of a moving object.

It is a schematic block diagram of the vehicle control system in which a position estimation device is mounted. It is a hardware block diagram of the electronic control device which is one Embodiment of a position estimation device. It is a functional block diagram of the processor of the electronic control device concerning the vehicle control processing including a position estimation device. (A) is a diagram showing an image taken by a camera, and (B) is a diagram showing candidate points detected on the image. It is a figure which shows the 1st collation range candidate and the 2nd collation range candidate. It is a flowchart explaining operation of a collation range setting part. (A) is a diagram showing an example in which a sufficient number of candidate points can be detected in the collation range, and (B) is a diagram showing an example in which the number of candidate points included in the collation range is small. It is a figure explaining that the collation range in a lateral direction is changed according to the curvature of a lane marking line. It is a figure explaining that a part of the detected candidate points is excluded from a collation range. It is a figure explaining that the area around a vehicle is divided into a plurality of areas, and the collation range is set for each divided area.

Hereinafter, the position estimation device will be described with reference to the drawings. This position estimation device uses a plurality of area data (for example, an image) representing a feature in a predetermined area around the moving object acquired by using a sensor such as a camera mounted on the moving object to represent the feature. Detect candidate points. The position estimation device counts the number of candidate points in the first collation range candidate set for the area data from a plurality of candidate points, and the first collation range from the plurality of candidate points. The number of candidate points in the second collation range candidate set for the data area, which includes the candidates and is larger than the first collation range candidate, is counted. The position estimation device collates with the corresponding feature on the map based on at least one of the number of candidate points in the first collation range candidate or the number of candidate points in the second collation range candidate. Is set for the data area, and the position of the moving object is estimated by collating the candidate points included in the collation range from the plurality of candidate points with the corresponding features on the map. As a result, when a sufficient number of candidate points are detected within the collation range, the position estimator can excessively widen the collation range and set the collation range so that the wrong candidate points are not collated. The correct position of a moving object can be estimated by collating the candidate points of. Further, when a sufficient number of candidate points are not detected within the collation range, the position estimation device collates an appropriate number of candidate points with the map by expanding the collation range and setting the collation range. In this way, the position estimation device appropriately sets the collation range so that an appropriate number of candidate points are collated with the map.

In the following, an example in which the position estimation device is applied to the vehicle control system will be described. In this example, the position estimation device executes a position estimation process based on an image (image data) which is an example of area data acquired by a camera mounted on a vehicle which is an example of a moving object. A collation range for collating a map with a candidate point representing a lane marking line, which is an example of an object, is set for an image. Then, the position estimation device estimates the position of the vehicle based on the candidate points included in the collation range. However, the technical scope of the present invention is not limited to those embodiments, but extends to the inventions described in the claims and their equivalents.

FIG. 1 is a schematic configuration diagram of a vehicle control system in which a position estimation device is mounted. FIG. 2 is a hardware configuration diagram of an electronic control device, which is one embodiment of the position estimation device.

In the present embodiment, the vehicle control system 1 mounted on the vehicle 10 and controlling the vehicle 10 has a camera 2 for acquiring an image in front of the vehicle and a positioning information receiver 3. Further, the vehicle control system 1 stores wide area map information of a predetermined area, and based on the positioning information output by the positioning information receiver 3, map information of a narrow area including the current position represented by the positioning information. It has a first electronic control device (ECU) 4 that outputs. Further, the vehicle control system 1 is an example of a position estimation device, and includes a second electronic control device (ECU) 5 that estimates the position of the vehicle 10 using the map information output by the first ECU 4.

The camera 2 captures an image in front of the vehicle 10 as area data. The captured image shows a feature such as a lane marking line included in a predetermined area in front of the vehicle 10. The camera 2 is an example of an imaging unit, and is a two-dimensional detector composed of an array of photoelectric conversion elements having sensitivity to visible light such as CCD or C-MOS, and an image capture target on the two-dimensional detector. It has an imaging optical system that forms an image of a region. Then, the camera 2 is attached to, for example, the interior of the vehicle 10 so as to face the front of the vehicle 10. Then, the camera 2 photographs the front region of the vehicle 10 at predetermined photographing cycles, and generates an image in which the front region is captured. Each time the camera 2 generates an image, the camera 2 outputs the image and the information acquisition time at which the image was acquired to the second ECU 5 via the in-vehicle network 6. The image generated by the camera 2 may be a color image or a gray image.

The positioning information receiver 3 acquires positioning information representing the current position of the vehicle 10. For example, the positioning information receiver 3 can be a GPS receiver. Then, each time the positioning information receiver 3 acquires the positioning information in a predetermined reception cycle, the positioning information and the time when the positioning information is acquired are output to the first ECU 4.

The first ECU 4 has a storage device (not shown) such as a magnetic disk drive or a non-volatile semiconductor memory, and this storage device has a wide range including the current position of the vehicle 10 (for example, a range of several km square). Stores wide area map information of. This wide area map information is preferably high-precision map information including information indicating the types and positions of features and structures such as lane marking lines on the road. The positions of features and structures on the road are represented by, for example, a world coordinate system whose origin is a predetermined reference position in real space. The first ECU 4 has, for example, a wireless communication unit (not shown), receives wide area map information from an external server using the wireless communication unit according to the current position of the vehicle 10, and stores it in a storage device. To do. Each time the first ECU 4 inputs the positioning information input from the positioning information receiver 3, the first ECU 4 refers to the stored wide area map information and refers to a narrow area including the current position represented by the positioning information (for example, several tens to several tens). The map information (within a range of several hundred meters square) and the time when the positioning information was acquired are output to the second ECU 5 via the in-vehicle network 6. The first ECU 4 is an example of a storage unit that stores a map showing a feature.

The second ECU 5 controls the vehicle 10. In the present embodiment, the second ECU 5 estimates the position of the vehicle 10 based on the image output by the camera 2 and controls the vehicle 10 so as to automatically drive the vehicle 10. Therefore, the second ECU 5 has a communication interface 21, a memory 22, and a processor 23.

The communication interface 21 is an example of a communication unit, and has an interface circuit for connecting the second ECU 5 to the in-vehicle network 6. That is, the communication interface 21 is connected to the camera 2 and the like via the in-vehicle network 6. Then, for example, each time the communication interface 21 receives an image and information acquisition time from the camera 2, the received image and information acquisition time is passed to the processor 23. Further, each time the communication interface 21 receives the positioning information, the time when the positioning information is acquired, and the map information from the first ECU 4, the communication interface 21 passes the received positioning information, the time when the positioning information is acquired, and the map information to the processor 23. Further, the communication interface 21 passes the vehicle speed and the yaw rate received from the vehicle speed sensor and the yaw rate sensor (not shown) to the processor 23.

The memory 22 is an example of a storage unit, and includes, for example, a volatile semiconductor memory and a non-volatile semiconductor memory. The memory 22 is used for various data used in the position-determining process executed by the processor 23 of the second ECU 5, for example, installation position information such as the optical axis direction and the mounting position of the camera 2, the focal length of the imaging optical system, and Stores internal parameters such as the angle of view. Further, the memory 22 stores an image received from the camera 2 or the like, positioning information received from the first ECU 4, the time when the positioning information is acquired, map information, and the like.

The processor 23 is an example of a control unit, and includes one or a plurality of CPUs (Central Processing Units) and peripheral circuits thereof. The processor 23 may further include other arithmetic circuits such as a logical operation unit, a numerical operation unit, or a graphic processing unit. When the processor 23 has a plurality of CPUs, each CPU may have a memory. Then, each time the image is generated by the camera 2, the processor 23 estimates the position of the vehicle 10 at the information acquisition time based on the candidate points included in the collation range set for the image. Execute the process. Further, the processor 23 is based on the position of the vehicle 10 at the previous position determination time, the position of the vehicle 10 at the information acquisition time, the vehicle speed, and the yaw rate at the position determination time set in a predetermined cycle. The position of the vehicle 10 at the position-fixing time is estimated. Further, the processor 23 controls the vehicle 10 so as to automatically drive the vehicle 10 based on the relative positional relationship between the estimated position of the vehicle 10 and other objects around the vehicle 10.

FIG. 3 is a functional block diagram of the processor 23 of the second ECU 5 regarding vehicle control processing including position estimation processing. The processor 23 includes a candidate point detection unit 31, a candidate point counting unit 32, a collation range setting unit 33, a position estimation unit 34, an object detection unit 35, an operation planning unit 36, and a vehicle control unit 37. .. Each of these parts of the processor 23 is, for example, a functional module realized by a computer program running on the processor 23. Alternatively, each of these parts included in the processor 23 may be a dedicated arithmetic circuit provided in the processor 23. Further, among these units of the processor 23, the candidate point detection unit 31, the candidate point counting unit 32, the collation range setting unit 33, and the position estimation unit 34 execute the position estimation process.

Next, the process in which the candidate point detection unit 31, the candidate point counting unit 32, and the collation range setting unit 33 cooperate to set the collation range in the image is described with reference to FIGS. 4 to 6. This will be described below. It is preferable that the process of setting the collation range is performed every time an image is output from the camera 2.

The candidate point detection unit 31 detects a plurality of candidate points representing a lane marking line, which is an example of a feature, from an image generated by the camera 2. FIG. 4A is a diagram showing an image 400 generated by the camera 2. First, the candidate point detection unit 31 applies an edge detection filter such as a Sobel filter to the image generated by the camera 2, detects pixels to be edges, and determines predetermined pixels among the detected pixels. Pixels having a brightness equal to or higher than the threshold value are detected as candidate points. Further, the candidate point detection unit 31 may detect a candidate point in the image by inputting the image into the classifier, for example. As the discriminator, for example, a deep neural network (DNN) trained in advance to detect a feature such as a lane marking line in the input image can be used. The candidate point detection unit 31 detects a pixel in the region identified as a feature in the image as a candidate point. FIG. 4B is a diagram showing candidate points 411 detected on the image 410. The candidate point detection unit 31 notifies the candidate point counting unit 32 of the position information of the candidate points in the detected image. The position information of the candidate point can be represented by, for example, an image coordinate system having an xi axis extending to the right and a yi axis extending downward with the position of the upper left corner of the image 400 as the origin. In addition, the candidate point detection unit 31 notifies the candidate point counting unit 32 of the image used for detecting the candidate point and the information acquisition time at which the image was acquired.

The candidate point counting unit 32 counts the number of candidate points in the first collation range candidate set for the image generated by the camera 2. Further, the candidate point counting unit 32 includes the number of candidate points in the second collation range candidate set for the image generated by the camera 2, which includes the first collation range candidate and is larger than the first collation range candidate. To count.

First, the candidate point counting unit 32 represents the coordinates of the candidate points represented by the image coordinate system in the camera coordinate system with the position of the camera 2 as the origin. In the camera coordinate system, the center of the imaging surface is the origin, the traveling direction of the vehicle 10 is the zc axis, the direction orthogonal to the zc axis and parallel to the ground is the xc axis, and the vertical direction is the yc axis. The origin is at a height above the ground where the camera 2 is installed. The lane markings on the ground taken by the camera 2 are assumed to be located on the ground where the yc coordinates are constant. The candidate point detection unit 31 executes a viewpoint conversion process on the coordinates of the candidate point represented by the image coordinate system by using information such as the installation position information of the camera 2 and internal parameters, so that the coordinates of the candidate point can be obtained. Is represented by the camera coordinate system.

Then, the candidate point counting unit 32 represents the coordinates of the candidate points represented by the camera coordinate system in the world coordinate system. The candidate point counting unit 32 determines the position of the vehicle 10 at the previous position determination time notified from the position estimation unit 34, and the movement amount and movement direction of the vehicle 10 between the previous position determination time and the information acquisition time. Based on this, the position and traveling direction of the vehicle 10 at the information acquisition time are estimated. Here, the candidate point counting unit 32 calculates the movement amount and the movement direction of the vehicle 10 between the previous position determination time and the information acquisition time based on the vehicle speed and the yaw rate notified during that time. The candidate point counting unit 32 estimates the assumed position and assumed posture of the camera 2 at the information acquisition time by using the information such as the installation position information and the internal parameters of the camera 2 and the estimated position and traveling direction of the vehicle 10. To do. The candidate point counting unit 32 obtains a conversion formula from the camera coordinate system to the world coordinate system according to the assumed position and assumed posture of the camera 2 at the information acquisition time. Such a transformation formula is represented by a combination of a rotation matrix representing rotation between coordinate systems and a translation vector representing translation between coordinate systems. Then, the candidate point counting unit 32 converts the coordinates of the candidate points represented by the camera coordinate system into the coordinates of the world coordinate system according to the conversion formula.

Then, the candidate point counting unit 32 sets the first collation range candidate for the image acquired at the information acquisition time notified from the candidate point detecting unit 31 for the image generated by the camera 2. The first collation range candidate is an area in the image used to count the candidate points. The candidate point counting unit 32 cuts out a region assumed to be represented by the image generated by the camera 2 at the information acquisition time from the map information according to the assumed position and the assumed posture of the camera 2 at the information acquisition time, and cuts out a reference image. To generate. The candidate point counting unit 32 sets the first collation range candidate so as to include the area corresponding to the lane marking line on the reference image with respect to the image generated by the camera 2. Here, the coordinates of the region representing the first collation range candidate are represented by the world coordinate system. FIG. 5 is a diagram in which first collation range candidates 503a and 503b are set for the left and right lane marking lines 501a and 501b of the vehicle 10 by using the reference image 500.

Further, the candidate point counting unit 32 sets a second collation range candidate including the first collation range candidate and larger than the first collation range candidate for the image generated by the camera 2. Here, the coordinates of the region representing the second collation range candidate are represented in the world coordinate system. The candidate point counting unit 32 may set the second collation range candidate by, for example, expanding the contour of the first collation range candidate by a predetermined number of pixels. As shown in FIG. 5, the candidate point counting unit 32 laterally orthogonals the contours of the first collation range candidates 503a and 503b to the direction in which the lane marking lines 501a and 501b extend for each of the first collation range candidates 503a and 503b. The second collation range candidates 504a and 504b may be set by expanding in the direction. Further, the candidate point counting unit 32 may set the outline of the first collation range candidate symmetrically enlarged with respect to the direction in which the lane marking line extends for each of the second collation range candidates 504a and 504b. , Or it may be enlarged and set asymmetrically. When one side of the lane marking line is adjacent to the road shoulder, the lane-side contour of the lane marking line may be worn more than the road shoulder side contour to be difficult to detect as a candidate point. Therefore, the candidate point counting unit 32 may set each of the second collation range candidates 504a and 504b so that the expansion amount on the lane side of the first collation range candidate is larger than that on the road shoulder side. This makes it easier to detect the lane marking line as a candidate point. Further, the candidate point counting unit 32 may be set so that the expansion amount on the road shoulder side of the first collation range candidate becomes zero for each of the second collation range candidates 504a and 504b. This prevents the road shoulder from being detected as a candidate point.

The candidate point counting unit 32 creates a virtual image taken by the camera 2 based on the map information represented by the world coordinate system, and includes an area corresponding to the lane marking line on the virtual image. 1 The collation range candidate may be set for the image generated by the camera 2. For example, the candidate point counting unit 32 sets the shooting direction set from the virtual shooting point with the position of the camera 2 at the information acquisition time when the image is acquired as the virtual shooting point and the optical axis direction of the camera 2 as the shooting direction. Generates a virtual image of the lane markings on the road included in the map information. The virtual image preferably has the same size as the image generated by the camera 2, for example. The candidate point counting unit 32 superimposes the image generated by the camera 2 on the virtual image, and the first collation range candidate is generated by the camera 2 so as to include an area corresponding to the lane marking line on the virtual image. It can be set for the image.

Then, the candidate point counting unit 32 counts the number of candidate points in the first collation range candidate set for the image generated by the camera 2. For example, the candidate point counting unit 32 counts the number A of the candidate points 502 located inside the first collation range candidates 503a and 503b in the reference image 500 shown in FIG. The candidate point counting unit 32 collates the number A of the candidate points, the position information of the candidate points, the information representing the first collation range candidate, the image in which the first collation range candidate is set, and the information acquisition time. Notify the setting unit 33.

Further, the candidate point counting unit 32 counts the number B of candidate points in the second collation range candidate set for the image generated by the camera 2. The candidate point counting unit 32 counts the number B of the candidate points 502 located inside the second collation range candidates 504a and 504b in the image 500 shown in FIG. The candidate point counting unit 32 collates the number B of candidate points, the position information of the candidate points, the information representing the second collation range candidate, the image in which the second collation range candidate is set, and the information acquisition time. Notify the setting unit 33.

The collation range setting unit 33 determines the position of the corresponding lane marking line on the map based on at least one of the number of candidate points in the first collation range candidate or the number of candidate points in the second collation range candidate. Set the collation range for collating with the image. FIG. 6 is a flowchart illustrating the operation of the collation range setting unit. The collation range setting unit 33 first determines whether or not the number of candidate points in the first collation range candidate is larger than the first threshold value Th1 (step S601 in FIG. 6). The first threshold value Th1 is determined based on, for example, the number of pixels included in the region of the first collation range candidate. Specifically, the collation range setting unit 33 sets the product of the number of pixels included in the region of the first collation range candidate and a predetermined coefficient (0 <coefficient = <1) as the first threshold value Th1. You may ask.

When the number of candidate points in the first collation range candidate is larger than the first threshold value Th1 (step S601-Yes), the collation range setting unit 33 determines the first collation range candidate as the collation range (step S601-Yes). Step S603). The collation range setting unit 33 notifies the position estimation unit 34 of the information representing the determined collation range, the position information of the candidate points, the image in which the collation range is set, and the information acquisition time. As a result, when a sufficient number of candidate points can be obtained within a relatively narrow collation range, the collation range setting unit 33 is prevented from excessively expanding the collation range, so that an inappropriate feature is selected as a candidate point. The collation range can be set so that it is not used for collation. For example, the collation range setting unit 33 can relatively narrow the collation range when a clear lane marking line can be photographed. When the candidate point counting unit 32 counts the number of candidate points in the first collation range candidate, the candidate point counting unit 32 notifies the collation range setting unit 33 of the number of the counted candidate points, and the collation range setting unit 33 performs step S601. May be made to determine. In this case, when the number of candidate points in the first collation range candidate is larger than the first threshold value Th1, the candidate point counting unit 32 sets the second collation range candidate and the second collation. The process of counting the number of candidate points in the range candidates may be omitted. On the other hand, when the number of candidate points in the first collation range candidate is equal to or less than the first threshold value Th1, the candidate point counting unit 32 sets the second collation range candidate and sets the second collation range candidate. Execute the process of counting the number of candidate points in.

On the other hand, when the number of candidate points in the first collation range candidate is equal to or less than the first threshold value Th1 (step S601-No), the collation range setting unit 33 determines the number of candidate points in the second collation range candidate. It is determined whether or not the ratio (A / B) of the number A of the number of candidate points in the first collation range candidate to B is larger than the second threshold value (step S605). The second threshold value is determined based on, for example, the number of pixels included in the area of the first collation range candidate and the number of pixels included in the area of the second collation range candidate. Specifically, the collation range setting unit 33 has a ratio of the number of pixels included in the second collation range candidate area to the number of pixels included in the first collation range candidate area, and a predetermined coefficient (0 <coefficient = <. The product with 1) may be obtained as the second threshold value.

When the ratio (A / B) of the number of candidate points A in the first collation range candidate to the number B of candidate points in the second collation range candidate is larger than the second threshold value (step S605-Yes). The collation range setting unit 33 determines the first collation range candidate as the collation range (step S607). The collation range setting unit 33 notifies the position estimation unit 34 of the information representing the determined collation range, the position information of the candidate points, the image in which the collation range is set, and the information acquisition time. For example, depending on the condition of a feature such as a lane marking line, the absolute number of candidate points obtained from an appropriate feature may be small in the first place. Even in such a case, the collation range setting unit 33 collates if the ratio (A / B) of the number A of the number of candidate points in the first collation range candidate to the number B of the candidate points in the second collation range candidate is large. By setting the range to a relatively narrow range, it is possible to prevent the collation range from being excessively widened, so that the collation range can be set so that an inappropriate feature is not used as a candidate point for collation. For example, even if the lane marking line is deteriorated and faded due to wear or the like, the collation range setting unit 33 does not excessively widen the collation range.

On the other hand, when the ratio (A / B) of the number of candidate points A in the first collation range candidate to the number B of candidate points in the second collation range candidate is equal to or less than the second threshold value (step S605-No). , The collation range setting unit 33 determines the second collation range candidate as the collation range (step S609). The collation range setting unit 33 notifies the position estimation unit 34 of the information representing the determined collation range, the position information of the candidate points, the image in which the collation range is set, and the information acquisition time. As a result, when the estimated position of the vehicle 10 shifts and the number of candidate points detected in the first collation range is small, the collation range setting unit 33 expands the collation range to obtain a sufficient number of candidate points. Can be detected.

The above is the description of the process of setting the collation range.

The position estimation unit 34 estimates the position of the vehicle 10 by collating the candidate points detected within the collation range from the plurality of candidate points with the corresponding lane marking lines on the map. The position estimation unit 34 estimates the position of the vehicle 10 at the information acquisition time by using the image and the collation range each time the collation range setting unit 33 notifies the image and the collation range for the image. The position estimation unit 34 projects the coordinates of the candidate points included in the collation range on the map represented by the map information according to the assumed position and the assumed posture of the camera 2 at the information acquisition time, and is represented in the map information. The degree of coincidence between the lane marking lines around the vehicle 10 and the candidate points included in the collation range is calculated. The position estimation unit 34 performs a plurality of processes of changing the position from the camera coordinate system to the world coordinate system and calculating the degree of coincidence in the same manner as described above while changing the assumed position and the assumed posture by a predetermined amount. For each of the assumed position and the assumed posture, the degree of coincidence between the lane markings around the vehicle 10 shown in the map information and the candidate points included in the collation range is calculated. The position estimation unit 34 specifies the assumed position and assumed posture when the degree of coincidence is maximized, and based on the assumed position and assumed posture of the camera 2, the position estimation unit 34 of the vehicle 10 at the information acquisition time when the camera 2 took an image. Estimate the position and direction of travel. The first ECU 4 outputs map information so as to include an area in which the vehicle 10 can travel between the information acquisition time when the image is acquired and the time when the positioning information used for generating the map information is acquired. Is preferable. As a result, the position estimation unit 34 can calculate the position and the traveling direction of the vehicle 10 using the latest map information.

The position estimation unit 34 sets the position of the vehicle 10 at the position determination time at the position determination time set in a predetermined cycle, the position of the vehicle 10 at the information acquisition time, and the vehicle between the position determination time and the information acquisition time. Estimate based on the amount of movement and the direction of movement of 10. The position estimation unit 34 calculates the movement amount and movement direction of the vehicle 10 between the position determination time and the information acquisition time based on the vehicle speed and the yaw rate input during that time.

When the cycle of the position-fixing time does not match the update interval of the information acquisition time (for example, when the cycle of the position-fixing time is shorter than the update interval of the information acquisition time), the position estimation unit 34 uses the previous position estimation unit 34. Based on the position of the vehicle 10 estimated at the position-fixing time and the amount and direction of movement of the vehicle 10 between the previous position-fixing time and the current position-fixing time, the vehicle 10 at the current position-fixing time Estimate the position. The position estimation unit 34 calculates the movement amount and the movement direction of the vehicle 10 between the previous position determination time and the current position determination time based on the vehicle speed and the yaw rate input during that time.

The object detection unit 35 acquires an image generated by the camera 2 and detects other objects around the vehicle 10 based on this image. The object detection unit 35 detects the object represented by the image by inputting the image into the classifier, for example. As the discriminator, for example, a deep neural network (DNN) trained in advance to detect an object represented by the image from the input image can be used. The object detection unit 35 may use a classifier other than the DNN. For example, the object detection unit 35 inputs a feature amount (for example, Histograms of Oriented Gradients, HOG) calculated from a window set on the image as a discriminator, and the object to be detected is displayed in the window. A support vector machine (SVM) trained in advance to output a certain degree of certainty may be used. Alternatively, the object detection unit 35 may detect the object region by performing template matching between the template representing the object to be detected and the image. In addition, the object detection unit 35 tracks the object detected from the latest image by associating the object detected from the latest image with the object detected from the past image according to the tracking process based on the optical flow. May be good. Then, the object detection unit 35 may estimate the relative velocity of the object with respect to the vehicle 10 based on the change in the size of the object being tracked on the image with the passage of time. Then, the object detection unit 35 determines the same object for the objects detected based on each image, and determines that the objects determined to be the same are one object. The object detection unit 35 notifies the operation planning unit 36 of information indicating the position of the detected object. The object detection unit 35 may detect other objects around the vehicle 10 based on the measurement results measured by a sensor that acquires a distance image such as a rider sensor.

The driving planning unit 36 acquires the position of the vehicle 10 at the position determination time, the information indicating the position of the detected object, and the map information. The driving planning unit 36 generates one or more planned traveling routes of the vehicle 10 based on the information. The planned travel route is represented as, for example, a set of target positions of the vehicle 10 at each time from the current time to a predetermined time ahead. The driving planning unit 36 determines the relative position of the vehicle 10 and the other object according to the position of the vehicle 10 and the positional relationship between the structure on the road shown on the map and the detected other object. Estimate the relationship. For example, the driving planning unit 36 identifies the lane in which the other object is traveling according to the positional relationship between the lane displayed on the map and the other object, so that the other object and the vehicle 10 can be connected to each other. Determine if you are in the same lane. For example, the driving planning unit 36 determines that the other object is traveling in a lane specified by two adjacent lane marking lines located so as to sandwich the horizontal center position of the other object. Similarly, the driving planning unit 36 determines that the vehicle 10 is traveling in a lane specified by two adjacent lane marking lines located so as to sandwich the vehicle 10. Then, the driving planning unit 36 determines whether or not the lane in which the vehicle 10 is traveling and the lane in which another object is traveling are the same. Also, since the distance between two adjacent lane markings shown on the map is known and the internal parameters such as the focal length of the camera are known, the distance between the two adjacent lane markings on the image is known. The distance from the vehicle 10 can be estimated from the interval. Therefore, the driving planning unit 36 estimates the distance from the vehicle 10 to the other object based on the distance between the two adjacent lane marking lines shown on the map at the position of the other object on the image. You may. In this way, the driving planning unit 36 estimates the relative positional relationship between the other object and the vehicle 10 based on the positional relationship with the structure on the road shown on the map. Therefore, the driving planning unit 36 can accurately estimate the relative positional relationship between the other object and the vehicle 10 even if the features on the road such as the lane marking line shown in the image are unclear.

The driving planning unit 36 either travels in a different lane from the other object and the vehicle 10 based on the lane in which the other object is traveling and the relative distance, or the relative between the vehicle 10 and the other object. The planned travel route of the vehicle 10 is generated so that the distance is equal to or greater than a predetermined distance. The operation planning unit 36 may generate a plurality of planned travel routes. In this case, the driving planning unit 36 may select the route that minimizes the sum of the absolute values of the accelerations of the vehicle 10 from the plurality of planned traveling routes. The operation planning unit 36 notifies the vehicle control unit 37 of the generated planned travel route.

The vehicle control unit 37 makes the vehicle travel along the notified planned travel route based on the position of the vehicle 10 at the position determination time, the vehicle speed and the yaw rate, and the notified planned travel route. Each part of 10 is controlled. For example, the vehicle control unit 37 obtains the steering angle, acceleration, and angular acceleration of the vehicle 10 according to the notified planned travel route and the current vehicle speed and yaw rate of the vehicle 10, and the steering angle, acceleration, and angular acceleration thereof. The steering amount, accelerator opening degree or brake amount is set so as to be. Then, the vehicle control unit 37 outputs a control signal according to the set steering amount to an actuator (not shown) that controls the steering wheels of the vehicle 10. Further, the vehicle control unit 37 obtains a fuel injection amount according to the set accelerator opening degree, and outputs a control signal corresponding to the fuel injection amount to the fuel injection device (not shown) of the engine of the vehicle 10. Alternatively, the vehicle control unit 37 outputs a control signal according to the set brake amount to the brake (not shown) of the vehicle 10.

As described above, this position estimation device detects a candidate point representing a feature from the area data representing the feature in a predetermined area around the moving object. The position estimation device counts the number of candidate points in the first collation range candidate set in the data area represented by the area data, includes the first collation range candidate, and is more than the first collation range candidate. The number of the candidate points in the large second collation range candidate set in the data area is counted. The position estimation device collates with the position of the corresponding feature on the map based on at least one of the number of candidate points in the first collation range candidate or the number of candidate points in the second collation range candidate. Set the collation range for the data area. Then, the position estimation device estimates the position of the moving object based on the candidate points included in the collation range. As a result, the collation range can be set so as to include candidate points representing appropriate features, so that the correct position of the moving object can be estimated based on the detected candidate points.

Next, another example of setting the first collation range candidate will be described below with reference to FIGS. 7 to 10.

As described above, the first collation range candidate is set based on the estimated position and posture of the vehicle 10 when the image is taken by the camera 2. If the estimated position of the vehicle 10 is correct, the first collation range candidate is set to correspond to a feature such as a lane marking line included in the map information, and therefore, the first collation range candidate is included in the first collation range candidate. , A sufficient number of candidate points can be detected. FIG. 7A is a diagram showing an example in which a sufficient number of candidate points can be detected within the collation range set in the image of the curved road in front. The first collation range candidate 702 is set so as to include the lane marking line 701 located in front of the vehicle 700, and a large number of candidate points 703 are detected in the first collation range candidate 702.

On the other hand, an error may occur in the estimated position of the vehicle 10. For example, if the estimated position in the traveling direction of the vehicle 10 is deviated from the correct position, the first collation range candidate deviates from the lane demarcation line when detecting the lane lane marking of the curved road ahead. There is a risk of FIG. 7B is a diagram showing an example in which there are few candidate points included in the collation range set in the image of the curved road in front. Since the estimated position in the traveling direction of the vehicle 700 is deviated from the correct position, the number of candidate points 703 detected in the first collation range candidate 702 is reduced.

Therefore, the candidate point counting unit 32 estimates the curvature of the lane marking line in the image from the curve formed by the plurality of candidate points detected in the image generated by the camera 2, and the larger the curvature, the more. The first collation range candidate may be set so that the width of the first collation range candidate is widened.

The candidate point counting unit 32 uses the coordinates (xw, zw) of a plurality of candidate points represented by the world coordinate system as a clothoid approximation formula: xw = P1 + P2 × zw + (P3 / 2) × zw ² + (P4 / 6). Enter in × zw ³ . Here, P1 is the xw coordinate of the lane marking line at zw = 0, P2 is the azimuth angle of the lane marking line at zw = 0, P3 is the curvature of the lane marking line at zw = 0, and P4 is the curvature change rate. (Constant on clothoid curve). It is assumed that the coordinates yw of the candidate points in the vertical direction with respect to the ground are constant.

The candidate point counting unit 32 obtains P1, P2, P3, and P4 by applying multivariate analysis such as the least squares method to the clothoid approximation formula. As a result, the collation range setting unit 33 obtains the curvature P4 of the lane marking line.

The width of the candidate point counting unit 32 in the lateral direction, which is a direction orthogonal to the traveling direction of the vehicle 10 in the first collation range candidate, increases as the curvature of the lane marking line increases with respect to the image generated by the camera 2. The first collation range candidate is set as described above. The candidate point counting unit 32 sets the second collation range candidate so as to include the first collation range candidate and be larger than the first collation range candidate. The collation range setting unit 33 sets the collation range based on at least one of the first collation range candidate and the second collation range candidate set in this way. FIG. 8 is a diagram for explaining that the collation range in the lateral direction orthogonal to the traveling direction of the vehicle is changed according to the curvature of the lane marking line. As shown in FIG. 8, when the road in front of the vehicle 10 is curved and the lane marking line 801 is also curved, the lateral width of the collation range 802 increases as the distance from the vehicle 10 increases. It is set large. As described above, the candidate point counting unit 32 can detect a sufficient number of candidate points by expanding the collation range as the curvature of the lane marking line increases. Here, when the curvature of the lane marking line is equal to or less than a predetermined threshold value, the candidate point counting unit 32 determines that the lane marking line is a straight line, and sets the width of the first collation range candidate in the lateral direction to be constant. May be good.

Further, the candidate point counting unit 32 is traveling in a lane based on the position and traveling direction of the vehicle 10 at the information acquisition time when the image was taken by the camera 2 and the position information of the lane marking line included in the map information. The curvature of the lane marking line may be obtained as the curvature of, and the first collation range candidate may be set so that the larger the curvature, the wider the width of the first collation range candidate in the lateral direction. The traveling direction of the vehicle 10 at the information acquisition time may be determined based on the time course of the position at the position determination time or the yaw rate at the information acquisition time.

Further, the candidate point counting unit 32 may exclude a region in the image including the lane marking line having a curvature equal to or higher than the third threshold value from the collation range. The candidate point counting unit 32 can obtain this third threshold value as follows, for example. First, it is assumed that the deviation Δz in the traveling direction of the vehicle 10 determined based on the positioning information is known. Further, the candidate point counting unit 32 obtains the distance Δx between the lane marking line and the adjacent road shoulder based on the position information of the lane marking line and the road shoulder adjacent to the lane marking line included in the map information. The candidate point counting unit 32 shifts the position of the lane lane marking included in the map information by Δz in the traveling direction, obtains the curvature of the lane lane marking at the position where the lane lane marking overlaps with the adjacent shoulder, and obtains the curvature of the lane marking line. Let the curvature be the third threshold.

FIG. 9 is a diagram for explaining that a part of the candidate points detected in the image 900 generated by the camera 2 is excluded from the collation range. Image 900 includes a lane marking line 901 and a collation range 902. The candidate point counting unit 32 excludes the region 1003 in the image including the lane marking line having a curvature equal to or higher than the third threshold value from the collation range in the image. As a result, the candidate point counting unit 32 excludes features such as shoulders from the collation range to prevent collation as candidate points.

Further, the processor 23 may have a region dividing unit 38 that divides the image generated by the camera 2 into a plurality of regions along the traveling direction or the lateral direction of the vehicle 10 (see FIG. 3). The area division unit 38 divides the image into a plurality of areas and notifies the candidate point detection unit 31 of information representing the plurality of areas. FIG. 10 is a diagram illustrating that the image 1000 generated by the camera 2 is divided into three regions 1000a, 1000b, and 1000c in the traveling direction of the vehicle 10 and a collation range is set for each of the divided regions. .. The candidate point counting unit 32 counts the number of candidate points in the first collation range candidate set for each of the plurality of regions 1000a, 1000b, 1000c, and is set for each of the plurality of regions 1000a, 1000b, 1000c. The number of the candidate points in the second collation range candidate is counted. The collation range setting unit 33 is based on at least one of the number of candidate points in the first collation range candidate or the number of candidate points in the second collation range candidate for each of the plurality of regions 1000a, 1000b, and 1000c. , The collation range 1001, 1002, 1003 is set.

In the present invention, the position estimation device of the above-described embodiment can be appropriately modified as long as the gist of the present invention is not deviated.

For example, in the above-described embodiment, the area data is an image taken by a camera and has two-dimensional information. This region data may be a distance image representing a feature obtained based on a reflected wave reflected from the feature by irradiating an electromagnetic wave such as a laser or millimeter wave. In this case, the area data has three-dimensional information.

Further, in the above-described embodiment, the candidate point detected in the image is the lane marking line, but the candidate point detected in the image may be a road shoulder, a wall, or a guardrail. For example, when a road shoulder is detected as a candidate point, the collation range is preferably set so that the guardrail adjacent to the road shoulder is not collated as a candidate point.

Further, in the above-described embodiment, the number of candidate points in the third collation range candidate set in the area data, which includes the second collation range candidate and is larger than the second collation range candidate, is counted and the first collation range candidate is counted. Corresponding features on the map based on at least one of the number of candidate points in the collation range candidate, the number of candidate points in the second collation range candidate, or the number of candidate points in the third collation range candidate. A collation range for collating with the position of may be set for the area data.

Further, in the above-described embodiment, the collation range is set so as to include the left lane division line in the area data (image), and the collation range is set so as to include the right lane division line in the area data. You may. As a result, the candidate points in the left and right collation ranges in the area data can be independently detected.

1 Vehicle control system 2 Camera 3 Positioning information receiver 4 1st electronic control device 5 2nd electronic control device 6 In-vehicle network 10 Vehicle 21 Communication interface 22 Memory 23 Processor 31 Candidate point detection unit 32 Candidate point counting unit 33 Collation range setting unit 34 Position estimation unit 35 Object detection unit 36 Operation planning unit 37 Vehicle control unit

Claims (8)

A memory unit that stores a map showing features and
A detection unit that detects a plurality of candidate points representing the feature from the area data representing the feature in a predetermined area around the moving object acquired by using a sensor attached to the moving object.
From the plurality of candidate points, the number of the candidate points in the first collation range candidate set for the area data is counted, and from the plurality of candidate points, the first collation range. A candidate point counting unit that includes candidates and is larger than the first collation range candidate and counts the number of the candidate points in the second collation range candidate set for the data area.
Collation with the position of the corresponding feature on the map based on at least one of the number of candidate points in the first collation range candidate or the number of candidate points in the second collation range candidate. A collation range setting unit that sets the collation range to be performed for the data area,
A position estimation unit that estimates the position of the moving object by collating the candidate point included in the collation range from the plurality of candidate points with the corresponding feature on the map.
A position estimator having. The collation range setting unit is
The number of the candidate points in the first collation range candidate is larger than the first threshold value, or the number of the candidate points in the first collation range candidate is equal to or less than the first threshold value. When the ratio of the number of the candidate points in the first collation range candidate to the number of the candidate points in the second collation range candidate is larger than the second threshold value, the first collation The range candidate is determined as the collation range, and
The number of the candidate points in the first collation range candidate is equal to or less than the first threshold value, and the number of the candidate points in the second collation range candidate is in the first collation range candidate. The position estimation device according to claim 1, wherein when the ratio of the number of candidate points is equal to or less than the second threshold value, the second collation range candidate is determined as the collation range. The position estimation device according to claim 1 or 2, wherein the collation range setting unit sets the collation range so that the width of the collation range increases as the distance from the moving object increases. The feature is a lane marking line
The collation range setting unit is
The curvature of the lane marking line in the data area is estimated from the curve formed by the plurality of candidate points detected in the data area.
The position estimation device according to any one of claims 1 to 3, wherein the collation range is set so that the width of the collation range becomes wider as the curvature is larger. The feature is a lane marking line
The map shows the location of the lane markings
The collation range setting unit is
Based on the current position and traveling direction of the moving object estimated by the position estimation unit and the position of the lane marking line represented on the map, the curvature of the lane marking line in the data area is obtained.
The position estimation device according to any one of claims 1 to 3, wherein the collation range is set so that the width of the collation range becomes wider as the curvature is larger. The position estimation device according to claim 4 or 5, wherein the collation range setting unit excludes a region in the region data including a lane marking line having the curvature equal to or higher than a third threshold value from the collation range. It has a region dividing portion that divides the region data into a plurality of regions along the traveling direction of the moving object.
The detection unit detects the candidate points for each of the plurality of candidate points from the plurality of candidate points.
The candidate point counting unit counts the number of the candidate points in the first collation range candidate set for each of the plurality of regions, and the second collation range candidate set for each of the plurality of regions. Count the number of the candidate points,
The collation range setting unit is based on at least one of the number of the candidate points in the first collation range candidate or the number of candidate points in the second collation range candidate for each of the plurality of regions. The position estimation device according to claim 1 or 2, wherein the collation range is set. From the area data representing a feature in a predetermined area around the moving object acquired by using a sensor attached to the moving object, a plurality of candidate points representing the feature can be detected.
From the plurality of candidate points, the number of the candidate points in the first collation range candidate set for the area data is counted, and from the plurality of candidate points, the first collation range. Counting the number of the candidate points in the second collation range candidate set for the data area, which includes the candidates and is larger than the first collation range candidate.
Corresponding said features on a map representing the feature based on at least one of the number of candidate points in the first collation range candidate or the number of candidate points in the second collation range candidate. Setting a collation range to match the position of the feature with respect to the data area
To estimate the position of the moving object by collating the candidate point included in the collation range with the corresponding feature on the map from the plurality of candidate points.
A computer program that causes the processor to execute.

Images