JP6477348B2 - Self-position estimation apparatus and self-position estimation method - Google Patents

Description

  The present invention relates to a self-position estimation apparatus and a self-position estimation method for estimating the position of a vehicle on a map.

  In Patent Document 1 (Japanese Patent Laid-Open No. 2009-163714), a position of a sign existing around the host vehicle is estimated, and the estimated sign position is compared with a sign position on a map used for a navigation device. It is disclosed to correct the marker position included in the information.

  In Patent Document 1, light that is periodically switched on and off is projected from a projector, and the reflected light of the light is captured by a camera, and a difference image between the images captured at the time of lighting and extinguishing is generated. Based on this difference image, it is shown that the presence / absence and position of a sign is calculated by setting a region having a reflection intensity equal to or higher than a predetermined value and having an area corresponding to a target sign (for example, a rectangle). ing.

JP 2009-163714 A

  However, in the conventional example disclosed in Patent Document 1 described above, it is included in an image captured by a camera in accordance with the presence of a light source that is turned on and off at a cycle similar to the light to be projected and the angle of the sign surface with respect to the vehicle. The reflected light changes. For this reason, there is a problem in that the sign position included in the image captured by the camera cannot be collated with the sign position included in the map information, and the self-position of the vehicle cannot be estimated with high accuracy.

  The present invention has been made to solve such a conventional problem, and an object of the present invention is to provide a self-position estimation device and a self-position estimation capable of estimating the self-position of a vehicle with high accuracy. It is to provide a method.

In order to achieve the above object, according to one embodiment of the present invention, a light projecting unit that projects light around a vehicle, an image capturing unit that captures an area around the vehicle including a region where light is projected from the light projecting unit, and imaging Based on the reflected light extracted from the reflected light extracted from the image captured by the imaging unit and the reflected light extracted by the reflected light extracting unit , the image area of the reflector is set from the image captured by the imaging unit, Based on the area of the image area, the center coordinates, the distribution of the received intensity, and the integrated value of the received intensity, the apparatus has a sign position estimating unit that estimates a marker position existing around the vehicle. Further, a map information acquisition unit that acquires map information, a self-position estimation unit that estimates a position of the vehicle on the map based on the sign position included in the map information and the sign position estimated by the sign position estimation unit Prepare.

  According to the present invention, the sign position existing around the vehicle is estimated from the area and intensity distribution of the reflected light extracted from the captured image, and further on the map of the vehicle based on the sign position included in the map information. Therefore, it is possible to estimate the self-position of the vehicle with high accuracy.

It is a block diagram which shows the structure of the self-position estimation apparatus which concerns on one Embodiment of this invention. It is a flowchart which shows the treatment procedure of the self-position estimation of the self-position estimation apparatus which concerns on one Embodiment of this invention. It is a flowchart which shows the treatment procedure of the respreading process of the self-position estimation apparatus which concerns on one Embodiment of this invention. (A) shows the relationship between the vehicle position and the intensity distribution of the projected light, and (b) is a characteristic diagram showing the relationship between the angle of view of the imaging unit and the reflection coefficient. It is explanatory drawing which shows an example of the measurement image produced | generated by the respreading process. It is a characteristic diagram which shows the relationship between the x coordinate of a pixel, and reception intensity, (a) shows distribution of the reception intensity acquired in the reflected light extraction part, (b) is reception intensity obtained by the respreading process The distribution of. It is explanatory drawing which shows typically the process which multiplies the measured image and the pixel value of a some reference image, calculates these addition values, and selects the reference image with the largest addition value.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings. FIG. 1 is a block diagram showing a configuration of a self-position estimation apparatus 100 according to an embodiment of the present invention. The self-position estimation apparatus 100 according to the present embodiment detects the sign positions existing around the vehicle with high accuracy, and collates the detected sign positions with the sign positions included in the map information. Estimate the vehicle's self-position. In addition, the “signs” shown below are concepts including those provided on the road side such as traffic signs and those provided on the road surface such as a reflector provided together with Botsdots.

  As shown in FIG. 1, the self-position estimation apparatus 100 according to the present embodiment includes a light projecting unit 11, an imaging unit 12, a reflected light extraction unit 13, and a marker position estimation unit 14. Furthermore, a GPS 17 that acquires vehicle position information, a reference image generation unit 16, and a self-position estimation unit 15 are provided.

  The light projecting unit 11 is mounted at an appropriate position of the vehicle, and projects flashing light that repeatedly turns on and off at a predetermined cycle toward the periphery of the vehicle.

  The imaging unit 12 is, for example, a CCD camera or a CMOS camera mounted on the vehicle, and images light that is irradiated from the light projecting unit 11 and reflected by a reflecting object P such as a sign existing around the vehicle.

  The reflected light extraction unit 13 acquires an image captured by the imaging unit 12 in synchronization with the cycle of the blinking light projected by the light projecting unit 11. And only the reflected light of the light projected by the light projecting unit 11 is extracted by calculating the difference between the image when the light projecting unit 11 is turned on and the image when the light projecting unit 11 is turned off.

  Based on the reflected light detected by the reflected light extraction unit 13, the marker position estimation unit 14 measures the position, region, and reflected light intensity of the marker in the image captured by the imaging unit 12. And the normalization process mentioned later is implemented and the image (henceforth "measurement image") containing a label | marker is produced | generated. That is, the sign position estimating unit 14 has a function of estimating the sign position existing around the vehicle from the distribution of the area and intensity of the reflected light extracted by the reflected light extracting unit 13.

  The reference image generation unit 16 is based on the current position information of the vehicle acquired from the GPS 17 and the map information including the marker positions existing around the vehicle, and the vehicle surroundings captured by the imaging unit 12 from a plurality of predicted vehicle postures. Are generated as reference images. At this time, the plurality of reference images are generated so that the x, y, and z coordinates, the yaw, the roll, and the pitch are different from each other. As a result, a plurality of reference images that are expected to be captured by the imaging unit 12 are generated. That is, the reference image generation unit 16 has a function as a map information acquisition unit that acquires map information including sign positions existing around the vehicle, and generates a plurality of reference images based on the map information.

  The self-position estimation unit 15 executes a matching process between the plurality of reference images generated by the reference image generation unit 16 and the measurement image generated by the marker position estimation unit 14, and determines the degree of coincidence between the images. Calculate. Then, the reference image having the highest degree of coincidence is selected. Further, the vehicle position on the map and the attitude angle of the vehicle are estimated based on the x, y, z coordinates set in the selected reference image, and the yaw, roll, and pitch. That is, the self-position estimating unit 15 has a function of estimating the position of the vehicle on the map based on the sign position included in the map information and the sign position estimated by the sign position estimating unit 14.

  In the above matching process, for example, as shown in FIG. 7, multiplication of each pixel of the measurement image M and each pixel of the reference image N is performed by the multiplier 31, and each multiplication result is added. Then, the addition values for the respective reference images N are compared, and it is determined that the reference image N having the largest addition value is the image having the highest degree of coincidence with the measurement image. Details will be described later.

  Here, the reflected light extraction unit 13, the sign position estimation unit 14, the self-position estimation unit 15, and the reference image generation unit 16 illustrated in FIG. 1 are, for example, a central processing unit (CPU), a RAM, a ROM, a hard disk, or the like. It can be configured as an integrated computer comprising storage means.

  Next, the operation of the self-position estimation apparatus 100 according to the present embodiment will be described with reference to the flowcharts shown in FIGS. First, in step S11 of FIG. 2, flashing light is projected from the light projecting unit 11 around the vehicle. In step S12, the imaging unit 12 images the surroundings of the vehicle. The captured image includes light that is irradiated with blinking light and reflected by a sign (reflector).

  In step S <b> 13, the reflected light extraction unit 13 calculates a difference between an image captured by the imaging unit 12 when the light projecting unit 11 is lit and an image captured by the imaging unit 12 when the light projecting unit 11 is turned off. Calculate. As a result, the reception intensity of each pixel of the image captured by the imaging unit 12 is obtained. The marker position estimation unit 14 acquires the reception intensity.

  In step S14, the sign position estimation unit 14 sets an area having a reception intensity equal to or higher than a certain level as an area where a sign exists. For example, when a square sign is present in front of the vehicle, the light projected from the light projecting unit 11 is reflected by the sign and is imaged by the imaging unit 12, so that an image area corresponding to the sign is displayed. As a result, the reception strength exceeds a certain level. Therefore, this image area is set as an area where a sign exists. As a result, as shown in FIG. 5, for example, two image regions R1 and R2 are set in the image captured by the imaging unit 12.

  In step S15, the marker position estimation unit 14 calculates the center coordinates, the integrated value of the received intensity, the area, and the distribution of the received intensity set in the process of step S14. Specifically, for the image region R1 shown in FIG. 5, the center coordinates (x1, y1), the received intensity integral value ΣA1, and the area D1 are calculated, and further, the received intensity distribution is calculated. Further, for the image region R2, the center coordinates (x2, y2), the received intensity integral value ΣA2, and the area D2 are calculated, and further, the received intensity distribution is calculated. The areas D1 and D2 can be calculated from, for example, the number of pixels equal to or greater than a predetermined reception intensity value set in advance.

  In step S <b> 16, the sign position estimation unit 14 performs a re-spreading process of the received intensity information based on each information calculated in the process of step S <b> 15, and a direction orthogonal to the traveling direction of the vehicle V <b> 1 (image horizontal direction). The variation of the light intensity with respect to (direction) is corrected. Hereinafter, the reason for executing the respreading process will be described with reference to the explanatory diagram shown in FIG.

  FIG. 4 (a) shows a situation in which the vehicle V1 is traveling on the traveling path X1, the sign Q2 is present on the right front side of the traveling path X1, and the sign Q1 is further on the far side. When blinking light is projected from the vehicle V1, the intensity of the light is highest at the front of the vehicle V1 and gradually decreases in the left-right direction, as shown by the curve q1. Accordingly, the reception intensity of the reflected light is affected by this characteristic.

  For example, as shown in FIG. 4B, for the sign Q1, there is no significant change in the light reflection coefficient with respect to the change in the angle of view. That is, there is no great difference between the reflection coefficient shown at the left end p1 of the sign Q1 and the reflection coefficient shown at the right end p2. On the other hand, for the sign Q2, the light reflection coefficient changes greatly with respect to the change in the angle of view. That is, the reflection coefficient shown at the right end p4 is significantly lower than the reflection coefficient shown at the left end p3 of the sign Q2. Therefore, when the light reflected by the sign Q2 is picked up by the image pickup unit 12, the reception intensity is greatly biased at each pixel in the image area, and the sign position cannot be accurately estimated.

  Therefore, in step S16 in FIG. 2, the reception intensity re-spreading process is executed to correct the variation in the light intensity in the direction orthogonal to the traveling direction of the vehicle V1 (the image lateral direction). Hereinafter, with reference to the flowchart shown in FIG. 3, a detailed processing procedure of the respreading process will be described. Note that the signs Q1 and Q2 shown in FIG. 4A correspond to the image areas R1 and R2 shown in FIG. 5, respectively. Here, the re-spreading process for the image region R2 illustrated in FIG. 5 will be described as an example.

  First, in step S31 of FIG. 3, the marker position estimation unit 14 acquires the integral value ΣA2 of the reception intensity of the image region R2 set in step S14 of FIG.

  In step S32, the sign position estimation unit 14 acquires the barycentric coordinates Xc (see FIG. 6B) of the reception intensity based on the distribution of the reception intensity of each pixel in the image region R2.

In step S33, the sign position estimation unit 14 respreads the reception intensity so that the image vertical direction is a constant value and the image horizontal direction is a normal distribution image. That is, the signal intensity detected in the image region R2 has a bias in the image region R2, as described with reference to FIG. Specifically, as shown in FIG. 6A, the reception intensity on the left side of the image region R2 is large, and the reception intensity decreases as it moves to the right side. The marker position estimation unit 14 normalizes the reception intensity and respreads the reception intensity on each pixel under the condition that the integral value ΣA2 of the reception intensity in the image region R2 is constant. Specifically, the probability density function f (x) is calculated by the following equation (1) using the average value μ and the standard deviation σ.

  In step S34, the marker position estimation unit 14 generates an image after re-scattering based on the probability density function f (x). In the image obtained by the respreading, fluctuations in reception intensity due to variations in the light distribution of the light projecting unit 11 are made uniform. That is, as shown in FIG. 6B, the image is converted into a measurement image having a reception intensity distribution having a normal distribution.

  Thereafter, in step S35, the self-position estimation unit 15 compares the measurement image calculated in the process of step S34 with the plurality of reference images generated by the reference image generation unit 16, thereby determining the self-position of the vehicle. presume. Specifically, as shown in FIG. 7, the multiplier 31 performs multiplication of each pixel included in the measurement image M and each pixel included in the reference image N. Further, each multiplication result is added to obtain an added value. This calculation is performed for all the reference images N.

  The higher the degree of coincidence between the measurement image M and the reference image N, the larger the added value. Therefore, the reference image having the largest added value is selected from the reference images N, and the x, y, z coordinates, yaw, pitch, and roll of the selected reference image are acquired. Based on the acquired x, y, z coordinates and the yaw, pitch, and roll, the position of the vehicle on the map can be estimated with high accuracy.

  Thus, in the self-position estimation apparatus 100 according to the present embodiment, based on the self-position of the vehicle detected by the GPS 17, an image expected to be captured at this self-position is obtained for each of x, y, and z. A plurality of coordinates, yaw, pitch, and roll are appropriately changed to create a reference image. Then, multiplication of each pixel is performed between each reference image image and the measurement image obtained by the redispersion process, and each multiplication value is added. An image having a high degree of coincidence of images has a large added value. Therefore, it can be determined that the reference image having the highest added value between the plurality of reference images and the measurement image is a reference image having a high degree of coincidence with the measurement image. Therefore, the self-position of the vehicle is estimated based on the x, y, and z coordinates of the reference image determined to have the highest degree of coincidence, and the yaw, pitch, and roll. Thus, the self-position of the vehicle is estimated with high accuracy.

  In other words, when detecting signs on the road of the vehicle and estimating the vehicle position on the map, many signs use reflective material, and when using a blinking light source, its shape and received intensity Information varies with reflectivity and distance. At that time, the reception intensity of the reflected light is easily affected by the distance, the inclination of the reflecting object, the presence or absence of optical regular reflection, and the like when the multiplication of the measurement image and a plurality of reference images is performed. May be an error factor.

  In this embodiment, the pixel value of the image obtained by normal distribution correction of the reflection intensity information on the reflecting surface of the sign is multiplied by the pixel value of the reference image generated from the own vehicle candidate position. The total value of matching increases near the center of gravity of the reflector. Therefore, the position information can be specified without depending on the shape of the reflector.

  Here, in the above-described embodiment, a reference image in which the x, y, and z coordinates, the yaw, the pitch, and the roll are appropriately changed is created, and the reference image having the largest added value is specified by matching processing with the reference image. Although the example to do was demonstrated, a matching process can also be performed using the particle filter which is a well-known technique.

  That is, the self-position estimation unit 15 shown in FIG. 1 sets a predetermined particle presence distribution range using a particle filter. Then, the particle filter is dispersed within the set presence distribution range, and the sign position included in the map information and the sign position estimation unit 14 are estimated based on a plurality of vehicle postures predicted by each particle filter. It is also possible to estimate the position of the vehicle on the map and the attitude angle by performing matching processing with the sign position.

  In this way, the self-position estimation apparatus 100 according to the present embodiment normalizes the reflection intensity information of the reflecting surface, and performs the matching process with the reference image using the normalized measurement image. Therefore, it is possible to generate a measurement image with high accuracy without being affected by optical characteristics such as the shape of the sign and the reflectance, and it is possible to estimate the self-position with high accuracy.

  In addition, the light projecting unit 11 emits flashing light that is switched on and off periodically, and from the image when the flashing light is turned on and the image when the flashing light is turned off, from the reflector. The reflected light of is detected. Therefore, the influence of other light sources and the like can be reduced, and the marker position can be detected with high accuracy.

  Further, the marker position estimation unit 14 detects the marker position based on the area of the reflected light extracted in the horizontal direction and the distribution of the reflection intensity in the extracted region. In other words, when detecting the intensity distribution, it is possible to measure the position with high accuracy from the integrated values of the area and intensity by suppressing the influence of the intensity distribution by evaluating the horizontal direction where the influence of the optical positional relationship is large. Is possible.

  In the above-described embodiment, a plurality of reference images having different x, y, and z coordinates, yaw, pitch, and roll are generated, and the degree of coincidence between each reference image and the measurement image is obtained to determine the self-position of the vehicle. Although an example of estimating the above has been described, it is also possible to use a particle filter.

  Furthermore, when the degree of coincidence with the measurement image is obtained using a particle filter, when performing a search within a predetermined range, the reflection intensity is included in the weighting to search for a reflector that can obtain a stronger reflection. A corresponding self-position measurement result is obtained. Therefore, it is possible to further improve the self-position estimation accuracy.

[Description of modification]
In the above-described embodiment, the example in which the blinking light is projected to detect the area where the sign is present has been described. However, the light that is periodically changed is projected, and the light that is synchronized with the period is captured from the captured image. It is also possible to use a synchronous detection process that extracts a signal to generate a synchronous image. Specifically, a signal transmitted from the light projecting unit 11 is S (t), and this is modulated with sin (ωt). Further, the image data captured by the imaging unit 12 is multiplied by sin (ωt). Then, the signal S (t) can be extracted from the relationship of the following equation (2) by removing the high-frequency component with a low-pass filter.
S (t) * sin (ωt) * sin (ωt) = S (t) * (1-cos (2ωt)) / 2 (2)
Since the synchronous detection method is a well-known technique, detailed description thereof is omitted. Then, by using the synchronous detection process, it is possible to estimate the marker position with higher accuracy.

  That is, the light projecting unit 11 projects light whose luminance periodically changes, and the reflected light extracting unit 13 extracts reflected light synchronized with the periodic change in luminance from the image acquired by the imaging unit 12. To do. And a synchronous image is produced | generated based on this reflected light, and this synchronous image is used as a measurement image. Therefore, in order to accurately measure the intensity of the reflected light, a synchronous detection circuit is provided to detect the periodic modulation of the reflected light in synchronization with the light projecting unit 11 and observe the amplitude intensity. It is possible to detect an existing marker position with higher accuracy.

  The self-position estimation apparatus and self-position estimation method of the present invention have been described based on the illustrated embodiment. However, the present invention is not limited to this, and the configuration of each unit is an arbitrary function having the same function. It can be replaced with that of the configuration.

DESCRIPTION OF SYMBOLS 11 Light projection part 12 Imaging part 13 Reflected light extraction part 14 Mark position estimation part 15 Self-position estimation part 16 Reference image generation part 17 GPS
31 Multiplier 100 Self-position estimation device

Claims (5)

A light projecting unit mounted on the vehicle and projecting light around the vehicle;
An imaging unit that is mounted on a vehicle and that images a vehicle periphery including an area where light is projected from the light projecting unit;
A reflected light extraction unit that extracts reflected light of light projected from the light projecting unit from an image captured by the imaging unit;
Based on the reflected light extracted by the reflected light extraction unit , an image area of a reflecting object is set from the image captured by the imaging unit, and the area of the image area, the center coordinates, the distribution of received intensity, and the reception A sign position estimator for estimating a sign position existing around the vehicle based on the integrated value of the intensity ;
A map information acquisition unit for acquiring map information including sign positions existing around the vehicle;
A self-position estimating unit that estimates a position of the vehicle on the map based on the sign position included in the map information and the sign position estimated by the sign position estimating unit;
A self-position estimation apparatus comprising: The light projecting unit projects light whose luminance periodically changes,
The self-position estimation apparatus according to claim 1, wherein the reflected light extraction unit extracts reflected light that is synchronized with a periodic change in luminance from an image acquired by the imaging unit. 3. The self-position estimation according to claim 2, wherein the sign position estimation unit detects the sign position based on an area of reflected light extracted in a horizontal direction and a distribution of reflection intensity in the extracted region. apparatus. The self-position estimation unit sets a predetermined particle presence distribution range using a particle filter,
Particles dispersed within a set presence distribution range, a plurality of vehicle postures predicted from the positional relationship of the particles after the vehicle has advanced for a predetermined time, a marker position included in map information, and the marker position estimation unit The self-position estimation apparatus according to any one of claims 1 to 3, wherein a position and a posture angle of the vehicle on the map are estimated based on the sign position estimated in (4). Projecting light around the vehicle;
Imaging a vehicle periphery including a region where the light is projected;
Extracting reflected light of the light from an image around the vehicle;
Based on the extracted reflected light, to set the image area of the reflector from the vehicle around the image, the area of the image region, the center coordinates, the reception intensity distribution, and based on the integrated value of the reception intensity, vehicle Estimating a surrounding marker position;
Obtaining map information including sign positions present around the vehicle;
Estimating the position of the vehicle on the map based on the sign position included in the map information and the estimated sign position;
A self-position estimation method comprising:

Images