JP2019003606A - Map change point detection device - Google Patents

Abstract

To provide a map change point detection device which can accurately detect the change point even using a camera with the low resolution.SOLUTION: A map change point detection device includes: a camera 110 which images the periphery of a vehicle 1; an overhead view conversion unit which converts the image photographed by the camera 110 into an overhead view image; a map storage unit 150 which stores a highly accurate map 151 including a road surface map; and a collation processing unit 164 which decides whether or not the change point being position where the change occurs on the actual road surface exists with respect to the road surface map by collating the overhead view image with the road surface map. The map change point detection device can have the image in which the road width is the constant width like the actual road regardless of the distance from a vehicle 1 by converting the image photographed by the camera 110 into the overhead view image, remove various objects such as buildings and trees that exist in the place which is not the road surface, and accurately detect the change point by making the white line intensity image generated from the overhead view image the image to be collated with the road surface map being a portion of the highly accurate map 151.SELECTED DRAWING: Figure 1

Description

  The present invention relates to a map change point detection device for detecting a change point of a road map.

  A technique for measuring a white line position with high accuracy by running a measurement carriage is known. The measurement cart described in Patent Literature 1 includes an odometry device, three GPS (Global Positioning System), three gyro sensors, a laser radar, a camera, and a computer. The white line position measured by the measuring carriage is used for a high-precision map. On the high-precision map, road markings such as white lines are accurately drawn. Patent Document 2 is a document disclosing a top view conversion process.

JP 2007-218705 A JP 2009-212734 A

  New roads may be opened. If a new road opens, the high-precision map will need to be updated. In addition, even on existing roads, there may be changes that require updating the high-precision map. For example, the rightmost lane is changed to a right turn lane.

  The high-precision map is preferably updated promptly in response to actual road changes. In the following, a change point is a position where a change has occurred in an actual road with respect to the road shown in the high-precision map.

  A change point can be detected by frequently running the measurement truck and comparing the prepared high-accuracy map with the map obtained by the measurement truck. However, the measurement cart is expensive because it includes the various devices described above, and it is difficult to prepare a large number of vehicles and frequently travel on various roads. That is, it is difficult to quickly detect the change point using the measurement carriage.

  In general vehicles, devices equipped with cameras such as drive recorders are increasingly installed. Therefore, it is conceivable to detect a change point using an image captured by a camera mounted on a general vehicle. However, it is not easy to detect a change point using an image captured by a camera mounted on a general vehicle. The reason is as follows.

  Even if the road shown in the image captured by the camera is actually the same road width, it will appear narrower as the distance increases. Therefore, the image captured by the camera cannot be simply compared with the image of the road portion drawn on the high-precision map.

  In addition, a camera mounted on a general vehicle is provided to capture an obstacle existing around the vehicle on which the camera is mounted, such as another vehicle. Various obstacles that are not on the road are shown. In order to detect the change point with high accuracy, it is preferable to remove an obstacle unnecessary for detection of the change point from the image captured by the camera.

  However, the resolution of a camera mounted on a general vehicle is not higher than that mounted on a measurement vehicle. For this reason, there is a possibility that the obstacle cannot be detected and removed from the image with high accuracy.

  For these reasons, it has not been easy to accurately detect a change point using an image captured by a low resolution camera mounted on a general vehicle.

  The present disclosure has been made based on this situation, and an object of the present disclosure is to provide a map change point detection device that can detect a change point with high accuracy even when a camera with low resolution is used. It is in.

  The above object is achieved by a combination of the features described in the independent claims, and the subclaims define further advantageous embodiments. Reference numerals in parentheses described in the claims indicate a correspondence relationship with specific means described in the embodiments described later as one aspect, and do not limit the technical scope.

One of the disclosures for achieving the above object is:
A camera (110) provided in the vehicle for imaging the periphery of the vehicle;
An overhead conversion unit (S6) that converts an image captured by the camera into an overhead image;
A map storage unit (150) for storing a road map (151) including a road surface map;
A map change provided with a matching processing unit (164) that collates the bird's-eye view image with the road surface map and determines whether or not there is a changing point that is a position where the actual road surface has changed with respect to the road surface map. It is a point detection device.

  This map change point detection device converts an image around a vehicle imaged by a camera into an overhead image. By converting to a bird's-eye view image, the road width becomes an image having a constant width as with an actual road regardless of the distance from the vehicle. Also, by converting to a bird's-eye view image, it is possible to remove various objects such as buildings and trees that exist in places other than the road surface. Since this bird's-eye view image is used as an image to be checked against a road map, a change point can be detected with high accuracy even if the resolution is low.

It is a block diagram which shows the structure of the road map information update system 100 of 1st Embodiment. It is a flowchart which shows the process which the image process part 162 of FIG. 1 performs. It is a figure which shows the front image 180 which the camera 110 images. It is a figure which shows the corrected front image 181 produced | generated by S2 of FIG. It is a figure which shows the relationship between real image height Y, ideal image height Y0, incident angle (theta), and the focal distance f. It is a figure which shows the relationship between each point (u, v) of the image before correction | amendment, and the orthogonal coordinate (u0, v0) of an ideal image. It is a figure which shows the front image. It is a figure which shows the distortion correction image 185. FIG. It is a figure explaining the image before distortion correction. It is a figure explaining the image after volume distortion correction. It is a figure which shows the relationship between the division line 182 and the vanishing point 183. FIG. It is a figure which shows the bird's-eye view image before removing an obstruction. It is a figure which shows the bird's-eye view image after removing an obstruction. It is a flowchart which shows the process which the collation area | region specific | specification part 163 performs. It is a figure explaining the method of calculating the white line intensity | strength G. FIG. It is a figure explaining the method of detecting the white line center from the white line intensity | strength G. FIG. It is a figure which shows the relationship between the pattern of the detected line segment, and a travel lane. It is a flowchart which shows the process which the collation process part 164 performs. It is a figure which shows the high precision map before carrying out distance conversion. It is a figure which shows the high precision map after carrying out distance conversion. It is a figure which shows the expansion / contraction process performed in S22 of FIG. It is a figure which shows the projective transformation performed in S22 of FIG. It is a figure explaining the scoring performed by S23 of FIG. It is a block diagram which shows the structure of the road map information update system 200 of 2nd Embodiment. It is a flowchart which shows the process which the map production | generation part 265 of FIG. 20 performs. It is a figure which shows the bird's-eye view image for collation processing produced | generated by the image process part. It is a figure explaining the image extracted by S31 of FIG. 21 from FIG. It is a flowchart which shows the process which the image process part 162 performs in 3rd Embodiment. It is a flowchart which shows the detailed process of S4A of FIG. It is a vanishing point region image 386 one frame before. It is the vanishing point region image 386 obtained in the latest frame. It is a figure explaining pitch variation | change_quantity (delta) y. It is a bird's-eye view image without pitch correction. It is the bird's-eye view image which performed pitch correction. It is a flowchart which shows the process which the collation process part 164 performs in 4th Embodiment. It is a figure which shows the relationship between the first collation area | region 401 and the partial area 402 of the 1st time. It is a figure which shows the relationship between the partial area 402a of the 1st time, and the partial area 403 of the 2nd time.

  Hereinafter, embodiments will be described with reference to the drawings. FIG. 1 is a block diagram showing a configuration of a road map information update system 100 having a function as a map change point detection device.

[overall structure]
The road map information update system 100 includes a camera 110, a parameter storage unit 120, a GPS receiver 130, a behavior detection sensor 140, a map storage unit 150, a calculation unit 160, and an accumulated data storage unit 170. The road map information update system 100 is mounted on the vehicle 1.

  The camera 110 is attached in such a direction that the front of the vehicle 1 can be imaged, and images the front of the vehicle 1 in the periphery of the vehicle 1. Hereinafter, an image captured by the camera 110 is referred to as a front image 180. FIG. 3 shows an example of the front image 180.

  In the present embodiment, the camera 110 is a camera included in the drive recorder. If the vehicle 1 is equipped with a drive recorder, the road map information update system 100 does not need to include the camera 110 separately. Since the camera 110 is included in the drive recorder, the camera 110 includes a wide-angle lens and has a wide angle of view.

  The parameter storage unit 120 stores parameters of the camera 110 necessary for image processing. Specifically, the parameters are the focal length f of the camera 110, the yaw angle rx and the tilt angle ry of the camera optical axis. The focal length f of the camera 110 is a value determined by the product and is stored in the parameter storage unit 120 in advance.

  On the other hand, the yaw angle rx and tilt angle ry of the camera optical axis vary depending on the mounting posture of the camera 110. Therefore, in the present embodiment, the front image 180 is analyzed, and the yaw angle rx and the tilt angle ry are calculated. Then, the calculation result is stored in the parameter storage unit 120. In the parameter storage unit 120, a writable storage unit is used so that the calculation result can be stored.

  The GPS receiver 130 receives a navigation signal transmitted from a navigation satellite provided in a GPS (Global Positioning System) which is one of satellite navigation systems, and sequentially calculates a current position based on the received navigation signal.

  The behavior detection sensor 140 is a sensor that detects the behavior of the vehicle 1. An acceleration sensor, a vehicle speed sensor, and a gyro sensor are specific examples of the behavior detection sensor 140.

  The map storage unit 150 stores a high-precision map 151. The high-accuracy map 151 is a map measured by running a measurement carriage, and road markings such as road lane markings are shown in a detailed state like an image. The high-precision map 151 also shows details of three-dimensional objects above the road surface, such as utility poles, guardrails, and road signs. For convenience of the following description, the road surface portion of the high-precision map 151 is referred to as a road surface map. The high-accuracy map 151 including a road surface map corresponds to a road map.

  The arithmetic unit 160 is a computer including a CPU, a ROM, a RAM, and the like. The CPU uses a temporary storage function of the RAM, and the non-transitory tangible storage medium such as a ROM. Run the stored program. Accordingly, the calculation unit 160 functions as a vehicle position detection unit 161, an image processing unit 162, a collation area specifying unit 163, and a collation processing unit 164. When these functions are executed, a method corresponding to the program is executed.

  Note that some or all of the functional blocks included in the arithmetic unit 160 may be realized using one or a plurality of ICs (in other words, as hardware). In addition, some or all of the functional blocks included in the calculation unit 160 may be realized by a combination of execution of software by the CPU and hardware members.

  The accumulated data storage unit 170 is a storage unit that accumulates the change points detected by the processing of the matching processing unit 164 and road information regarding the change points. The change point is a position where a change has occurred in an actual road with respect to the road shown in the high-accuracy map 151. The road information regarding the change point is information indicating what kind of change has occurred in the road at the change point. For example, a road marking character newly added to the road is road information regarding the change point.

  Next, the process of each part with which the calculating part 160 is provided is demonstrated.

[Processing of Vehicle Position Detection Unit 161]
The vehicle position detection unit 161 sequentially detects the current position of the vehicle 1 by hybrid navigation combining satellite navigation and self-contained navigation. The satellite navigation is a method of estimating the current position based on a signal supplied from the GPS receiver 130, and the self-contained navigation is a method of detecting a relative locus of the vehicle position based on the vehicle behavior detected by the behavior detection sensor 140. It is. However, the present invention is not limited to this, and the current position of the vehicle 1 may be sequentially detected only by satellite navigation. The vehicle position detection unit 161 outputs the detected current position to the image processing unit 162.

[Processing of Image Processing Unit 162]
The image processing unit 162 processes the front image 180 sequentially captured by the camera 110 to generate an overhead image. The processing executed by the image processing unit 162 will be described with reference to the flowchart shown in FIG. The image processing unit 162 periodically executes the process shown in FIG. 2 in the activated state of the road map information update system 100.

  In step (hereinafter, step is omitted) S1, the front image 180 captured by the camera 110 is input. In S2, a process of correcting the distortion of the front image 180 input in S1 is performed to generate a corrected front image 181 illustrated in FIG. S2 corresponds to a distortion correction processing unit.

  Since the front image 180 is captured by a wide-angle lens, a large distortion occurs. When the distortion is large, the detection accuracy of the change point is lowered. Therefore, the distortion of the front image 180 is corrected. The distortion here is a phenomenon in which the image is attracted to the center of the image in accordance with the incident angle to the camera 110 and is called distortion. In the case of a wide-angle lens, a barrel-shaped image is drawn toward the vertical center or the horizontal center of the image as the image corners are distorted.

Image projection methods include orthographic projection and equidistant projection, and distortion correction processing corresponding to the projection method is performed. In this embodiment, distortion correction processing corresponding to equidistant projection is performed. The equidistant projection relational expression is expressed by Expression 1. In Equation 1, Y is the real image height, Y0 is the ideal image, f is the focal length, and θ is the incident angle. FIG. 5 illustrates the relationship among these real image height Y, ideal image height Y0, incident angle θ, and focal length f.

When the orthogonal coordinates (u, v) of each point of the pre-correction image are expressed using the real image height Y and the angle φ in the polar coordinate system, Expression 2 is obtained.

Further, the orthogonal coordinates (u0, v0) of the ideal image are expressed by Equation 3.

  FIG. 6 shows the relationship between each point (u, v) of the pre-correction image, orthogonal coordinates (u0, v0) of the ideal image, and polar coordinates. In the distortion correction process, first, (u, v) is converted into polar coordinates (Y, φ) by Equation 2. Next, the polar coordinate Y is replaced with the ideal image height Y0 obtained from Equation 1. Thereafter, the orthogonal coordinates (u0, v0) are obtained from Equation 3.

  The image obtained by the processing so far is set as a distortion corrected image 185. FIG. 7 shows a front image 180 different from FIG. 3, and FIG. 8 shows a distortion corrected image 185 obtained by correcting the front image 180 of FIG.

  As can be seen from the relationship between Y and Y0 in FIG. 6, the distortion-corrected image 185 shown in FIG. 8 is an image in which a three-dimensional object projected at the end of the image is pulled in the end direction of the image. This is called a volume distortion image.

  In the distortion correction process, a process for correcting the volume distortion image is also performed. In order to correct the volume distortion image, as shown in FIG. 10, nonlinear transformation is performed so that the peripheral portion of the image is closer to the center of the image. This conversion is called volume distortion image correction. The volume distortion image correction is a correction for converting the straight line shown in FIG. 9 into a curve shown in Expression 48 of FIG. Further, by this correction, the size in the left-right direction of the image is reduced to the broken line shown in FIG. 10, and the reduction rate is higher at the left and right ends of the image.

  The image after the volume distortion image correction is the corrected forward image 181 shown in FIG. In the corrected front image 181, the roads and bridges are almost straight, and the building projected on the periphery of the image stands straight.

  In S3, a straight line is detected from the corrected front image 181. In order to detect a straight line, an edge detection process for detecting the density of the image is performed. The detected edge is indicated by a set of points. A straight line such as a road lane marking is detected by Hough transforming the set of edge points.

  In S4, the vanishing point 183 of the corrected forward image 181 is detected. S4 corresponds to a vanishing point detector. In order to detect the vanishing point 183, first, parallel lines extending near the center in the corrected front image 181 are extracted from the straight line detected in S3. As shown in FIG. 4, when four lane lines 182 can be detected as straight lines, for example, two lane lines 182 that divide the lane in which the vehicle 1 travels are used. A point where these two lane markings 182 intersect is defined as a vanishing point 183. FIG. 11 illustrates the relationship between the partition line 182 and the vanishing point 183.

  In S5, the orientation of the camera 110 is detected. S5 corresponds to a camera orientation detection unit. The direction of the camera 110 means the direction of the optical axis of the camera 110. Since the camera 110 is a camera included in the drive recorder, it is often attached after purchasing the vehicle, and the orientation of the camera 110 can be changed after the attachment. Therefore, the orientation of the camera 110 is different for each vehicle 1. Therefore, in S5, the orientation of the camera 110 is detected.

Specifically, the orientation of the camera 110 is a yaw angle rx and a tilt angle ry. The yaw angle rx and the tilt angle ry can be expressed by Expression 4 and Expression 5, respectively. In Expression 4, Δx is a distance in the x direction between the image center 184 and the vanishing point 183 in the corrected forward image 181 as shown in FIG. In Expression 5, Δy is the distance in the y direction between the image center 184 and the vanishing point 183 in the corrected forward image 181 as shown in FIG.

  If the orientation of the camera 110 can be detected in S5, next, in S6, a top view conversion process (that is, a bird's eye conversion) is performed on the corrected forward image 181 to generate an overhead image with a viewpoint above the vehicle 1. . S6 corresponds to an overhead conversion unit.

  The top view conversion process is a process of converting the optical axis of the camera 110 in a direction from above to below. The top view conversion process is described in Patent Document 2 and the like.

  When the corrected front image 181 is converted into a bird's-eye view image, a portion of the corrected front image 181 that can be seen when viewed from above the vehicle 1 is converted into a bird's-eye view image. That is, the road surface and its surroundings in the corrected front image 181 are converted into an overhead image. The range to be converted into the overhead image is determined based on the vanishing point 183 and the angle of view of the camera 110.

  In S7, the obstacle is removed from the overhead image generated in S6. S7 corresponds to an obstacle removal processing unit. By making the bird's-eye view image, of the obstacles that appear in the corrected front image 181, the obstacles that appear outside the range to be converted into the bird's-eye view image are removed. As a result, many of the obstacles that cause a decrease in detection accuracy of change points can be removed. However, there are cases where the bird's-eye view image includes objects other than road installations such as preceding vehicles and falling objects. Therefore, a process of removing the obstacle from the overhead image is performed.

  The method of removing the obstacle is a method of extracting a specified color and making the other colors black. The specified colors are road markings that you do not want to remove and white and orange colors that correspond to road markings.

  FIG. 12 shows a bird's-eye view image of a preceding vehicle that is an obstacle, and FIG. 13 shows an image after the obstacle is removed by executing S7. By converting the preceding vehicle portion to black, the preceding vehicle is removed in the image of FIG. Obstacles smaller than a certain size may be regarded as having no hindrance in subsequent processing, and may be restored to the original color.

  In S8, the overall luminance of the overhead image is adjusted so that the highest luminance portion in the overhead image has white luminance. The reason for adjusting the brightness is that the brightness of the bird's-eye view image is affected by the surrounding environment during camera imaging, such as the weather, time zone, and tunnels. Because. The bird's-eye view image after the luminance adjustment in S8 is a bird's-eye view image for collation processing.

  In S <b> 9, the current position supplied from the vehicle position detection unit 161 and the overhead view image for verification processing obtained by the processing in S <b> 8 are output to the verification region specifying unit 163.

[Processing of collation area specifying unit 163]
The collation area specifying unit 163 specifies a collation area from the overhead image for the collation process generated by the image processing unit 162. The collation area specifying unit 163 detects road lane markings from the overhead image for collation processing, and determines the range of the collation area in the vehicle width direction based on the number of road lane markings.

  FIG. 14 shows processing executed by the matching area specifying unit 163. Detailed processing of the collation area specifying unit 163 will be described with reference to FIG. The collation area specifying unit 163 executes the process shown in FIG. 14 when a bird's-eye view image for collation processing is generated.

  In FIG. 14, white line detection processing is executed in S11. In the white line detection processing, the following filter processing is performed on each pixel of the overhead image to digitize the white line likelihood. This value is the white line intensity G.

  The white line intensity G is a value obtained by dividing the sum of the pixel values within the range of the assumed white line width S around the target pixel (m, n) by the area. The target pixel (m, n) is a pixel for calculating the white line intensity G, and the white line intensity G is sequentially calculated for all the pixels.

The white line intensity G is expressed by Equation 6, Equation 7, and Equation 8. Equation 6 shows the left white line intensity _GL . The numerator of the first term on the right side of Equation 6 is the sum of the pixel values within the range of the assumed white line width S, and the numerator of the second term on the right side of Equation 6 is the sum of the out-of-range pixel values on the left side of the assumed white line width S. It is.

Equation 7 shows a right side white line intensity G _R. The numerator of the first term on the right side of Equation 7 is the sum of the pixel values within the range of the assumed white line width S as in the first term on the right side of Equation 6. The numerator of the second term on the right side of Equation 7 is the sum of out-of-range pixel values on the right side of the assumed white line width S. FIG. 15 shows the range of total pixel values calculated by the first and second numerators on the right side of Equations 6 and 7.

  The clearer the white line on the image, the larger the value of the first term on the right side and the smaller the value of the second term on the right side in Equations 6 and 7. Therefore, Expression 6 and Expression 7 indicate the white line intensity G.

Equation 8, calculates the square root of sum of squares to put together a left white line intensity G _L and the right white line intensity G _R into a single value. In image processing, a method that uses the square sum of squares as a method of calculating edge strength is common.

  FIG. 15 shows the white line intensity G in the horizontal direction of the screen. In S11, in addition to the white line intensity G in the horizontal direction of the screen, the white line intensity G in the vertical direction of the screen is also calculated.

  After calculating the white line intensity G, the center of the white line is then detected. To detect the center of the white line, the white line strength G in the horizontal direction is searched for a portion where the intensity is one line in the vertical direction, and the white line strength G in the horizontal direction is searched for a portion where the intensity is one line in the vertical direction And FIG. 16 shows the white line intensity G in the horizontal direction line by line. The peak shown in FIG. 16 is the center of the white line.

  In S12, the travel lane is recognized. In the travel lane recognition process, a portion in which the white line center detected in S11 is continuous in the vertical, horizontal, and diagonal directions is grouped (that is, merged) into one line segment. Let the image after grouping be a white line intensity image. Further, the accumulated value of the white line intensities G included in the merged line segment is set as the merged line white line intensity G, and the line segment having the strongest merged white line intensity G is defined as the reference line.

  A line segment is searched in the vicinity of the assumed lane width positions on the left and right sides from the reference line. Line segments are searched up to the second on the left and right with reference to the vehicle position. Also, a solid line and a broken line are distinguished from each other by the length of the line segment.

  Then, the traveling lane is recognized from the detected line segment pattern according to the relationship shown in FIG. The information to be recognized is how many lanes the road on which the vehicle is traveling is, and on which travel lane the vehicle 1 is traveling.

  In S13, a collation area is extracted from the white line intensity image generated in S12. The region to be extracted is fixed in the vertical direction, that is, in the front-rear direction of the host vehicle, and the horizontal direction is variable. The range in the vertical direction of the matching area is a certain range from the end of the white line intensity image on the own vehicle side.

  The range in the horizontal direction of the verification area corresponds to the travel lane recognized in S12. For example, when the number of travel lanes recognized in S12 is three lanes, the entire white line intensity image is set as the width of the collation area. When the number of travel lanes recognized in S12 is two lanes, the center of the line segment at both ends is set as the center of the extraction region, and the assumed lane width + α for two lanes is set as the lateral range of the extraction region. The image extracted in S13 is a rectangular image. As a result, an overhead image having an appropriate size according to the number of road lane markings that can be detected from the overhead image can be used as the verification region. In addition, you may make it not use the part removed by the obstruction removal of S7, such as the part which the front vehicle and the parallel running vehicle are reflected, as a collation area | region.

  In S14, a matching target area is specified from the road surface map of the high-precision map 151. The matching target area is a fixed area determined with the current position as a reference. The vertical and horizontal sizes of the matching target area are set to be larger than the matching area of the white line intensity image so that the collation determination can be performed while shifting the matching area of the white line intensity image with respect to the matching target area. ing.

[Processing of collation processing unit 164]
The matching processing unit 164 detects a change point by matching the white line intensity image of the matching area extracted in S13 with the road surface map of the high-accuracy map 151 of the matching target area specified in S14. Detailed processing of the verification processing unit 164 will be described with reference to FIG. The collation processing unit 164 executes the process shown in FIG. 18 when the collation area and the matching target area are determined.

  In FIG. 18, in S21, distance conversion is performed on the high-accuracy map 151 of the matching target area specified in S14. The distance conversion here is the same as the distance conversion widely used in image processing, and the distance from 0 pixel to the nearest non-zero pixel is calculated for all 0 pixels, and the pixel of 0 pixel is calculated. A transformation that replaces a value with its distance.

  If the high-accuracy map 151 and the white line intensity image are matched as they are, the correlation value becomes low even if the lane widths are slightly different. Therefore, distance conversion is performed on the high-precision map 151 of the matching target area.

Equation 9 shows the equation for distance conversion. In Equation 9, the distance replaced by the left side, x is the horizontal distance to the non-zero pixel, y is the vertical distance to the non-zero pixel, and N is a preset coefficient.

  19 and 20 show maps before and after distance conversion. FIG. 19 shows a map before distance conversion, that is, a high-precision map 151. When the distance conversion is performed, the white line is blurred as shown in FIG.

  In S22, while changing the position of the matching area of the white line intensity image by a predetermined value, the correlation value between the matching target area of the high-accuracy map 151 after the distance conversion in S21 and the matching area of the white line intensity image is calculated. . A correlation value map can be created by calculating the correlation value while changing the position of the collation region of the white line intensity image by a predetermined value. The correlation value map represents the relationship between the relative position of the matching area with respect to the matching target area and the correlation value.

  In the calculation of the correlation value, if the interval between the road lane markings drawn on the white line intensity image does not match the interval between the road lane markings drawn on the high-precision map 151 after the distance conversion, the white line intensity Enlarge / reduce the image or projective transform the white line intensity image. FIG. 21 shows a diagram for enlarging and reducing the white line intensity image, and FIG. 22 shows a diagram for projective transformation of the white line intensity image. The overall size of the white line intensity image is adjusted so that the lane markings of the high-precision map 151 and the lane markings of the white line intensity image coincide with each other by such enlargement / reduction or projective transformation. S22 corresponds to a correlation value map creation unit.

  In S23, scoring is performed. Scoring is to quantify the probability of the position of the collation region of the white line intensity image. Specifically, in S23, first, a scoring map is created. The scoring map is a map in which the coefficient is set to 1 assuming that the current position of the vehicle 1 detected by the vehicle position detection unit 161 is most likely, and the coefficient decreases as the distance from the current position increases.

  As shown in FIG. 23, scoring is performed by multiplying this scoring map and the correlation value map created in S22. The position with the highest score is set as the matching position. S23 corresponds to a matching position determination unit.

  In S24, the white line intensity image is converted to gray scale, and the absolute value of the difference between the high accuracy map 151 corresponding to the position of the white line intensity image determined in S23 and the white line intensity image is obtained. If this absolute value is greater than or equal to a predetermined value, the change point is determined.

  Further, the absolute value of the difference between the high-precision map 151 and the white line intensity image is replaced with the HSV color space and output. Specifically, when replacing with the HSV space, S (saturation) and V (lightness) are fixed, and the value of H (hue) is converted. Also, the absolute value of the difference is normalized to 0-255. Let this normalized value be the value of H. Then, red and blue of H are exchanged so that blue is 0 and red is 255. As a result, a change point such as an increase or decrease in paint is represented by a color difference as road information.

  The change points detected in this way are accumulated in the accumulated data storage unit 170. The position of the change point is provided to the update information utilization system 2 that is an external system. The update information utilization system 2 plots change points on a map, generates a list, and manages change points, that is, road update information indicating that a road has been updated. Further, the road update information may be compared by period so that it can be confirmed in which period a new road is opened.

[Effect of the embodiment]
As described above, the road map information update system 100 converts the front image 180 captured by the camera 110 into an overhead image. By converting to a bird's-eye view image, the road width becomes an image having a constant width, similar to an actual road, regardless of the distance from the vehicle 1. Also, by converting to a bird's-eye view image, it is possible to remove various objects such as buildings and trees that exist in places other than the road surface.

  Since the white line intensity image generated from the bird's-eye view image is used as an image to be compared with a road surface map that is a part of the high-accuracy map 151, the change point can be detected with high accuracy even when the resolution of the camera 110 is low.

  In the present embodiment, the obstacle is removed from the overhead image by executing the process of S7. Thereby, a change point can be detected more accurately.

  Further, in this embodiment, the scoring map determined based on the current position measured by the vehicle position detection unit 161 without directly setting the position of the white line intensity image having the highest correlation value in the matching process as the matching position. The correlation value is corrected by Thereby, the change point detection accuracy is further improved.

Second Embodiment
Next, a second embodiment will be described. In the following description of the second embodiment, elements having the same reference numerals as those used so far are the same as elements having the same reference numerals in the previous embodiments unless otherwise specified. Further, when only a part of the configuration is described, the above-described embodiment can be applied to the other parts of the configuration.

  In FIG. 24, the structure of the road map information update system 200 of 2nd Embodiment is shown. The road map information update system 200 of the second embodiment is different from the road map information update system 100 of the first embodiment in that the calculation unit 260 includes a map generation unit 265. The accumulated data storage unit 170 stores a generated map image 171. The generated map image 171 is a map obtained by combining overhead images.

  When the collation processing unit 164 detects a change point, the map generation unit 265 generates a map reflecting the change point information. Detailed processing of the map generation unit 265 will be described with reference to FIG.

  In FIG. 25, in S31, an overhead image for superimposition is extracted. Specifically, an area corresponding to the collation area of the white line intensity image extracted in S13 is extracted from the overhead image for collation processing generated by the image processing unit 162. FIG. 26 shows a bird's-eye view image for collation processing generated by the image processing unit 162, and FIG. 27 shows a bird's-eye view image for superposition extracted in S31. Note that a portion where an object other than road surface information is reflected in the extracted overhead image is excluded from the use area. The image extracted in S31 is a map image.

  In S32, the position where the overhead image extracted in S31 is added to the generated map image 171 is determined. This position is the matching position determined in S23.

  In S33, an averaged image is generated for a portion where the image already exists in the generated map image 171 among the additional positions determined in S32. The averaged image is an image obtained by averaging an existing image and an image to be added this time. By averaging, the influence of a temporary change caused by an object other than the road surface such as a shadow of the vehicle 1 can be reduced.

  Further, at the time of averaging, as shown in FIG. 27, the weight is increased toward the side closer to the camera 110, and the averaging process is performed with the existing image. This is because the resolution of the image is higher as it is closer to the camera 110 due to the characteristics of the overhead view image.

  In S34, images are superimposed and the generated map image 171 is updated. At the time of superimposition, the averaged image generated in S33 is superimposed on the generated map image 171 for the portion where the image already exists in the generated map image 171. For portions where the image does not yet exist in the generated map image 171, the overhead image extracted in S31 is directly superimposed on the generated map image 171.

  Since the generated map image 171 that is sequentially updated in this way includes the image of the change point, the change at the change point can be recognized from the image. For this reason, since the change point can be recognized without running the measurement carriage, the update cost of the high-accuracy map 151 can be reduced. In addition, the updated content can be used as temporary information for vehicle control such as automatic driving.

<Third Embodiment>
In the third embodiment, the image processing unit 162 executes the process shown in FIG. In the process shown in FIG. 28, S4A is added between S4 and S5 with respect to the process shown in FIG. Further, S5A is executed instead of S5.

  In S4A, the pitch fluctuation amount of the corrected forward image 181 is calculated. The detailed process of S4A is shown in FIG. In FIG. 29, the vanishing point region image 386 is determined in S41. As illustrated in FIGS. 30 and 31, the vanishing point region image 386 is an image of a region that is determined based on the vanishing point in the corrected forward image 181. The vanishing point region image 386 shown in FIGS. 30 and 31 is a horizontally long rectangular region centered on the vanishing point 183 (see FIG. 11), with a pair of short sides being vertical and a pair of long sides being horizontal. .

  In S42, the vanishing point region image 386 obtained in the previous frame, that is, the vanishing point region image 386 determined in the previous execution of FIG. 28, and the vanishing point region image 386 obtained in S41 executed immediately before S42 are obtained. A matching process for matching is performed. 30 is a vanishing point region image 386 obtained in the previous frame, and FIG. 31 is a vanishing point region image 386 obtained in the current frame. In the matching process, the vanishing point region image 386 obtained in the current frame is moved on the basis of the vanishing point region image 386 obtained in the previous frame, and the two vanishing point region images 386 are most closely matched. The vanishing point region image 386 obtained in the frame is moved.

  In S43, the movement amount in S42 of the vanishing point region image 386 obtained in the current frame is calculated. This amount of movement is used as a value that means pitch fluctuation that has occurred in the vehicle 1 during one frame. S41 to S43 so far correspond to a movement amount calculation unit.

  In S44, the movement amounts calculated in S43 are sequentially integrated in the processes of FIGS. 28 and 29 that are repeatedly executed each time the front image 180 is input. Hereinafter, the integrated value is referred to as a positional deviation integrated value. The integrated value of the position shift includes a position shift of the vanishing point region image 386 due to pitch variation and a change in the position of the vanishing point region image 386 due to a static reason other than the pitch variation. The static reason other than the pitch fluctuation is, for example, a change in the camera optical axis due to the vehicle 1 traveling uphill. S44 corresponds to an integral value calculation unit.

  In S45, a moving average value of the positional deviation integrated value calculated in S44 is calculated. The section for moving average is set in advance. The moving average value is used to indicate the position of the vanishing point region image 386 when the vehicle is stationary. The positional deviation integrated value includes a change in value due to a static reason other than pitch fluctuation. The moving average value is used to calculate a value resulting from this static reason, and the moving average section is set to a section in which a value resulting from this static reason can be calculated.

  In S46, the difference between the displacement integrated value calculated in S44 and the moving average value calculated in S45 is calculated. This difference is defined as a pitch fluctuation amount δy. Whether the pitch fluctuation amount δy is set to the positional deviation integral value−moving average value or the moving average value−position deviation integral value may be any as long as it is determined in advance. S45 and S46 correspond to the pitch fluctuation amount calculation unit. FIG. 32 shows the positional deviation integrated value, moving average value, and pitch fluctuation amount δy.

After executing S46, the process proceeds to S5A in FIG. In S5A, the orientation of the camera 110 is detected. The direction of the camera 110 is specifically the yaw angle rx and the tilt angle ry as described in the first embodiment. The yaw angle rx is obtained from Equation 4 described in the first embodiment. The tilt angle ry is obtained from Equation 10. As shown in Expression 10, the yaw angle rx is calculated from the distance Δy in the y direction between the image center 184 and the vanishing point 183 and the pitch fluctuation amount δy.

  The processing after executing S5A is the same as in the first embodiment. Steps S <b> 6 to S <b> 9 are executed, and the bird's-eye view image for matching processing and the current position are output to the matching region specifying unit 163.

[Effect of pitch correction]
FIG. 33 shows an overhead image without pitch correction, and FIG. 34 shows an overhead image with pitch correction. The bird's-eye view image after the pitch correction is the bird's-eye view image obtained by executing FIG. In the bird's-eye view image that has not been pitch-corrected, the interval between the white lines increases from the lower side toward the upper side. On the other hand, in the bird's-eye view image that has been subjected to pitch correction, the white line intervals are substantially parallel.

  By correcting the pitch in this way, the white line intervals in the overhead image can be made parallel. Therefore, the accuracy of subsequent processing performed based on this overhead view image is improved.

<Fourth embodiment>
Next, a fourth embodiment will be described. In the first embodiment, the matching processing unit 164 separately determines the matching position for each frame. On the other hand, in the fourth embodiment, the matching position is finally determined for each frame, but as the preprocessing, the matching position is determined for an image obtained by combining a plurality of frames. Thereafter, the image for determining the matching position is sequentially reduced. This will be described in detail below.

  In the fourth embodiment, as described above, a matching position is determined for an image obtained by combining a plurality of frames. Therefore, the image processing unit 162 executes the process when the front image 180 for a plurality of frames is input. The contents of the processing executed by the image processing unit 162 in the fourth embodiment are the same as those in the previous embodiments, except that the front image 180 to be processed is for a plurality of frames.

  The number of the plurality of frames can be set as appropriate. For example, it can be the total number of frames recorded in one event in the drive recorder.

  The collation area specifying unit 163 specifies a collation processing area for all of the collation processing overhead images generated by the image processing unit 162. In the fourth embodiment, the contents of the processing executed by the matching processing unit 164 are the same as those in the previous embodiments except that the overhead image to be processed is a plurality of frames.

  In the fourth embodiment, the collation processing unit 164 executes the process shown in FIG. The collation processing unit 164 executes the process shown in FIG. 35 when the collation area specifying unit 163 specifies the collation area.

  In S51, the first collation area 401 (see FIG. 36) is determined. The first matching area 401 is an area obtained by combining matching areas of white line intensity images for a plurality of frames. The first collation area 401 shown in FIG. 36 is an area obtained by combining the collation areas of the white line intensity images respectively determined from all the frames included in the image data for one event of the drive recorder.

  In S52, the same processing as S21 of FIG. 18 is performed on the matching target area of the high-precision map determined based on the first matching area 401 determined in S51. As described in S <b> 14 of FIG. 14, the matching area specifying unit 163 determines a matching target area from the road map of the high-precision map 151 based on the matching area. That is, the matching area specifying unit 163 determines the matching target area of the road map for the matching area for one frame. Therefore, in S52, the matching target area of the road surface map for the matching area for one frame determined by the matching area specifying unit 163 is added to all frames included in the first matching area 401 determined in S51. The distance conversion is performed on it.

  In S53, the collation areas of all the white line intensity images included in the first collation area 401 determined in S51 are overlaid. The process of superimposing is the same process as the map generation unit 265 described in the second embodiment.

  In S54, matching processing is performed on the collation region, and a correlation value map is created. The collation area is the first collation area 401 determined in S51 if it is the first execution of S54 after the execution of S51 to S53. The subsequent collation area is determined by executing S59 described later. The process of S54 is the same as the process of S22 of FIG. 18 except that the areas are different. Therefore, S54 corresponds to a correlation value map creation unit.

  In S55, scoring and matching position determination are performed. The process of S55 is the same as S23 of FIG. Therefore, S55 corresponds to a matching position determination unit.

  In S56, it is determined whether or not the determination of the matching positions of all the collation areas has been completed. Although the initial collation area 401 is one, a plurality of partial areas are determined from one collation area by executing S59 described later. Each partial area becomes the next collation area. Therefore, there may be a plurality of collation areas. When there are a plurality of collation areas, the matching positions are sequentially determined. Therefore, in S56, it is determined whether or not the determination of the matching position for all the collation regions has been completed.

  If the determination in S56 is NO, in S57, the matching area for determining the matching position is changed to a matching area for which the matching position has not yet been determined. Thereafter, S54 and S55 are executed to determine the matching position.

  If judgment of S56 is YES, it will progress to S58. In S58, it is determined whether or not the collation region for which the matching position has been determined is for one frame. One frame corresponds to the final number of frames. In the present embodiment, the final frame number is one frame, but a larger frame number (for example, two frames) may be the final frame number. If judgment of S58 is NO, it will progress to S59.

  In S59, each verification area is divided to create a partial area. FIG. 36 shows an initial collation area 401 and partial areas 402a, 402b, 402c, 402d, and 402e. FIG. 36 also shows a road map of the high accuracy map 151. When the partial areas 402a, 402b, 402c, 402d, and 402e are not distinguished, they are described as partial areas 402. The partial area 402 is a part that becomes the second verification area.

  The partial area 402 is an area included in the first collation area 401 and is a partial area of the first collation area 401. The partial area 402 is a range including a portion that may be a lane marking from the first collation area 401. The possibility of being a lane marking is determined from, for example, a change in edge strength.

  The size of the partial area 402 with respect to the first collation area 401 is, for example, half or more. This is because if the partial area 402 is too small with respect to the first collation area 401, it is difficult to obtain an effect of suppressing the erroneous matching position by performing the positioning little by little. The number of partial areas 402 is one or more, and can be set arbitrarily. However, it is preferable to set a plurality. In the example of FIG. 36, the number of partial areas 402 is five.

  In the case where the number of partial areas 402 is set to a plurality, it is preferable that the partial areas 402 partially overlap each other. This is because if a part of a certain partial region 402 is also a part of another partial region 402, it is possible to suppress the matching position with respect to the partial region 402 from being an incorrect position.

  The created partial area becomes the next collation area. After determining the next collation area, S54 and subsequent steps are executed. In the second and subsequent executions of S55, the matching target area is a collation area that is a target for creating a partial area in the immediately preceding S59. By executing S54 to S59, the matching position of the matching area is determined while sequentially reducing the matching area until the matching area reaches one frame.

  FIG. 37 illustrates second partial areas 403a, 403b, 403c, 403d, and 403e created from the first partial area 402a. When the partial areas 403a, 403b, 403c, 403d, and 403e are not distinguished, they are described as partial areas 403. The partial area 403 is a part that becomes the third collation area.

  The partial areas 403a, 403b, 403c, 403d, and 403e are partial areas of the partial area 402a. Each of the partial areas 403a, 403b, 403c, 403d, and 403e is a range including a portion that may be a lane marking. In addition, when the part that may be a lane marking is near the edge of the collation area for which the partial area 403 is to be determined, the partial area is so that a part of the partial area 403 is located outside the collation area. 403 may be determined. That is, a part of the partial area determined from the collation area may be located outside the collation area. This is because by making the lane marking closer to the center than the edge, the accuracy of the matching position is improved as compared with the case where the lane marking is positioned at the edge of the partial region 403. 37, the matching target area for the partial areas 403a, 403b, 403c, 403d, and 403e is the partial area 402a.

  As with the first partial area 402, five second partial areas 403 are created so that some of them overlap each other. The third and subsequent partial areas (that is, the fourth and subsequent collation areas) are also created in the same manner as the first and second partial areas 402 and 403.

  When a plurality of collation areas are created in S59 and some of the collation areas overlap each other, for the overlapping area, the matching positions are obtained from the plurality of collation areas having the overlap areas. That is, a plurality of matching positions are obtained for the overlapping region. The position of the overlapping region is determined in consideration of both of the plurality of matching positions, such as averaging the plurality of matching positions. Among the verification areas, the areas other than the overlapping area are determined in accordance with the position of the overlapping area. In this way, the position of the overlapping area is determined in consideration of the matching positions of a plurality of matching areas, and the matching position of the matching area is incorrect compared to the case where the matching area is determined so that there is no overlapping area. It can suppress determining to a position.

  When the collation area is one frame, the determination in S58 is YES and the process proceeds to S60. In S60, a change point is extracted. The process of S60 is the same as S24 of FIG.

  As described above, the matching processing unit 164 of the fourth embodiment determines a matching position in a large matching area, and sequentially determines a matching position while reducing the matching area. As a result, in a small area, even if an image similar to an image in a certain collation area exists at a position different from the original position of the collation area, the matching position may be determined as an incorrect position. Can be suppressed.

  Although the present embodiment has been described above, the present disclosure is not limited to the above-described embodiment, and the following modified examples are also included in the disclosure range, and various modifications can be made without departing from the scope other than the following. Can be changed and implemented.

<Modification 1>
A part of the processing performed by the map generation unit 265 of the second embodiment may be performed by a center outside the vehicle 1. In this case, the image extracted in S31 is transmitted to the center. Then, the processing after S32 is performed at the center.

<Modification 2>
The generated map image 171 of the second embodiment uses a bird's-eye view image for matching processing as an image for generating the generated map image 171. However, a white line intensity image may be used as an image for generating the generated map image 171.

<Modification 3>
The center includes the functions of the image processing unit 162, the collation region specifying unit 163, and the collation processing unit 164, and the vehicle 1 sequentially transmits the front image 180 and the current position of the vehicle 1 detected by the vehicle position detection unit 161 to the center. You may make it do. Alternatively, the processing up to the image processing unit 162 may be performed by the vehicle 1, and the processing of the matching area specifying unit 163 and the matching processing unit 164 may be performed at the center. Further, the processing up to the collation area specifying unit 163 may be performed by the vehicle 1, and the processing of the collation processing unit 164 may be performed at the center. In these cases, the vehicle 1 includes a communication unit for communicating with the center.

  In addition, when the process of the collation area | region specific | specification part 163 is performed with the vehicle 1, the center can also designate the area which processes to the vehicle 1. FIG. Then, only the high-precision map 151 for the designated area may be transmitted to the vehicle 1. Thereby, the vehicle 1 can reduce the capacity | capacitance and communication amount for memorize | storing the high precision map 151, and the center can also reduce processing load.

<Modification 4>
A camera that captures the rear of the vehicle may be used as the camera 110.

1: Vehicle 2: Update information utilization system 100: Road map information update system 110: Camera 120: Parameter storage unit 130: GPS receiver 140: Behavior detection sensor 150: Map storage unit 151: High-precision map 160: Calculation unit 161: Vehicle position detection unit 162: Image processing unit 163: Collation area specifying unit 164: Collation processing unit 170: Accumulated data storage unit 171: Generated map image 180: Front image 181: Front image after correction 182: Grading line 183: Vanishing point 184 : Image center 185: Distortion correction image 200: Road map information update system 260: Calculation unit 265: Map generation unit 386: Vanishing point region image 401: First collation region 402: Partial region S2: Distortion correction processing unit S4: Disappearance inspection Unit S5: Camera orientation detection unit S6 Overhead conversion unit, S7 Obstacle removal processing unit S22, S54: Correlation value map creation unit S23, S55: Matching position determination unit S41, S42, S43: Movement amount calculation unit S44: Integration value calculation Part S45, S46: Pitch fluctuation amount calculation part δy: Pitch fluctuation quantity

Claims (9)

A camera (110) provided in the vehicle for imaging the periphery of the vehicle;
An overhead conversion unit (S6) that converts an image captured by the camera into an overhead image;
A map storage unit (150) for storing a road map (151) including a road surface map;
A collation processing unit (164) that collates the bird's-eye view image with the road surface map and determines whether or not there is a change point that is a position where the actual road surface has changed with respect to the road surface map; Map change point detection device provided.   The map change point detection device according to claim 1, further comprising a collation area specifying unit (163) that determines a region in a vehicle width direction used for collation from the overhead image based on the number of road lane markings that can be detected from the overhead image. . Obstacle removal that removes obstacles included in the overhead image by specifying the color of the object included in the road map for the overhead image and converting the color other than the specified color to black A processing unit (S7),
The map change point detection device according to claim 1 or 2, wherein the collation processing unit collates the overhead image after the obstacle is removed with the road map.   The map change point detection device according to claim 1, wherein the collation processing unit performs distance conversion on the road map, and collates the road map after the distance conversion with the overhead image. In claim 2,
The collation area specifying unit extracts a collation area that is an area used for collation from the bird's-eye view image used for collation based on the number of road lane markings that can be detected from the bird's-eye view image. A certain area determined based on the
The collation processing unit
Relative position of the matching area with respect to the matching target area by calculating the correlation value of the matching area of the overhead image used for matching with the matching target area of the road map while changing the position of the matching area A correlation value map creating unit (S22, S54) for creating a correlation value map that is a relationship between the correlation value and the correlation value;
A score obtained by multiplying the correlation map and the scoring map in which the current position is most likely and the coefficient decreases as the distance from the current position increases, is calculated for each position of the matching area with respect to the matching target area, A matching position determining unit (S23, S55) that determines the position of the matching region having the highest score as a matching position;
The map change point detection apparatus which collates the bird's-eye view image for collation and the road surface map of the road surface map by using the position of the collation area as the matching position. In claim 5,
The collation processing unit
A plurality of frames captured by the camera are combined to generate an initial collation area (401),
The creation of the correlation value map by the correlation value map creation unit (S54) and the determination of the matching position by the matching position determination unit (S55) are started from the first collation region,
After the determination of the matching position by the matching position determination unit, if the number of frames included in one verification area is larger than the final frame number, a partial area (402) is determined from the verification area,
Using the determined partial area as the collation area, the correlation value map creation unit creates the correlation value map and the matching position decision unit decides the matching position. The number of frames included in the collation area is the final frame. Map change point detection device that repeats until the number is reached. A distortion correction processing unit (S2) that performs distortion correction and volume distortion correction on an image captured by the camera;
A vanishing point detection unit (S4) for detecting a vanishing point from the image after being corrected by the distortion correction processing unit;
A camera direction detection unit (S5, S5A) for detecting the direction of the optical axis of the camera based on the difference in position between the vanishing point and the image center detected by the vanishing point detection unit;
The overhead view conversion unit converts an image captured by the camera into the overhead view image by converting the direction of the optical axis of the camera detected by the camera direction detection unit from the upper side to the lower side. The map change point detection apparatus of any one of -5. In claim 7,
A vanishing point region image which is a part of an image after being corrected by the distortion correction processing unit and is an image of a region determined based on the vanishing point detected by the vanishing point detection unit is sequentially determined, and time When matching two vanishing point region images that are consecutive in succession, the vanishing point region image at a later time is taken up and down with respect to the vanishing point region image at the previous time among the two vanishing point region images. A movement amount calculation unit (S41, S42, S43) for calculating a movement amount to be moved in the direction;
An integral value calculation unit (S44) for calculating a position shift integral value that is an integral value of the movement amount in the vertical direction of the image calculated by the movement amount calculation unit;
A pitch fluctuation amount (δy) that is a difference between the moving average value and the moving amount calculated by the moving amount calculating unit is sequentially calculated as a moving average value of the positional deviation integrated value calculated by the integral value calculating unit. And a pitch fluctuation amount calculation unit (S45, S46) for calculating
The camera orientation detection unit (S5A) detects the orientation of the optical axis of the camera based on the position difference between the vanishing point and the image center detected by the vanishing point detection unit and the pitch fluctuation amount. Map change point detection device. A vehicle position detector (161) for detecting the position of the vehicle;
A map generation unit (265) that generates a map from the overhead image by setting the map image determined based on the overhead image as the map image at the position of the vehicle when the image that generated the overhead image is captured. The map change point detection apparatus of any one of Claims 1-6 provided with this.