JP2014106704A - In-vehicle image processing device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To reliably detect a road marker even when a three-dimensional object is present.SOLUTION: An image converting and synthesizing unit 20 converts an image including a road surface taken by an imaging unit 10 into a bird's-eye view image. An end point position detecting unit 30 scans the bird's-eye view image in at least one predetermined direction so as to detect a position of a first point where the brightness varies from dark to bright, and a position of a second point where the brightness varies from bright to dark, within a predetermined distance from the first point in the predetermined direction; and the end point position detecting unit detects positions of both end points of a center line M obtained by connecting a plurality of center points between the first points and the second points, the center points detected along a direction orthogonal to the predetermined direction. A radial edge detecting unit 40 detects, from line segments connecting the end points, a line segment extending from a point corresponding to the position where the imaging unit 10 is installed, toward a peripheral edge of the bird's-eye view image. A road marker detecting unit 50 detects a cross-walk Z (road marker) based on the line segments excluding the line segment detected by the radial edge detecting unit 40, in the line segments connecting both end points of the center line M.

Description

  The present invention relates to an in-vehicle image processing apparatus that detects a road marker such as a pedestrian crossing or a road sign drawn on a road surface by an in-vehicle camera.

  In recent years, a camera has been installed in a vehicle, the surroundings of the vehicle are observed with the camera, and pedestrian crossings and road signs represented by white lines and yellow lines drawn on the road surface are detected from the observed images. Thus, a system for supporting the safe driving by transmitting the detected information to the driver has been researched and developed.

  In such a system, for example, when recognizing a road surface sign such as a pedestrian crossing, edge information constituting a white line indicating a road boundary of a road is suppressed to reliably detect a pedestrian crossing (for example, a patent) Reference 1).

JP 2011-154480 A

  However, according to the invention disclosed in Patent Document 1, when recognizing a pedestrian crossing, it is possible to suppress white line information indicating the road boundary of a road, which may be erroneously recognized as a pedestrian crossing, In addition to the white line indicating the road boundary, there are structures on the road that cause misrecognition when recognizing a road sign such as a pedestrian crossing (hereinafter referred to as a road marker). For example, an edge constituting a three-dimensional object on a road, a water droplet attached to a lens, or the like has a problem that it is erroneously recognized as a road marker when its light and dark pattern is similar to a road marker. Therefore, it is difficult to stably and reliably detect a road marker such as a pedestrian crossing simply by suppressing the information on the white line indicating the road boundary.

  The present invention has been made in view of the above problems, and removes in advance information that becomes noise when detecting a road marker such as edge information constituting a three-dimensional object on a road surface or a water droplet attached to a lens. It is an object of the present invention to provide an in-vehicle image processing apparatus that can stably and reliably detect a road marker such as a pedestrian crossing or a road sign drawn on a road surface.

  An in-vehicle image processing apparatus according to the present invention is installed in a vehicle, observes the surroundings of the vehicle, captures an image including at least a road surface, and captures an image captured by the imaging unit. From the first point and the position of the first point at which the luminance changes from dark to bright while scanning the overhead image in at least one predetermined direction while converting the overhead image to a bird's eye view looking down from above A line made up of a plurality of first points detected in a direction perpendicular to the predetermined direction by detecting a position of a second point within a predetermined distance in the predetermined direction, the luminance of which changes from light to dark. The position of both end points of the element, the position of both end points of the line element composed of a plurality of second points, or the position of both end points of the center line connecting the midpoints of the first point and the second point From the end point position detection unit to be detected and the line segment connecting the both end points, A radial edge detection unit that detects a line segment extending from the point corresponding to the installation position of the imaging unit toward the periphery of the overhead image, and a line detected by the radial edge detection unit among the line segments that connect the both end points And a road marker detection unit that detects a road marker based on the line segment excluding the minutes.

  According to the in-vehicle image processing device according to the present invention configured as described above, an image including a road surface imaged by the imaging unit is converted into an overhead image by the image conversion unit, and the end point position detection unit converts the overhead image into an overhead image. The first point where the luminance changes from dark to bright while scanning in at least one predetermined direction and the first point where the luminance changes within a predetermined distance from the first point in the predetermined direction. The positions of the two end points of the line element consisting of a plurality of first points and the ends of the line element consisting of a plurality of second points, which are detected in the direction orthogonal to the predetermined direction. The positions of the points or the positions of both end points of the center line connecting the midpoints of the first point and the second point are detected, and the radial edge detection unit detects both ends detected by the end point position detection unit. From the line segment connecting the points, from the point corresponding to the installation position of the imaging unit, A line segment is detected by detecting a line segment extending toward the edge, and the road marker detection unit excludes the line segment detected by the radial edge detection unit from the line segments connecting the both end points detected by the end point position detection unit. Because the road marker is detected based on the image, it is possible to remove the influence of the edges that make up a three-dimensional object on the road and the water droplets attached to the lens, thereby stabilizing the road marker drawn on the road surface. And can be reliably detected.

  According to the vehicle-mounted image processing apparatus according to the present invention, a road marker drawn on a road surface can be detected stably and reliably.

It is a figure explaining the vehicle by which Example 1 of the vehicle-mounted image processing apparatus which concerns on this invention was mounted. It is a block diagram which shows the whole structure of Example 1 of this invention. (A) is a figure showing an example of a concrete road environment. (B) is an example of the image which image | photographed with the imaging part which mounted the road environment of (a) in-vehicle, converted into the bird's-eye view image, and was synthesize | combined. (A) is a figure explaining the characteristic of the edge which comprises a solid object. (B) is a figure explaining the detection method of a radial edge. It is a figure explaining the detection method of the pedestrian crossing which is an example of a road marker. It is a flowchart explaining the flow of a process of Example 1 of this invention. It is a figure explaining the example which removes the influence of the water drop adhering to the image.

  Embodiments of an in-vehicle image processing apparatus according to the present invention will be described below with reference to the drawings. In the following description, the gray value stored in the image is referred to as a luminance value. Also, the luminance value of the pixel (x, y) of the image I is represented by I (x, y).

  In this embodiment, the in-vehicle image processing apparatus according to the present invention detects a road marker such as a pedestrian crossing, a stop line, a parking frame, or a road sign drawn on the road surface, and alerts based on the detected road marker. It is an example applied to a system that provides driving support information such as driving advice.

  First, the configuration of the in-vehicle image processing apparatus according to the present invention will be described with reference to FIGS. 1 and 2. The in-vehicle image processing device 5 (see FIG. 2) is mounted on a vehicle 1 (see FIG. 1), and as shown in FIG. 1, a front camera 10A that images the front of the vehicle 1 and a left door mirror of the vehicle 1 A left camera 10B that is mounted and images the left direction of the vehicle, a right camera 10C that is mounted on the right door mirror of the vehicle 1 and images the right direction of the vehicle, and a rear camera 10D that images the rear of the vehicle 1 have. The road surface around the vehicle 1 can be observed by the imaging unit 10 constituted by these four cameras. Each camera is composed of an optical system having a light collecting function such as a lens and a photoelectric conversion element such as a C-MOS.

  FIG. 2 is a block diagram illustrating the overall configuration of the first embodiment. The in-vehicle image processing device 5 is mounted on the vehicle 1 (see FIG. 1), and the imaging unit 10 that captures an area including the road surface around the vehicle 1, and the front camera 10A and the left camera 10B that configure the imaging unit 10. , Image conversion that combines the images captured by the right camera 10 </ b> C and the rear camera 10 </ b> D into a bird's-eye view image looking down from above the vehicle 1, and combines them into a single image, and image conversion An end point position detection unit 30 that detects positions of both end points of a line segment connecting the midpoints of a plurality of + edges and − edges that are close to each other in a predetermined direction from the image synthesized by the synthesis unit 20; Of the plurality of end points detected by the end point position detection unit 30, the line segment connecting the both end points is the peripheral portion of the image captured by the image capturing unit 10 from the installation position of the image capturing unit 10 in the overhead image. Check for radial extension toward A radial edge detector 40, an end point list 45 for storing the positions of the detected end points, and a road marker database storing road marker shape features such as pedestrian crossings, lanes, road markings, and parking frames 48, a road marker detection unit 50 for detecting a road marker based on the positions of both end points stored in the end point list 45 and the road marker database 48, and the driver of the vehicle 1 according to the detected road marker. And an information output unit 60 including a display device and a voice output device.

Hereinafter, the point which is a point in the operation | movement outline | summary of the vehicle-mounted image processing apparatus 5 is demonstrated.
<Description of generation and synthesis method of overhead image>
As shown in FIG. 3B, an image captured by the front camera 10A, an image captured by the left camera 10B, an image captured by the right camera 10C, and an image captured by the rear camera 10D are converted into an image. The synthesizing unit 20 performs coordinate conversion to an overhead image obtained by looking down the vehicle 1 from directly above, and further, the image is synthesized as one image I.

This image I includes an overhead image I _{1 obtained} by converting an image captured by the front camera 10A and an overhead image I _{2 obtained} by converting an image captured by the left camera 10B around a region V representing the position of the vehicle 1. Then, the overhead image I ₃ converted from the image captured by the right camera 10C and the overhead image I ₄ converted from the image captured by the rear camera 10D are combined in a direction corresponding to the installation direction of each camera. It has been done. That is, for example, the bird's-eye view image I ₁ increases as the distance from the region V increases in the direction corresponding to the front of the vehicle 1 in the region V from the point O ₁ on the bird's-eye view image corresponding to the position where the front camera 10A is installed. Arranged so that a wide range is reflected.

Similarly, the bird's-eye view image I ₂ , the bird's-eye view image I ₃ , and the bird's-eye view image I ₄ are each viewed from the position on the bird's-eye view image corresponding to the position where the left camera 10B, the right camera 10C, and the rear camera 10D are installed. In the direction corresponding to the left side of the vehicle 1 in V, the direction corresponding to the right side, and the direction corresponding to the rear side, the larger the distance from the region V, the larger the range is displayed.

  In addition, since the coordinate conversion to this bird's-eye view image is a well-known technique that has recently been put into practical use as a monitoring system around the vehicle, a detailed description of the processing method is omitted.

<Description of detection method of road marker candidate area>
Next, a method for detecting the end point position of the road marker candidate region will be described.

  When the vehicle 1 is on the road R shown in FIG. 3A, a pedestrian crossing Z drawn on the road surface of the road R is included in the image I (FIG. 3B) captured by the imaging unit 10. The image Z ′ is reflected, and the telephone pole S installed on the road R is reflected as the image S ′.

  The upper left of the image I is the origin (0, 0), the horizontal direction is x, and the vertical direction is y. Then, in the end point position detection unit 30 (see FIG. 2), while referring to the luminance value I (x, y) stored in the image I from the left to the right in the image I, two adjacent points are detected. The difference between the luminance values of the pixels is calculated. For the referenced pixel (x, y), the luminance difference I (x−1, y) −I (x, y) and the luminance difference I (x, y) −I (x + 1, y) are represented as an image I. The calculation is sequentially performed for all the pixels.

The pixel luminance value changes significantly brighter than a predetermined value, i.e., the luminance difference threshold _{I th} set in advance, satisfy I (x, y) -I ( x-1, y)> I th A pixel, that is, a point where the luminance changes from dark to bright by a predetermined value or more is detected as the first point. This pixel is called a + edge.

Further, a pixel whose luminance value changes darker than a predetermined value, that is, a pixel where I (x, y) −I (x + 1, y)> I _th , that is, a luminance changes by a predetermined value or more from light to dark. A point is detected as a second point. This pixel is called -edge.

  In the case of a road marker, since the interval between the pixels constituting the + edge and the pixels constituting the − edge is substantially equal to the width of the white line constituting the road marker, the + edge detected from the image I is constituted. When an interval w between a pixel and a pair of pixels constituting an −edge detected adjacent to the + edge (referred to as an edge pair) is within a predetermined distance, the edge pair may constitute a road marker. It is determined that the pixel is high, and a pixel constituting the + edge and a pixel constituting the − edge are detected.

For example, when the luminance value stored in the image I is referred from the left to the right in FIG. 3B, for example, the image S ′ of the telephone pole S having a high luminance with respect to the road surface area of the road R. in the region, as shown in FIG. 4 (a), + the line segment L ₁ composed pixel is formed continuously with the edge, - the line segment constituted by pixels formed at the edge is continuously L ₂ is detected.

That is, in the area of the image S ′ of the telephone pole S, the area sandwiched between the + edge and the − edge is continuously detected in the direction orthogonal to the left-right direction with reference to the luminance. Thus formed, the line segment L ₁ constituted by consecutive pixels composed + edge - road region sandwiched by the configured line L ₂ formed pixels are continuously at the edge This is called a marker candidate area.

  Such detection of the road marker candidate region is performed on the entire image I.

<Description of end point detection method of road marker candidate area>
Next, a method for detecting the end point from the detected road marker candidate region will be described.

+ A segment L ₁ composed pixel is formed continuously with the edge, - from the line segment L ₂ consists pixel is formed continuously at the edge, when searching an image I in the transverse direction A center line M is detected by connecting the midpoints of pixels formed by + edges and pixels formed by −edges that are detected adjacent to each other. Thereby, for example, from the region of the image S ′ of the telephone pole S, as shown in FIG. 4A, a center line M connecting the points m ₁ and _mn is detected. The end points (point m ₁ and point m _n ) of the center line M detected in this way are called end points of the road marker candidate region.

In practice, not only the end points m ₁ and _mn but also more midpoints are detected from one road marker candidate region, and these midpoints are connected to form the center line M.

Such end points are detected for the entire image I excluding the inside of the region V, and the detected end points are stored in the end point list 45.
<Description of radial edge detection method>
Since the above-mentioned telephone pole S stands vertically from the road surface, both end points m ₁ and _{mn of} the center line M detected from the area of the image S ′ and the point O ₁ are on one straight line.

  This is because the coordinate conversion performed when generating the above-described bird's-eye view image is performed on the assumption that all points reflected on the camera are points on the road surface, so that a target standing upright from the road surface is formed. This is because the higher the position of the point, the more the point is coordinate-transformed as being on a road surface far from the camera.

Further, the points constituting the left and right vertical edges of the telephone pole S are coordinate-converted into the line segment L ₁ and the line segment L ₂ on the overhead image, respectively, but at this time, the line segment L ₁ and the line segment L ₂ are converted. Is converted so that the distance from the point O ₁ increases as the distance from the point O ₁ increases. That is, a pair of vertical edges that rise vertically from the road surface in the actual environment is coordinate-converted on the overhead image to a region that spreads radially from a point corresponding to the position where the imaging unit is installed.

  In the present invention, this feature is used to determine the presence or absence of vertical edges in the real environment. Then, the road marker is detected without being affected by the presence of the three-dimensional object by removing the edge corresponding to the condition from the overhead image.

The detection of the edge that spreads radially is performed by the radial edge detector 40. A specific detection method will be described with reference to FIGS. 4 and 5, taking as an example a case where a radial edge is detected inside the overhead image I ₁ .

The coordinates (for example, the points m ₁ and m _n ) of the both ends of the center line M stored in the end point list 35 are read, and the coordinates of the point O ₁ corresponding to the installation position of the front camera 10A and the point m ₁ Using the coordinates and the coordinates of the point _mn , as shown in FIG. 4B, the angle θa formed by the line segment O _{1 mn} with respect to the horizontal direction of the image I and the horizontal direction of the image I The absolute value of the difference between the angles θb formed by the line segments m _{1 mn} is obtained.

At this time, when the absolute value of the difference between the angle θa and the angle θb is equal to the predetermined angle θc, it is determined that the line segment m _{1 mn} is an edge extending radially from the point O ₁ .

Actually, when the angle θc has a predetermined width and the absolute value of the difference between the angle θa and the angle θb is within the width, the line segment m _{1 mn} extends radially from the point O _1. It is determined that it is an edge.

This determination method is based on the point O ₁ corresponding to the installation position of the front camera 10A and the end point _mn on the side close to the installation position of the front camera 10A among the end points m _n and m ₁ on both sides of the center line M. In other words, the angle θc formed by the line segment O _{1 mn} connecting O ₁ and the line segment m _{1 mn} connecting the end points m _n and m ₁ on both sides is within a predetermined value.

  The edges constituting the target drawn on the road surface like a pedestrian crossing are not coordinate-converted into a radially extending shape in the overhead view image like the left and right vertical edges of the telegraph pole S, and thus described above. When the evaluation is performed based on the angle of the line segment, the angle θc does not fall within a predetermined width range. Therefore, it is determined that the shape does not extend radially.

Similar processing is performed for all the endpoints detected in the interior of the overhead image I _1. In addition, the same processing is performed on the inside of the overhead images I ₂ , I ₃ , and I _{4. In} this case, instead of using the point O ₁ , the left camera 10 B, the right camera 10 C, and the rear camera 10 D are used. The position of the point in the image I corresponding to each installation position is used.

  In this way, when it is determined that the center line M extends radially, the end points on both sides of the center line M are deleted from the end point list 35.

<Description of pedestrian crossing detection method>
Next, a method for detecting the pedestrian crossing Z drawn on the road surface will be described. There are various methods for detecting the pedestrian crossing Z, and any of these methods may be applied. Here, a detection method using the shape feature of the pedestrian crossing Z will be described.

  The shape features of the pedestrian crossing Z are, for example, the following (1) to (2).

  (1) In the pedestrian crossing Z, a white line having a length D along the direction in which the road R extends is drawn at an interval W.

  (2) The pedestrian crossing Z is composed of N white lines, and these white lines are all parallel to each other.

  These shape features are illustrated in FIG. Such shape features are stored in the road marker database 48 in advance.

  Of the shape features described above, the values of the length D and the interval W vary depending on the range in which the image I is observed. However, since the parameters for converting the captured image into a bird's-eye view image are known in advance, the length D and the interval W of the white line that forms the pedestrian crossing can be easily estimated. The sidewalk Z is detected.

  The road marker detection unit 50 reads the position of the end point of the center line M stored in the end point list 45 and collates it with the information stored in the road marker database 48, so that the pedestrian crossing Z is displayed in the image I. It is determined whether or not the image Z ′ is reflected.

  Specifically, based on the position of the end point of the center line M stored in the end point list 45, the length of each center line M is compared with the expected length D, and the interval between adjacent center lines M is expected. Comparison with the interval W, the parallelism between adjacent centerlines M and evaluation, the number of centerlines M that satisfy all the conditions of length D, the interval W, and the parallelism, and there are crosswalks A comparison is made with the number N of centerlines M expected at this time to determine whether an image Z ′ of the pedestrian crossing Z is reflected.

  When detecting this pedestrian crossing, the edges that spread radially are already removed, so the number of endpoints to be processed can be reduced, thereby preventing false detection and non-detection of the pedestrian crossing. Detection can be performed in a short time

<Description of Operation of Example 1>
Next, a flow of a series of operations of the in-vehicle image processing apparatus 5 according to the first embodiment will be described with reference to the flowchart of FIG.

  (Step S <b> 10) The periphery of the vehicle 1 is imaged by the imaging unit 10.

  (Step S20) In the image conversion / synthesis unit 20, the image captured by the imaging unit 10 is converted into an overhead view, and further combined into a single image I as shown in FIG.

  (Step S30) The end point position detection unit 30 detects, from the image I, an edge pair that is a pair of a pixel that forms a + edge and a pixel that forms a −edge, which are adjacent to each other in the left-right direction at a predetermined interval.

  (Step S <b> 40) Further, the end point position detection unit 30 detects the center line M formed by the midpoints of the two pixels constituting the edge pair.

  (Step S50) Then, in the end point position detection unit 30, the positions of the end points on both sides of the center line M are obtained.

  (Step S60) The positions of the detected end points are stored in the end point list 45.

  (Step S70) In the radial edge detection unit 40, the positions of the end points on both sides of the center line M stored in the end point list 45 are read, and the center line M forms an edge that spreads radially by the method described above. It is determined whether or not. If it is determined that a radially extending edge is formed, the process proceeds to step S80. If it is determined that a radially extending edge is not formed, the process proceeds to step S90.

  (Step S80) The end points on both sides of the center line M determined to have a radial edge are deleted from the end point list 45.

  (Step S90) The road marker detection unit 50 detects an image Z ′ of the pedestrian crossing Z from the image I. Here, the shape feature of the pedestrian crossing stored in the road marker database 48 is read out, and a group of center lines M having the shape feature of the pedestrian crossing stored therein are stored in the end point list 45. It is determined whether or not it exists in the data representing the positions of the end points on both sides of the line M. Specifically, the positions of the end points on both sides of the center line M stored in the end point list 45 are read out, the length of the center line M, the interval between the adjacent center lines M, and the parallel between the adjacent center lines M. The degree is evaluated, and the number of center lines M satisfying the condition is further evaluated to determine the presence or absence of the image Z ′ of the pedestrian crossing Z.

  (Step S100) When the image Z ′ of the pedestrian crossing Z is detected, the information output unit 60 outputs information indicating that the pedestrian should be noted before the pedestrian crossing by screen display or voice guide, and the driver Call attention.

  As described above, according to the in-vehicle image processing device 5 of the present invention configured as described above, an image including a road surface imaged by the imaging unit 10 is overlooked by the image conversion / synthesis unit 20 (image conversion unit). The position of the first point that is converted into an image and whose luminance changes from dark to bright while the end point position detection unit 30 scans the overhead image from left to right (in at least one predetermined direction). And a position of a second point within a predetermined distance in the predetermined direction from the first point where the luminance changes from light to dark, and a plurality of points detected in a direction orthogonal to the predetermined direction are detected. The position of both end points of the line element consisting of the first point, the position of both end points of the line element consisting of the plurality of second points, or the center connecting the midpoints of the first point and the second point The positions of both end points of the line M are detected, and the radial edge detection unit 40 detects the end point positions. 30 detects a line segment extending from the point corresponding to the installation position of the imaging unit 10 toward the periphery of the bird's-eye view image from among the line segments configured by the two end points detected in 30, the road marker detection unit 50, In order to detect the pedestrian crossing Z (road marker) based on the line segment formed by the end point position detection unit 30 excluding the line segment detected by the radial edge detection unit 40 among the line segments formed by the both end points. The influence of the edges constituting the three-dimensional object existing on the road can be removed, whereby the pedestrian crossing Z (road marker) drawn on the road surface can be detected stably and reliably.

Further, according to the in-vehicle image processing device 5 of the present invention, the radial edge detection unit 40 is located on the side close to the point O ₁ that is the installation position of the front camera 10A among the two end points of the center line M in the overhead view image. and a line segment O ₁ m _n connecting point and O ₁ is a installation position of the end point m _n and front camera 10A, the difference in angle between the line segment m ₁ m _n connecting the two sides of the end point within a predetermined angle θc Therefore, since the line segment m _{1 mn} connecting the two end points of the center line M is a radial edge, the positional relationship among the three points O ₁ , _mn , and m ₁ is very simple. Radial edges can be detected by processing.

  In the first embodiment, the telegraph pole S is taken as an example, and the example of removing the influence of the edge constituting the three-dimensional object existing on the road has been described. However, when there is another three-dimensional object, for example, a guardrail, It goes without saying that those edges can be removed in the same manner when there is a stopped vehicle.

  In addition to the three-dimensional object, there is a thing whose edge shape spreads radially. For example, water droplets attached to the lens of the camera are not three-dimensional objects, but are attached to the lens surface, thereby causing radial edges on the image like the image S ′ of the telephone pole S shown in FIG. Configure.

  This is because the water droplets adhering to the lens flow down in the direction of the road surface (downward) due to gravity, and thus have an edge component in the vertical direction in the image before the overhead conversion, like the three-dimensional object.

7, in the overhead view image I ₁ captured by the front camera 10A, showing a state in which water drops are attached. As shown in FIG. 7, the water droplets Ma and Mb are attached to the lens, the line segment La ₁ having the end point m _a11 and the end point m _a1n , the end point m _a21, and the end point m, which are the edges of the water drop Ma. line La ₂ having _a2n, and radial edges consisting of a line segment Lb ₂ with line Lb ₁ and end point _{m b21} and end points _{m B2n} having end points _{m b11} and end points _{m b1n} a peripheral edge of the water droplet Mb Form.

  Such a radial edge can also be detected by using the method described in the first embodiment. And when detecting a road marker, the radial edge which generate | occur | produced with the water droplet is also deleted, and it processes, Even if it is raining, the stable road marker can be detected.

In the example shown in FIG. 7, water droplets adhere to the area Ma sandwiched between the line segment La ₁ and the line segment La ₂ and the area Mb sandwiched between the line segment Lb ₁ and the line segment Lb _2. Therefore, the reliability of the detection result of the road marker can be increased by not using the information of not only the line segment but also the area sandwiched between the line segments for the detection of the road marker.

  Since the area Ma and the area Mb to which water droplets are attached are transparent areas, information on roads and backgrounds may be reflected through the water droplets, which may cause a pseudo edge. Therefore, by not using the information of the region sandwiched between the radial edges for the detection of the road marker, it is possible to prevent erroneous detection of the road marker due to such a pseudo edge. It should be noted that a radial edge sandwiched between water droplets may be determined by calculating the luminance distribution inside the region and, for example, when the edge intensity calculated from the luminance distribution is weak, that a water droplet is present.

Specifically, in FIG. 7, the region Ma surrounded by the end point m _a11 , the end point m _a1n , the end point m _a21 , and the end point m _a2n , and the end point m _b11 , the end point m _b1n , the end point m _b21 , and the end point m _b2n are surrounded. When the edge intensity is calculated from the luminance distribution inside the region Mb and predicted to be a water droplet, the luminance value inside the region is not used for detection of the road marker.

  Thus, according to the in-vehicle image processing device 5 of the present invention, the luminance information of the region where the luminance distribution satisfies the predetermined condition among the regions sandwiched between the plurality of radial edges detected by the radial edge detection unit 40 is obtained. In addition, since the road marker is detected, it is possible to prevent erroneous detection of the road marker due to the pseudo edge appearing inside the area sandwiched between the radial edges.

  As described above, as examples in which a radial edge appears, an example in which a three-dimensional object is present and an example in which water droplets adhere to the lens surface have been described. However, sunlight or street light is reflected on the road surface and captured by the imaging unit 10. Radial edges may also be observed when road surface reflections occur due to this. For such road surface reflection, the reliability of detection of the road marker can be increased by deleting the radial edge from the road marker detection process in the same manner as the three-dimensional object or water drop.

  In the first embodiment, the center line M of the road marker candidate region is detected and removed from the road marker candidates when the center line M forms a radial edge. When a line segment composed of + edges constituting the road marker candidate area is detected instead of the center line M of the area, and the line segment forms a radial edge, the line segment composed of the + edges is detected from the road marker candidates. The road marker candidate area may be removed, or when a line segment including an edge is detected and the line segment forms a radial edge, the line segment including the edge is detected by the road marker. Even if it is removed from the candidates, the same effect can be obtained.

  Further, in the first embodiment, while scanning from the left to the right in the image I, the pixels forming the + edge and the pixels forming the − edge are detected. While detecting the pixels constituting the + edge and the pixels constituting the − edge and combining the pixels constituting the + edge and the pixels constituting the − edge detected in a plurality of directions, the pixels constituting the + edge -You may make it handle as a pixel which comprises an edge.

  Thus, by detecting the pixels constituting the + edge and the pixels constituting the − edge in a plurality of directions, the road marker extending in an arbitrary direction in the image I can be reliably detected. Can do.

  In the first embodiment, the example of detecting the image Z ′ of the pedestrian crossing Z has been described. However, this is not limited to the pedestrian crossing. In other words, a road marker drawn on the road surface can be detected even if it is a road marker other than a pedestrian crossing such as a stop line, a parking frame, or a road sign. It is assumed that the shape feature of the road marker detected at this time is registered in the road marker database 48 in advance.

  As mentioned above, although the Example of this invention was explained in full detail with drawing, since an Example is only an illustration of this invention, this invention is not limited only to the structure of an Example. Of course, changes in design and the like within a range not departing from the gist are included in the present invention.

DESCRIPTION OF SYMBOLS 5 Car-mounted image processing apparatus 10 Image pick-up part 10A Front camera 10B Left camera 10C Right camera 10D Rear camera 20 Image conversion and composition part 30 End point position detection part 40 Radial edge detection part 45 End point list 48 Road marker database 50 Road marker detection part 60 Information output section

Claims (3)

An imaging unit that is installed in a vehicle, observes the surroundings of the vehicle, and captures an image including at least a road surface;
An image conversion unit that converts an image captured by the image capturing unit into an overhead image viewed from directly above the vehicle;
While scanning the overhead image in at least one predetermined direction, the position of the first point where the luminance changes from dark to bright and within a predetermined distance from the first point in the predetermined direction, the luminance is from light to dark The position of the second point that changes to the predetermined direction is detected, and the positions of both end points of the line element composed of the plurality of first points detected in the direction orthogonal to the predetermined direction are composed of the plurality of second points. An end point position detection unit for detecting positions of both end points of the line element, or positions of both end points of the center line connecting the midpoints of the first point and the second point;
A radial edge detector that detects a line segment extending from a point corresponding to an installation position of the imaging unit to a peripheral edge of the overhead image, among line segments connecting the both end points;
A road marker detection unit that detects a position of a road marker based on a line segment obtained by removing the line segment detected by the radial edge detection unit from the line segment connecting the both end points. In-vehicle image processing device.   In the overhead image, the radial edge detection unit includes a line segment connecting an end point of the both end points closer to the installation position of the imaging unit and the installation position of the imaging unit, and a line connecting the both end points. The in-vehicle image processing apparatus according to claim 1, wherein a line segment connecting the both end points is a radial edge when a difference in angle with the minute is within a predetermined value.   Among areas sandwiched between a plurality of radial edges detected by the radial edge detection unit, when a luminance distribution of the area satisfies a predetermined condition, road marker detection is performed except for luminance information inside the area. The in-vehicle image processing apparatus according to claim 1, wherein the on-vehicle image processing apparatus is performed.