JP2008210084A - Road marking recognition device - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To more accurately recognize a road marking regardless of a shape of the road marking. <P>SOLUTION: A road face on the periphery of a vehicle is periodically photographed, and imaged image is converted into a bird's eye image, and is stored in a memory 14b. An extraction area is specified to a part on a vehicle main body side of the bird's eye image (S112, S114), an image of the extraction target area is sequentially from a plurality of bird's eye images stored in the memory 14b, and a composite image is generated (S116), and the road marking drawn on the road face with the composition as a target is image-recognized (S118). <P>COPYRIGHT: (C)2008,JPO&INPIT

Description

  The present invention relates to a road marking recognition apparatus that recognizes an image of a road marking drawn on a road surface.

  2. Description of the Related Art Conventionally, various devices have been proposed in which a road is photographed with an in-vehicle camera, and a road marking drawn on a road surface such as a white line or a pedestrian crossing is recognized from the photographed image using an image processing technique.

As such a device, an image obtained by photographing a front road surface is scanned in the horizontal direction to detect an edge position, and an object is evaluated by evaluating the object property of a window set in the vicinity of the operation position and surrounding a predetermined range of the image. There is one in which a good window position is obtained, and a road marking drawn on a road surface is recognized based on the edge position and the window position having good objectivity (see, for example, Patent Document 1).
Japanese Patent No. 3153845

  The device described in Patent Document 1 can recognize a road sign having a target property such as a pedestrian crossing or a pedestrian crossing notice, but recognizes an asymmetric road sign indicating a speed limit, for example. There is a problem that can not be.

  In addition, when a road sign drawn on the road surface is photographed as a still image using an in-vehicle camera, and a still image is to be recognized using this still image, a still image including the entire road sign is photographed. There is a need. However, the still image taken so as to include the entire road marking in this way has a small road marking size.

  Also, when shooting road markings using an in-vehicle camera in this way, the road markings are shot diagonally upward, so the portion near the road marking vehicle is large and the portion away from the road marking vehicle is Small projection. In particular, the road markings drawn on the road surface have a vertically long shape along the road traveling direction so that the driver can easily recognize the vehicle even when traveling at a high speed. As described above, there is a problem in that the portion of the captured image that is away from the vehicle of the road marking becomes very small, and the road marking may not be recognized correctly.

  The present invention has been made in view of the above problems, and an object of the present invention is to make it possible to recognize a road marking more accurately regardless of the shape of the road marking.

  According to a first aspect of the present invention, there is provided an imaging unit that is mounted on a vehicle and that periodically transmits an image of a road surface around the vehicle, and a captured image that is transmitted from the imaging unit is displayed on the ground surface from the vertical direction. An image conversion unit that converts the bird's-eye view image representing the bird's-eye view that is looked down into the storage unit, and an extraction target region specifying unit that specifies an extraction target region in a part of the bird's-eye view image converted by the image conversion unit on the vehicle body side; Synthetic image generation means for generating a composite image obtained by extracting images of extraction target regions in time series from a plurality of bird's-eye images stored in the storage means and connecting them in the vertical direction; and a composite image generated by the composite image generation means Image recognition processing means for recognizing a road marking drawn on a road surface for an image.

  In such a configuration, a part of the bird's-eye view image on the vehicle main body side is used as an extraction target region, and a composite image is generated by extracting and connecting the images of the extraction target region in time series from a plurality of bird's-eye images. That is, the composite image is generated by connecting images in a region near the vehicle body with high accuracy among the bird's-eye images. And since the image recognition of the road marking drawn on the road surface for this composite image is performed, the road marking can be recognized more accurately regardless of the shape of the road marking.

  In addition, the second feature of the present invention is that the extraction target area specifying means sets a reference line in a horizontal direction at a position closest to the vehicle main body side of the bird's-eye view image, and a rectangle having the reference line as one side in the bird's-eye view image. Is to specify the extraction target area.

  In such a configuration, a reference line is set at a position closest to the vehicle main body side of the bird's-eye view image, and a rectangular extraction target region having this reference line as one side is specified in the bird's-eye view image. A composite image can be generated by connecting images in a region near a vehicle body having a high height.

  A third feature of the present invention is that the extraction target area specifying unit determines whether or not the bird's-eye image includes a shadow of the vehicle itself. A reference line is set in the horizontal direction at a position closest to the vehicle main body side of the bird's-eye view image, a rectangular extraction target region having the reference line as one side is specified in the bird's-eye view image, and the shadow of the vehicle itself is included in the bird's-eye view image In this case, the reference line is set in a horizontal direction at a position where the shadow of the vehicle itself is excluded from the extraction target area and closest to the vehicle body side of the bird's-eye view image, and the reference line is included in the bird's-eye view image. Is to specify a rectangular extraction target area having one side as.

  When the bird's-eye view image includes a shadow of the vehicle itself, the possibility of misrecognizing the road marking due to the shadow of the vehicle itself may increase. In such a configuration, the bird's-eye view image includes the shadow of the vehicle itself. If it is determined that the reference line is set in a horizontal direction at a position where the shadow of the vehicle itself is excluded from the extraction target area and closest to the vehicle body side of the bird's-eye view image, the reference line is set in the bird's-eye view image. Since a rectangular extraction target area having a line as one side is specified, it is possible to reduce the possibility of erroneously recognizing a road marking due to the shadow of the vehicle itself.

  Further, the fourth feature of the present invention is that the extraction target area specifying unit includes the low-luminance area on the vehicle side of the bird's-eye image converted by the image conversion unit. It is to determine whether or not a shadow is included.

  In this way, it is possible to determine whether or not the bird's-eye view image includes the shadow of the vehicle itself based on whether or not the low-luminance region is included on the vehicle side of the bird's-eye view image.

  Further, a fifth feature of the present invention is provided with azimuth change determining means for determining whether or not the azimuth of the vehicle has changed by a predetermined azimuth amount or more, and the extraction target area specifying means is provided by the azimuth change determining means. When it is determined that the azimuth direction has changed more than a predetermined azimuth amount, the extraction target area is specified again.

  In such a configuration, when it is determined that the direction of the vehicle has changed by a predetermined amount or more, the extraction target area is reset to a position where the shadow of the vehicle itself is excluded from the extraction target area. Even if the direction of the vehicle changes and the shadow of the vehicle itself included in the bird's-eye view image changes, the road marking can be image-recognized without being affected by the shadow of the vehicle itself included in the bird's-eye view image.

  In addition, a sixth feature of the present invention is provided with solar radiation determining means for determining whether or not the presence or absence of solar radiation irradiated to the vehicle has changed, and the extraction target area specifying means is irradiated to the vehicle by the solar radiation determining means. When it is determined that the presence or absence of solar radiation has changed, the extraction target area is identified again.

  In such a configuration, when it is determined that the presence or absence of solar radiation irradiated to the vehicle has changed, the extraction target region is reset to a position where the shadow of the vehicle itself is excluded from the extraction target region. Even if there is a change in the presence or absence of solar radiation and the shadow of the vehicle itself included in the bird's-eye view image, the road marking can be recognized without being affected by the shadow of the vehicle itself included in the bird's-eye view image. It is.

  Further, a seventh feature of the present invention is provided with an inclination determination unit that determines whether or not the inclination of the vehicle has changed by a predetermined angle or more, and the extraction target area specifying unit is configured such that the inclination of the vehicle is determined by the inclination determination unit. When it is determined that the angle has changed by a predetermined angle or more, the extraction target area is specified again.

  In such a configuration, when it is determined that the inclination of the vehicle has changed by a predetermined angle or more, the extraction target region is reset to a position where the shadow of the vehicle itself is excluded from the extraction target region. For example, even when the inclination of the vehicle approaches a slope and the shadow of the vehicle itself included in the bird's-eye image changes, the road marking can be recognized without being affected by the shadow of the vehicle itself included in the bird's-eye image. Is possible.

  The eighth feature of the present invention is that the photographing means photographs a road surface around the vehicle using a wide-angle lens, and the image converting means corrects image distortion caused by the wide-angle lens and This is to convert the transmitted captured image into a bird's-eye view image.

  In such a configuration, the image distortion caused by the wide-angle lens is corrected, so that the recognition rate of the road sign image recognition can be improved.

  In addition, a ninth feature of the present invention is provided with vehicle speed detection means for detecting the vehicle speed of the vehicle, and the extraction target area specifying means extracts images of the extraction target area in time series from a plurality of bird's-eye images, in the vertical direction. The higher the vehicle speed detected by the vehicle speed detection means, the higher the extraction target area height, and the lower the vehicle speed, the lower the extraction target area height, so that the road surface shooting areas are continuous when connected. That is.

  In such a configuration, when the image of the extraction target area is extracted in time series from a plurality of bird's-eye images and connected in the vertical direction, the higher the vehicle speed, the higher the extraction target area. Since the height of the extraction target area is lower as the height is higher and the vehicle speed is lower, it is possible to obtain a composite image in which shooting areas on the road surface are continuous.

  The configuration of a road marking recognition apparatus according to an embodiment of the present invention is shown in FIG. The road marking recognition device 1 includes an in-vehicle camera 10, a solar radiation sensor 11, a gyroscope 12, a vehicle speed sensor 13, and an image processing device 14.

  The in-vehicle camera 10 is mounted on a vehicle and sends an image obtained by periodically capturing a road surface behind the vehicle to the image processing device 14. Note that the in-vehicle camera 10 in the present embodiment is configured as a so-called rear view camera for photographing the rear of the vehicle, and the photographed image is horizontally reversed. Moreover, a wide-angle lens is used as the lens of the in-vehicle camera 10 so that the field of view is widened. For this reason, a rounded distortion occurs in the captured image captured by the in-vehicle camera 10.

  The solar radiation sensor 11 detects the amount of solar radiation irradiated into the interior of the vehicle, and outputs a signal corresponding to the amount of solar radiation to the image processing device 14.

  The gyroscope 12 detects the azimuth change amount of the vehicle and the tilt change amount of the vehicle in the front-rear direction, and outputs a signal corresponding to the azimuth change amount and the tilt change amount to the image processing device 14.

  The vehicle speed sensor 13 outputs a vehicle speed signal corresponding to the vehicle speed to the image processing device 14.

  The image processing apparatus 14 is configured as a computer including a CPU 14a, a memory 14b, and the like.

  The memory 14b includes a ROM for storing a program, a RAM for storing temporary data, and a VRAM for storing image data. The CPU 14a performs various processes in accordance with programs stored in the ROM of the memory 14b.

  As processing of the image processing device 14, the vehicle is irradiated based on an image recognition process for recognizing a road display drawn on a road surface from a captured image sent from the in-vehicle camera 10, and a signal input from the solar radiation sensor 11. From the gyroscope 12, a solar radiation determination process for determining whether there is solar radiation, a direction change determination process for determining whether the vehicle orientation has changed by a predetermined amount or more based on a signal input from the gyroscope 12, Inclination determination processing for determining whether or not the inclination in the longitudinal direction of the vehicle has changed by a predetermined angle or more based on the input signal, and detecting the vehicle speed based on the vehicle speed signal input from the vehicle speed sensor 13 There is a vehicle speed detection process.

  Next, image recognition processing by the image processing device 14 will be described with reference to FIG. The image processing apparatus 14 operates in a low power consumption mode when the ignition switch is off, and enters an activated state when the ignition switch is on.

  First, it is determined whether or not the ignition switch is turned on. (S100).

  When the ignition switch is off, the determination in S100 is repeated. When the ignition switch is turned on, the activation state is set, and first, the shadow area detection flag is set to 0 (S102). This shadow area detection flag is provided in the RAM area of the memory 14b.

  Next, one frame of the original image is read (S104). Specifically, one frame of the latest original image of the captured image sent from the in-vehicle camera 10 is stored in the VRAM of the memory 14b.

  Next, distortion correction is performed on the original image and bird's-eye view conversion is performed to generate a bird's-eye view image (S106). Specifically, after performing a process for correcting distortion caused by the wide-angle lens of the in-vehicle camera 10 on the original image, the distortion-corrected image is reversed left and right and up and down, and the ground surface is viewed from the vertical direction. The image is converted into a bird's-eye image representing a bird's-eye view looking down and stored in the storage means. Here, the reason that the distortion-corrected image is reversed left and right is to restore the captured image transmitted from the in-vehicle camera 10 while being reversed left and right. Further, the reason why the image subjected to the distortion correction is turned upside down is to make the road sign photographed by turning the vehicle mounted camera 10 upside down to the original direction.

  As a technique for converting a captured image transmitted from the in-vehicle camera 10 into a bird's-eye view image, a conventional technique (for example, JP-A-10-211849) can be used. The processing for correcting distortion caused by the wide-angle lens of the in-vehicle camera 10 is also a well-known technique.

  FIG. 3 shows an example of an original image and a bird's-eye view image. In the figure, the left side is the original image and the right side is the bird's-eye view image. The road marking in the figure indicates a speed limit of 50 kilometers per hour. The road marking in the original image is distorted by the wide-angle lens, and the road marking is inverted vertically and horizontally. Further, the road marking of the bird's-eye view image is corrected for distortion caused by the wide-angle lens, and the road marking is corrected to the original direction. In this way, the original image captured by the in-vehicle camera 10 is converted into a bird's-eye image representing a bird's-eye view looking down on the ground surface from the vertical direction.

  Next, it is determined whether or not the shadow area detection flag is 1 (S108). Here, since the shadow area detection flag is 0, the determination in S108 is NO. Next, the shadow area is detected from the original image, and the shadow area detection flag is set to 1 (S110). Specifically, it is determined whether or not the bird's-eye image includes a shadow of the vehicle itself based on whether or not the bird's-eye image includes a low-luminance region, and the bird's-eye image includes the shadow of the vehicle itself. When the determination is made, a low-luminance area included in the bird's-eye view image is detected as a shadow area, and the shadow area detection flag is set to 1. Note that the shadow region detection flag is also set to 1 when it is determined that the bird's-eye view image does not include the shadow of the vehicle itself.

  The original image and the bird's-eye view image shown in FIG. 3 each include a shadow S of the vehicle B itself. Thus, when the shadow S of the vehicle B itself is included in the bird's-eye view image, a low-luminance region included in the bird's-eye view image is detected as a shadow region.

  Next, a horizontal reference line is set at the reference position of the bird's-eye view image (S112). If the shadow of the vehicle itself is not included in the bird's-eye view image, as shown on the left side of FIG. 4, a reference line is set in the horizontal direction at a position closest to the vehicle body side of the bird's-eye view image, and When a shadow is included, as shown on the right side of FIG. 4, the reference line is at a position where the shadow S of the vehicle itself is excluded from the extraction target region and closest to the vehicle main body side of the bird's-eye view image. Is set horizontally.

  Next, a rectangular extraction target area E is specified in the bird's-eye view image and stored in the memory (S114). In the present embodiment, since the rear of the vehicle is photographed by the in-vehicle camera 10, a rectangular extraction target area E having a reference line as an upper side is specified in the bird's-eye view image. Further, the vehicle speed is detected based on the vehicle speed signal input from the vehicle speed sensor 13, and as shown in FIG. 5, the height H of the extraction target area E increases as the vehicle speed increases, and the extraction target area E decreases as the vehicle speed decreases. The height H of the extraction target region E is changed according to the vehicle speed so that the height H of In this way, the height H of the extraction target area E is changed in accordance with the vehicle speed in the next S116 by extracting images of the extraction target area E in time series from a plurality of bird's-eye images and connecting them in the vertical direction. This is because the shooting area of the road surface is continuous when the composite image is generated.

  In this way, a rectangular extraction target area having a horizontal reference line as one side is specified in a part of the bird's-eye view image on the vehicle body side.

  As shown in FIG. 6, the image processing device 14 performs a process for correcting distortion caused by the wide-angle lens on all frames of the captured image sent from the in-vehicle camera 10, and flips it up, down, left, and right. After that, it is converted into a bird's-eye view image and stored in the VRAM of the memory 14b one by one. The VRAM of the memory 14b is allocated with a memory area (for example, several tens of bird's-eye images) necessary for recognizing road markings, and a plurality of bird's-eye images are overwritten in this memory area. It has come to be.

  In the next S116, a composite image is generated by extracting images of the extraction target region E in time series from a plurality of bird's-eye images stored in the VRAM of the memory 14b and connecting them in the vertical direction and storing them in another region of the VRAM. . Specifically, an image of the extraction target area E is extracted in time series from a plurality of bird's-eye images stored in the VRAM, and an image obtained by connecting the extracted images in the vertical direction is drawn in another area of the VRAM to be a composite image. Is generated. When the vehicle is moving forward, a composite image is generated by connecting the image extracted later to the upper side of the previously extracted image. When the vehicle is moving backward, Next, images extracted later are connected to generate a composite image.

  FIG. 7 shows a state in which a composite image is generated by connecting images of the extraction target region E extracted from a plurality of bird's-eye images when the vehicle is moving forward. In order to eliminate the shadow S of the vehicle itself from the extraction target region E, images of the extraction target region E are extracted from a plurality of bird's-eye images in the order of (a)... (B). An image extracted later is connected to the upper side of the generated image to generate a composite image.

  In addition, when the image of the extraction target area E is extracted from a plurality of bird's-eye images in a state where the extraction target area E includes the shadow S of the vehicle B itself, as shown in FIG. 8, the combined image includes the shadow S of the vehicle B itself. In this embodiment, the image of the extraction target region E is extracted from the plurality of bird's-eye images so that the shadow S of the vehicle itself is excluded from the extraction target region E. Therefore, it is possible to suppress a decrease in the recognition rate of the road marking due to the shadow S of the vehicle itself.

  If the height H of the extraction target area E is a fixed value and the vehicle stops, the same area of the road marking is repeatedly extracted as the extraction target area E as shown in FIG. In this embodiment, the height H of the extraction target region E is changed according to the vehicle speed, and the extraction target regions are extracted in time series from a plurality of bird's-eye images. When the images are extracted and connected in the vertical direction, the shooting area on the road surface can be made continuous.

  Next, the road marking drawn on the road surface for the composite image stored in the VRAM is recognized (S118). Specifically, the road sign is recognized by collating the composite image with a road sign template prepared in advance, and the recognized result is output to an external device (not shown). The external device notifies the driver of the speed limit, for example, according to the recognition result.

  Next, it is determined whether or not the ignition switch is turned off (S120). If the ignition switch remains on, the determination in S120 is NO, and a process of setting the shadow area detection flag to 0 is performed when the following conditions (1) to (3) are satisfied (S122).

(1) When it is determined that the azimuth of the vehicle has changed by more than a predetermined amount of azimuth (2) When the presence or absence of solar radiation irradiated to the vehicle has changed (3) The inclination of the vehicle has changed by more than a predetermined angle In the case where one of the conditions (1) to (3) is satisfied, the shadow of the vehicle itself included in the bird's-eye view image changes, so the shadow region detection flag is set so that the extraction target region is specified again. If it is set to 0 and none of the conditions (1) to (3) is satisfied, the shadow area detection flag is set to 1 and the process returns to S104.

  Therefore, when at least one of the conditions (1) to (3) is satisfied and the shadow area detection flag is set to 0, the determination in S108 is NO, and the extraction target area is specified in S110, S112, and S114. The processing of S116 and S118 is performed again. If none of the conditions (1) to (3) is satisfied, the setting of the shadow area detection flag remains 1 and the determination in S108 is YES, and the extraction target area is not changed, and S116 and S118. The process is implemented. That is, the extraction target area is specified again only when at least one of the conditions (1) to (3) is satisfied, so that the processing load on the image processing device 14 can be reduced.

  According to the configuration described above, a composite image is generated in which a part of the bird's-eye image on the vehicle main body side is used as the extraction target region, and the images of the extraction target region are extracted and connected in time series from the plurality of bird's-eye images. That is, the composite image is generated by connecting images in a region near the vehicle body with high accuracy among the bird's-eye images. And since the image recognition of the road marking drawn on the road surface for this composite image is performed, the road marking can be recognized more accurately regardless of the shape of the road marking.

  In addition, when the rear side of the vehicle is photographed by the in-vehicle camera, it is possible to recognize the road marking earlier by extracting a region close to the vehicle body of the bird's-eye view image and generating a composite image.

  In addition, since a reference line is set at a position closest to the vehicle main body side of the bird's-eye view image and a rectangular extraction target area having this reference line as one side is specified in the bird's-eye view image, the vehicle body having high accuracy among the bird's-eye images Can be combined to generate a composite image.

  In addition, when the bird's-eye image includes a shadow of the vehicle itself, there is a high possibility that the road marking is erroneously recognized due to the shadow of the vehicle itself. In such a configuration, the shadow of the vehicle itself is included in the bird's-eye image. When it is determined that the image is included, the reference line is set in the horizontal direction at a position where the shadow of the vehicle itself is excluded from the extraction target area and closest to the vehicle main body side of the bird's-eye view image. Since a rectangular extraction target region having a reference line as one side is specified, it is possible to reduce the possibility of erroneously recognizing a road marking due to the shadow of the vehicle itself.

  Note that it is possible to determine whether or not the bird's-eye view image includes the shadow of the vehicle itself based on whether or not the low-luminance region is included on the vehicle side of the bird's-eye view image.

  In addition, when it is determined that the vehicle orientation has changed by a predetermined amount or more, the extraction target area is reset to a position where the shadow of the vehicle itself is excluded from the extraction target area. Even when the shadow changes and the shadow of the vehicle itself included in the bird's-eye view image changes, the road marking can be image-recognized without being affected by the shadow of the vehicle itself included in the bird's-eye view image.

  In addition, when it is determined that the presence or absence of solar radiation applied to the vehicle has changed, the extraction target region is reset to a position where the shadow of the vehicle itself is excluded from the extraction target region, so that the vehicle is irradiated. Even if the presence or absence of solar radiation changes and the shadow of the vehicle itself included in the bird's-eye view image changes, it is possible to recognize the road marking without being affected by the shadow of the vehicle itself included in the bird's-eye view image.

  In addition, when it is determined that the inclination of the vehicle has changed by a predetermined angle or more, the extraction target area is reset to a position where the shadow of the vehicle itself is excluded from the extraction target area. Even if the inclination of the vehicle changes and the shadow of the vehicle itself included in the bird's-eye view image changes, the road marking can be recognized without being affected by the shadow of the vehicle itself included in the bird's-eye view image.

  In addition, since the image distortion caused by the wide-angle lens is corrected when the captured image is converted into the bird's-eye view image, the recognition rate of the image recognition of the road marking can be improved.

  In addition, the higher the vehicle speed, the higher the height of the extraction target area so that when the images of the extraction target area are extracted in time series from a plurality of bird's-eye images and connected in the vertical direction, the shooting area of the road surface is continuous. As the vehicle speed is lower, the height of the extraction target area is lower, so that it is possible to obtain a composite image in which shooting areas on the road surface are continuous.

  In addition, this invention is not limited to the said embodiment, Based on the meaning of this invention, it can implement with a various form.

  For example, in the above-described embodiment, an example in which a road marking recognition device recognizes a road marking and outputs the recognized result to an external device has been described. For example, the navigation device includes a road marking recognition device, and the road marking recognition device The travel guidance may be performed using the recognition result. For example, when a road sign indicating a temporary stop line is recognized, if a road sign indicating that there is a temporary stop line or a right turn lane is recognized before an intersection to go straight ahead, You can be guided to run on the lane.

  In the above-described embodiment, the configuration example in which the vehicle-mounted camera using the wide-angle lens is mounted on the vehicle is shown. However, the vehicle-mounted camera using a normal lens instead of the wide-angle lens may be mounted on the vehicle. In such a configuration, the captured image sent from the in-vehicle camera can be converted into a bird's-eye view image without correcting the image distortion by the wide-angle lens as performed in S106.

  Moreover, although the example which mounted the vehicle-mounted camera which image | photographs the back of a vehicle was shown in the said embodiment, it is good also as a structure mounted with the vehicle-mounted camera which image | photographs the front of a vehicle. In the above embodiment, when the captured image is converted into the bird's-eye view image, the image captured by the in-vehicle camera is inverted upside down, but the image captured in front of the vehicle by the in-vehicle camera is a road marking. There is no need to flip the top, bottom, left and right.

  Moreover, in the said embodiment, since it was the structure which image | photographs the vehicle back by the vehicle-mounted camera 10, the example which pinpoints the rectangular extraction object area | region E which makes a reference line an upper side in a bird's-eye view image was shown, In the case where the front is photographed with the in-vehicle camera, a rectangular extraction target region E having a reference line as a lower side may be specified in the bird's-eye view image.

  The correspondence relationship between the configuration of the above embodiment and the configuration of the claims will be described. The in-vehicle camera 10 corresponds to an imaging unit, and the captured image sent from S106 and the in-vehicle camera 10 is converted into a bird's-eye view image. The processing stored in the memory 14b one by one corresponds to the image conversion means, S112 and S114 correspond to the extraction target area specifying means, S116 corresponds to the composite image generation means, and S118 corresponds to the image recognition processing means. A process for determining whether or not the direction of the vehicle has changed by a predetermined amount or more based on a signal input from the gyroscope 12 corresponds to the direction change determination means, and is based on a signal input from the solar radiation sensor 11. The process for determining the presence or absence of solar radiation applied to the vehicle corresponds to the solar radiation determining means, and the vehicle is based on the signal input from the gyroscope 12. The process for determining whether or not the forward / backward inclination of the vehicle has changed more than a predetermined angle corresponds to the inclination determination means, and the process of detecting the vehicle speed based on the vehicle speed signal input from the vehicle speed sensor 13 is the vehicle speed. It corresponds to detection means.

It is a figure which shows the structure of the road marking recognition apparatus which concerns on one Embodiment of this invention. It is a flowchart which shows the image recognition process of an image processing apparatus. It is a figure which shows an original image and a bird's-eye view image. It is a figure for demonstrating the setting position of a reference line when the shadow of a vehicle itself is not included in a bird's-eye view image, and when the shadow of a vehicle itself is included in a bird's-eye view image. It is a figure for demonstrating the process which changes the height of an extraction object area | region according to a vehicle speed. It is a figure for demonstrating the picked-up image and bird's-eye view image of a vehicle-mounted camera. It is a figure which shows a mode that the image of the extraction object area | region extracted from the several bird's-eye view image is connected, and a synthesized image is produced | generated. It is a figure for demonstrating the synthesized image at the time of extracting the image of an extraction object area | region from several bird's-eye view images in the state in which the shadow of the vehicle itself is included in an extraction object area | region. It is a figure for demonstrating the synthesized image when the height of an extraction object area | region is made into a fixed value, and also a vehicle stops. It is a figure for demonstrating a subject.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 ... Road marking recognition apparatus, 10 ... Car-mounted camera, 11 ... Solar radiation sensor,
DESCRIPTION OF SYMBOLS 12 ... Gyroscope, 13 ... Vehicle speed sensor, 14 ... Image processing apparatus.

Claims (9)

An imaging means that is mounted on a vehicle and sends out an image obtained by periodically imaging the road surface around the vehicle;
An image conversion means for converting the captured image sent from the imaging means into a bird's-eye image representing a bird's-eye view looking down on the ground surface from the vertical direction and storing the image in the storage means;
Extraction target area specifying means for specifying an extraction target area in a part on the vehicle body side of the bird's-eye view image converted by the image conversion means;
Synthetic image generation means for generating a composite image obtained by extracting images of the extraction target region in time series from the plurality of bird's-eye images stored in the storage means and connecting them in the vertical direction;
A road sign recognition device, comprising: an image recognition processing means for recognizing a road sign drawn on a road surface for the composite image generated by the composite image generation means. The extraction target area specifying means sets the reference line in the horizontal direction at a position closest to the vehicle main body side of the bird's-eye view image, and the rectangular extraction target area having the reference line as one side in the bird's-eye view image The road marking recognition apparatus according to claim 1, wherein: The extraction target area specifying unit determines whether or not the bird's-eye image includes a shadow of the vehicle itself. If the bird's-eye image does not include a shadow of the vehicle itself, the extraction target area specifying unit The reference line is set in a horizontal direction at a position closest to the main body side, and the rectangular extraction target region having the reference line as one side is specified in the bird's-eye view image, and the shadow of the vehicle itself is added to the bird's-eye view image. When included, the reference line is set in the horizontal direction at a position where the shadow of the vehicle itself is excluded from the extraction target area and at a position closest to the vehicle main body side of the bird's-eye view image. The road marking recognition apparatus according to claim 1, wherein the extraction target area having a rectangular shape with the reference line as one side is specified in the bird's-eye view image. The extraction target area specifying means determines whether or not the bird's-eye view image includes a shadow of the vehicle itself based on whether or not a low-luminance area is included on the vehicle side of the bird's-eye view image converted by the image conversion means. The road marking recognition device according to claim 3, wherein the road marking recognition device is determined. Comprising an azimuth change determining means for determining whether the azimuth of the vehicle has changed by a predetermined amount or more;
The extraction target area specifying means re-executes specification of the extraction target area when it is determined by the azimuth change determining means that the azimuth of the vehicle has changed by a predetermined azimuth amount or more. Item 5. A road marking recognition device according to item 3 or 4. Comprising solar radiation determination means for determining whether or not the presence or absence of solar radiation irradiated to the vehicle has changed,
The extraction target area specifying means re-specifies the extraction target area when it is determined by the solar radiation determining means that the presence or absence of the solar radiation irradiated to the vehicle has changed. 5. The road marking recognition apparatus according to any one of 5 above. Inclination determining means for determining whether or not the inclination of the vehicle has changed by a predetermined angle or more,
The extraction target area specifying unit re-specifies the extraction target area when the inclination determination unit determines that the inclination of the vehicle has changed by a predetermined angle or more. The road marking recognition apparatus according to any one of 6 to 6. The photographing means photographs a road surface around the vehicle using a wide-angle lens,
8. The image conversion unit according to claim 1, wherein the image conversion unit corrects image distortion caused by the wide-angle lens and converts the captured image sent from the imaging unit into the bird's-eye view image. 9. The road marking recognition device described. Vehicle speed detecting means for detecting the vehicle speed of the vehicle,
The extraction target area specifying unit is configured to detect the vehicle speed so that when the images of the extraction target area are extracted in time series from the plurality of bird's-eye images and connected in the vertical direction, the shooting areas of the road surface are continuous. The height of the extraction target area is increased as the vehicle speed detected by the means is higher, and the height of the extraction target area is decreased as the vehicle speed is lower. The road marking recognition device described.

Images