JP2014191475A - Driving support device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a driving support device capable of suppressing deterioration in visibility of an image after processing, and capable of suppressing deterioration in accuracy of driving support.SOLUTION: A driving support device 10 comprises: a plurality of cameras 21-24 that capture the periphery of its own vehicle; and an image processing unit 35 that processes respective images captured by the plurality of cameras 21-24. The driving support device 10 further comprises an extraction unit 33 that extracts an inappropriate image which is inappropriate for processing among the respective images. The image processing unit 35 excludes the inappropriate image.

Description

  The present invention relates to a driving support device.

  Conventionally, as a driving support device for a vehicle such as an automobile, for example, a plurality of images captured by a plurality of cameras mounted on the vehicle are processed and combined, and an object around the host vehicle (for example, a forward vehicle or an obstacle) ) And the like are displayed on a display to guide the route of the host vehicle.

  By the way, in general, a plurality of cameras mounted on a vehicle are attached to different positions in order to obtain information on the surroundings of the vehicle. For this reason, the images picked up by the respective cameras differ in luminance, saturation, contrast, and the like due to, for example, differences in light insertion. Therefore, when a plurality of images taken by each camera are combined and displayed on the display, a combined image having a predetermined brightness, saturation, contrast, etc. cannot be obtained by simply combining the images. Visibility may deteriorate.

  In order to solve the above-described problem, for example, Patent Document 1 discloses a pixel data of an image captured by an adjacent camera among a plurality of cameras as a method of processing an image captured by a plurality of cameras of a driving support device. A camera image composition display method is described in which a difference between the images is corrected to be small and a composite image of a plurality of cameras is displayed on a screen. Specifically, an image closest to an average value of each image among a plurality of images captured by a plurality of (eight in Patent Document 1) cameras is set as a reference image. Then, the image of the adjacent camera is corrected so that the brightness of the image captured by the camera adjacent to the reference camera that captures the reference image is equal to the brightness of the reference image. Thereby, it is supposed that a composite image that is easy to visually recognize can be generated in real time from images captured by a plurality of in-vehicle cameras.

JP 2002-324235 A

  However, in Patent Document 1, for example, when there are a plurality of images with extremely high luminance and poor visibility, an image with poor visibility may be selected as a reference image as an image closest to the average value of each image. There is. For this reason, when each image is corrected and processed based on a reference image with poor visibility, the visibility of the processed image may be deteriorated, and the accuracy of driving assistance using the processed image May decrease.

  Therefore, the present invention has been made in view of the above-described circumstances, and can provide a driving assistance device that can suppress deterioration in the visibility of an image after processing and can suppress deterioration in driving assistance accuracy. Is an issue.

  In order to solve the above-mentioned problem, the driving support device according to claim 1 of the present invention (for example, the driving support device 10 in the embodiment) images the periphery of the host vehicle (for example, the host vehicle 1 in the embodiment). A plurality of imaging units (for example, the front camera 21, the rear camera 22, the left side camera 23, and the right side camera 24 in the embodiment) and each image captured by the plurality of the imaging units (for example, the front camera in the embodiment) An image processing unit (for example, the image processing unit 35 in the embodiment) that processes the image 21a of the rear camera 22, the image 22a of the rear camera 22, the image 23a of the left side camera 23, and the image 24a of the right side camera 24). In the driving support device, out of each of the images, an inappropriate image inappropriate for the processing (for example, the image 21a of the front camera 21 in the embodiment) is extracted. Extractor for (e.g., the extraction unit 33 in the embodiment) further wherein the image processing unit is characterized to exclude the inappropriate images.

  According to the present invention, the image processing unit includes an extraction unit that extracts an inappropriate image inappropriate for processing of each image, and the image processing unit excludes the inappropriate image, so that each image can be used without using the inappropriate image. Can be processed. Thereby, since each image can be processed without being affected by an inappropriate image, it is possible to suppress deterioration of the visibility of the processed image. Moreover, it can suppress that the precision of driving assistance falls by using the image after the process by which the deterioration of visibility was suppressed in the case of the driving assistance by a driving assistance device.

  The invention according to claim 2 of the present invention is characterized in that each of the plurality of imaging units images different directions around the own vehicle.

  According to the present invention, it is possible to vary the influence of light such as sunlight between the images by imaging different directions around the host vehicle with a plurality of imaging units. Therefore, the degree of influence of light such as sunlight in each image can be accurately detected by comparing each image, and the number of inappropriate images generated by being significantly affected by light such as sunlight can be calculated. Can be reduced. That is, since the number of inappropriate images to be excluded can be reduced, the accuracy of the processed image can be improved and the visibility can be further suppressed from deteriorating.

  Moreover, the invention according to claim 3 of the present invention includes a calculation unit (for example, the calculation unit 31 in the embodiment) that calculates a numerical value due to the brightness of each of the images captured by the imaging unit. The extraction unit extracts the inappropriate image based on a result of the calculation unit.

  According to the present invention, since the operation unit that calculates the numerical value resulting from the brightness of each image captured by the image capturing unit is provided, the influence of light such as sunlight is affected by, for example, the brightness and saturation of the image, It can be quantified as contrast or the like. Further, since the extraction unit extracts an inappropriate image based on the result of the calculation unit, it is possible to reliably extract the inappropriate image based on the digitized data. Therefore, an inappropriate image generated by being significantly affected by light such as sunlight can be surely excluded, so that the accuracy of the processed image can be improved and the visibility can be further suppressed from deteriorating.

  In the invention according to claim 4 of the present invention, the arithmetic unit is configured to obtain an average value of numerical values resulting from the brightness of each of the images (for example, average luminance values K1, K2, K3, K4 in the embodiment). The extraction unit extracts, as the inappropriate image, the image whose average value is equal to or greater than a predetermined value (for example, the predetermined value α in the embodiment) among the images. .

  According to the present invention, an inappropriate image is extracted using a predetermined value as a threshold value, and therefore an inappropriate image generated by being significantly affected by light such as sunlight can be reliably excluded.

  In the invention according to claim 5 of the present invention, the arithmetic unit is configured to obtain an average value of numerical values resulting from the brightness of each of the images (for example, average luminance values K1, K2, K3, K4 in the embodiment). The extraction unit calculates the average value of one of the images (for example, the image 21a of the front camera 21 in the embodiment) and another image (for example, the image of the rear camera 22 in the embodiment). 22a, the total value of the difference between the average value of the image 23a of the left-side camera 23 and the image 24a) of the right-side camera 24, or the difference between the average value of the one image and the average value of the other image When the average value is equal to or greater than a predetermined value (for example, the predetermined values D1 and D2 in the embodiment), the one image is extracted as the inappropriate image.

  According to the present invention, the total value of the differences between the average value of one image and the average value of the other image, or the average value of the difference between the average value of the one image and the average value of the other image is a predetermined value or more. In this case, since one image is extracted as an inappropriate image, the one image is compared with another image obtained by capturing a different direction from the one image, Can be definitely excluded.

  In the invention according to claim 6 of the present invention, the image processing unit acquires straight line information in each of the images (for example, straight line information N1, N2, N3, N4, N5, N6 in the embodiment). The extraction unit extracts the inappropriate image based on the straight line information.

  According to the present invention, since linear noise in an image can be acquired as straight line information, an inappropriate image can be extracted.

  Moreover, the invention according to claim 7 of the present invention is a map information acquisition unit (for example, a roadway center line L1 and a roadway outer line L2 in the embodiment) that acquires the map information of the vicinity where the host vehicle is located The map information acquisition unit 45) according to the embodiment is provided, and the extraction unit extracts the inappropriate image based on the straight line information and the map information.

  According to the present invention, since the extraction unit extracts an inappropriate image based on the straight line information and the map information, it is possible to accurately select erroneous straight line information due to noise among a plurality of straight line information and An appropriate image can be reliably extracted.

  Further, in the invention according to claim 8 of the present invention, the extraction unit includes the straight line information and the map information in one of the images (for example, the image 21a of the front camera 21 in the embodiment). When the number of the straight line information that matches the map information is a predetermined number or more, or when the straight line information that matches the map information does not exist at all, the one image is The image is extracted as the inappropriate image.

  According to the present invention, since one image is extracted as an inappropriate image by comparing the straight line information and the map information, for example, erroneous straight line information caused by noise can be accurately selected. In particular, even when there is no straight line information, since one image is extracted as an inappropriate image, for example, an inappropriate image with a small amount of received light and a remarkably low visibility can be surely excluded.

  According to the present invention, the image processing unit includes an extraction unit that extracts an inappropriate image inappropriate for processing of each image, and the image processing unit excludes the inappropriate image, so that each image can be used without using the inappropriate image. Can be processed. Thereby, since each image can be processed without being affected by an inappropriate image, it is possible to suppress deterioration of the visibility of the processed image. Moreover, it can suppress that the precision of driving assistance falls by using the image after the process by which the deterioration of visibility was suppressed in the case of the driving assistance by a driving assistance device.

It is a system block diagram of a driving assistance device. It is a top view of the own vehicle carrying a driving assistance device. It is explanatory drawing of the calculation by a calculating part, and the extraction by an extraction part. It is a flowchart of the process of the image which the driving assistance device of 1st embodiment performs. It is explanatory drawing of extraction of the inappropriate image in each modification of 1st embodiment. It is explanatory drawing of the process of the image which the driving assistance device of 2nd embodiment performs. It is a flowchart of the process of the image which the driving assistance device of 2nd embodiment performs.

Hereinafter, the driving support apparatus according to the first embodiment will be described with reference to the drawings.
FIG. 1 is a system configuration diagram of the driving support device 10, and FIG. 2 is a plan view of the host vehicle 1 on which the driving support device 10 is mounted.
As shown in FIG. 1, the driving support device 10 according to the present embodiment mainly includes an external camera unit 25, a navigation system 40, a calculation unit 31, an extraction unit 33, an image processing unit 35, and a storage unit 37. Device 30. As shown in FIG. 2, the driving support device 10 is mounted on the host vehicle 1 and displays an image around the host vehicle 1 to the driver. Below, the detail of each component which comprises the driving assistance apparatus 10 is demonstrated. Note that the directions such as front, rear, left and right in the following description are the same as those in the host vehicle 1 shown in FIG. 2 unless otherwise specified.

  The external camera unit 25 is for imaging different directions in the vicinity of the host vehicle 1, and is, for example, a front camera 21 having a peripheral region in front of the host vehicle 1 as an imaging region ("imaging unit" in claims) ), A rear camera 22 (corresponding to “imaging unit” in the claims) having a peripheral region behind the host vehicle 1 as an imaging region, and a peripheral region on the left side of the host vehicle 1 as an imaging region. A left-side camera 23 (corresponding to “imaging unit” in claims) and a right-side camera 24 (corresponding to “imaging unit” in claims) whose imaging region is a peripheral region on the right side of the host vehicle 1. I have. Each of the cameras 21 to 24 is a camera provided with an image sensor such as a CCD (Charge-Coupled Device) or a CMOS (Complementary Metal Oxide Semiconductor, complementary metal oxide semiconductor).

  As shown in FIG. 1, the external camera unit 25 outputs an image obtained by imaging with the cameras 21 to 24 to the control device 30. Each image obtained by the imaging of each of the cameras 21 to 24 is synthesized by an image processing unit 35 described later provided in the control device 30. Thereby, an image (hereinafter referred to as “overhead synthesized image”) obtained by performing viewpoint conversion so as to overlook the host vehicle 1 (see FIG. 2) is obtained, and the overhead view synthesized image is transmitted via the control device 30 to the navigation system 40. To be displayed on the display 43.

The navigation system 40 is for displaying the current position of the host vehicle 1 (see FIG. 2), or for guiding the host vehicle 1 (see FIG. 2) by searching for a route from the current position to the destination. The navigation control unit 41 and the display 43 are mainly provided. As shown in FIG. 2, the navigation system 40 is accommodated in an instrument panel (not shown) of the host vehicle 1, for example.
The navigation control unit 41 mainly includes a current position detection unit (not shown), a map data storage unit, and a navigation processing unit.
The current position detection unit receives a positioning signal such as a GPS (Global Positioning System) signal using an artificial satellite, and detects the current position of the host vehicle 1.
The map data storage unit is configured by recording map data on a computer-readable storage medium such as a magnetic disk device such as a hard disk device or an optical disk device such as a CD-ROM or DVD-ROM. As the map data, for example, road data such as road and intersection positions and shapes, building data such as building positions and shapes, and parking lot data such as parking positions and shapes are stored.
The navigation processing unit searches for the travel route of the host vehicle 1 based on the current position detected by the current position detection unit, the destination input to the navigation system 40, the map data read from the map data storage unit, and the like. To do.

  The display 43 is a liquid crystal display, for example, and is installed in an instrument panel (not shown) of the host vehicle 1 so that the driver can visually recognize it. The display 43 displays an image based on the map data of the navigation control unit 41, an overhead view composite image of the host vehicle 1 output from the control device 30, and the like.

  In addition, the driving support device 10 may include a map information acquisition unit 45. The map information acquisition unit 45 is a mobile information terminal such as a mobile phone using an existing mobile communication network. The map information acquisition unit 45 is provided with map information such as a parking lot provided from an information server (not shown) (for example, map information of a white line defining the parking lot) and map information such as a road (for example, a roadway on a road) Map information of lane markings such as Chuo Line and roadway outside line). The map information acquisition unit 45 is connected to the navigation system 40 by existing communication means and has an application function for sharing map information with the navigation system 40.

As shown in FIG. 1, the control device 30 includes a calculation unit 31, an extraction unit 33, an image processing unit 35, and a storage unit 37, and the external camera unit 25 and the navigation system 40 described above. Is connected.
By connecting the external camera unit 25, images around the host vehicle 1 (see FIG. 2) obtained by the cameras 21 to 24 are input to the control device 30. Further, the control device 30 outputs an image around the host vehicle 1 to the navigation system 40 and displays it on the display 43.

FIG. 3 is an explanatory diagram of calculation by the calculation unit 31 (see FIG. 1) and extraction by the extraction unit 33 (see FIG. 1). Note that the horizontal axis in FIG. 3 indicates luminance.
The calculation unit 31 calculates a numerical value resulting from the brightness of each image captured by each camera 21 to 24. More specifically, as shown in FIG. 3, for example, the luminance in each of the images 21 a to 24 a is calculated as a numerical value resulting from the brightness of each of the images 21 a to 24 a captured by the cameras 21 to 24. Average values K1 to K4 (corresponding to “average value of numerical values attributed to image brightness” in claims, hereinafter referred to as “average luminance value”) are calculated for each of the images 21a to 24a. In FIG. 3, as an example, the case where the average luminance value K1 in the image 21a of the front camera 21 is the highest is illustrated.

  In the present embodiment, luminance is described as an example of the “numerical value resulting from brightness” calculated by the calculation unit 31, but the “numerical value resulting from brightness” is not limited to luminance. That is, the “numerical value resulting from brightness” is a numerical value related to the visibility of each of the images 21a to 24a and refers to brightness, brightness, saturation, contrast, and the like. Therefore, the calculation unit 31 may calculate brightness, saturation, contrast, and the like as numerical values resulting from the brightness of the images 21a to 24a captured by the cameras 21 to 24.

  The extraction unit 33 (see FIG. 1) is inappropriate for processing among the images 21a to 24a captured by the cameras 21 to 24 based on the calculation result of the calculation unit 31 (see FIG. 1). Extract images. The extraction unit 33 (see FIG. 1) of the present embodiment does not recognize the image 21a of the front camera 21 whose average luminance value is equal to or greater than the predetermined value α among the images 21a to 24a, as in the example illustrated in FIG. Extract as an appropriate image. The predetermined value α for extracting an inappropriate image is, for example, a luminance value that significantly reduces the visibility of the image, and when strong light is incident on each of the cameras 21 to 24 and blooming or halation occurs. It is determined corresponding to the average luminance value in the image.

  The image processing unit 35 (see FIG. 1) processes the images 21a to 24a captured by the cameras 21 to 24. The image processing unit 35 (see FIG. 1) mainly corrects the luminance of the images 21a to 24a captured by the cameras 21 to 24 (hereinafter referred to as “brightness correction processing”), for example, as the processing of the images 21a to 24a. And a process of synthesizing the images 21a to 24a to create an overhead view composite image (hereinafter referred to as "overhead view composite image creation process").

In the brightness correction process, the brightness of each of the images 21a to 24a captured by the cameras 21 to 24 is corrected. As a correction method, the average luminance values K1, K2, K3, and K4 of the image 21a of the front camera 21, the image 22a of the rear camera 22, the image 23a of the left side camera 23, and the image 24a of the right side camera 24 are further An average value (hereinafter referred to as “overall average value”) is obtained. Then, using the overall average value as a target value, the average luminance value K1 of the image 21a of the front camera 21, the average luminance value K2 of the image 22a of the rear camera 22, the image 23a of the left camera 23, the average luminance value K3, and the right camera 24 The average luminance value K4 of the image 24a is corrected.
In the overhead view composite image creation process, the image 21a of the front camera 21, the image 22a of the rear camera 22, the image 23a of the left side camera 23, and the image 24a of the right side camera 24 are synthesized to create an overhead view composite image.
In addition, the image processing unit 35 filters and removes obstacle data that hinders the calculation of the average luminance value of the image, such as data of a light source of another vehicle that locally increases the luminance in the image.

Here, the image processing unit 35 (see FIG. 1) of the present embodiment is an inappropriate image (in the example illustrated in FIG. 3 of the present embodiment) among the images 21a to 24a captured by the cameras 21 to 24. If the image 21a) of the front camera 21 exists, the process is performed after removing the inappropriate image.
Specifically, in the luminance correction process, the image 21a of the front camera 21 that is an inappropriate image is excluded from the target of the luminance correction process. Next, the average luminance values K2, K3, and K4 of the image 22a of the rear camera 22, the image 23a of the left camera 23, and the image 24a of the right camera 24, which are images other than the image 21a of the front camera 21, are further averaged. A value (hereinafter referred to as “overall average value”) is obtained. Then, the average luminance value K2 of the image 22a of the rear camera 22, the average luminance value K3 of the image 23a of the left side camera 23, and the average luminance value K4 of the image 24a of the right side camera 24 are corrected using the overall average value as a target value. Thereby, the brightness | luminance of each other images 22a-24a other than the image 21a of the front camera 21 can be correct | amended, without being influenced by the image 21a of the front camera 21 made into the inappropriate image.

  In the overhead view composite image creation process, the image 21a of the front camera 21 that is an inappropriate image is excluded, and the image 22a of the rear camera 22 and the image 23a of the left-side camera 23 that are images other than the image 21a of the front camera 21 are excluded. Then, an overhead view composite image is created from the image 24a of the right side camera 24. Thereby, a bird's-eye view synthesized image can be created without displaying the image 21a of the front camera 21 which is an inappropriate image with poor accuracy. In addition, while excluding the image 21a of the front camera 21 that is an inappropriate image from the target of the brightness correction process, the image 21a of the front camera 21 and the rear side are excluded without excluding the image 21a of the front camera 21 in the overhead view composite image creation process. An overhead view composite image may be created from the image 22a of the camera 22, the image 23a of the left camera 23, and the image 24a of the right camera 24.

  The storage unit 37 includes, for example, a memory such as a flash memory, a ROM (Read Only Memory), an EEPROM (Electrically Erasable Programmable Read-Only Memory), and a RAM (Random Access Memory). The storage unit 37 stores various data output from the external camera unit 25, the navigation system 40, and the like to the control device 30, for example.

FIG. 4 is a flowchart of image processing performed by the driving support apparatus according to the first embodiment.
Next, image processing performed by the driving support apparatus according to the present embodiment will be described with reference to the flowchart shown in FIG. In addition, please refer to FIG. 1 to FIG.

In step S <b> 1, the images 21 a to 24 a captured by the cameras 21 to 24 are stored in the storage unit 37 of the control device 30, for example.
In step S <b> 3, for each of the images 21 a to 24 a stored in the storage unit 37, for example, it is determined whether or not failure data such as light source data of another vehicle exists. If there is no failure data in step S3, the determination is “NO” and the process proceeds to step S7. On the other hand, in step S3, when failure data exists, it determines with "YES" and progresses to step S5, and after filtering and removing the failure data of the target image, it progresses to step S7.
In step S7, average luminance values K1 to K4 of the images 21a to 24a captured by the cameras 21 to 24 are calculated.

In step S9, the average luminance values K1 to K4 of the images 21a to 24a are compared with a predetermined value α to determine whether there is an image having an average luminance value equal to or larger than the predetermined value α. In step S9, if none of the average luminance values K1 to K4 of the images 21a to 24a is greater than or equal to the predetermined value α, it is determined as “NO” and the process proceeds to step S11.
In step S11, each image 21a-24a imaged by each camera 21-24 is processed. Specifically, after correcting the brightness of each of the images 21a to 24a, the images 21a to 24a are combined to create an overhead view composite image. When the processing of each of the images 21a to 24a is finished, the image processing performed by the driving support device 10 is finished.

On the other hand, if the average luminance value of at least one of the average luminance values K1 to K4 of the images 21a to 24a is greater than or equal to the predetermined value α in step S9, “YES” is determined. Then, the process proceeds to step S13. For example, as in the example illustrated in FIG. 3, when the average luminance value K1 of the image 21a of the front camera 21 is equal to or greater than a predetermined value α, “YES” is determined. Then, it is determined that the image 21a of the front camera 21 is an inappropriate image, and the process proceeds to step S13.
In step S13, inappropriate images (for example, the image 21a of the front camera 21 in the example shown in FIG. 3) are excluded.
In step S15, an image other than an inappropriate image (for example, in the example shown in FIG. 3, the image 22a of the rear camera 22, the image 23a of the left camera 23, and the image 24a of the right camera 24) is processed. Specifically, after correcting the brightness of the images 22a to 24a other than the inappropriate images, the images 22a to 24a other than the inappropriate images are combined to create an overhead view composite image. When the processing of each of the images 22a to 24a is completed, the image processing performed by the driving support device 10 is completed. In step S15, the image 21a of the front camera 21 that is an inappropriate image is excluded from the target of the brightness correction process, and the image 21a of the front camera 21 is not excluded in the overhead view composite image creation process. An overhead view composite image may be created from the image 21a, the image 22a of the rear camera 22, the image 23a of the left camera 23, and the image 24a of the right camera 24.

  According to the present embodiment, the image processing unit 35 includes the extraction unit 33 that extracts an inappropriate image inappropriate for processing of each image, and the image processing unit 35 excludes the inappropriate image, so that the inappropriate image is not used. Each image can be processed. Thereby, since each image can be processed without being influenced by an inappropriate image, it is possible to suppress deterioration of the visibility of the overhead view composite image after processing. Moreover, it can suppress that the precision of a driving assistance falls by using the bird's-eye view synthesized image in which the deterioration of visibility was suppressed in the case of the driving assistance by the driving assistance apparatus 10. FIG.

  Moreover, the influence of light, such as sunlight, can be varied between each image by imaging the different direction in the periphery of the own vehicle 1 with the some cameras 21-24. Therefore, the degree of influence of light such as sunlight in each image can be accurately detected by comparing each image, and the number of inappropriate images generated by being significantly affected by light such as sunlight can be calculated. Can be reduced. That is, since the number of inappropriate images to be excluded can be reduced, it is possible to improve the accuracy of the overhead view composite image after processing and further suppress deterioration in visibility.

  Moreover, since the calculating part 31 which calculates the brightness | luminance of each image imaged with each camera 21-24 is provided, the influence of light, such as sunlight, can be quantified. Moreover, since the extraction part 33 extracts an inappropriate image based on the result of the calculating part 31, it can extract an inappropriate image reliably based on the digitized data. Therefore, an inappropriate image generated by being significantly affected by light such as sunlight can be surely excluded, so that it is possible to improve the accuracy of the overhead view composite image and further suppress the deterioration of the visibility.

  In addition, since an inappropriate image is extracted with the predetermined value α as a threshold, an inappropriate image generated by being significantly affected by light such as sunlight can be reliably excluded.

(Modification of the first embodiment)
Next, driving support devices according to modifications of the first embodiment will be described. In addition, below, description is abbreviate | omitted about the component similar to 1st embodiment, and only a different part is demonstrated.
As shown in FIG. 3, in the first embodiment, among the images 21a to 24a, an image having an average luminance value equal to or greater than a predetermined value α (an image 21a of the front camera 21 in the example shown in FIG. 3) is not used. It was extracted as an appropriate image. On the other hand, the technique for extracting inappropriate images is not limited to the above technique.

(First modification of the first embodiment)
FIG. 5 is an explanatory diagram of extraction of inappropriate images in each modification of the first embodiment. In addition, please refer to FIG. 1 to FIG.
As shown in FIG. 5, in the first modification example of the first embodiment, the extraction unit 33 uses the average luminance value of one of the images 21 a to 24 a (for example, the front camera in the example shown in FIG. 5). 21 and the average luminance value of other images (for example, in the example shown in FIG. 5, the average luminance value K2 of the image 22a of the rear camera 22 and the average luminance of the image 23a of the left-side camera 23). When the total value of the difference between the value K3 and the average brightness value K4 of the image 24a of the right side camera 24 is equal to or greater than a predetermined value, one image (for example, the image of the front camera 21 in the example shown in FIG. 5). 21a) is extracted as an inappropriate image.
That is, when the predetermined value is D1,
(K1-K2) + (K1-K3) + (K1-K4) ≧ D1
Is satisfied, the image 21a of the front camera 21 is extracted as an inappropriate image.

(Second modification of the first embodiment)
Further, in the second modification of the first embodiment, the average luminance value of one image (for example, the average luminance value K1 of the image 21a of the front camera 21 in the example shown in FIG. 5) and the average luminance value of other images. (For example, in the example shown in FIG. 5, the average luminance value K2 of the image 22a of the rear camera 22, the average luminance value K3 of the image 23a of the left side camera 23, and the average luminance value K4 of the image 24a of the right side camera 24) If the average value of the differences is greater than or equal to a predetermined value, one image (for example, the image 21a of the front camera 21 in the example shown in FIG. 5) is extracted as an inappropriate image.
That is, when the predetermined value is D2,
((K1-K2) + (K1-K3) + (K1-K4)) / 3 ≧ D2
Is satisfied, the image 21a of the front camera 21 is extracted as an inappropriate image.

  According to each modification of the first embodiment, the total value of the difference between the average value of one image and the average value of the other image, or the difference between the average value of one image and the average value of the other image When the average value is greater than or equal to the predetermined value, one image is extracted as an inappropriate image, so compare one image with another image obtained by capturing a different direction from the one image. Thus, inappropriate images can be surely excluded.

(Second embodiment)
Next, the driving support device according to the second embodiment will be described. Note that the system configuration diagram of the driving support apparatus of the second embodiment is the same as FIG. Moreover, the code | symbols, such as a component common in 1st embodiment in 2nd embodiment, are the same as the codes | symbols of each component in 1st embodiment.
The driving support apparatus 10 according to the first embodiment performs image processing by extracting and excluding inappropriate images when creating a bird's-eye view composite image from the images of the cameras 21 to 24.
On the other hand, in the driving support device 10 according to the second embodiment, an inappropriate image is used when recognizing the road center line and the road outer line provided on the road from the images of the cameras 21 to 24, for example. Are extracted and excluded to perform image processing. In addition, below, description is abbreviate | omitted about the component similar to 1st embodiment, and only a different part is demonstrated.

FIG. 6 is an explanatory diagram of image processing performed by the driving support apparatus of the second embodiment. In FIG. 6, the image of the imaging region R1 of the front camera 21 is illustrated by a two-dot chain line, and the images of the imaging regions of the other cameras 22 to 24 are not illustrated.
As shown in FIG. 6, the driving assistance device 10 (see FIG. 1) is for driving the host vehicle 1 along the road center line L1 and the road outer line L2 provided on the road W on which the host vehicle 1 travels. When a necessary steering assist torque is output to an electric power steering unit (not shown) and the own vehicle 1 protrudes from the road center line L1 and the road outer line L2, An alarm is output. That is, the driving support device 10 is a device that performs so-called LKAS (Lane Keeping Assistance System) control and LDW (Lane Departure Warning) control.

As shown in FIG. 1, the driving support device 10 includes a map information acquisition unit 45. The map information acquisition unit 45 of the second embodiment acquires map information of the road center line L1 and the road outer line L2 (see FIG. 6) provided on the road W from an external information server (not shown).
The image processing unit 35 acquires straight line information from the images of the cameras 21 to 24 stored in the storage unit 37. Here, the straight line information in the present embodiment is information on the road center line L1 and the road outer line L2 provided on the road W on which the host vehicle 1 travels (see FIG. 6). In this embodiment, straight line information necessary for LKAS control is obtained mainly from images captured by the front camera 21 and the rear camera 22, and straight line information necessary for LDW control is mainly obtained by the left camera 23 and the right camera. It is acquired from the image captured by the direction camera 24.
The extraction unit 33 extracts inappropriate images based on the straight line information acquired from the images of the cameras 21 to 24 and the map information acquired by the map information acquisition unit 45. A specific method of extracting inappropriate images by the extraction unit 33 will be described later.

FIG. 7 is an explanatory diagram of image processing performed by the driving support apparatus of the second embodiment.
Next, image processing performed by the driving support device of the present embodiment will be described with reference to the flowchart shown in FIG. In addition, please refer to FIG. 1 and FIG.

First, in step S <b> 21, each image captured by each camera 21 to 24 is stored in the storage unit 37 of the control device 30, for example.
In step S23, straight line information existing on the road W is obtained from each image stored in the storage unit 37. Here, when linear image noise exists in the image, the image noise is also acquired as straight line information. In the example shown in FIG. 6, in the image of the imaging region R1 of the front camera 21, straight line information corresponding to the road center line L1, straight line information corresponding to the road outer line L2, and straight image noise are straight line information N1 to N6. To be acquired. Similarly, straight line information is acquired in the images of the cameras 22 to 24 other than the front camera 21.

  In step S25, the map information is compared with the straight line information of each image. In the example shown in FIG. 6, comparison is made between the roadway center line L1 and the roadway outer line L2 that are map information, and the straight line information N1 to N6 acquired from the image of the front camera 21. Similarly, the straight line information acquired from the images of the cameras 22 to 24 other than the front camera 21 is compared with the map information.

  In step S27, straight line information matching the map information is detected. In the example shown in FIG. 6, the roadway center line L1 is calculated from the degree of matching (matching degree) between the roadway center line L1 and the roadway outer line L2 that are map information and the straight line information N1 to N6 acquired from the image of the front camera 21. The straight line information matching the roadway outer line L2 is detected. Note that the matching in the present embodiment is determined, for example, based on whether or not the straight line information intersects the roadway center line L1 and the roadway outer line L2 at a predetermined angle (for example, about 30 °) or less. That is, when the straight line information intersects the roadway center line L1 and the roadway outer line L2 at 30 ° or less, it is detected as straight line information that matches the map information.

Of the straight line information N1 to N6 acquired from the image of the front camera 21, the straight line information N1 and the straight line information N6 intersect with the roadway center line L1 based on the map information at an angle of 30 ° or less, respectively. Therefore, the two pieces of straight line information of the straight line information N1 and the straight line information N6 are detected as straight line information matching the roadway center line L1. Of the straight line information N1 to N6 acquired from the image of the front camera 21, the straight line information N2 and the straight line information N5 intersect with the roadway outer line L2 based on the map information at an angle of 30 ° or less, respectively. Therefore, the two pieces of straight line information of the straight line information N2 and the straight line information N5 are detected as straight line information that matches the roadway outer line L2. Similarly, straight line information matching the map information is detected for each straight line information acquired from the images of the cameras 22 to 24 other than the front camera 21.
Of the straight line information N1 to N6, the straight line information N3 and the straight line information N4 do not intersect the roadway center line L1 and the roadway outer line L2 based on the map information, respectively. Therefore, the two pieces of straight line information of the straight line information N3 and the straight line information N4 may be detected as straight line information based on image noise that does not match the roadway center line L1 and the roadway outer line L2.

In step S29, it is determined whether or not the number of straight line information matching the map information is equal to or greater than a predetermined number (two in this embodiment).
In the example illustrated in FIG. 6, two pieces of straight line information N1 and straight line information N6 out of straight line information N1 to N6 acquired from the image of the front camera 21 are straight line information that matches the roadway center line L1. While being detected, the two pieces of straight line information of the straight line information N2 and the straight line information N5 are detected as straight line information that matches the roadway outer line L2. That is, in the image of the front camera 21, two pieces of straight line information matching the roadway center line L1 and the roadway outer side line L2, which are map information, are respectively detected. Therefore, in step S29, it determines with "YES" and progresses to step S31. Similarly, for the images of the cameras 22 to 24 other than the front camera 21, it is determined whether or not the number of straight line information matching the map information is equal to or greater than a predetermined number (two in the present embodiment).

  In step S31, images having a predetermined number (two in the present embodiment) of straight line information matching the map information are excluded as inappropriate images. In the present embodiment, two pieces of straight line information matching the roadway center line L1 and the roadway outer side line L2, which are map information, are respectively detected in the image of the front camera 21. Therefore, the image of the front camera 21 is excluded as an inappropriate image.

  In step S33, switching between LKAS control and LDW control is performed as necessary. In the present embodiment, since the image of the front camera 21 is excluded as an inappropriate image, from the LKAS control mainly using the images captured by the front camera 21 and the rear camera 22, the left camera 23 and the right camera are mainly used. The mode is switched to the LDW control using the image captured by the direction camera 24. When step S33 ends, the image processing performed by the driving support device 10 ends.

On the other hand, if the number of straight line information matching the map information is less than the predetermined number (two in this embodiment) in step S29, the determination is “NO” and the process proceeds to step S35.
In step S35, the straight line information in the images of the cameras 21 to 24 is recognized as road information (in this embodiment, the road center line L1 and the road outer line L2). When step S35 ends, the image processing performed by the driving support device 10 ends. In step S35, the flow is ended in a state where at least one of the selected LKAS control and LDW control is adopted without switching between LKAS control and LDW control.

  According to the second embodiment, for example, linear noise in an image generated by the influence of light such as sunlight can be acquired as straight line information, so that an inappropriate image can be extracted.

  In addition, the extraction unit 33 extracts an inappropriate image based on the straight line information and the roadway center line L1 and the roadway outer line L2, which are map information. Therefore, among the plurality of straight line information, incorrect straight line information caused by noise is extracted. In addition to being able to select with high accuracy, it is possible to reliably extract inappropriate images.

  Moreover, since one image is extracted as an inappropriate image by comparing the straight line information with the roadway center line L1 and the roadway outer line L2 which are map information, erroneous straight line information caused by noise can be accurately selected. .

  The technical scope of the present invention is not limited to the above embodiment, and various modifications can be made without departing from the spirit of the present invention.

  The system configuration of the driving support device 10 is not limited to each embodiment. For example, the number of external camera units 25 is not limited to four. The means for displaying images captured by the cameras 21 to 24 is not limited to the display 43 of the navigation system 40, and may be a so-called head-up display or the like, for example. Moreover, the map information acquisition part 45 is not limited to a portable information terminal.

  In the first embodiment, the image 22a of the rear camera 22 other than the image 21a of the front camera 21 is left in the overhead view composite image creation process while excluding the image 21a of the front camera 21 that is an inappropriate image from the target of the brightness correction process. The luminance correction processing was performed on the image 23a of the side camera 23 and the image 24a of the right side camera 24. On the other hand, for example, a recognized inappropriate image may be excluded from the target of image processing, and for example, white line may not be recognized at all.

  It is good also as the driving assistance device 10 which combined 1st embodiment and 2nd embodiment. Specifically, image processing may be performed by extracting and excluding inappropriate images, and switching between LKAS control and LDW control may be performed in response to an image determined to be an inappropriate image. Therefore, for example, when the image 21a of the front camera 21 is determined to be an inappropriate image, control may be performed so as to switch from LKAS control to LDW control.

  The driving support device 10 according to the second embodiment is a device that performs so-called LKAS control and LDW control in which the host vehicle 1 travels along the road center line L1 and the road outer line L2 provided on the road W. However, for example, a parking assist device for detecting a parking space defined by a white line and calculating a route when the host vehicle 1 is parked in the parking space and causing the host vehicle 1 to travel along the route. It may be. For example, in a parking assistance apparatus, you may control so that the white line in an inappropriate image is not recognized.

  In addition, the driving support device 10 of the second embodiment extracts an image in which the number of straight line information matching the map information is equal to or greater than a predetermined number (two in the present embodiment) as an inappropriate image. On the other hand, an image having no straight line information matching the map information may be extracted as an inappropriate image. Thereby, even when there is no straight line information, it can be extracted as an inappropriate image, so that an inappropriate image with a small amount of received light and a remarkably low visibility can be surely excluded.

  In addition, it is possible to appropriately replace the components in the above-described embodiments with known components without departing from the spirit of the present invention.

1 Self-vehicle 21 Front camera (imaging part)
21a Image (unsuitable image)
22 Rear camera (imaging part)
22a Image 23 Left-side camera (imaging unit)
23a Image 24 Right-side camera (imaging unit)
24a Image 31 Calculation unit 33 Extraction unit 35 Image processing unit 45 Map information acquisition unit D1, D2 Predetermined values K1, K2, K3, K4 Average luminance value (average value)
L1 Roadway Chuo Line (map information)
L2 roadway outside line (map information)
N1, N2, N3, N4, N5, N6 Straight line information α Predetermined value

Claims (8)

A plurality of imaging units for imaging the periphery of the host vehicle;
An image processing unit that processes each image captured by the plurality of imaging units;
In a driving support device equipped with
An extraction unit that extracts an inappropriate image inappropriate for the processing among the images;
The driving support device, wherein the image processing unit excludes the inappropriate image. The driving support device according to claim 1,
The plurality of imaging units respectively capture different directions in the vicinity of the host vehicle. The driving support device according to claim 1 or 2,
An arithmetic unit that calculates a numerical value resulting from the brightness of each of the images captured by the imaging unit;
The said extraction part extracts the said inappropriate image based on the result of the said calculating part, The driving assistance apparatus characterized by the above-mentioned. The driving support device according to claim 3,
The calculation unit calculates an average value of numerical values resulting from the brightness of each of the images,
The said extraction part extracts the said image whose said average value is more than predetermined value among each said image as said inappropriate image. The driving support device according to claim 3,
The calculation unit calculates an average value of numerical values resulting from the brightness of each of the images,
The extraction unit includes a sum of differences between the average value of one image and the average value of another image among the images, or the average value of the one image and the average of the other image. When the average value of the difference from the value is equal to or larger than a predetermined value, the one image is extracted as the inappropriate image. The driving support device according to claim 1,
The image processing unit acquires straight line information in each of the images,
The driving support device, wherein the extraction unit extracts the inappropriate image based on the straight line information. The driving support device according to claim 6,
A map information acquisition unit for acquiring map information around the vehicle where the vehicle is located;
The driving support device, wherein the extraction unit extracts the inappropriate image based on the straight line information and the map information. The driving support device according to claim 7,
The extraction unit compares the straight line information and the map information in one image among the images, and when the straight line information matching the map information is a predetermined number or more, or When there is no straight line information that matches map information, the one image is extracted as the inappropriate image.

Images