JP5066558B2 - Vehicle shadow recognition device - Google Patents

Description

  The present invention relates to a vehicle shadow recognition apparatus.

  Conventionally, in the vehicle shadow recognition device, smear (smear generated when the sun is reflected in the image taken by the in-vehicle camera) When shooting a subject extremely bright from the surroundings with a camera using a CCD image sensor ) Is used to estimate the position of the sun and identify the position of the shadow of the vehicle (own vehicle shadow). An example of a technique related to this description is disclosed in Patent Document 1.

JP 2007-300559 A

In the above-described conventional apparatus, there is a need for specifying the own vehicle shadow region with high accuracy.
The objective of this invention is providing the own vehicle shadow recognition apparatus which can pinpoint the own vehicle shadow area | region from an image with high precision.

  In the present invention, the position of the sun is calculated from information on the current position of the host vehicle and information on the current date and time, and the host vehicle is calculated from the position information of the sun, the traveling direction information of the host vehicle, and the three-dimensional shape information of the host vehicle. The region where the shadow is generated is estimated, and the own vehicle shadow region is specified from the image obtained by the own vehicle periphery imaging device based on the coordinate information obtained by the estimation.

  According to the present invention, the own vehicle shadow region can be specified from the image with high accuracy.

It is a block diagram of the own vehicle shadow recognition apparatus 11 of Example 1. FIG. 4 is a diagram illustrating an arrangement example and an imaging range of a vehicle periphery imaging device 300. FIG. 4 is a diagram illustrating an arrangement example and an imaging range of a vehicle periphery imaging device 300. FIG. FIG. 6 is a diagram showing an example of arrangement of a vehicle periphery ranging device 350. 5 is a flowchart showing a flow of own vehicle shadow removal processing executed by the own vehicle shadow recognition control device 100 according to the first embodiment. 6 is a flowchart illustrating a flow of information acquisition processing according to the first exemplary embodiment. It is a flowchart which shows the flow of a starting determination process. 3 is a flowchart illustrating a flow of solar position calculation processing according to the first embodiment. It is a figure which shows the coordinate system of the celestial sphere applied when calculating the position of the sun. It is a flowchart which shows the flow of the own vehicle shadow area estimation process of Example 1. FIG. It is a figure which shows the calculation method of the own vehicle shadow estimation area | region. It is a flowchart which shows the flow of the own vehicle shadow estimation area | region correction process of Example 1. FIG. It is a figure which shows the method of calculating the own vehicle shadow estimation area | region when a solid object exists around the own vehicle. It is a flowchart which shows the flow of the own vehicle shadow area | region detection process of Example 1. FIG. 3 is a flowchart illustrating a flow of image brightness correction processing according to the first exemplary embodiment. 6 is a diagram showing a method for determining a solar radiation region 6. FIG. It is a block diagram of the own vehicle shadow recognition apparatus 12 of Example 2. FIG. 10 is a flowchart illustrating a flow of own vehicle shadow removal processing executed by the own vehicle shadow recognition control device 100 according to the third embodiment. It is a flowchart which shows the flow of the additional protrusion shadow area | region detection process of Example 4. FIG. It is a figure which shows the content of the additional protrusion shadow area | region detection process of Example 4. FIG. It is a figure which shows the effect | action of the additional protrusion shadow area | region detection process of Example 4. FIG. 10 is a flowchart showing a flow of own vehicle shadow removal processing executed by the own vehicle shadow recognition control device 100 of the fifth embodiment.

  EMBODIMENT OF THE INVENTION Hereinafter, the form for implementing the own vehicle shadow recognition apparatus of this invention is demonstrated based on the Example shown on drawing. In addition, the Example demonstrated below is examined so that it can adapt to many needs, and it is one of the needs examined that the own vehicle shadow area | region can be specified with high precision from an image. The following embodiments are further adapted to needs for cost reduction and power saving.

Example 1
First, the configuration will be described.
〔overall structure〕
FIG. 1 is a configuration diagram of the own vehicle shadow recognition apparatus 11 according to the first embodiment. The vehicle shadow recognition device 11 includes a vehicle shadow recognition control device 100, a vehicle periphery imaging device (vehicle periphery imaging device) 300, a video output device (vehicle display device) 310, a user interface 320, and an in-vehicle network 330. An obstacle recognition device 340, a white line recognition device 341, a vehicle peripheral distance measurement device (distance measurement device) 350, an illuminance acquisition device (illuminance sensor) 351, and a car navigation device 400.
The own vehicle shadow recognition control device 100 identifies the position of the own vehicle shadow (own vehicle shadow) from the captured image of the range including the road surface on which the own vehicle is traveling, obtained by the vehicle surrounding imaging device 300, and An image from which a shadow is removed is generated. A sun position calculating unit 110, a vehicle shadow region estimating unit 120, a vehicle shadow region detecting unit 130, an image correcting unit (vehicle shadow region luminance correcting unit) 140, , An image memory 150 and a vehicle three-dimensional shape database 160 are provided.
The solar position calculation means 110 is based on the information on the latitude, longitude, current date and current time of the current position of the vehicle obtained from the car navigation device 400, and the standard meridian determined from the current position of the vehicle. Calculate the altitude and azimuth of the sun.
The own vehicle shadow region estimating means 120 includes the information on the own vehicle traveling direction obtained by the car navigation device 400 in addition to the altitude and azimuth angle of the sun obtained by the sun position calculating means 110 and the three-dimensional shape database 160 of the own vehicle. Calculate the estimated value of the direction in which the vehicle shadow appears and the estimated shadow length at each measurement point based on the vehicle solid shape data from the saved vehicle solid shape data (vehicle solid shape information) To do. Furthermore, these information is integrated and the own vehicle shadow estimation area | region is produced | generated. Here, the own vehicle shadow estimation area is a point on the road surface corresponding to each point constituting the own vehicle solid shape data stored in the own vehicle solid shape database 160. Therefore, it is handled as data of n two-dimensional coordinates (X, Y), and is configured as an array represented by SP [n] → {X, Y}.
The own vehicle shadow area detection means 130 detects the own vehicle shadow area from the image captured by the vehicle periphery imaging device 300 using the own vehicle shadow estimation area obtained by the own vehicle shadow area estimation means 120.
The image correction unit 140 corrects the brightness and contrast of the vehicle shadow area detected by the vehicle shadow area detection unit 130, thereby removing the vehicle shadow from the image obtained from the vehicle periphery imaging device 300. .
The image memory 150 captures (saves) an image obtained from the vehicle periphery imaging device 300.
The own vehicle three-dimensional shape database 160 stores own vehicle three-dimensional shape data which is three-dimensional data of the own vehicle. The vehicle's three-dimensional shape data is n pieces of data obtained by discretizing the outer shape of the vehicle as three-dimensional coordinates (X, Y, Z), and is expressed as VP [n] → {X, Y, Z}. Configured as an array.

A car navigation device 400, a vehicle periphery imaging device 300, a video output device 310, an obstacle recognition device 340, a white line recognition device 341, a user interface 320, and an in-vehicle network 330 are directly connected to the own vehicle shadow recognition control device 100. . Further, a vehicle peripheral distance measuring device 350 and an illuminance acquisition device 351 are connected via the in-vehicle network 330.
The vehicle periphery imaging device 300 is a camera installed on the vehicle body so as to output an image obtained from at least one image sensor and capture an image of a range where the vehicle is in contact with the road surface. For example, as shown in FIG. 2, a rear imaging device 300Rr may be attached to the rear of the host vehicle 1 so as to capture a range 301Rr behind the host vehicle. In addition, as shown in FIG. 3, imaging elements (300Fr, 300Rr, 300L, 300R) are arranged on the front, rear, left and right sides of the own vehicle 1, and the entire periphery of the own vehicle 1 is used as an imaging range (301Fr, 301Rr, 301L, 301R). Also good. In Example 1, the structure which has arrange | positioned the image pick-up element in the front and rear, right and left of the own vehicle 1 shown in FIG. 3 is employ | adopted. Further, the vehicle periphery imaging device 300 according to the first embodiment has a function of combining the captured images obtained from the four imaging elements and converting the images into an image (overhead image) as seen from directly above the vehicle (overhead view). Equivalent to image creation means).
The video output device 310 displays the video in which the own vehicle shadow area is corrected by the image correction means 140.
The user interface 320 includes switches and indicators for a user (driver) to operate the own vehicle shadow recognition system by the own vehicle shadow recognition device 11. For example, it has a function of expressing the system activation state with the LED lighting state, forcibly starting the system, or forcibly disabling it.
The in-vehicle network 330 is a network for sharing information between a plurality of control units mounted on a vehicle, and typical standards include CAN (controller area network) and Flex Ray. The in-vehicle network 330 includes, for example, vehicle speed information, steering angle, shift position information, ON / OFF states of various lamps, and wiper operation state information. In addition to these pieces of information, any controller or sensor connected to the in-vehicle network 330 can acquire all of the information.
The obstacle recognizing device 340 recognizes the obstacle from the image from which the own vehicle shadow has been removed by the own vehicle shadow recognition control device 100 and prompts the user to give a warning.
The white line recognition device 341 recognizes the white line painted on the road from the image from which the own vehicle shadow has been removed by the own vehicle shadow recognition control device 100, and warns the own vehicle not to deviate from the lane, and from the lane. Control to prevent deviation.
The vehicle periphery distance measuring device 350 includes a distance measuring sensor that measures a distance to a three-dimensional object existing around the host vehicle, and has a function of calculating the size and shape of the three-dimensional object from information of the distance measuring sensor. In the first embodiment, as shown in FIG. 4, ultrasonic sensors 5 a to 5 j are arranged around the subject vehicle as distance measuring sensors, and the distances to the three-dimensional objects existing on the front, rear, left and right of the subject vehicle 1 are measured.

[Self-vehicle shadow removal processing]
FIG. 5 is a flowchart showing the flow of the vehicle shadow removal process executed by the vehicle shadow recognition control apparatus 100 of the first embodiment. Each step will be described below.
In step S100, an information acquisition process is performed, and the process proceeds to step S200.
In step S200, an activation determination process is performed, and the process proceeds to step S300. Step S200 corresponds to activation determination means.
In step S300, the solar position calculation means 110 performs a solar position calculation process, and the process proceeds to step S400.
In step S400, the own vehicle shadow region estimation means 120 performs own vehicle shadow region estimation processing, and the process proceeds to step S500.
In step S500, the own vehicle shadow area detection means 130 performs own vehicle shadow area detection processing, and the process proceeds to step S600.
In step S600, the image correction means 140 performs image brightness correction processing, and the process proceeds to return.
Details of each process will be described below.

[Information acquisition processing]
FIG. 6 is a flowchart showing the flow of information acquisition processing according to the first embodiment. Each step will be described below.
In step S110, various types of information shared by the in-vehicle network 330 are buffered, and the process proceeds to step S120.
In step S120, the current position information provided from car navigation device 400 is received, and the process proceeds to step S130.
In step S130, information from sensors not connected to the in-vehicle network 330 is acquired by various sensors mounted on the vehicle, and this control is terminated.
As described above, in the information acquisition process, various information for performing the own vehicle shadow removal process is acquired from the in-vehicle network 330, the car navigation device 400, and the like.

[Startup judgment processing]
FIG. 7 is a flowchart showing the flow of the activation determination process according to the first embodiment. Each step will be described below.
In step S205, it is determined whether or not the “automatic determination mode” is set. If YES, the process moves to step S210, and if NO, the process moves to step S250. The “automatic determination mode” is a mode for automatically determining whether or not the own vehicle shadow recognition system can be activated according to the result of the subsequent determination processing (steps S210, S215, S220, S230, and S240).
In step S210, it is determined whether or not the current solar altitude is higher than a predetermined threshold. If YES, the process proceeds to step S215. If NO, the vehicle shadow removal process is terminated. Here, the threshold value is a value equal to or higher than the solar altitude at sunrise.
In step S215, it is determined whether or not the illuminance acquisition device 351 is equipped. If YES, the process proceeds to step S220, and if NO, the process proceeds to step S230.
In step S220, it is determined whether the information of the illuminance acquisition device 351 obtained from the in-vehicle network 330 is higher than a predetermined threshold. If YES, this control is terminated, and if NO, the vehicle shadow removal process is terminated. Here, the threshold value is the amount of solar radiation that causes the own vehicle shadow.
In step S230, based on the wiper operation state information obtained from the in-vehicle network 330, it is determined whether or not the wiper has been stopped for a predetermined time. If YES, the process proceeds to step S240. If NO, the vehicle shadow removal process is terminated. If the wiper is stopped for a predetermined time continuously, it can be determined that it is not raining.
In step S240, it is determined whether or not the headlight of the host vehicle is turned off. If YES, this control is terminated, and if NO, the vehicle shadow removal process is terminated. When the headlight is turned off, it can be determined that the surrounding area of the host vehicle is sufficiently bright and that the possibility of occurrence of a shadow of the host vehicle is high.
In step S250, it is determined whether or not the manual activation SW (switch) is ON. If YES, this control is terminated, and if NO, the vehicle shadow removal process is terminated. Here, the manual activation SW is a switch for the user to activate the vehicle shadow recognition system, and the ON / OFF state of the switch is stored in the internal memory.
As described above, in the activation determination process, in the “automatic determination mode”, when it is determined that the vehicle shadow will occur after sunrise, or after the sunrise, it is not raining and the surroundings of the vehicle is sufficiently bright When it is determined, the process proceeds to the sun position calculation process (step S300), and otherwise, the vehicle shadow removal process ends (returns to step S100). On the other hand, when not in the “automatic determination mode”, when the user activates the manual activation SW, the process proceeds to the sun position calculation process. When the manual activation SW is OFF, the vehicle shadow removal process is terminated.

[Sun position calculation processing]
FIG. 8 is a flowchart illustrating the flow of the solar position calculation process according to the first embodiment. Each step will be described below.
In step S310, the latitude and longitude information of the current position provided from the car navigation device 400 is acquired, and the process proceeds to step S320.
In step S320, the current date and time provided from the car navigation device 400 are acquired, and the process proceeds to step S330.
In step S330, the standard meridian information of the current position provided from the car navigation device 400 is acquired, and the process proceeds to step S340.
In step S340, the sun's altitude and azimuth are calculated based on the information acquired in steps S310, S320, and S330, and this control is terminated.
As described above, in the solar position calculation process, based on the latitude and longitude information of the current position provided from the car navigation device 400, the current date and time information, and the standard meridian information of the current position, The azimuth, that is, the position of the sun is calculated.

Hereinafter, a method for calculating the altitude and azimuth of the sun in step S340 will be described.
When calculating the position of the sun, the coordinate system of the celestial sphere shown in FIG. 9 is used. Take the center of gravity of your car 1 at the origin O, South S is + Y, North N is -Y, West W is + X, East E is -X.
The solar altitude h and azimuth angle A can be defined as negative toward the south and positive at the east, respectively, and the respective calculation formulas are expressed by the following formulas (1) and (2).
h = sin ^-1 (sin (φ) sin (δ) + cos (φ) cos (δ) cos (t))… (1)
A = sin ^-1 {(cos (δ) sin (t)) / cos (h)}… (2)
Here, φ is the latitude of the current position. Further, δ represents the solar declination, t represents the hour angle, and the respective calculation formulas are shown in the following formulas (3) and (4).
δ = 0.33281-22.984cos (ωJ)-0.34990cos (2ωJ)-0.13980cos (3ωJ)
+ 3.7872sin (ωJ) + 0.0325sin (2ωJ) + 0.07187sin (3ωJ)… (3)
t = 15 ° T-180 °… (4)
However, ω = 2π / 365 (in the case of leap years, ω = 2π / 366), and J = total number of days from the first day +0.5.
Here, T is true solar time, and the calculation formula is shown in the following formula (5).
T = Ts + (L-Ms) / 15 ° + e… (5)
Here, the current Ts is Central Standard Time, the latitude of the current position, and Ms is the standard meridian (135 ° in Japan). Further, e is a time difference, and the calculation formula is shown in the following formula (6).
e = 0.0072cos (ωJ)-0.0528cos (2ωJ)-0.0012cos (3ωJ)
-0.1229sin (ωJ)-0.1565sin (2ωJ)-0.0041sin (3ωJ)… (6)
However, ω = 2π / 365 (in the case of leap years, ω = 2π / 366), and J = total number of days from the first day +0.5.

[Self-vehicle shadow area estimation processing]
FIG. 10 is a flowchart showing the flow of the own vehicle shadow area estimation process according to the first embodiment. Each step will be described below.
In step S410, the solar altitude h and azimuth angle A obtained in the solar position calculation process (step S300) are read, and the process proceeds to step S420.
In step S420, own vehicle traveling direction information provided from car navigation device 400 is acquired, and the process proceeds to step S430.
In step S430, the three-dimensional coordinates VP [n] stored in advance in the vehicle three-dimensional shape database 160 as the vehicle three-dimensional shape data are read, and the process proceeds to step S440.
In step S440, an estimated region where the own vehicle shadow is generated is calculated based on each information acquired in steps S410, S420, and S430. A method of calculating the own vehicle shadow estimation area 3 will be described with reference to FIG. First, based on the altitude and azimuth A of the sun, the traveling direction of the host vehicle, and the three-dimensional shape data of the own vehicle, the estimated value θ _s ^ of the direction in which the shadow of the own vehicle 1 appears and the three-dimensional shape data of the own vehicle An estimated value L ^ [n] of the length of the shadow SP [n] at each measurement point VP [n] is calculated. Here, if the traveling direction of the vehicle is θ _v and the height of each measurement point based on the vehicle's solid shape data is H [n] (= VP [n] → Z), the estimated value θ of the vehicle shadow direction _s ^ and the estimated value of the length of the vehicle shadow at each measurement point VP [n] L ^ [n] (= {(SP [n] → X) ² + (SP [n] → Y) ² } ^{1 / 2} ) is calculated by the following equations (7) and (8).
θ _s ^ = A -θ _v ^ = sin-{(cos (δ) sin (t)) / cos (h)} -θ _v ^… (7)
L ^ [n] = H [n] cot (h)… (8)
By calculating the point cloud of the own vehicle shadow SP [n] by the above formulas (7) and (8) and integrating these point groups, the own vehicle shadow estimation region 3 is finally calculated.
In step S445, it is determined whether or not the own vehicle shadow estimation area calculated in step S440 is within the imaging range of the vehicle periphery imaging device 300. If YES, the process proceeds to step S450. If NO, the vehicle shadow removal process is terminated.
In step S450, the measurement values of the ultrasonic sensors 5a to 5j collected by the vehicle periphery ranging device 350 are acquired as distance information around the vehicle, and the process proceeds to step S455.
In step S455, based on the distance information around the vehicle acquired in step S455, it is determined whether or not an obstacle (a solid object such as a wall) having a size larger than a predetermined size exists in the own vehicle shadow estimation area. If YES, the process moves to step S470, and if NO, the process moves to step S460. Here, when there is an obstacle of a predetermined size or more in the own vehicle shadow estimation area, it can be estimated that the own vehicle shadow is projected on the obstacle other than on the road surface, so the process proceeds to step S460. It is necessary to perform processing for correcting the vehicle shadow estimation area.

次に、カメラを仮想的に移動させて、自車を真上から俯瞰する仮想カメラの射影変換行列をP'とすると、下記の式(10)に示すように、世界座標系の三次元点(X,Y,Z)は、俯瞰画像座標系(u',v')に射影される。

上記式(10)によって、世界座標系の三次元点(X,Y,Z)が俯瞰変換画像座標系(u',v')に変換される。
ステップS470では、ステップS455により自車影が障害物にも投影していると推定されたため、自車影推定領域補正処理を実施し、ステップS460へ移行する。実施例１の自車影推定領域補正処理の流れを図１２に示す。まず、ステップS471では、ステップS450で計測された障害物までの距離Dを取得し、続くステップS472では、障害物による自車影推定領域の変形量を算出し、変形量に応じて自車影推定領域を補正する。図１３に示すように、自車1の側方に壁10が存在し、この壁10に自車影2が射影している場合、自車影2は路面領域7の自車影2aと壁領域の自車影2bに分類できる。今、自車立体形状データの代表点VPの高さをH、壁10が存在しない場合の自車影2の推定領域の長さをLとする。また、路面領域7と壁10との接点をSP(X_SP,Y_SP,Z_SP)、壁領域における自車影2bの点をSPW(X_SPW,Y_SPW,Z_SPW)とする。ステップS471で取得された自車1と壁10との距離がDの場合、点SPと点SPWはそれぞれ下記の式(11),(12)で表される。なお、便宜的に世界座標系の原点を点SPとしている。

ここで、式(12)におけるLは、壁10が存在しない場合の自車影2の推定領域の長さであるため、式(8)で置き換えることができ、下記の式(13)で表される。

以上の計算式により、壁10に射影した自車影2bの点SPWを逐次演算することで、障害物がある場合は自車影推定領域が補正される。
ステップS480では、推定された自車影推定領域を自車影推定パターンとしてメモリに保存し、本制御を終了する。
以上のように、自車影領域推定処理では、太陽の高度hと方位角Aの情報、自車進行方向の情報、自車立体形状データに基づいて自車影推定領域を算出した後、撮像範囲に自車影が含まれると判断された場合、自車影推定領域を世界座標系から画像座標系に変換し、推定された領域の特徴を自車影推定パターンとして記憶する。このとき、自車影が障害物にも投影していると判断された場合、障害物による自車影の変形量に応じて自車影推定領域を補正した後、座標系の変換および自車影推定パターンの記憶を行う。 In step S460, the coordinate system of the own vehicle shadow estimation area calculated in step S440 is converted into an image coordinate system, and the process proceeds to step S480. A method of converting from a point in the world coordinate system to an image coordinate system will be described. As shown in the following equation (9), the three-dimensional point (X, Y, Z) in the world coordinate system is projected onto the point (u, v) in the projected image. In Equation (9), λ is a scale factor, and P is a projection matrix with 3 rows and 4 columns. The projection matrix P is calculated in advance by camera calibration. Camera calibration refers to associating world coordinates with the camera image coordinates of the vehicle periphery imaging device 300.

Next, let P ′ be the projective transformation matrix of the virtual camera that virtually moves the camera and looks down on the vehicle from directly above, as shown in the following equation (10), a three-dimensional point in the world coordinate system (X, Y, Z) is projected onto the overhead image coordinate system (u ′, v ′).

By the above equation (10), the three-dimensional point (X, Y, Z) in the world coordinate system is converted into the overhead view conversion image coordinate system (u ′, v ′).
In step S470, since it is estimated that the own vehicle shadow is also projected on the obstacle in step S455, the own vehicle shadow estimation area correction process is performed, and the process proceeds to step S460. FIG. 12 shows the flow of the vehicle shadow estimation area correction process according to the first embodiment. First, in step S471, the distance D to the obstacle measured in step S450 is acquired, and in the subsequent step S472, the deformation amount of the own vehicle shadow estimation area due to the obstacle is calculated, and the own vehicle shadow is calculated according to the deformation amount. Correct the estimated area. As shown in FIG. 13, when the wall 10 exists on the side of the vehicle 1 and the vehicle shadow 2 is projected on the wall 10, the vehicle shadow 2 is the same as the vehicle shadow 2 a of the road surface region 7 and the wall. The area can be classified as the own vehicle shadow 2b. Now, assume that the height of the representative point VP of the vehicle three-dimensional shape data is H, and the length of the estimated region of the vehicle shadow 2 when the wall 10 does not exist is L. Further, the contact point between the road surface region 7 and the wall 10 is SP (X _SP , Y _SP , Z _SP ), and the point of the vehicle shadow 2b in the wall region is SPW (X _SPW , Y _SPW , Z _SPW ). When the distance between the vehicle 1 and the wall 10 acquired in step S471 is D, the point SP and the point SPW are expressed by the following equations (11) and (12), respectively. For convenience, the origin of the world coordinate system is the point SP.

Here, L in Equation (12) is the length of the estimated region of the own vehicle shadow 2 when the wall 10 does not exist, so it can be replaced by Equation (8) and expressed by Equation (13) below. Is done.

By calculating the point SPW of the vehicle shadow 2b projected onto the wall 10 by the above formula, the vehicle shadow estimation region is corrected when there is an obstacle.
In step S480, the estimated vehicle shadow estimation area is stored in the memory as a vehicle shadow estimation pattern, and this control is terminated.
As described above, in the own vehicle shadow region estimation process, after calculating the own vehicle shadow estimation region based on the information on the sun altitude h and the azimuth angle A, the information on the own vehicle traveling direction, and the own vehicle solid shape data, When it is determined that the vehicle shadow is included in the range, the vehicle shadow estimation area is converted from the world coordinate system to the image coordinate system, and the characteristics of the estimated area are stored as the vehicle shadow estimation pattern. At this time, if it is determined that the own vehicle shadow is also projected on the obstacle, after correcting the own vehicle shadow estimation area according to the deformation amount of the own vehicle shadow caused by the obstacle, the coordinate system conversion and the own vehicle are corrected. The shadow estimation pattern is stored.

[Vehicle shadow area detection processing]
FIG. 14 is a flowchart showing the flow of the own vehicle shadow area detection process according to the first embodiment. Each step will be described below.
In step S510, the image captured by the vehicle periphery imaging device 300 is captured in the image memory 150, and the process proceeds to step S520.
In step S520, an area for searching for the own vehicle shadow is set based on the own vehicle shadow estimation pattern calculated in step S400, and the process proceeds to step S530. Here, the area for searching for the own vehicle shadow is an area wider than the own vehicle shadow estimation pattern calculated in step S400 by a predetermined pixel (for example, an area in which the own vehicle shadow estimation pattern is expanded by 5 pixels). Set to In addition, the set value (expansion rate or the like) of the area for searching for the own vehicle shadow may be variable in consideration of calculation errors in each step, sensor measurement errors, and the like. Thereby, even when the vehicle shadow estimation pattern calculated in step S400 is estimated as a narrower region than the actual vehicle shadow region due to various calculation errors, it can be set to a region including the vehicle shadow. In addition, by making the set value of the area for searching for the own vehicle shadow variable, it is possible to set the area including the own vehicle shadow more reliably.
In step S530, the vehicle shadow candidate region is extracted from the search region set in step S520 using the characteristics of luminance and shape, and the process proceeds to step S540. Here, as a feature of the brightness, the fact that the brightness of the own vehicle shadow area is relatively lower than the surrounding area is used. As a shape feature, the fact that the own vehicle shadow region always extends from the ground contact surface between the own vehicle and the road surface is used. As a result, the search area of the image can be limited by using the information of the own vehicle shadow estimation pattern obtained by the own vehicle shadow area estimating means 120 (step S520), so that the processing speed can be expected.
In step S540, pattern matching between the own vehicle shadow candidate area extracted in step S530 and the own vehicle shadow estimation pattern calculated in step S400 is performed, and the process proceeds to step S550.
In step S550, it is determined whether the similarity calculated as a result of pattern matching in step S540 is greater than a predetermined threshold. If YES, the process moves to step S570, and if NO, the process moves to step S560.
In step S560, the threshold values for extracting the brightness and shape of the vehicle shadow candidate region in step S530 are changed, and the process proceeds to step S530.
In step S570, the own vehicle shadow candidate area is determined as the own vehicle shadow area, and this control is terminated.
As described above, in the vehicle shadow area detection process, the area for searching for the vehicle shadow is set based on the vehicle shadow estimation pattern, and the luminance and shape characteristics of the vehicle shadow area are set with respect to the set search area. The vehicle shadow candidate region is extracted using the vehicle shadow candidate region, and when the similarity between the extracted vehicle shadow candidate region and the vehicle shadow estimation pattern exceeds a threshold, the vehicle shadow candidate region is determined as the vehicle shadow region. Here, when it is determined that the similarity is equal to or less than the threshold value, the luminance and shape threshold values used when extracting the own vehicle shadow candidate region are changed.

[Image brightness correction processing]
FIG. 15 is a flowchart showing the flow of image brightness correction processing according to the first embodiment. Each step will be described below.
At step S610, the obtaining the average luminance value I _AVRs inner ascertained vehicle shadow regions (internal) in step S500, the average luminance value I _Avrr insolation region outside (external) of Jikurumakage region. As shown in FIG. 16 (a), the solar radiation area 6 is an area in which the vehicle shadow area 4 is expanded by a predetermined pixel (for example, an area in which the vehicle shadow area 4 is expanded by 5 pixels). And By making the solar radiation area 6 an area that is expanded by a predetermined number of pixels from the vehicle shadow area 4, it is possible to reduce the effects of road surface paint, dirt, roadside structures, traveling vehicles, etc. Can be corrected.
In step S620, the equation (14) from the average luminance value I _Avrr the average luminance value I _AVRs and solar radiation region 6 of the calculated vehicle shadow region 4 the luminance correction parameter α of the vehicle shadow region 4 below at step S610 The process proceeds to step S630.
α = I _avrr / I _avrs … (14)
In step S630, the brightness of the vehicle shadow area 4 is corrected based on the brightness correction parameter α calculated in step S620, and this control is terminated. Assuming that the number of pixels in the own vehicle shadow region in the image is m and the luminance value of the point i in the own vehicle shadow region before correction is I _s [i], the corrected luminance value I _s ' [i] is expressed by the following equation: It is expressed by (15).
I _s ' [i] = αI _s [i] (i = 1,2,…, m)… (15)
As described above, in the brightness correction process, the brightness correction parameter α of the own vehicle shadow area is _obtained from the average brightness value I _avrs of the own vehicle shadow area 4 and the average brightness value I _avrr of the solar radiation area, and the brightness correction parameter α is used. The own vehicle shadow area 4 is corrected. As a result, as shown in FIG. 16 (a), the visibility of the own vehicle shadow region 4 can be improved even in a situation where the brightness value of the white line 8 is difficult to be seen due to the own vehicle shadow region 4 (see FIG. 16 (b)). )). Further, since the luminance value of only the own vehicle shadow region 4 is corrected, the road surface region 7 and the white line 8 in the solar radiation state are the same as the luminance before correction, and the visibility is not deteriorated.

Next, the operation will be described.
[Problems of conventional vehicle shadow recognition]
Until now, a device that has a camera mounted behind the vehicle and assists the user's field of view when the vehicle moves backward has been put into practical use. Furthermore, in recent years, a device in which a plurality of cameras are mounted on a vehicle and assists the entire field of view as well as the rear of the vehicle has been put into practical use. In these systems, the main purpose is to ensure the user's field of view, and the user directly checks the image displayed on the screen of a car navigation system or the like to confirm the safety of the surroundings. In addition, a system has been devised that automatically determines an obstacle from an image captured by a camera and alerts the user.
As described above, whether the user directly checks the image for safety confirmation or the system automatically recognizes an obstacle, the information that is the determination source is an image taken by the camera. That is, in both cases, the brightness, contrast, and the like of the captured images greatly affect the visibility. In particular, cameras mounted on vehicles often have a narrow dynamic range of luminance in order to reduce costs, and it is difficult to ensure sufficient visibility depending on the scene. For example, when there is a road area that is exposed to direct sunlight and a dark area due to the shadow of the vehicle, the camera exposure control is generally based on the brightness of the larger area. The brightness is corrected based on the road surface exposed to direct sunlight. As a result, the shadow area of the own vehicle becomes an image with low brightness and contrast, and visibility is lowered.
In order to solve such a problem, a technique for recognizing the shadow of the own vehicle has been devised. For example, in the invention described in Patent Document 1, smear (smear generated when the sun appears in an image captured by a vehicle-mounted camera) is used to capture a subject that is extremely bright from the surroundings with a camera using a CCD image sensor. The position of the vehicle shadow is specified by estimating the position of the sun using the phenomenon. As another means, a technique for identifying the position of the own vehicle shadow based on information on the current position of the vehicle, a database in which information on the position of the sun is registered, and time information is also known. Further, the brightness of the portion corresponding to the position of the own vehicle shadow specified by these means is corrected, and an image without a shadow is generated.

However, in the invention described in Patent Document 1, since only the position where the shadow is generated from the position of the sun, that is, the direction in which the shadow is generated is estimated, other than the own vehicle (tree, building, person, etc.) And other noise components may be misrecognized as part of the shadow of the vehicle and may be corrected to an unnecessary area. In addition, when a three-dimensional object similar to the estimated own vehicle shadow exists in the vicinity of the own vehicle, it may be erroneously recognized as the own vehicle shadow. Further, in a scene such as a narrow alley or a parking lot, it is conceivable that the shadow of the own vehicle is projected not only on the road surface but also on a structure such as a wall surface. When estimating the own vehicle shadow area in such a scene, it is necessary to measure the distance to the structure. Therefore, with the method described in Patent Document 1, it is difficult to accurately specify the own vehicle shadow area. is there. In addition, in the method using smear, the position of the sun cannot be calculated because smear does not occur in a camera using a CMOS sensor. In addition, when the altitude of the sun is high and does not fall within the imaging range of the camera, smear does not occur, so the position of the sun cannot be calculated.
On the other hand, in the case of storing the position (altitude, azimuth) information of the sun around the country as a database (discretized table data) and specifying the position of the own vehicle shadow based on the time information, the number of registration points If the amount is reduced, the amount of information can be reduced, but the resolution is lowered, so that the detection accuracy of the own vehicle shadow region is lowered. Conversely, when the number of registration points is increased, the resolution is increased, but the amount of information is enlarged, resulting in an increase in cost.
Furthermore, in the conventional vehicle shadow recognition device, since there is no start / end trigger for the function to identify the position of the vehicle shadow, it is assumed that it will be executed even in situations where no shadows occur, such as nighttime and bad weather. This leads to an increase in power.

(Self-vehicle shadow estimation effect)
On the other hand, the own vehicle shadow recognition apparatus 11 according to the first embodiment is based on the sun altitude h and the azimuth A, the own vehicle traveling direction θ _v, and the own vehicle solid shape data (vertex coordinates of the outline of the shadow). The estimated value θ _s ^ for the direction in which the vehicle shadow appears and the estimated value L ^ [n] for the length of the shadow SP [n] at each measurement point VP [n] based on the vehicle's solid shape data Then, the vehicle shadow area estimation means 120 for integrating the information and estimating the vehicle shadow estimation area is provided. In other words, because the shape of the vehicle shadow is estimated based not only on the direction in which the shadow is generated but also on the vehicle's solid shape data, shadows other than the vehicle and other noise components are recognized as part of the vehicle shadow. In this way, the estimation accuracy of the vehicle shadow region can be improved.
[Self-vehicle shadow detection]
In addition, the vehicle shadow recognition device 11 sets a region for searching for the vehicle shadow from the image captured by the vehicle periphery imaging device 300 based on the vehicle shadow estimation region (vehicle shadow estimation pattern), and sets the region in the region. On the other hand, the vehicle shadow region detection means 130 for comparing the vehicle shadow candidate region extracted using the characteristics of the brightness and shape of the vehicle shadow region with the vehicle shadow estimation pattern and specifying the vehicle shadow region is provided. In other words, because the vehicle shadow area is specified based on the highly accurate vehicle shadow estimation area, the vehicle shadow area is robust against noise such as shadows of other structures and road surfaces. It can be detected.
(Self-vehicle shadow area removal action)
Furthermore, Jikurumakage recognition device 11, the luminance correction parameter from the average luminance value I _AVRs the average luminance value I _Avrr and Jikurumakage region of finalized outside of the vehicle shadow region (insolation region) α _(I avrr / I _avrs ) is calculated, and image correction means 140 is provided for correcting the luminance of each pixel in the own vehicle shadow region with the luminance correction parameter α. Therefore, only the luminance of the own vehicle shadow region can be increased according to the luminance outside the own vehicle shadow region without deteriorating the visibility outside the own vehicle shadow region, thereby improving the visibility of the own vehicle shadow region. Can do.

[Solar position calculation function]
The own vehicle shadow recognition device 11 is based on information on the latitude, longitude, current date and current time of the current position of the own vehicle obtained from the car navigation device 400, and the standard meridian determined from the current position of the own vehicle. And solar position calculating means 110 for calculating the solar altitude h and the azimuth angle A. Therefore, since the position of the sun can be calculated without using smear, a CMOS sensor that is lower in power consumption and easier to miniaturize than the CCD image sensor can be used as the vehicle periphery imaging device 300. Further, the position of the sun can be calculated even when the sun does not enter the imaging range of the camera. Furthermore, a discretized database for storing the relationship between the latitude and longitude of the vehicle and the position of the sun is not necessary, and the memory capacity can be reduced, so that the cost can be suppressed.
The sun position calculation means 110 calculates the sun's altitude and azimuth in consideration of the standard meridian corresponding to the current position. Available. Furthermore, by using the car navigation device 400, not only the current position information (latitude and longitude) of the host vehicle but also date and time information can be acquired simultaneously. In addition, since the vehicle traveling direction can be acquired from the map matching function and the dead reckoning function provided in a general car navigation device, the cost of separately adding a device for acquiring the vehicle traveling direction can be reduced.
[Car shadow estimation area correction based on distance to obstacle]
Since the own vehicle shadow recognition device 11 includes a vehicle peripheral distance measuring device 350 having a distance measuring sensor (ultrasonic sensors 5a to 5j) that measures a distance to a three-dimensional object existing around the own vehicle, the shape of the own vehicle shadow It is possible to measure the distance to obstacles around the vehicle that affect the vehicle. Then, the own vehicle shadow area estimation means 120 estimates the deformation amount of the own vehicle shadow due to a three-dimensional object (obstacle) existing around the own vehicle, and corrects the own vehicle shadow estimation area according to the deformation amount. The shadow estimation area can be estimated with high accuracy. In addition, since the ultrasonic sensors 5a to 5j are inexpensive, a plurality of ultrasonic sensors 5a to 5j can be mounted around the vehicle, and the resolution of obstacle detection can be increased.
[System startup judgment]
The own vehicle shadow recognition device 11 acquires information such as the illuminance acquisition device 351 from the in-vehicle network 330, and when it is determined that the own vehicle shadow will occur after sunrise, or is not raining after sunrise, the surroundings of the own vehicle When it is determined that is sufficiently bright, the vehicle shadow removal process is executed. That is, the own vehicle shadow recognition system is activated only when it is determined that the own vehicle shadow occurs. Accordingly, the system can be started at an appropriate timing, and unnecessary power consumption can be suppressed because the system does not start in a situation where no shadow occurs, such as at night or bad weather. The user interface 320 allows the user to manually set whether or not to execute the vehicle shadow removal process regardless of whether or not the vehicle shadow is present. Can be determined.

Next, the effect will be described.
The own vehicle shadow recognition apparatus 11 according to the first embodiment has the following effects.
(1) A vehicle position imaging device 300 that captures a range including a road surface on which the host vehicle is traveling, and a solar position calculating unit that calculates the position of the sun from information on the current position of the host vehicle and information on the current date and time 110, own vehicle shadow region estimation means for estimating a region where the own vehicle shadow is generated from the sun position information obtained by the sun position calculation means 110, the traveling direction information of the own vehicle, and the three-dimensional shape data of the own vehicle 120, and own vehicle shadow area detection means 130 for identifying the own vehicle shadow area from the image obtained by the vehicle periphery imaging device 300 based on the coordinate information obtained from the own vehicle shadow area estimation means 120. . Thereby, since the own vehicle shadow area | region can be calculated | required accurately from the exact position of the sun, the own vehicle shadow area | region can be specified from an image with high precision.
(2) Since the sun position calculation means 110 calculates the position of the sun from the standard meridian information corresponding to the current position of the own vehicle, the own vehicle shadow recognition system can be used at any position in the world.
(3) Since the image correction means 140 for correcting the luminance of the own vehicle shadow region based on the luminance information inside the own vehicle shadow region and the luminance information outside the own vehicle shadow region is provided, it is possible to visually recognize the outside of the own vehicle shadow region. Therefore, it is possible to correct only the luminance of the own vehicle shadow region without deteriorating the performance, and to improve the visibility of the own vehicle shadow region.
(4) Since the image in which the brightness of the own vehicle shadow area is corrected by the image correcting means 140 is output to the video output device 310, a video around the own vehicle with improved visibility can be provided to the user.
(5) The vehicle surrounding distance measuring device 350 that measures the distance to the three-dimensional object existing around the own vehicle is provided, and the own vehicle shadow region estimation means 120 uses the information from the vehicle surrounding distance measuring device 350 to Since the shadow area is estimated, a more accurate own vehicle shadow estimation area can be calculated in consideration of the deformation amount of the own car shadow by the three-dimensional object.
(6) Start determination means (step S200) that determines whether or not to start processing for detecting the own vehicle shadow region based on information obtained from the illuminance acquisition device 351 mounted on the vehicle, When the value of the illuminance from the illuminance acquisition device 351 is equal to or less than a predetermined threshold value, the process for detecting the own vehicle shadow region is not executed. As a result, in the situation where no shadow occurs, such as at night or in bad weather, the detection of the own vehicle shadow area is not performed, so that unnecessary power consumption can be suppressed.
(7) Since the user interface 320 that allows the user to set whether or not to start the process for detecting the own vehicle shadow area is provided, it is possible to determine whether or not the own vehicle shadow area can be detected with priority given to the user's intention.
(8) Since the vehicle periphery imaging device 300 captures the front, rear, left and right of the host vehicle, it can provide the user with images of the entire circumference of the host vehicle.

(Example 2)
FIG. 17 is a configuration diagram of the own vehicle shadow recognition apparatus 12 according to the second embodiment.
In the own vehicle shadow recognition device 12 of the second embodiment, a current position acquisition device 200 having a GPS (Global Positioning System) function is replaced with the configuration shown in FIG. The difference is that a vehicle direction measuring device 210 having an electronic compass for measuring the vehicle is provided. Further, instead of the configuration for obtaining the standard meridian information from the car navigation device 400, a standard meridian database 170 for storing standard meridian data is provided in the own vehicle shadow recognition control device 100. Further, in the second embodiment, the illuminance acquisition device 351, the obstacle recognition device 340, and the white line recognition device 341 of the first embodiment are omitted.
Since other configurations are the same as those of the first embodiment, the description thereof is omitted.
Next, the operation will be described.
Since the current position acquisition device 200 can obtain information equivalent to that of the car navigation device, the vehicle position recognition can be performed even if the vehicle is not equipped with the car navigation device. In addition, since the vehicle direction measuring device 210 uses an electronic compass, the direction of the vehicle is highly accurate even in situations where the accuracy of map matching or dead reckoning of the car navigation device is reduced or GPS is not received. Can measure corners.
Further, in the second embodiment, since the standard meridian database 170 is stored in the own vehicle shadow recognition control device 100, reference to the standard meridian corresponding to the current position of the own vehicle from the information obtained from the current position acquisition device 200 is referred. The sun's altitude and azimuth can be calculated respectively.
In the second embodiment, the illuminance acquisition device 351 as in the first embodiment is not installed. That is, a wiper signal and a headlight signal are acquired from the in-vehicle network 330, and the possibility that the vehicle shadow will occur is determined in a composite manner. Thereby, the own vehicle shadow recognition system can be applied even to a vehicle not equipped with the illuminance acquisition device. Further, the obstacle recognition device 340 and the white line recognition device 341 of the first embodiment are omitted, and the result of the image correction unit 140 is output only to the video output device 310. This eliminates the need for hardware for performing recognition processing, thereby reducing costs.
Other functions and effects are the same as those of the first embodiment, and thus description thereof is omitted.

(Example 3)
The own vehicle shadow recognition apparatus according to the third embodiment adds a function of correcting the own vehicle traveling direction θ _v based on the azimuth angle of the finally detected own vehicle shadow to the configuration of the first embodiment. . Hereinafter, a configuration different from that of the first embodiment will be described.
[Self-vehicle shadow removal processing]
FIG. 18 is a flowchart showing the flow of the own vehicle shadow removal process executed by the own vehicle shadow recognition control apparatus 100 of the third embodiment, and only steps for performing processing different from the processing of the first embodiment shown in FIG. 5 will be described. .
In step S700, the host vehicle traveling direction correction process is performed, and the process proceeds to return. The own traveling direction correction processing, the final of the detected self Kurumakage azimuth angle theta _c in Jikurumakage region detection unit 130, by using the correction equation shown by the following equation (16), the corrected self The vehicle traveling direction θ _v ′ is calculated.
θ _v '= A-(θ _c + 180)… (16)
Step S700 corresponds to traveling direction information correction means.
Next, the operation will be described.
[Vehicle direction correction function]
In the own vehicle shadow recognition apparatus according to the third embodiment, the information on the traveling direction of the own vehicle by the car navigation apparatus 400 is corrected by using the fact that the azimuth angle of the own vehicle shadow is accurately obtained by image recognition. Thereby, the accuracy of the car navigation apparatus 400 can be improved by using the azimuth angle of the own vehicle shadow. Note that the correction of the traveling azimuth angle of the host vehicle according to the third embodiment can also be applied to the configuration of the second embodiment, and has the same effects as the third embodiment. That is, the accuracy of the vehicle direction measuring device 210 can be improved using the azimuth angle of the vehicle shadow.

Next, the effect will be described.
In addition to the effects (1) to (8) of the first embodiment, the own vehicle shadow recognition apparatus of the third embodiment has the following effects.
(9) Information on the direction and size of the vehicle shadow is calculated from the vehicle shadow region specified by the vehicle shadow region detection unit 130, and the direction correction information correction unit (step S700). Thereby, the accuracy of the traveling direction information of the own vehicle in the car navigation device 400 can be improved.

Example 4
The own vehicle shadow recognition apparatus according to the fourth embodiment is an example in which a function of correcting own vehicle solid shape data is added to the configuration of the first embodiment. Hereinafter, a configuration different from that of the first embodiment will be described.
[Additional protrusion shadow area detection processing]
FIG. 19 is a flowchart showing the flow of additional projection shadow region detection processing according to the fourth embodiment. Each step will be described below. This process is performed when the state in which the similarity when collated with the own vehicle shadow estimation pattern in step S550 is equal to or less than the threshold value continues for a predetermined time or longer in the own vehicle shadow area detection process shown in FIG. (If it continues for more than 30 seconds in the state).
In step S710, it is determined whether the vehicle speed is greater than a predetermined threshold. If YES, the process proceeds to step S720, and if NO, this control is terminated. Here, the threshold is a speed at which it can be determined that the vehicle is traveling.
In step S720, it is determined whether the steering angle of the steered wheels is less than a predetermined threshold. If YES, the process proceeds to step S730, and if NO, this control is terminated. Here, the threshold value is a rudder angle at which it can be determined that the vehicle is traveling straight.
In step S730, the image I (t) is captured, and the process proceeds to step S740.
In step S740, the image I (t) acquired in step S730 is binarized based on the luminance information of the area of the vehicle shadow estimation pattern obtained in step S400 to obtain a binary image I _b (t). Further, when binarizing, in order to remove fine noise components other than the shadow region, a filtering process is performed so as to remove a region having a predetermined area or less from the extracted region, and the process proceeds to step S750. . By this processing, only the shadow candidate region in the image is extracted.
In step S750, a logical product (AND) process between the binary image I _b (t) and the binary image of the previous frame is performed, and the process proceeds to step S760.

In step S760, it is determined whether or not steps S710 to S750 have been executed for a predetermined number of frames n (for example, 100 frames). If YES, the process moves to step S790, and if NO, the process moves to step S770. Thereby, the shadow of the road structure and the oncoming vehicle is removed, and the own vehicle shadow region including the additional protrusion is extracted.
In step S770, an exclusive OR of the own vehicle shadow estimation pattern obtained in step S400 and G _b (n) obtained in step S750 is calculated, and the process proceeds to step S780. As a result, only the additional protrusion shadow region is extracted.
In step S780, the coordinate data of the region of the binary image obtained in step S770 is stored in the memory as the additional projection shadow region, and this control is terminated. In this step, solar altitude information or current time and GPS information are recorded simultaneously and stored in a database. The three-dimensional shape of the additional protrusion can be estimated based on the accumulated coordinate information of the plurality of additional protrusion shadow areas, and can be added to the vehicle three-dimensional shape data. This eliminates the need to execute the additional protrusion shadow region detection process (step S700) every time. Note that when the additional protrusion 20 is removed from the vehicle, the similarity in step S550 decreases, so the original three-dimensional shape data of the host vehicle is used.
In step S790, the frame number n is incremented, and the process proceeds to step S710.
As described above, in the additional protrusion shadow region detection process, as shown in FIG. 20, the logical product of the binary image obtained by binarizing the captured image and the binary image of the previous frame is obtained. By performing the processing for a predetermined number of frames, the vehicle shadow region including the additional protrusion is extracted from the image. Subsequently, by calculating an exclusive logical sum of the extracted vehicle shadow area and the vehicle shadow estimation pattern, an additional protrusion shadow area is extracted and added to the vehicle three-dimensional shape data.

Next, the operation will be described.
[Additional protrusion shadow area detection]
For example, when an additional protrusion 20 such as a carrier box is added to the own vehicle 1 as shown in FIG. 21A, the data of the additional protrusion 20 is not stored in the own vehicle three-dimensional shape database 160. In the brightness correction (first brightness correction) for correcting only the brightness of the own vehicle shadow area 4 shown in FIG. 5, the shadow area 21 of the additional protrusion 20 cannot be removed (FIG. 21B).
Therefore, in the fourth embodiment, as the second luminance correction, an exclusive logical sum of the own vehicle shadow area including the additional protrusion 20 and the own vehicle shadow estimation pattern is calculated and the additional protrusion shadow area 21 is extracted. The brightness correction of the vehicle shadow area 4 including the additional protrusion shadow area 21 can be performed, and the additional protrusion shadow area 21 can be removed.
Here, in Example 4, when the own vehicle 1 is not in the traveling state, the second luminance correction is not performed. This is the logical product processing of binary images of a predetermined number of frames when the own vehicle 1 is stopped, and the shadow of a three-dimensional object (tree, building, person, etc.) around the own vehicle overlaps the own vehicle shadow. This is because it is difficult to determine whether the own vehicle shadow area obtained by the above includes the shadow of the additional protrusion or the shadow of the structure. When the host vehicle 1 is in a traveling state, the influence of a three-dimensional object that overlaps the host vehicle shadow region can be eliminated, so that the brightness of the host vehicle shadow region including the additional protrusion shadow region can be accurately corrected.
Further, in the fourth embodiment, the second luminance correction is not performed even when the vehicle 1 is not in a straight traveling state. This is because when the vehicle is not in a straight traveling state, the shape of the original vehicle shadow may change, and the extraction accuracy of the vehicle shadow region including the additional protrusion shadow region may be reduced. When the vehicle 1 is in a straight traveling state, the shape of the original vehicle shadow does not change, and therefore the vehicle shadow region including the additional protrusion shadow region can be extracted with high accuracy.

(Example 5)
The own vehicle shadow recognition apparatus according to the fifth embodiment adds a function of correcting the result of the vehicle periphery ranging device 350 based on the shape and azimuth of the finally detected own vehicle shadow to the configuration of the first embodiment. It is a thing. Hereinafter, a configuration different from that of the first embodiment will be described.
[Self-vehicle shadow removal processing]
FIG. 22 is a flowchart showing the flow of the own vehicle shadow removal process executed by the own vehicle shadow recognition control apparatus 100 according to the fifth embodiment. Only steps for performing processing different from the processing of the first embodiment shown in FIG. 5 will be described. .
In step S800, distance measurement result correction processing is performed, and the process proceeds to return. Note that this process uses the distance D to the obstacle that is determined to be affirmative (YES) in step S455 and acquired from the vehicle peripheral distance measuring device 350 in step S470 in the own vehicle shadow area estimation process shown in FIG. This is performed when the vehicle shadow estimation area correction process is executed, and is not performed when a negative (NO) determination is made in step S455.
In this distance measurement result correction process, the point SPW ′ () of the own vehicle shadow 2b in the wall region shown in FIG. 14 is calculated from the azimuth angle and shape of the own vehicle shadow finally detected by the own vehicle shadow region detecting means 130. X _SPW ', Y _SPW ', Z _SPW '), and the SPW (X _SPW , Y _SPW , Z _SPW ) calculated by equation (13) based on this and the distance D obtained from the vehicle periphery ranging device 350 And the distance D is corrected so as to eliminate the error. Step S800 corresponds to distance correction means.

Next, the operation will be described.
[Distance measurement result correction]
In the own vehicle shadow recognition apparatus according to the fifth embodiment, the distance to the obstacle existing around the own vehicle by the vehicle peripheral distance measuring device 350 using the fact that the azimuth angle and shape of the own vehicle shadow are accurately obtained by image recognition. Correct D. Thereby, the accuracy of the vehicle periphery ranging device 350 can be improved.
Next, the effect will be described.
In addition to the effects (1) to (8) of the first embodiment, the own vehicle shadow recognition apparatus of the fifth embodiment has the following effects.
(10) Distance correction means for calculating information on the direction and size of the own vehicle shadow based on the own vehicle shadow area specified by the own vehicle shadow area detection means 130 and correcting the result of the vehicle periphery ranging device 350 (Step S800). Thereby, the accuracy of the distance information to the obstacle in the vehicle peripheral distance measuring device 350 can be improved.

(Other examples)
As mentioned above, although the form for implementing this invention was demonstrated based on the Example, the specific structure of this invention is not limited to the structure shown in the Example, and does not deviate from the summary of invention. Any change in the design of the range is included in the present invention.
For example, in the embodiment, an example in which an ultrasonic sensor is used as a distance measuring sensor has been described, but an infrared sensor, a laser radar, a millimeter wave radar, or the like may be used as the distance measuring sensor.
In the fourth embodiment, the straight traveling state is determined from the steering angle of the steered wheel, but may be determined from the angle of the steering wheel.

The technical ideas other than the invention described in the scope of claims ascertained from the embodiments will be described below.
(a) a camera that images the road surface around the vehicle;
Own vehicle position calculating means for calculating the current position of the own vehicle;
Solar position calculating means for calculating the position of the sun based on the current date and time information;
3D shape information of the vehicle, which is information related to the 3D shape of the vehicle,
A vehicle shadow region estimation means for estimating the vehicle shadow region using the vehicle three-dimensional shape information and the calculated current position of the vehicle and the position of the sun;
Vehicle shadow area detection means for identifying the vehicle shadow area estimated by the vehicle shadow area estimation means from the data captured by the camera;
A vehicle shadow recognition device characterized by comprising:
According to this invention, since the own vehicle shadow area can be accurately obtained from the accurate sun position, the own vehicle shadow area can be specified with high accuracy from the captured data.
(b) In the vehicle shadow recognition device described in (a),
The own vehicle three-dimensional shape information is three-dimensional shape data relating to the vehicle shape, and the own vehicle shadow recognition device.
According to this invention, since the own vehicle shadow area is estimated based on the three-dimensional shape data, the shape of the own vehicle shadow can be estimated with high accuracy.
(c) In the vehicle shadow recognition device described in (b),
Comprising an overhead image creation means for creating an overhead image viewed from above the vehicle based on data captured by the camera;
The own vehicle shadow area detecting means identifies the own vehicle shadow area from the created overhead view image.
According to the present invention, it is possible to provide an image around the own vehicle that is easy for the user to understand and has high visibility.
(d) In the vehicle shadow recognition device described in (c),
A vehicle shadow comprising a vehicle shadow region luminance correction means for correcting the luminance of the vehicle shadow region based on luminance information inside the vehicle shadow region and luminance information outside the vehicle shadow region. Recognition device.
According to the present invention, the brightness outside the vehicle shadow area is the same as the brightness before correction, and thus the visibility of the vehicle shadow area is improved while preventing the deterioration of the visibility outside the vehicle shadow area. Can do.

(e) In the vehicle shadow recognition device described in (d),
An own vehicle shadow recognition device, wherein an image obtained by correcting the luminance of the own vehicle shadow region by the own vehicle shadow region luminance correction means is output to a vehicle display device.
According to the present invention, it is possible to provide a user with an image around the own vehicle with improved visibility.
(f) In the vehicle shadow recognition device described in (e),
The own vehicle area brightness correcting means increases the brightness of the own vehicle shadow area and corrects the brightness to substantially the same as the brightness outside the area.
According to the present invention, the visibility of the own vehicle shadow region can be improved even when the brightness inside the own vehicle shadow region is difficult to see.
(g) In the vehicle shadow recognition device described in (b),
A vehicle shadow comprising a vehicle shadow region luminance correction means for correcting the luminance of the vehicle shadow region based on luminance information inside the vehicle shadow region and luminance information outside the vehicle shadow region. Recognition device.
According to the present invention, the brightness outside the vehicle shadow area is the same as the brightness before correction, and thus the visibility of the vehicle shadow area is improved while preventing the deterioration of the visibility outside the vehicle shadow area. Can do.
(h) In the vehicle shadow recognition device described in (g),
The own vehicle area brightness correcting means increases the brightness of the own vehicle shadow area and corrects the brightness to substantially the same as the brightness outside the area.
According to the present invention, the visibility of the own vehicle shadow region can be improved even when the brightness inside the own vehicle shadow region is difficult to see.
(i) An image display method for processing data captured by a camera and displaying the data on a vehicle display device,
Calculate the current position of the vehicle and the current position of the sun, estimate the shadow area of the vehicle from both calculated positions and the 3D data of the vehicle shape of the vehicle,
An image display method, comprising: displaying an image after correcting the estimated brightness of the vehicle shadow region on the vehicle display device.
According to the present invention, the vehicle shadow region can be accurately obtained from the exact sun position. Therefore, the vehicle shadow region can be identified with high accuracy from the captured data, and the vehicle has improved visibility. It is possible to provide users with images around the car.
(j) In the image display method described in (i),
An image display method comprising correcting the luminance of the own vehicle shadow region based on luminance information outside the own vehicle shadow region.
According to the present invention, the brightness outside the vehicle shadow area is the same as the brightness before correction, and thus the visibility of the vehicle shadow area is improved while preventing the deterioration of the visibility outside the vehicle shadow area. Can do.

11 Vehicle shadow recognition device
110 Solar position calculation means
120 Vehicle shadow area estimation means
130 Vehicle shadow area detection means
300 Vehicle periphery imaging device (Self-vehicle periphery imaging device)

Claims (7)

A vehicle periphery imaging device that images front and rear, left and right of the vehicle within a range including a road surface on which the vehicle is traveling
Solar position calculating means for calculating the position of the sun from information on the current position of the own vehicle, information on the current date and time, and information on the standard meridian corresponding to the current position of the own vehicle ;
A vehicle shadow region estimation unit that estimates a region where the vehicle shadow is generated from the position information of the sun obtained by the sun position calculation unit, the traveling direction information of the host vehicle, and the three-dimensional shape information of the host vehicle;
Based on the coordinate information obtained from the vehicle shadow area estimation means, the vehicle shadow area detection means for identifying the vehicle shadow area from the image obtained by the vehicle periphery imaging device;
From the own vehicle shadow area specified by the own vehicle shadow area detecting means, calculating the direction and size information of the own vehicle shadow, and the traveling direction information correcting means for correcting the traveling direction information of the own vehicle,
A vehicle shadow recognition device characterized by comprising: The own vehicle shadow recognition device according to claim 1,
An own vehicle shadow comprising a own vehicle shadow region luminance correction means for correcting the luminance of the own vehicle shadow region based on the luminance information inside the own vehicle shadow region and the luminance information outside the own vehicle shadow region. Recognition device. The own vehicle shadow recognition apparatus according to claim 2 ,
An own vehicle shadow recognition device, wherein an image obtained by correcting the luminance of the own vehicle shadow region by the own vehicle shadow region luminance correction means is output to a vehicle display device. The own vehicle shadow recognition device according to claim 1,
It is equipped with a distance measuring device that measures the distance to a three-dimensional object existing around the vehicle.
The own vehicle shadow recognition device, wherein the own vehicle shadow region estimation means estimates the own vehicle shadow region using information from the distance measuring device. In the own vehicle shadow recognition device according to claim 4 ,
Based on the own vehicle shadow area specified by the own vehicle shadow area detecting means, it comprises distance correction means for calculating information on the direction and size of the own vehicle shadow and correcting the result of the distance measuring device. A car shadow recognition device. The own vehicle shadow recognition device according to claim 1,
An activation determining means for determining whether to activate a process for detecting the own vehicle shadow area based on information obtained from an illuminance sensor mounted on the vehicle;
The said vehicle determination part does not perform the process for detecting the own vehicle shadow area | region, when the value of the illumination intensity from the said illumination intensity sensor is below a predetermined value, The own vehicle shadow recognition apparatus characterized by the above-mentioned. The own vehicle shadow recognition device according to claim 1,
A host vehicle shadow recognition apparatus comprising a user interface that allows a user to set whether to activate a process for detecting a host vehicle shadow region.

Images