WO2010029592A1

WO2010029592A1 - Vehicle periphery monitoring apparatus

Info

Publication number: WO2010029592A1
Application number: PCT/JP2008/002484
Authority: WO
Inventors: 原田雅之; 都丸義広; 三木洋平; 藤本仁志
Original assignee: 三菱電機株式会社
Priority date: 2008-09-09
Filing date: 2008-09-09
Publication date: 2010-03-18
Also published as: JPWO2010029592A1; JP5295254B2

Abstract

A vehicle periphery monitoring apparatus has an image holding section (4) to acquire and hold an image P1 photographed by a camera (2) with a lamp (3) turned off, while acquiring and holding an image P2 photographed by the camera (2) with the lamp (3) turned on, a shadow detector section (5) to detect a shadow S of a solid object A that is made by the light from the lamp (3) and is projected on the image P2 by comparing the image P1 and the image P2 that are held by the image holding section (4), and a solid object detector section (6) to detect the solid object A using the shadow S of the solid object A that is detected by the shadow detector section (5).

Description

Vehicle perimeter monitoring device

The present invention relates to a vehicle surrounding monitoring device that detects a three-dimensional object existing around a vehicle.

A conventional vehicle periphery monitoring device is provided with a camera that captures an area that becomes a blind spot from the driver's seat of the vehicle, and the user can visually recognize the blind spot area by viewing an image captured by the camera.
At that time, it is important that the user can determine whether or not an object present on the image is in contact with the vehicle, but it is difficult to grasp the height of the object only by looking at a single image. Therefore, it is difficult to determine whether to contact the vehicle.

For example, as shown in FIG. 16B, when any one of objects A, B, and C having different heights exists around the vehicle, the object is photographed by a camera attached to the vehicle. All images taken by the camera are as shown in FIG.
When the actual object is C, the object C is on the road surface and is less likely to come into contact with the vehicle. However, when the object is A, the vehicle will come into contact with the vehicle as it moves backward. In driving, it is important to determine whether or not the object D on the image is in contact with the vehicle. On the image taken by the camera, the objects A, B, and C are displayed as the object D having the same length. Therefore, it is difficult to determine whether to contact the vehicle.

However, if it is possible to detect the height of the object D and confirm whether the substance D is the substance A, B, or C, it is possible to determine whether the object D is in contact with the vehicle. it can.
For example, Patent Document 1 below discloses a method for detecting that an object on an image is a three-dimensional object by comparing the moving distance of a vehicle and the moving distance of an object on an image captured by a camera. ing.

Japanese Unexamined Patent Publication No. 2007-087236 (paragraph numbers [0030] to [0034], FIG. 1)

Since the conventional vehicle periphery monitoring device is configured as described above, it is necessary to use the moving distance of the vehicle to detect that the object on the image is a three-dimensional object, and the vehicle is stopped. However, there is a problem that it cannot be detected that the object on the image is a three-dimensional object.

The present invention has been made to solve the above-described problems, and an object of the present invention is to obtain a vehicle periphery monitoring device that can detect a three-dimensional object even when the vehicle is stopped.

The vehicle surrounding monitoring apparatus according to the present invention acquires a first image taken by a camera while the lamp is turned off, and also obtains a second image taken by the camera while the lamp is turned on. A shadow for detecting a shadow of a three-dimensional object by a lamp projected on the second image by comparing the first image acquired by the image acquiring unit for acquiring an image and the second image acquired by the image acquiring unit. Detecting means, and the three-dimensional object detecting means detects the three-dimensional object using the shadow of the three-dimensional object detected by the shadow detecting means.

This has the effect that a three-dimensional object can be detected even when the vehicle is stopped.

It is a block diagram which shows the vehicle periphery monitoring apparatus by Embodiment 1 of this invention. It is explanatory drawing which shows a mode that the circumference | surroundings of a vehicle are monitored by the vehicle periphery monitoring apparatus by Embodiment 1 of this invention. It is a flowchart which shows the processing content of the vehicle periphery monitoring apparatus by Embodiment 1 of this invention. It is explanatory drawing which shows the image P1 image | photographed with the camera 2 in the state in which the lamp | ramp 3 was extinguished. It is explanatory drawing which shows the image P2 image | photographed with the camera 2 in the state in which the lamp | ramp 3 is lighting. It is explanatory drawing which shows the example of a display of the height h of the solid object A. FIG. It is a block diagram which shows the vehicle periphery monitoring apparatus by Embodiment 2 of this invention. It is explanatory drawing which shows a mode that the circumference | surroundings of a vehicle are monitored by the vehicle periphery monitoring apparatus by Embodiment 2 of this invention. It is a flowchart which shows the processing content of the vehicle periphery monitoring apparatus by Embodiment 2 of this invention. It is explanatory drawing which shows the image P1 image | photographed with the camera 2 in the state in which the lamp | ramp 3 was extinguished. It is explanatory drawing which shows the image P2 image | photographed with the camera 2 in the state in which the lamp | ramp 3 is lighting. It is a block diagram which shows the vehicle periphery monitoring apparatus by Embodiment 3 of this invention. It is explanatory drawing which shows a mode that the circumference | surroundings of a vehicle are monitored by the vehicle periphery monitoring apparatus by Embodiment 3 of this invention. It is explanatory drawing used for the detection result evaluation of the vehicle periphery monitoring apparatus by Embodiment 3 of this invention. It is a flowchart which shows the processing content of the vehicle periphery monitoring apparatus by Embodiment 3 of this invention. It is explanatory drawing which shows the image etc. which were image | photographed with the camera of the conventional vehicle periphery monitoring apparatus.

Hereinafter, in order to describe the present invention in more detail, the best mode for carrying out the present invention will be described with reference to the accompanying drawings.
Embodiment 1 FIG.
1 is a block diagram showing a vehicle surrounding monitoring apparatus according to Embodiment 1 of the present invention.
FIG. 2 is an explanatory diagram showing a situation in which the surroundings of the vehicle are monitored by the vehicle surrounding monitoring apparatus according to Embodiment 1 of the present invention.

1 and 2, the camera 2 is installed, for example, at the rear portion of the vehicle 1 and photographs the surroundings of the vehicle 1.
The lamp 3 is installed, for example, at the rear of the vehicle 1 and irradiates light toward the periphery of the vehicle 1 under the instruction of the control unit 7.

Under the instruction of the control unit 7, the image holding unit 4 acquires and holds the image P1 (first image) taken by the camera 2 with the lamp 3 turned off, and the lamp 3 is turned on. In this state, a process of acquiring and holding the image P2 (second image) taken by the camera 2 is performed.
The image holding unit 4 and the control unit 7 constitute an image acquisition unit.

Under the instruction of the control unit 7, the shadow detection unit 5 compares the image P1 and the image P2 held in the image holding unit 4, and determines the shadow S of the three-dimensional object A by the lamp 3 projected on the image P2. Perform the detection process.
The shadow detection unit 5 and the control unit 7 constitute a shadow detection unit.

The three-dimensional object detection unit 6 performs processing for detecting the three-dimensional object A using the shadow S of the three-dimensional object A detected by the shadow detection unit 5 under the instruction of the control unit 7. That is, the process of calculating the three-dimensional coordinates (x ₀ , y ₀ , z ₀ ) of the three-dimensional object A is performed using the shadow S of the three-dimensional object A detected by the shadow detection unit 5.
The three-dimensional object detection unit 6 and the control unit 7 constitute a three-dimensional object detection unit.

The control unit 7 is a processing unit that controls the operations of the lamp 3, the image holding unit 4, the shadow detection unit 5, the three-dimensional object detection unit 6, and the display unit 8, and the control unit 7 is a three-dimensional object detected by the three-dimensional object detection unit 6. object 3-dimensional coordinates of _{_{a (x 0, y 0,}} z 0) and converts the user information necessary to display on the display unit 8.
The display unit 8 is composed of a liquid crystal display or the like, and displays an image photographed by the camera 2 or information necessary for the user under the instruction of the control unit 7.

In FIG. 2, the coordinates (x ₁ , y ₁ , z ₁ ) are coordinates indicating the position where the lamp 3 is installed, and the coordinates (x ₀ , y ₀ , z ₀ ) are the coordinates of the vertex of the three-dimensional object A. is there.
The coordinates (x _b , y _b , z _b ) are the coordinates of the point where the solid object A intersects the road surface, and the coordinates (x _s , y _s , z _s ) are shadows corresponding to the vertices of the solid object A. The coordinates of the end point of S.
FIG. 3 is a flowchart showing the processing contents of the vehicle periphery monitoring apparatus according to Embodiment 1 of the present invention.

Next, the operation will be described.
First, the control unit 7 outputs an image acquisition command to the image holding unit 4 while the lamp 3 is turned off.
When receiving an image acquisition command from the control unit 7, the image holding unit 4 acquires and holds the image P1 captured by the camera 2 (step ST1).

Here, FIG. 4 is an explanatory view showing an image P1 taken by the camera 2 in a state where the lamp 3 is turned off.
In FIG. 4, coordinates (u ₀ , v ₀ ) are coordinates on the image P1 of points corresponding to the coordinates (x ₀ , y ₀ , z ₀ ) of the vertex of the three-dimensional object A in FIG.
The coordinates (u _b , v _b ) are the coordinates on the image P1 of the point corresponding to the coordinates (x _b , y _b , z _b ) of the intersection of the solid object A and the road surface in FIG.
However, since the image is taken with the lamp 3 turned off, the shadow S of the three-dimensional object A by the lamp 3 is not projected on the image P1 in FIG.

Next, the control unit 7 turns on the lamp 3 (step ST2), generates a shadow S of the three-dimensional object A by the lamp 3 on the road surface, and then outputs an image acquisition command to the image holding unit 4.
When receiving an image acquisition command from the control unit 7, the image holding unit 4 acquires and holds the image P2 captured by the camera 2 (step ST3).

Here, FIG. 5 is an explanatory diagram showing an image P2 taken by the camera 2 in a state where the lamp 3 is lit.
In FIG. 5, coordinates (u ₀ , v ₀ ) are coordinates on the image P2 of points corresponding to the coordinates (x ₀ , y ₀ , z ₀ ) of the vertex of the three-dimensional object A in FIG.
Coordinates (u _{_b,} v _b) is a three-dimensional object A and the road surface at the intersection of the coordinates _{_{(x b, y b, z}} b) the coordinates of the image P2 in a point corresponding to the in Figure 2.
The coordinates (u _s , v _s ) are the coordinates on the image P2 of the point corresponding to the coordinates (x _s , y _s , z _s ) of the end point of the shadow S in FIG.

When the image holding unit 4 acquires the images P 1 and P 2, the control unit 7 outputs a detection command for the shadow S of the three-dimensional object A by the lamp 3 to the shadow detection unit 5.
When the shadow detection unit 5 receives the detection command of the shadow S from the control unit 7, the shadow detection unit 5 compares the image P1 and the image P2 held by the image holding unit 4 and compares the image P1 with the lamp 3 projected on the image P2. The shadow S of the object A is detected (step ST4).

For example, each pixel value of the image P2 is subtracted from each pixel value of the image P1, and a portion where a pixel whose subtraction value is equal to or larger than a predetermined threshold exists is detected as the shadow S of the three-dimensional object A.
However, the method for detecting the shadow S is not particularly limited. For example, the user may compare the image P1 and the image P2 and specify the position at which the shadow S is determined to exist.
When the shadow detection unit 5 detects the shadow S of the three-dimensional object A by the lamp 3, the coordinates of the points in FIG. 5 (x _s , y _s , z _s ) corresponding to the coordinates (x _s , y _s , z _s ) of the shadow S in FIG. u _s , v _s ).

When the shadow detection unit 5 detects the shadow S of the three-dimensional object A by the lamp 3, the control unit 7 outputs a detection command for the three-dimensional object A to the three-dimensional object detection unit 6.
When the solid object detection unit 6 receives the detection command for the solid object A from the control unit 7, the solid object detection unit 6 detects the solid object A using the shadow S of the solid object A detected by the shadow detection unit 5.
That is, the three-dimensional object detection unit 6 uses the shadow S of the three-dimensional object A detected by the shadow detection unit 5 to calculate the coordinates (x ₀ , y ₀ , z ₀ ) of the vertex of the three-dimensional object A in FIG. (Step ST5).

Hereinafter, a calculation example of the three-dimensional coordinates (x ₀ , y ₀ , z ₀ ) of the three-dimensional object A in the three-dimensional object detection unit 6 will be specifically described.
First, with the optical center of the camera 2 as the coordinate origin, the Z axis that is the coordinate axis is the line-of-sight direction of the camera 2, the Y axis is orthogonal to the Z axis, and upwards, and the X axis is orthogonal to the Y and Z axes And
Since the camera 2 and the lamp 3 are fixed to the vehicle 1 and the positional relationship between the camera 2 and the lamp 3 is unchanged, the three-dimensional object detection unit 6 previously measures the position where the lamp 3 is installed. The coordinates (x _l , y _l , z _l ) of the position where the lamp 3 is installed are obtained in advance.

The three-dimensional object detection unit 6 also has coordinates (x _b , y _b , z _b ) of the point where the three-dimensional object A intersects the road surface, and coordinates (x _s) of the end point of the shadow S corresponding to the vertex of the three-dimensional object A. , Y _s , z _s ) exist on the same plane (the road surface of the vehicle), the relationship between the points on the image captured by the camera 2 in advance and the points on the road surface of the vehicle 1 (hereinafter, , Referred to as “camera image / road surface relationship”) by plane projective transformation or the like.
When the image holding unit 4 acquires the image P2, the three-dimensional object detection unit 6 determines the intersection of the three-dimensional object A and the road surface in FIG. 2 from the coordinates (u _b , v _b ) on the image P2 according to the camera image / road surface relationship. coordinates _{_{(x b, y b, z}} b) is calculated.

Further, the three-dimensional object detection unit 6 determines the coordinates (x _s , y _s , z _s ) of the end point of the shadow S in FIG. 2 from the coordinates (u _s , v _s ) on the image P2 according to the camera image / road surface relationship. Is calculated.
However, the coordinates (x ₀ , y ₀ , z ₀ ) of the vertex of the three-dimensional object A in FIG. 2 do not exist on the road surface of the vehicle 1, and are coordinates on the image P 2 according to the camera image / road surface relationship ( can not be calculated from u _0, v _0), the three-dimensional object detection unit 6 is calculated as below.

The three-dimensional object detection unit 6 uses coordinates (x ₁ , y ₁ , z ₁ ) and coordinates (x _s , y _s ) that have already been calculated for the coordinates (x ₀ , y ₀ , z ₀ ) of the three-dimensional object A. , since the position in the middle of the z _s), using the coordinates _{_{(x l, y l, z}} l) and the coordinates _{_{(x s, y s, z}} s), as shown in the following formula (1) The coordinates (x ₀ , y ₀ , z ₀ ) of the vertex of the three-dimensional object A are defined.

Also, three-dimensional object detection unit 6, coordinates x _0, for y ₀ is the focal length f and the coordinates of the camera 2 (x _0, y _0, z ₀₎ to the coordinates on the image P2 corresponding (u _0, v ₀ ), The coordinates x ₀ and y ₀ are defined as in the following formula (2).

However, since the focal length f is an eigenvalue of the camera 1, it is determined by measuring in advance.

When the three-dimensional object detection unit 6 defines Expression (1) and Expression (2) as described above, the following Expression (3) is derived from Expression (1) and Expression (2).

When the three-dimensional object detection unit 6 derives the expression (3), the value of “s” in the expression (3) is obtained, and the value of “s” is substituted into the expression (1), thereby obtaining the vertex of the three-dimensional object A. Coordinates (x ₀ , y ₀ , z ₀ ) are calculated.

When the three-dimensional object detection unit 6 detects the three-dimensional object A, the control unit 7 converts the three-dimensional coordinates (x ₀ , y ₀ , z ₀ ) of the three-dimensional object A into information necessary for the user and displays the information on the display unit 8. (Step ST6).
For example, when the height h of the three-dimensional object A is displayed, the height h is calculated from the three-dimensional coordinates (x ₀ , y ₀ , z ₀ ) of the three-dimensional object A as shown in the following equation (4). And displayed on the display unit 8.

FIG. 6 is an explanatory diagram illustrating a display example of the height h of the three-dimensional object A. FIG. 6 shows an example in which the height h of the three-dimensional object A is 25 cm.
In FIG. 6, the specific height h of the three-dimensional object A is displayed, but the information presentation method is not particularly limited.
For example, if it is sufficient to display information on whether or not to contact the vehicle 1, a three-dimensional object A having a height h or higher that the vehicle 1 cannot get over may be highlighted.

As apparent from the above, according to the first embodiment, the image P1 photographed by the camera 2 is acquired and held while the lamp 3 is turned off, and the lamp 3 is turned on. Thus, the image holding unit 4 that acquires and holds the image P2 captured by the camera 2 is compared with the image P1 and the image P2 that are held in the image holding unit 4, and the lamp that is projected on the image P2 3 to detect the shadow S of the three-dimensional object A, and the three-dimensional object detection unit 6 uses the shadow S of the three-dimensional object A detected by the shadow detection unit 5 to detect the three-dimensional object A. Thus, the three-dimensional object A can be detected even when the vehicle 1 is stopped.

Embodiment 2. FIG.
FIG. 7 is a block diagram showing a vehicle periphery monitoring apparatus according to Embodiment 2 of the present invention.
FIG. 8 is an explanatory diagram showing a situation in which the surroundings of the vehicle are monitored by the vehicle surroundings monitoring apparatus according to Embodiment 2 of the present invention.
7 and 8, the same reference numerals as those in FIGS. 1 and 2 indicate the same or corresponding parts, and thus description thereof is omitted.

Under the instruction of the control unit 14, the shadow detection unit 11 compares the image P1 held in the image holding unit 4 with the image P2, and detects the shadow S of the three-dimensional object A due to the sun projected on the image P1. Perform the process.
The shadow detection unit 11 and the control unit 14 constitute a shadow detection unit.

The solar information input unit 12 obtains the position of the vehicle 1 using, for example, GPS, calculates the position of the sun from the position of the vehicle 1 and the current time, and uses a unit vector indicating the direction of sunlight as the solar information. The process given to 13 is executed.
The three-dimensional object detection unit 13 uses the shadow S of the three-dimensional object A by the sun detected by the shadow detection unit 11 and the sun information provided from the sun information input unit 12 under the instruction of the control unit 14 to convert the three-dimensional object A. Perform the detection process.
The sun information input unit 12, the three-dimensional object detection unit 13, and the control unit 14 constitute a three-dimensional object detection unit.

The control unit 14 is a processing unit that controls operations of the lamp 3, the image holding unit 4, the shadow detection unit 11, the three-dimensional object detection unit 13, and the display unit 8, and the control unit 14 is a three-dimensional object detected by the three-dimensional object detection unit 13. object 3-dimensional coordinates of _{_{a (x 0, y 0,}} z 0) and converts the user information necessary to display on the display unit 8.

In FIG. 8, a vector (x _v , y _v , z _v ) is a unit vector indicating the direction of sunlight.
FIG. 9 is a flowchart showing the processing contents of the vehicle periphery monitoring apparatus according to Embodiment 2 of the present invention.

In the first embodiment, the shadow detection unit 5 detects the shadow S of the three-dimensional object A by the lamp 3, and the three-dimensional object detection unit 6 detects the three-dimensional object A by using the shadow S of the three-dimensional object A by the lamp 3. Although the thing was shown, when the shadow S of the solid object A is projected by the sun, the shadow detection part 11 detects the shadow S of the solid object A by the sun, and the solid object detection part 13 is the solid object A by the sun. The three-dimensional object A may be detected using the shadow S.

Next, the operation will be described.
First, the control unit 14 outputs an image acquisition command to the image holding unit 4 while the lamp 3 is turned off.
When receiving an image acquisition command from the control unit 14, the image holding unit 4 acquires and holds the image P1 captured by the camera 2 (step ST11).

Here, FIG. 10 is an explanatory view showing an image P1 taken by the camera 2 in a state where the lamp 3 is turned off.
In FIG. 10, coordinates (u ₀ , v ₀ ) are coordinates on the image P1 of points corresponding to the coordinates (x ₀ , y ₀ , z ₀ ) of the vertex of the three-dimensional object A in FIG.
The coordinates (u _b , v _b ) are the coordinates on the image P1 of the point corresponding to the coordinates (x _b , y _b , z _b ) of the intersection of the solid object A and the road surface in FIG.
In the state where the lamp 3 is turned off, no light is irradiated from the lamp 3 toward the three-dimensional object A, so that the shadow S of the three-dimensional object A by the sun is projected on the image P1 without being affected by the lamp 3. .
The coordinates (u _s , v _s ) are the coordinates on the image P1 of points corresponding to the coordinates (x _s , y _s , z _s ) of the end points of the shadow S in FIG.

Next, the control unit 14 turns on the lamp 3 (step ST12), and outputs an image acquisition command to the image holding unit 4.
When receiving an image acquisition command from the control unit 14, the image holding unit 4 acquires and holds the image P2 captured by the camera 2 (step ST13).
In the state where the lamp 3 is lit, the light irradiated from the lamp 3 is illuminated on the three-dimensional object A, so that the shadow S of the three-dimensional object A caused by the sun becomes inconspicuous.

Here, FIG. 11 is an explanatory diagram showing an image P2 taken by the camera 2 in a state where the lamp 3 is lit.
In FIG. 11, coordinates (u ₀ , v ₀ ) are coordinates on the image P2 of points corresponding to the coordinates (x ₀ , y ₀ , z ₀ ) of the vertex of the three-dimensional object A in FIG.
The coordinates (u _b , v _b ) are the coordinates on the image P2 of the points corresponding to the coordinates (x _b , y _b , z _b ) of the intersection of the solid object A and the road surface in FIG.

When the image holding unit 4 acquires the images P 1 and P 2, the control unit 14 outputs a detection command for the shadow S of the three-dimensional object A by the sun to the shadow detection unit 11.
When the shadow detection unit 11 receives the detection command of the shadow S from the control unit 14, the shadow detection unit 11 compares the image P 1 held in the image holding unit 4 with the image P 2, and the solid object by the sun projected on the image P 1. A shadow S of A is detected (step ST14).

For example, each pixel value of the image P2 is subtracted from each pixel value of the image P1, and a portion where a pixel whose subtraction value is equal to or larger than a predetermined threshold exists is detected as the shadow S of the three-dimensional object A.
However, the method for detecting the shadow S is not particularly limited. For example, the user may compare the image P1 and the image P2 and specify the position at which the shadow S is determined to exist.
When the shadow detection unit 11 detects the shadow S of the three-dimensional object A by the sun, the coordinates (u) of the point in FIG. 10 corresponding to the coordinates (x _s , y _s , z _s ) of the shadow S in FIG. _s , v _s ).

The solar information input unit 12 obtains the position (latitude and longitude) of the vehicle 1 using, for example, GPS, calculates the position of the sun from the position of the vehicle 1 and the current time, and indicates the direction of the sunlight as solar information. A vector (x _v , y _v , z _v ) is calculated (step ST15).

When the shadow detection unit 11 detects the shadow S of the three-dimensional object A due to the sun and the solar information input unit 12 calculates a unit vector (x _v , y _v , z _v ) indicating the direction of the sun rays, A detection command for the three-dimensional object A is output to the three-dimensional object detection unit 13.
When the solid object detection unit 13 receives a detection command for the solid object A from the control unit 14, the shadow S of the solid object A detected by the shadow detection unit 11 and the unit vector (x _v ) calculated by the sun information input unit 12. , Y _v , z _v ) to detect the three-dimensional object A.
That is, the three-dimensional object detection unit 13 uses the shadow S and the unit vector (x _v , y _v , z _v ) of the three-dimensional object A detected by the shadow detection unit 5 to coordinate the vertex of the three-dimensional object A in FIG. (X ₀ , y ₀ , z ₀ ) is calculated (step ST16).

Hereinafter, a calculation example of the three-dimensional coordinates (x ₀ , y ₀ , z ₀ ) of the three-dimensional object A in the three-dimensional object detection unit 13 will be specifically described.
First, with the optical center of the camera 2 as the coordinate origin, the Z axis that is the coordinate axis is the line-of-sight direction of the camera 2, the Y axis is orthogonal to the Z axis, and upwards, and the X axis is orthogonal to the Y and Z axes And

The three-dimensional object detection unit 13 coordinates the coordinates (x _b , y _b , z _b ) of the point where the three-dimensional object A intersects the road surface, and the coordinates (x _s , y) of the shadow S corresponding to the vertex of the three-dimensional object A. _{s 1} , z _s ) exist on the same plane (road surface of the vehicle), so the camera image / road surface relationship is obtained in advance by plane projective transformation or the like.
When the image holding unit 4 acquires the image P1, the three-dimensional object detection unit 13 determines the intersection of the three-dimensional object A and the road surface in FIG. 8 from the coordinates (u _b , v _b ) on the image P1 according to the camera image / road surface relationship. The coordinates (x _b , y _b , z _b ) are calculated.

In addition, the three-dimensional object detection unit 13 determines the coordinates (x _s , y _s , z _s ) of the end point of the shadow S in FIG. 8 from the coordinates (u _s , v _s ) on the image P1 according to the camera image / road surface relationship. Is calculated.
However, the coordinates (x ₀ , y ₀ , z ₀ ) of the vertex of the three-dimensional object A in FIG. 8 do not exist on the road surface of the vehicle 1, and the coordinates on the image P 1 according to the camera image / road surface relationship ( Since it cannot be calculated from u ₀ , v ₀ ), the three-dimensional object detection unit 13 calculates as follows.

The three-dimensional object detection unit 13 has the same coordinates in the vertex (x ₀ , y ₀ , z ₀ ) of the three-dimensional object A and the already calculated coordinates (x _s , y _s , z _s ), that is, solar information. Since the unit vector (x _v , y _v , z _v ) calculated by the input unit 12 exists in the direction indicated, the coordinates (x _s , y _s , z _s ) and the unit vector (x _v , y _v , z _v ) is used to define the coordinates (x ₀ , y ₀ , z ₀ ) of the vertex of the three-dimensional object A as shown in the following equation (5).

The three-dimensional object detection unit 13 defines the vertex coordinates (x ₀ , y ₀ , z ₀ ) of the three-dimensional object A as in equation (5), and performs the same process as the three-dimensional object detection unit 6 of the first embodiment. Then, the equation (2) is defined, and the value of “t” in the equation (5) is obtained using the equations (5) and (2), whereby the coordinates (x ₀ , y _{_0,} z ₀₎ is calculated.

When the three-dimensional object detection unit 13 detects the three-dimensional object A, the control unit 14 uses the three-dimensional coordinates (x ₀ , y ₀ , z ₀ ) of the three-dimensional object A to the user as in the control unit 7 of the first embodiment. Is converted into necessary information and displayed on the display unit 8 (step ST17).
For example, when displaying the height h of the three-dimensional object A, the height h is calculated from the three-dimensional coordinates (x ₀ , y ₀ , z ₀ ) of the three-dimensional object A using the above equation (4). To display.

As apparent from the above, according to the second embodiment, the image P1 photographed by the camera 2 is acquired and held while the lamp 3 is turned off, and the lamp 3 is turned on. The image holding unit 4 that acquires and holds the image P2 captured by the camera 2 is compared with the image P1 and the image P2 that are held in the image holding unit 4, and the sun projected on the image P1. The shadow detection unit 11 for detecting the shadow S of the three-dimensional object A is provided, and the three-dimensional object detection unit 13 calculates the shadow S of the three-dimensional object A detected by the shadow detection unit 11 and the unit vector calculated by the sun information input unit 12. Since the three-dimensional object A is detected using (x _v , y _v , z _v ), the three-dimensional object A can be detected even when the vehicle 1 is stopped. .

Embodiment 3 FIG.
12 is a block diagram showing a vehicle surrounding monitoring apparatus according to Embodiment 3 of the present invention.
FIG. 13 is an explanatory diagram showing a situation in which the surroundings of the vehicle are monitored by the vehicle surrounding monitoring apparatus according to Embodiment 3 of the present invention.
12 and FIG. 13, the same reference numerals as those in FIG. 1 and FIG.

The lamp 3a, which is the first lamp, is installed at the rear of the vehicle 1, for example, and irradiates light toward the periphery of the vehicle 1 under the instruction of the control unit 24.
The lamp 3b, which is the second lamp, is installed at a position different from the lamp 3a at the rear part of the vehicle 1, for example, and irradiates light from the position different from the lamp 3a toward the periphery of the vehicle 1 under the instruction of the control unit 24. .

Under the instruction of the control unit 24, the image holding unit 21 acquires and holds the image P1 (first image) taken by the camera 2 with the

lamps

3a and 3b turned off, and the lamp 3a With the lamp 3b turned off, the image P2 (second image) taken by the camera 2 is acquired and held, and the lamp 3a is turned off and the lamp 3b is turned on. In this state, a process of acquiring and holding the image P3 (third image) taken by the camera 2 is performed.
The image holding unit 21 and the control unit 24 constitute an image acquisition unit.

Under the instruction of the control unit 24, the shadow detection unit 22 compares the image P1 and the image P2 held in the image holding unit 21, and calculates the shadow Sa of the three-dimensional object A by the lamp 3a projected on the image P2. While detecting, the image P1 currently hold | maintained at the image holding | maintenance part 21 and the image P3 are compared, and the process which detects the shadow Sb of the solid object A by the lamp | ramp 3b currently projected on the image P3 is implemented.
The shadow detection unit 22 and the control unit 24 constitute a shadow detection unit.

The three-dimensional object detection unit 23 detects the three-dimensional object A using the shadow Sa of the three-dimensional object A detected by the shadow detection unit 22 under the instruction of the control unit 24, and the three-dimensional object detected by the shadow detection unit 22. The process of detecting the three-dimensional object A is performed using the shadow Sb of A.
The three-dimensional object detection unit 23 and the control unit 24 constitute a three-dimensional object detection unit.

The control unit 24 is a processing unit that controls operations of the

lamps

3 a and 3 b, the image holding unit 21, the shadow detection unit 22, the three-dimensional object detection unit 23, and the display unit 25, and the control unit 24 is operated by the three-dimensional object detection unit 23. The three-dimensional object detected by using the shadow Sa of the three-dimensional object A by the lamp 3b and the three-dimensional object detected by using the shadow Sb of the three-dimensional object A by the lamp 3b are collated, and the validity of the detection result of the three-dimensional object detection unit 23 Implement a process to evaluate That is, if the coordinates of the three-dimensional object detected using the shadow Sa of the three-dimensional object A by the lamp 3a and the coordinates of the three-dimensional object detected using the shadow Sb of the three-dimensional object A by the lamp 3b are different, the vehicle 1 The certification that the road surface R1 that exists and the road surface R2 on which the shadow S of the three-dimensional object A exists does not match is performed.
The control unit 24 constitutes a detection result evaluation unit.

The display unit 25 is composed of a liquid crystal display or the like. Under the instruction of the control unit 24, the display unit 25 displays images taken by the camera 2 and information necessary for the user. When it is recognized that the existing road surface R1 and the road surface R2 on which the shadow S of the three-dimensional object A does not match, for example, information for calling attention is displayed.

In the first and second embodiments, in order to detect the accurate three-dimensional coordinates (x ₀ , y ₀ , z ₀ ) of the three-dimensional object A, the road surface R1 on which the vehicle 1 exists and the shadow S of the three-dimensional object A are present. It is necessary that the road surface R 2 where the road is present matches.
For example, as shown in FIG. 13, when the road surface R1 on which the vehicle 1 exists and the road surface R2 on which the shadow S of the three-dimensional object A is not coplanar, the shadow of the three-dimensional object A projected on the road surface R2 Since S is shorter than the shadow S of the three-dimensional object A in FIG. 2, the accurate three-dimensional coordinates (x ₀ , y ₀ , z ₀ ) of the three-dimensional object A cannot be calculated.

If an incorrect three-dimensional coordinate (x ₀ , y ₀ , z ₀ ) of the three-dimensional object A is presented to the user, the vehicle 1 should not come into contact with the three-dimensional object A, but problems such as contact will occur. Therefore, it is important to be able to verify whether or not the three-dimensional coordinates (x ₀ , y ₀ , z ₀ ) of the three-dimensional object A are accurately calculated.
Therefore, in the third embodiment, it is possible to verify whether or not the three-dimensional coordinates (x ₀ , y ₀ , z ₀ ) of the three-dimensional object A are accurately calculated.

FIG. 14 is an explanatory diagram used for detection result evaluation of the vehicle periphery monitoring device according to Embodiment 3 of the present invention.
In FIG. 14, a point D1 is an end point of the shadow Sa projected by the lamp 3a if the shadow S of the three-dimensional object A exists on the road surface R1.
A point D2 is a point calculated from an image photographed by the camera 2 based on the shadow Sa of the three-dimensional object A when the shadow Sa of the three-dimensional object A is actually projected on the road surface R2 by the lamp 3a. .
The point D3 is an end point of the shadow Sa of the three-dimensional object A that is actually projected on the road surface R2 by the lamp 3a.

The point D4 is an end point of the shadow Sb projected on the road surface by the lamp 3b if the shadow S of the three-dimensional object A exists on the road surface R1.
The point D5 is an end point of the shadow Sb of the three-dimensional object A when it is actually projected on the road surface R2 by the lamp 3b.

When the three-dimensional coordinates (x ₀ , y ₀ , z ₀ ) of the three-dimensional object A are obtained using one camera 2, the method described in Embodiment 1 above is used on the image photographed by the camera 2. When the point D3 is converted into three-dimensional coordinates, the coordinates of the point D2 are obtained.
The reason for this is that in the first embodiment, since the vehicle 1 and the shadow S of the three-dimensional object A are obtained using the same road surface R1, the same as the point D3 when viewed from the camera 2. This is because the point D2 on the road surface R1 at the coordinates is converted.

In this case, in the equation (1), it is necessary to correctly determine that the coordinate of the three-dimensional object A is in the middle between the ramp 3a and the point D3, but in actuality it is in the middle between the point D2 on the road surface R1 and the ramp 3a. Since the coordinates are obtained, different coordinates are obtained.
Similarly, when the coordinates of the three-dimensional object A are obtained by using another lamp 3b, it is necessary to obtain the coordinates between the lamp 3b and the point D5. Since the coordinates are obtained as intermediate coordinates, different coordinates are obtained.

When the shadow S of the three-dimensional object A exists on the road surface R1, the coordinates of the three-dimensional object A obtained from the shadow Sa by the lamp 3a coincide with the coordinates of the three-dimensional object A obtained from the shadow Sb by the lamp 3b. .
However, when the shadow S of the three-dimensional object A does not exist on the road surface R1, as described above, since the obtained coordinates are different, it is detected that the vehicle 1 and the shadow S do not exist on the same road surface. can do.
FIG. 15 is a flowchart showing the processing contents of the vehicle periphery monitoring apparatus according to Embodiment 3 of the present invention.

Next, the operation will be described.
First, the control unit 24 outputs an image acquisition command to the image holding unit 21 in a state where the

lamps

3 a and 3 b are turned off.
When receiving an image acquisition command from the control unit 24, the image holding unit 21 acquires and holds the image P1 captured by the camera 2 (step ST21).

Next, the control unit 24 turns on the lamp 3a (step ST22), and outputs an image acquisition command to the image holding unit 21. At this time, the lamp 3b is not turned on and is kept off.
When receiving an image acquisition command from the control unit 24, the image holding unit 21 acquires and holds the image P2 captured by the camera 2 (step ST23).

Next, the control unit 24 turns off the lamp 3a and turns on the lamp 3b (step ST24), and then outputs an image acquisition command to the image holding unit 21.
When receiving an image acquisition command from the control unit 24, the image holding unit 21 acquires and holds the image P3 captured by the camera 2 (step ST25).

When the image holding unit 21 acquires the images P1, P2, and P3, the control unit 24 outputs detection commands for the shadows Sa and Sb of the three-dimensional object A by the

lamps

3a and 3b to the shadow detection unit 22.
When receiving the shadow Sa, Sb detection command from the control unit 24, the shadow detection unit 22 compares the image P1 and the image P2 held in the image holding unit 21 and projects the lamp 3a projected on the image P2. The shadow Sa of the three-dimensional object A is detected.
Further, the image P1 and the image P3 held in the image holding unit 21 are compared, and the shadow Sb of the three-dimensional object A by the lamp 3b projected on the image P3 is detected (step ST26).
Since the shadow detection method by the shadow detection unit 22 is the same as that of the shadow detection unit 5 according to the first embodiment, detailed description thereof is omitted.

When the shadow detection unit 22 detects the shadow Sa of the three-dimensional object A by the lamp 3a and the shadow Sb of the three-dimensional object A by the lamp 3b, the control unit 24 outputs a detection command for the three-dimensional object A to the three-dimensional object detection unit 23. .
When the solid object detection unit 23 receives the detection command for the solid object A from the control unit 24, the solid object detection unit 23 detects the solid object A using the shadow Sa of the solid object A detected by the shadow detection unit 22.
That is, the three-dimensional object detection unit 23 uses the shadow Sa of the three-dimensional object A detected by the shadow detection unit 22 in the same manner as the three-dimensional object detection unit 6 of the first embodiment, and the coordinates ( x ₀ , y ₀ , z ₀ ) is calculated (step ST27).

In addition, when the solid object detection unit 23 receives a detection command for the solid object A from the control unit 24, the solid object detection unit 23 detects the solid object A using the shadow Sb of the solid object A detected by the shadow detection unit 22.
That is, the three-dimensional object detection unit 23 uses the shadow Sb of the three-dimensional object A detected by the shadow detection unit 22 in the same manner as the three-dimensional object detection unit 6 of the first embodiment, and the coordinates of the vertex of the three-dimensional object A ( x ₀ , y ₀ , z ₀ ) is calculated (step ST27).

When the three-dimensional object detection unit 23 calculates the coordinates (x ₀ , y ₀ , z ₀ ) of the three-dimensional object A, the control unit 24 detects the three-dimensional object A detected using the shadow Sa of the three-dimensional object A by the lamp 3a. compared with the coordinates of the vertices _{_{(x 0, y 0, z}} 0), the vertex coordinates of the detected three-dimensional object a using a shadow Sb of the three-dimensional object a by the lamp 3b of the _{_{(x 0, y 0, z}} 0) of (Step ST28).
If the two coordinates (x ₀ , y ₀ , z ₀ ) are the same, the control unit 24 matches the road surface R1 where the vehicle 1 exists and the road surface R2 where the shadow S of the three-dimensional object A exists. Authenticate.
On the other hand, if the two coordinates (x ₀ , y ₀ , z ₀ ) are different, the road surface R1 on which the vehicle 1 exists and the road surface R2 on which the shadow S of the three-dimensional object A do not match. Authorize.

When the control unit 24 recognizes that the road surface R1 on which the vehicle 1 exists and the road surface R2 on which the shadow S of the three-dimensional object A coincides, like the control unit 7 in the first embodiment, The three-dimensional coordinates (x ₀ , y ₀ , z ₀ ) of the three-dimensional object A calculated by the three-dimensional object detection unit 23 are converted into information necessary for the user and displayed on the display unit 25 (step ST29).
On the other hand, when recognized as road shadow S of the road surface R1 and the three-dimensional object A vehicle 1 is present is present R2 do not match, accurate three-dimensional coordinates of the three-dimensional object A (x _0, y _0, Attention information indicating that z ₀ ) cannot be calculated is displayed on the display unit 25 (step ST30).

As apparent from the above, according to the third embodiment, the image P1 held in the image holding unit 21 is compared with the image P2, and the three-dimensional object A formed by the lamp 3a projected on the image P2 is compared. The shadow detection unit 22 detects the shadow Sa and detects the shadow Sb of the three-dimensional object A by the lamp 3b projected on the image P3 by comparing the image P1 and the image P3 held in the image holding unit 21. The solid object A is detected using the shadow Sa of the solid object A detected by the shadow detection unit 22, and the solid object A is detected using the shadow Sb of the solid object A detected by the shadow detection unit 22. A three-dimensional object detection unit 23 for detection, and the control unit 24 detects the coordinates of the three-dimensional object detected using the shadow Sa of the three-dimensional object A by the lamp 3a and the shadow Sb of the three-dimensional object A by the lamp 3b. If the coordinates of the created solid object are different Since the road surface R1 on which the vehicle 1 exists and the road surface R2 on which the shadow S of the three-dimensional object A does not match each other, the road surface R1 on which the vehicle 1 exists and the solid surface When the accurate three-dimensional coordinates (x ₀ , y ₀ , z ₀ ) of the three-dimensional object A cannot be calculated because the road surface R2 on which the shadow S of the object A exists does not match, An effect that can alert the user and avoid unexpected contact is achieved.

As described above, the vehicle surroundings monitoring device according to the present invention is suitable for an application that detects a three-dimensional object existing around and avoids an unexpected contact.

Claims

A camera that captures the periphery of the vehicle, a lamp that irradiates light toward the periphery of the vehicle, a first image captured by the camera in a state where the lamp is turned off, and the lamp In a state where is lit, the image acquisition means for acquiring the second image taken by the camera is compared with the first image acquired by the image acquisition means and the second image, and the first image A shadow detection means for detecting the shadow of the three-dimensional object by the lamp projected on the image of 2, and a three-dimensional object detection means for detecting the three-dimensional object using the shadow of the three-dimensional object detected by the shadow detection means. And a vehicle surrounding monitoring device.
2. The vehicle surrounding monitoring apparatus according to claim 1, wherein the three-dimensional object detection means calculates the height of the three-dimensional object using the shadow of the three-dimensional object detected by the shadow detection means.
The shadow detecting means detects a shadow of a three-dimensional object by the sun instead of a shadow of the three-dimensional object by a lamp when the shadow of the three-dimensional object is projected on the first image by the sun. The vehicle surrounding monitoring apparatus according to claim 1.
The vehicle according to claim 3, wherein the three-dimensional object detection means detects the direction of light emitted from the sun, and detects the three-dimensional object using the direction of the light and the shadow of the three-dimensional object by the sun. Ambient monitoring device.
A camera that captures the periphery of the vehicle, a first lamp that irradiates light toward the periphery of the vehicle, and a second lamp that irradiates light toward the periphery of the vehicle from a position different from the first lamp In a state where the first and second lamps are turned off, the first image taken by the camera, the first lamp is turned on, and the second lamp is turned off. The second image captured by the camera and the third image captured by the camera in a state where the first lamp is turned off and the second lamp is turned on are acquired. The image acquisition means and the first image acquired by the image acquisition means are compared with the second image, and the shadow of the three-dimensional object by the first lamp projected on the second image is detected. And acquired by the image acquisition means. The shadow detection unit that compares the first image with the third image and detects the shadow of the three-dimensional object by the second lamp projected on the third image, and the shadow detection unit detects A three-dimensional object for detecting the three-dimensional object using the shadow of the three-dimensional object detected by the shadow detecting means and detecting the three-dimensional object using the shadow of the three-dimensional object by the first lamp. The three-dimensional object detected by the detection means and the solid object detected by the three-dimensional object detection means using the shadow of the three-dimensional object by the first lamp and the three-dimensional object detected by using the shadow of the three-dimensional object by the second lamp are collated. A vehicle periphery monitoring device comprising: a detection result evaluation unit that evaluates the validity of the detection result of the three-dimensional object detection unit.
The detection result evaluation means is a three-dimensional object coordinate detected by the three-dimensional object detection means using the three-dimensional object shadow detected by the first lamp and the three-dimensional object shadow detected by the second lamp. 6. The vehicle periphery monitoring device according to claim 5, wherein when the two are different, the road surface on which the vehicle exists and the road surface on which the shadow of the three-dimensional object does not coincide are identified.