JP2002036954A - Vehicle surroundings monitor and vehicle surroundings monitoring system - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a vehicle surroundings monitor and a vehicle surroundings monitoring system free from the dead angle. SOLUTION: This vehicle surroundings monitor includes an omnidirectional visual sensor 2 attached to the right front of the vehicle, a visual sensor 3 attached to the left rear of the vehicle, an image generation portion, which is connected to both the visual sensor 2 and the visual sensor 3 to generate new image data by processing image data output from both the visual sensor 2 and the visual sensor 3, an image display portion which is connected to the image generation portion 6 to display the image data generated by the image generation portion 6, and a mode selection portion to which driver's operation is applied to exchange the mode of the image generated by the image generation portion.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

The present invention relates to a vehicle periphery monitoring device and a vehicle periphery monitoring system, and more particularly to a vehicle periphery monitoring device and a vehicle periphery monitoring system having no blind spot.

[0002]

2. Description of the Related Art Referring to FIG. 17, a conventional vehicle periphery monitoring device includes a camera 100 installed behind a vehicle and a monitor device (not shown) connected to camera 100. The monitor device is provided at a position where the driver can easily recognize the monitor device, and displays an image captured by the camera 100. The driver can drive the vehicle while checking the rear by looking at the monitor device.

[0003]

However, in this vehicle periphery monitoring device, the field of view is limited to the hatched area in FIG. For this reason, there was a blind spot, and it was difficult to drive safely.

[0004] Further, it is impossible to record the video imaged by the vehicle periphery monitoring device. Therefore, even if an accident occurs, it is not easy to determine the cause of the accident.

Further, it is impossible to generate a warning when an obstacle or a suspicious person approaches.

SUMMARY OF THE INVENTION The present invention has been made to solve the above-mentioned problems, and an object of the present invention is to provide a vehicle periphery monitoring device and a vehicle periphery monitoring system having no blind spot.

Another object of the present invention is to provide a vehicle periphery monitoring system that can easily determine the cause of an accident.

Still another object of the present invention is to provide a vehicle periphery monitoring system capable of issuing a warning to an approaching suspicious person.

[0009]

According to one aspect of the present invention, there is provided a vehicle periphery monitoring device comprising: two omnidirectional vision sensors disposed at both ends on a diagonal line of a vehicle; an image display unit for displaying an image; An image generation unit connected to the two omnidirectional vision sensors and the image display unit, the output unit converts the outputs of the two omnidirectional vision sensors according to a predetermined image conversion method, and displays the created image on the image display unit. Each of the two omnidirectional vision sensors includes a camera and a light condensing unit that is arranged at a position having a predetermined relationship with the camera and condenses an omnidirectional image to the camera.

[0010] Two omnidirectional vision sensors are located at diagonally opposite ends of the vehicle. Therefore, by using two omnidirectional vision sensors at the same time, 360 degrees around can be monitored simultaneously, and the blind spot of the vehicle is eliminated. Therefore, the driver can safely drive.

Preferably, the vehicle periphery monitoring device further includes a mode switching unit connected to the image generation unit and switching an image conversion method in the image generation unit.

[0012] The image generating method in the image generating section can be switched. Therefore, an image desired by the user can be displayed on the image display unit in various modes.

[0013] More preferably, the image generating section is connected to the converting means for converting the outputs of the two omnidirectional vision sensors according to a predetermined method, and cuts out an image from the output of the converting means. Image generating means for connecting to the image extracting means for creating an image by combining the extracted images; and Display control means for displaying on the display unit.

For example, creating an image viewed from multiple viewpoints,
These images can be displayed in combination.

According to another aspect of the present invention, there is provided a vehicle periphery monitoring device connected to two omnidirectional vision sensors and two omnidirectional vision sensors disposed at both ends on a diagonal line of a vehicle. An image processing unit that converts the output of the visual sensor according to a predetermined image conversion method, an information storage unit that stores data, and an image processing unit that is connected to the image processing unit and the information storage unit. And a storage control unit for storing in the storage unit. Each of the two omnidirectional vision sensors includes a camera and a light condensing unit that is disposed at a position having a predetermined relationship with the camera and condenses an omnidirectional image to the camera.

The image captured by the omnidirectional vision sensor is stored in the information storage unit. Therefore, even if a traffic accident or the like occurs, the cause of the accident can be easily determined.

A vehicle periphery monitoring device according to still another aspect of the present invention is provided with two omnidirectional vision sensors arranged at both ends on a diagonal line of a vehicle and two omnidirectional vision sensors connected to each other. An image processing unit that converts the output of the azimuth visual sensor according to a predetermined image conversion method, a determining unit connected to the image processing unit, and determining whether an abnormality has occurred around the vehicle; and a determining unit. And an alarm generation unit for generating an alarm based on the output of the determination unit. Each of the two omnidirectional vision sensors includes a camera and a light condensing unit that is disposed at a position having a predetermined relationship with the camera and condenses an omnidirectional image to the camera.

An alarm is generated when an abnormality is detected. For this reason, it is possible to prevent excessive passing during driving,
It is possible to support safe driving such as avoiding obstacles. In addition, a warning can be issued to a suspicious person approaching.

An image periphery monitoring system according to still another aspect of the present invention is connected to a vehicle periphery monitoring device for monitoring a situation around the vehicle, and connected to the vehicle periphery monitoring device through a communication line.
A remote monitoring device that remotely controls the vehicle periphery monitoring device. The vehicle periphery monitoring device includes two omnidirectional vision sensors arranged at both ends on a diagonal line of the vehicle, and a transmission unit that transmits images captured by the two omnidirectional vision sensors to a remote monitoring device. Each of the two omnidirectional vision sensors
The camera includes a camera and a light condensing unit that is disposed at a position having a predetermined relationship with the camera and condenses an omnidirectional image to the camera. The remote monitoring device includes an information storage unit that stores data, and a storage control unit that receives an image from a transmission unit via a communication line and stores the image in the information storage unit.

The image picked up by the omnidirectional vision sensor is stored in the information storage unit. Therefore, even if a traffic accident or the like occurs, the cause of the accident can be easily determined.

An image periphery monitoring system according to still another aspect of the present invention is connected to a vehicle periphery monitoring device for monitoring a situation around the vehicle and a vehicle periphery monitoring device through a communication line.
A remote monitoring device that remotely controls the vehicle periphery monitoring device. The remote monitoring device includes a monitoring region setting unit that sets a monitoring region in the vehicle periphery monitoring device, and a transmission unit that is connected to the monitoring region setting unit and transmits the monitoring region to the vehicle periphery monitoring device. The vehicle periphery monitoring device is connected to a monitoring area receiving unit that receives the monitoring area from the transmitting unit, two omnidirectional vision sensors arranged at both ends on a diagonal line of the vehicle, and two omnidirectional vision sensors. An image processing unit that converts the outputs of the three omnidirectional vision sensors according to a predetermined image conversion method, and is connected to the image processing unit and the monitoring area receiving unit;
The determination unit includes a determination unit that determines whether an abnormality has occurred in the monitoring area, and an alarm generation unit that is connected to the determination unit and generates an alarm based on an output of the determination unit. Each of the two omnidirectional vision sensors includes a camera and a light condensing unit that is disposed at a position having a predetermined relationship with the camera and condenses an omnidirectional image to the camera.

A monitoring area is set by the remote monitoring device, and an alarm is generated when an abnormality is detected in the set monitoring area. For this reason, safe driving can be supported, for example, it is possible to prevent unreasonable overtaking during driving and avoid obstacles. In addition, a warning can be issued to a suspicious person approaching.

[0023]

[First Embodiment] FIGS. 1 and 2
Referring to, the vehicle surroundings monitoring device according to Embodiment 1 of the present invention includes an omnidirectional vision sensor 2 provided on the right front of the vehicle.
An omnidirectional vision sensor 3 provided on the left rear side of the vehicle,
An image generation unit 6 connected to the omnidirectional vision sensors 2 and 3 for processing the image data output from the omnidirectional vision sensors 2 and 3 to generate new image data; An image display unit 7 for displaying image data generated by the generation unit 6 and a mode switching unit 5 operated by a driver to switch the mode of the image generated by the image generation unit 6 are included.

The image display unit 7 includes a monitor device of a car navigation system, and is provided at a position where the driver can easily recognize the image.

Referring to FIG. 3, omnidirectional vision sensor 2 comprises
A hyperboloid mirror 8 is provided at the tip of the right end pole 12 of the bumper 14 at the front of the vehicle, and is provided vertically downward, and is provided vertically upward so as to be coaxial with the hyperboloid mirror 8 below the hyperboloid mirror 8. And a cylindrical transparent cover 10 covering the hyperboloid mirror 8 and the camera 4.

Regarding the omnidirectional visual sensor, see “Kazumasa Yamazawa et al .:“ A Omnidirectional Visual Sensor for Navigation of Mobile Robots ”, IEICE Transactions D-IIVol.
J79-D-II No. 5pp. 698-707 (19
May 1996) ".

Hereinafter, the schematic configuration of the omnidirectional vision sensor 2 will be described. Referring to FIG. 4, hyperboloid mirror 8 uses a hyperboloid in a region of Z> 0 among two-lobe hyperboloid as a mirror. The two-lobed hyperboloid is a curved surface obtained by rotating a hyperbola around a real axis (Z axis). The two-lobe hyperboloid has two focal points, (0, 0, + c) and (0, 0, -c). However,

[0028]

(Equation 1)

Is as follows. Note that the constants a and b define the shape of the hyperbola, b is the absolute value of the Z coordinate at the intersection of the hyperboloid and the Z axis, and a is the value obtained by ascending the asymptotic surface of the hyperboloid to the vertex of the hyperboloid Is the length of the vertical line.

Here, consider a three-dimensional coordinate system O-XYZ having the Z axis as a vertical axis as shown in FIG. At this time, the two-lobe hyperboloid is expressed by the following equation (1).

[0031]

(Equation 2)

Referring to FIG. 5, the omnidirectional vision sensor 2 comprises:
It is composed of a hyperboloid mirror 8 installed vertically in a region of Z> 0 and a camera (not shown) installed vertically below it. At this time, the focal point OM of the hyperboloid mirror 8 and the lens center OC of the camera are respectively the two focal points (0, 0, + c) and (0, 0,-) of the bilobal hyperboloid.
The hyperboloid mirror 8 and the camera are arranged so as to be located at c). The image plane xy is a plane parallel to the XY plane and separated from the lens center OC of the camera by the focal length f of the camera. Reflection surface of hyperboloid mirror 8, hyperboloid mirror 8
Of the focal point OM and the lens center OC of the camera are
Is represented by

[0033]

(Equation 3)

Referring to FIG. 6, an arbitrary point P in the space
The mapping point on the image for (X, Y, Z) is p (x,
When y), the azimuth θ of the point P is expressed by the following equation (3).

Tan θ = Y / X = y / x (3) That is, the azimuth θ of the point P defined by Y / X is y / x
Is obtained by calculating the azimuth angle θ of the mapping point p determined by. Thus, the azimuth angle θ of the target object in the 360-degree panoramic region directly appears as the azimuth of the mapping on the image plane of the object.

Referring to FIG. 7, assuming a vertical section including point P and the Z axis, a point P and a mapping point p are located between:
The following equation (4) holds.

[0037]

(Equation 4)

That is, the azimuth θ and the depression α of the point P from the focal point OM of the hyperboloid mirror 8 are determined by the lens center O of the camera.
By providing C at the focal position of the hyperboloid, the mapping point p (x,
y), it is uniquely obtained. At this time, the hyperboloid mirror 8
Since the focal point OM is fixed, the input image can be converted into an image obtained by rotating the camera as viewed from the focal point OM of the hyperboloid mirror 8 around a vertical axis or an image of a normal camera.

For example, it is assumed that an image as shown in FIG. When this image data is converted into a panoramic image, an image as shown in FIG. 9 is obtained, and it is possible to overlook 360 degrees around the image. Also,
Using the method described above and converting it to a normal camera image,
An image as shown in FIG. 10 is obtained. 10A is an image of the front left of the vehicle, FIG. 10B is an image of the front of the vehicle,
FIG. 10C is a right front image of the vehicle. These image conversion processes are performed by the image generation unit 6 in FIG.

The omnidirectional vision sensor 3 is provided at the tip of the leftmost pole of the bumper at the rear of the vehicle. The omnidirectional vision sensor 3 has the same hardware configuration as the omnidirectional vision sensor 2. Therefore, detailed description thereof will not be repeated here.

The omnidirectional vision sensor 3 can perform the same image conversion as that of the omnidirectional vision sensor 2 to obtain a normal camera image. FIG. 11 is an example of an image captured by the omnidirectional vision sensor 3. When this image data is converted into a panoramic image, an image as shown in FIG. 12 is obtained, and it is possible to overlook 360 degrees around. When the image is converted into a normal camera image using the above-described method, an image as shown in FIG. 13 is obtained. FIG.
(A) is an image of the rear right with respect to the traveling direction of the vehicle, FIG.
13B is an image right behind the vehicle, and FIG. 13C is an image left behind with respect to the traveling direction of the vehicle. These image conversion processes are performed by the image generation unit 6 in FIG.

Referring again to FIG. 1, the omnidirectional vision sensor 2
Is provided on the right front of the vehicle. Therefore, the omnidirectional vision sensor 2 can monitor the left front, the front, the right front, and the right rear of the vehicle except for the region where the vehicle itself is captured. The omnidirectional vision sensor 2 has approximately 270
Degree areas can be monitored.

The omnidirectional vision sensor 3 is provided at the rear left of the vehicle. Therefore, the omnidirectional vision sensor 3 can monitor the rear right, the rear right, the rear left, and the front left of the vehicle except for the region where the vehicle itself is captured. The omnidirectional vision sensor 3 can monitor an area of approximately 270 degrees.

The omnidirectional vision sensors 2 and 3 are arranged on a diagonal line of the vehicle. Therefore, 360 degrees around the vehicle can be monitored by combining the omnidirectional vision sensors 2 and 3.

The image generating section 6 shown in FIG. 2 generates various images by combining the converted panoramic image or the image viewed from a normal camera, and displays the generated various images on the image display section 7.

For example, referring to FIG. 14, image generating section 6 arranges a panoramic image in front of the vehicle on upper surface 30A, a front left image on lower left surface 30B, and a front right image on lower right surface 30C. An image is created and displayed on the image display unit 7. The panoramic image in front of the vehicle is created by cutting out a part of the panoramic image shown in FIG.

Referring to FIG. 15, image generation unit 6
Shows a panoramic image of the rear of the vehicle on the upper surface 32A and the lower left surface 3
An image in which the rear left image is arranged in 2B and the rear right image is arranged in the lower right surface 32C may be created and displayed on the image display unit 7. The panoramic image behind the vehicle is shown in Fig. 1.
It is created by cutting out a part of the panoramic image shown in FIG.

Further, referring to FIG.
Creates an image in which the left front image is arranged on the left upper surface 34A, the right front image is arranged on the right upper surface 34B, the left rear image is arranged on the left lower surface 34C, and the right rear image is arranged on the right lower surface 34D. You may make it display on the display part 7.

The mode arrangement section 5 arranges these images.
Can be switched by operating.

The way of arranging the images is not limited to these, and it goes without saying that other combinations may be used.

In the example shown in FIG. 3, the omnidirectional vision sensor 3
Was attached to the tip of the pole 12, but the camera 4
May be embedded in the bumper 14, and the omnidirectional vision sensor 3 may be mounted on the bumper 14.

Furthermore, an infrared camera or a high-sensitivity camera may be used as the camera 4 in order to enable shooting at night.

As described above, according to the embodiment of the present invention, the omnidirectional vision sensors 2 and 3 are arranged on a diagonal line of the vehicle. Therefore, by using the omnidirectional vision sensors 2 and 3 at the same time, it is possible to simultaneously monitor the surroundings of 360 degrees and eliminate the blind spot of the vehicle.
Therefore, the driver can safely drive.

Also, images viewed from multiple viewpoints can be created, and these images can be combined and displayed. Therefore, the driver can safely drive.

The present invention is particularly effective when driving a vehicle having a large blind spot, such as a large truck or a large bus.

[Second Embodiment] Referring to FIG. 18, a vehicle periphery monitoring system according to a second embodiment of the present invention is installed in a vehicle and monitors the periphery of the vehicle.
And a remote monitoring device 24 that is connected to the vehicle periphery monitoring device 22 through a communication line and remotely controls the vehicle periphery monitoring device 22 and records an image captured by the vehicle periphery monitoring device 22.

The vehicle periphery monitoring device 22 is connected to the omnidirectional imaging unit 32 for capturing an omnidirectional image around the vehicle, and receives an analog video signal output from the omnidirectional imaging unit 32. A / D (Analog to Digital) conversion unit 34, an image processing unit 36 connected to the A / D conversion unit 34 for processing the image data output from the A / D conversion unit 34, and an alarm by sound or light Alarm generation unit 4 that generates
0 is included.

The vehicle periphery monitoring device 22 further includes an input unit 44 used by a user to input commands, parameters, and the like, a monitor device of a car navigation system, and the like. An image display unit 7 for displaying the image data, and an information recording unit 4 for recording the image data and various information such as the image capturing time and information on whether or not an alarm has been generated.
8 and a transmission / reception unit 42 for transmitting / receiving data between the remote monitoring device 24 and the image processing unit 36, the input unit 44, the image display unit 7, the alarm generation unit 40, the information recording unit 48, and the transmission / reception unit 42. And a determination unit 38 that determines whether an abnormality has occurred around the vehicle based on the outputs of the image processing unit 36 and the information recording unit 48, and controls connected devices.

The omnidirectional imaging unit 32 includes the omnidirectional visual sensor 2 provided at the front right of the vehicle described in the first embodiment,
And an omnidirectional vision sensor 3 provided on the left rear side of the vehicle.

The remote monitoring device 24 is a vehicle periphery monitoring device 2
, An image processing unit 50 for processing image data, and an input unit 56 used by a user to input commands, parameters, and the like.
An image display unit 58 for displaying image data; an image recording unit 60 for recording the image data and various information such as information on the image capturing time and whether or not an alarm has occurred; and an image processing unit 50 , An input unit 56, an image display unit 58, an information recording unit 60, and a transmission / reception unit 54,
A determination unit 52 that performs various determinations and controls connected devices based on data received by the transmission / reception unit 54, data recorded in the information recording unit, an image processed by the image processing unit 50, and the like.

Referring to FIG. 19, each unit of remote monitoring device 24 operates as follows. The user operates the input unit 56 to set remote monitoring conditions in the vehicle periphery monitoring device 22 (S2). The process of S2 will be described later in detail.

The determination unit 52 determines whether or not S2 is
Is transmitted to the vehicle periphery monitoring device 22 (S4). The determination unit 52 determines whether a warning information transmission request is set by the vehicle periphery monitoring device 22 in the process of S2 (S6). A warning information transmission request is
When an abnormality occurs in the vehicle periphery monitoring device 22, the image and various information (the abnormality occurrence time, the abnormality occurrence position, the external sensor information, and the like) are information on whether or not to transmit to the remote monitoring device 24.

If the warning information transmission request has not been set (NO in S6), determination section 52 terminates all the processes without doing anything. If the warning information transmission request is set (YES in S6), the transmission / reception unit 54 receives the image data and various information at the time of occurrence of the abnormality from the vehicle periphery monitoring device 22 (S8). The image processing unit 50 converts the received image data into an image viewed from a normal camera according to the method described in Embodiment 1, and displays the image on the image display unit 58 (S1).
0).

The determination section 52 determines whether or not a storage request for an image and various information has been set in the process of S2 (S12). If the storage request has not been set (NO in S12), the determination unit 52 ends all the processing without doing anything. If the storage request is set (S12
The determination unit 52 stores the image and various information received from the transmission / reception unit 54 in the information recording unit 60 (S1).
4).

Referring to FIG. 20, the processing in S2 of FIG. 19 will be described in detail. The user sets whether to store the image and various information in the information recording unit 48 of the vehicle periphery monitoring device 22 and the information recording unit 60 of the remote monitoring device 24 (S22). The user sets a monitoring area in the omnidirectional imaging unit 32 (S24). The method of setting the monitoring area will be described later in detail.

The user sets whether or not to generate an alarm by using the alarm generator 40 when an abnormality occurs in the vehicle periphery monitoring device 22 (S26). The user sets whether or not to display the image captured by the omnidirectional imaging unit 32 and the warning content on the image display unit 7 in the vehicle (S28).
The user sets whether to request transmission of the above-described warning information from the vehicle periphery monitoring device 22 to the remote monitoring device 24 (warning information transmission) (S30).

The user sets an image compression ratio (image quality and number of frames) when transmitting an image from the vehicle periphery monitoring device 22 to the remote monitoring device 24 (S32). The user sets a warning generation condition (threshold) for detecting occurrence of an abnormality. A method for detecting the occurrence of an abnormality will be described later.

Referring to FIG. 21, vehicle periphery monitoring device 22
Works as follows. The determination unit 38 includes a transmission / reception unit 42
The monitoring conditions set in S2 of FIG. 19 are received from the remote monitoring device 24 via (S42). The determination unit 38 sets the received monitoring condition for each connected device (S4).
4).

The omnidirectional imaging section 32 captures an omnidirectional image of the set monitoring area (S46). Image processing unit 3
Reference numeral 6 denotes an omnidirectional image taken by the omnidirectional imaging unit 32 as A
While being received via the / D conversion unit 34, the image is converted into a normal camera image according to the method described in the first embodiment (S48).

The determining unit 38 determines whether an abnormality has occurred around the vehicle (S50). As a method of detecting the occurrence of abnormality, a conventionally known method is used. For example, when the vehicle is stationary, images captured by the omnidirectional imaging unit 32 are accumulated for each time, and a difference is calculated between temporally preceding and succeeding images. If the difference image is binarized and the area of the change area is larger than a predetermined threshold value, it can be detected that a suspicious person is approaching the vehicle and an abnormality has occurred.

When the vehicle is moving, the positional relationship of the omnidirectional imaging unit 32 at the time of continuous shooting can be known from the steering direction of the vehicle, the speed, and the like. For this reason, stereo measurement is performed between two consecutive images having a known positional relationship to detect the position of an obstacle. When the obstacle approaches closer than a predetermined threshold value, it can be detected that an abnormality has occurred. This makes it possible to detect an abnormality when the inter-vehicle distance is too short or to detect an abnormality when approaching a falling object in front of the vehicle.

Even without performing the stereo measurement between the two images, the position of the image picked up by the omnidirectional image pickup unit 32 is corrected, and the difference between the corresponding pixels of the continuous images is obtained. The binarization may be used to detect the occurrence of an abnormality from the area of the change region.

The threshold for determining whether or not an abnormality has occurred is set in the process of S34 in FIG.

If no abnormality has occurred around the vehicle (NO in S50), the flow returns to S46. If an abnormality has occurred around the vehicle (YES in S50), the determination unit 38 determines whether or not a request to generate an alarm has been set in the process of S26 in FIG. 20 (S52). If a warning request is set (YES in S52), a warning is issued from the warning generation unit 40 to the approaching person (suspicious person) and the approaching object (obstacle) by sound and light (S54).

If the alarm generation request is not set (S
(NO at 52) or after the processing of S54,
In the processing of S30 in FIG. 20, it is determined whether a warning information transmission request has been set (S56).

If a warning information transmission request is set (S
(YES in 56), the determination unit 38 transmits the image at the time of occurrence of the abnormality and the various information (abnormality occurrence time, abnormality occurrence position, external sensor information, etc.) to the remote monitoring device 24 via the transmission / reception unit 42. (S58).

If the warning information transmission request has not been set (NO in S56) or after the processing in S58, the judgment unit 38
Determines whether a request to display an image or the like on the image display unit 7 is set in the process of S28 in FIG. 20 (S6).
0).

If the display request has been set (S6
(YES at 0), as described in the first embodiment, an image around the vehicle is displayed on image display unit 7 in the vehicle (S6).
2).

If the display request is not set (S60
NO) or after the processing of S52, the determination unit 38
In the process of S22 of "0", it is determined whether or not a request for storing an image and various information is set (S64).

If a storage request has been set (Y in S64)
ES), the image and various information transmitted in S58 are stored in the information recording unit 48 (S66). Then, the process returns to S44. If the storage request has not been set (N in S64)
O), and return to S44.

Next, a method of setting a monitoring area in S24 of FIG. 20 will be described. FIG. 22 is a diagram illustrating the relationship between the omnidirectional vision sensor 2 and the monitoring area in the three-dimensional space. FIG. 23 is a diagram illustrating the relationship between the input image captured by the omnidirectional vision sensor 2 and the monitoring area. The same applies to the relationship between the omnidirectional vision sensor 3 and the monitoring area. Therefore, in the following description, the details of the relationship between omnidirectional vision sensor 3 and the monitoring area will not be repeated.

Referring to FIG. 22, P (X, Y, Z) is
The coordinates in the three-dimensional space when the focal point OM of the hyperboloid mirror 8 is set as the origin. The monitoring area is hatched in the figure. When the monitoring area to be set is projected on the input image, the monitoring area becomes a fan-shaped monitoring area as shown in FIG. The coordinates of the four corners in the three-dimensional space of the monitoring area are represented by E11,
E12, E21 and E22. When the coordinates of these four corners are projected on the input image, the result is as shown in FIG. The angle of the sector from the x-axis is sandwiched between θ1 and θ2.

The coordinates of E11 in the three-dimensional space are represented by (X1
1, Y11, Z1) and E12 in the three-dimensional space are (X12, Y12, Z1), E21 in the three-dimensional space are (X21, Y21, Z1), and E22 are in the three-dimensional space. Are defined as (X22, Y22, Z1).

A horizontal plane with the ground passing through the focal point OM and a straight line OM
Assuming that the angle between E11 and α11 is α1, the relationship of Expression (5) holds. Equation (6) is derived from equation (5). The angle between the horizontal plane with the ground passing through the focal point OM and the straight line OME12 is α
Assuming that 2, the relationship of equation (7) holds. Equation (8) is derived from equation (7).

The angles α and β shown in FIG.
It can be expressed by equation (9) and equation (10), respectively.

[0086]

(Equation 5)

Referring to FIG. 23, the coordinates of the projection point on the input image of E11 are (E11x, E11y), the coordinates of the projection point on the input image of E12 are (E12x, E12y), and E2
(E21x, E21x)
y), the coordinates of the projection point on the input image in E22 are set to (E22
x, E22y).

From the above relationship, E11x and E11y
Is represented by equations (11) and (12), respectively.
E12x and E12y are represented by equations (13) and (14), respectively. Similarly, E21x, E2
1y, E22x and E22y can also be determined.

Therefore, assuming that the azimuth from the x-axis on the input image is θ, the fan-shaped monitoring area is expressed by the following equations (15) to (1).
7). Therefore, in order for the user to set a fan-shaped monitoring area, the azimuth angles θ1 and θ2 and the coordinates E11, E12, and E in the three-dimensional space of the four corners of the fan-shaped area are set.
21 and E22 may be input.

[0090]

(Equation 6)

Next, a method of setting a monitoring area having a more complicated shape will be described. As shown in FIG. 24, even in the case of a monitoring area having a complicated shape, it can be regarded as a set of fan-shaped areas. For this reason, by inputting the azimuth angle and the coordinates of the four corners in the three-dimensional space in the manner described above, a complicated monitoring area can be set for each sector area.

As shown in FIG. 25, omnidirectional vision sensors 2 and 3 are installed on a diagonal line of the vehicle. Therefore, depending on how the monitoring area is set, monitoring can be performed by both omnidirectional visual sensors, or monitoring can be performed by only one omnidirectional visual sensor.

In the above description, various settings are made by the remote monitoring device 24. However, various settings may be made by the vehicle periphery monitoring device 22.

According to the present embodiment, a monitoring area is set in advance, and when an abnormality is detected in the monitoring area, the video and various information at that time can be recorded. Therefore, even if a traffic accident or the like occurs, the cause of the accident can be easily investigated.

When an abnormality is detected, an alarm is issued. For this reason, safe driving can be supported, for example, it is possible to prevent unreasonable overtaking during driving and avoid obstacles. In addition, a warning can be issued to a suspicious person approaching.

The embodiments disclosed this time are to be considered in all respects as illustrative and not restrictive. The scope of the present invention is defined by the terms of the claims, rather than the description above, and is intended to include any modifications within the scope and meaning equivalent to the terms of the claims.

[0097]

As described above, 360 degrees around the vehicle can be monitored at the same time, and the driver can safely drive the vehicle.

[Brief description of the drawings]

FIG. 1 is a diagram for explaining an arrangement position and a monitorable area of a vehicle periphery monitoring device according to a first embodiment of the present invention.

FIG. 2 is a block diagram illustrating a hardware configuration of the vehicle periphery monitoring device.

FIG. 3 is an external view of an omnidirectional vision sensor provided at the front right of the vehicle.

FIG. 4 is a diagram for explaining a two-lobed hyperboloid.

FIG. 5 is a diagram showing an arrangement position of an omnidirectional vision sensor.

FIG. 6 is a first diagram illustrating a relationship between an arbitrary point in space and a mapped point on an image.

FIG. 7 is a second diagram illustrating the relationship between an arbitrary point in space and a mapping point on an image.

FIG. 8 is a diagram illustrating an example of an image captured by an omnidirectional vision sensor at the front right of the vehicle.

FIG. 9 is a diagram illustrating an example of a panoramic image obtained by converting an image captured by an omnidirectional vision sensor at the front right of the vehicle.

FIG. 10 is a diagram illustrating an example of a normal camera image obtained by converting an image captured by an omnidirectional vision sensor at the front right of the vehicle.

FIG. 11 is a diagram illustrating an example of an image captured by an omnidirectional vision sensor at the rear left of the vehicle.

FIG. 12 is a diagram illustrating an example of a panoramic image obtained by converting an image captured by the omnidirectional vision sensor at the rear left of the vehicle.

FIG. 13 is a diagram illustrating an example of a normal camera image obtained by converting an image captured by an omnidirectional vision sensor at the rear left of the vehicle.

FIG. 14 is a diagram illustrating an example of an image generated by the image generation unit 6.

FIG. 15 is a diagram illustrating an example of an image generated by the image generation unit 6.

FIG. 16 is a diagram illustrating an example of an image generated by the image generation unit 6.

FIG. 17 is a diagram for explaining an arrangement position and a monitorable area of a conventional vehicle periphery monitoring device.

FIG. 18 is a block diagram illustrating a hardware configuration of a vehicle periphery monitoring system according to a second embodiment of the present invention.

FIG. 19 is a flowchart of a process executed by the remote monitoring device.

FIG. 20 is a flowchart of a remote monitoring condition setting process.

FIG. 21 is a flowchart of a process executed by the vehicle periphery monitoring device.

FIG. 22 is a diagram illustrating a relationship between an omnidirectional vision sensor and a monitoring area in a three-dimensional space.

FIG. 23 is a diagram showing a monitoring area in an input image.

FIG. 24 is a diagram showing a monitoring area in an input image.

FIG. 25 is a diagram illustrating a relationship between a monitoring area of an omnidirectional vision sensor and an imaging position.

[Explanation of symbols]

2,3 Omnidirectional vision sensor, 4,100 camera, 5
Mode switching unit, 6 image generation unit, 7 image display unit, 8
Hyperboloid mirror, 10 transparent cover, 12 poles, 14
Bumper.

──────────────────────────────────────────────────続 き Continued on the front page (51) Int.Cl. ⁷ Identification symbol FI Theme coat ゛ (Reference) B60R 21/00 622 B60R 21/00 622F 624 624C 626 626G 626A 630 630F G06T 1/00 330 G06T 1/00 330Z 400 400Z H04N 5/225 H04N 5/225 C 7/18 7/18 J F term (reference) 5B047 AA19 BC09 5B057 AA16 BA02 CA12 CA16 CB12 CB16 CD11 CE08 CE09 DA04 DA15 DA16 5C022 AA04 AB61 AC01 AC18 AC42 AC51 AC54 AC695C05 A AA05 CA04 CC03 EA01 EA05 EA07 FA00 FC11 FF07 HA30

Claims (14)

[Claims] 1. An omnidirectional visual sensor disposed at each of two ends on a diagonal line of a vehicle, an image display unit displaying an image, and connected to the two omnidirectional visual sensors and the image display unit, An image generation unit that converts the outputs of the two omnidirectional vision sensors according to a predetermined image conversion method and displays the created image on the image display unit. A vehicle periphery monitoring device, comprising: a camera; and a condensing unit that is disposed at a position having a predetermined relationship with the camera and condenses an omnidirectional image to the camera. 2. The vehicle periphery monitoring device according to claim 1, further comprising a mode switching unit connected to the image generation unit and switching an image conversion method in the image generation unit. 3. The image generation unit is connected to a conversion unit for converting the outputs of the two omnidirectional vision sensors according to a predetermined method, and is configured to convert an image from the output of the conversion unit. An image cutting unit for cutting out, an image forming unit connected to the image cutting unit and forming an image by combining the cut out images, and an image forming unit connected to the image forming unit and forming the image 3. The vehicle surroundings monitoring device according to claim 1, further comprising a display control unit for displaying an output of the unit on the image display unit. 4. Two omnidirectional vision sensors respectively disposed at both ends on a diagonal line of the vehicle; and a predetermined image connected to the two omnidirectional vision sensors and outputting the outputs of the two omnidirectional vision sensors. An image processing unit that performs conversion according to a conversion method; an information storage unit that stores data; a storage that is connected to the image processing unit and the information storage unit and stores an image created by the image processing unit in the information storage unit A control unit, wherein each of the two omnidirectional vision sensors is arranged at a position having a predetermined relationship with the camera, and condensing means for converging an omnidirectional image to the camera. A vehicle periphery monitoring device. 5. The vehicle according to claim 4, wherein said storage control unit includes means for storing an image created by said image processing unit and a time at which said image was taken in said storage control unit. Perimeter monitoring device. 6. Two omnidirectional vision sensors arranged at both ends on a diagonal line of a vehicle, respectively, connected to the two omnidirectional vision sensors, and output a predetermined image of the output of the two omnidirectional vision sensors. An image processing unit that performs conversion in accordance with a conversion method; a determination unit that is connected to the image processing unit and determines whether an abnormality has occurred around the vehicle; and an output of the determination unit that is connected to the determination unit. And an alarm generation unit that generates an alarm based on the camera. Each of the two omnidirectional vision sensors is arranged at a position having a predetermined relationship with the camera, and collects an omnidirectional image to the camera. A vehicle periphery monitoring device, comprising: a light condensing means for emitting light. 7. The monitoring device according to claim 1, wherein the determining unit includes: a monitoring area setting unit configured to set a monitoring area around the vehicle; and an abnormality occurrence determining unit configured to determine whether an abnormality has occurred in the monitoring area. The alarm generation unit is connected to the abnormality occurrence determination means,
The vehicle periphery monitoring device according to claim 6, further comprising a unit for generating an alarm based on an output of the abnormality occurrence determination unit. 8. The two omnidirectional vision sensors include: an omnidirectional vision sensor provided in front of the vehicle on the same side as the driver's seat; and an omnidirectional vision sensor provided behind the vehicle on the side opposite to the driver's seat. A vehicle periphery monitoring device according to any one of claims 1 to 7. 9. The light condensing means is a hyperboloid mirror,
A vehicle periphery monitoring device according to any one of claims 1 to 8. 10. A vehicle periphery monitoring device, comprising: a vehicle periphery monitoring device that monitors a situation around a vehicle; and a remote monitoring device that is connected to the vehicle periphery monitoring device through a communication line and remotely controls the vehicle periphery monitoring device. The apparatus includes: two omnidirectional vision sensors arranged at both ends on a diagonal line of a vehicle; and a transmission unit that transmits an image captured by the two omnidirectional vision sensors to the remote monitoring device. Each of the three omnidirectional vision sensors includes a camera, and a condensing unit that is disposed at a position having a predetermined relationship with the camera and condenses an omnidirectional image to the camera. A vehicle periphery monitoring system, comprising: an information storage unit that stores data; and a storage control unit that receives the image from the transmission unit through the communication line and stores the image in the information storage unit. 11. The transmitting unit on both sides of the person includes means for transmitting the image and the time at which the image was captured to the remote monitoring device, and the storage control unit transmits the image via the communication line from the transmitting unit. The vehicle periphery monitoring system according to claim 10, further comprising a unit configured to receive the image and a time at which the image was captured, and to store the image in the information storage unit. 12. The remote monitoring device, comprising: a vehicle peripheral monitoring device that monitors a situation around the vehicle; and a remote monitoring device that is connected to the vehicle peripheral monitoring device through a communication line and remotely controls the vehicle peripheral monitoring device. Comprises a monitoring area setting unit that sets a monitoring area in the vehicle periphery monitoring device; and a transmission unit that is connected to the monitoring region setting unit and transmits the monitoring region to the vehicle periphery monitoring device. A monitoring area receiving unit configured to receive the monitoring area from the transmission unit; two omnidirectional vision sensors disposed at both ends on a diagonal line of the vehicle; and An image processing unit that converts outputs of the two omnidirectional vision sensors according to a predetermined image conversion method, and is connected to the image processing unit and the monitoring area receiving unit;
A determination unit that determines whether an abnormality has occurred in the monitoring area; and an alarm generation unit that is connected to the determination unit and generates an alarm based on an output of the determination unit. A vehicle periphery monitoring system, wherein each of the azimuth vision sensors includes a camera, and a light condensing unit disposed at a position having a predetermined relationship with the camera, and condensing omnidirectional images to the camera. 13. The two omnidirectional vision sensors include: an omnidirectional vision sensor provided in front of the vehicle on the same side as the driver's seat; and an omnidirectional vision sensor provided behind the vehicle on the side opposite to the driver's seat. The vehicle periphery monitoring system according to claim 10. 14. The vehicle periphery monitoring system according to claim 10, wherein said focusing means is a hyperboloid mirror.