JPH08305999A - On-vehicle camera system - Google Patents

Abstract

PURPOSE: To prevent danger without using a device which is complicated and hard to handle such as a radar by detecting an obstacle according to an image picked up by a camera and discriminating its kind. CONSTITUTION: An image extraction processor 304 binarizes the image 303 picked up by the camera 302 fitted to a vehicle 301 so as to pick up an image of the circumferential state so that the bodies and background are separated, and codes the sizes and distances of the recognized bodies. A character selecting circuit 307 calls respective characters registered in a memory 310 corresponding to the codes of bodies previously and sends a picture 311 of an obstacle to an image compositing device 312. A screen generation part 309, on the other hand, generates a figure 313 obtained by viewing the vehicle from right above by CG, and measures the position viewed from the top of the body on the basis of the sent direction and distance information and sends it to the image compositing device 312. The image compositing device 312 generates an image 314 having the picture 311 of the obstacle put together at the position of the obstacle measured in the top elevation 313 and displays the image on a monitor 315 installed in the console of this vehicle.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a vehicle-mounted camera system, and more particularly to a camera system for detecting obstacles by the camera.

[0002]

2. Description of the Related Art As a conventional camera for detecting an object on a road, there is known a camera for detecting a white line (JP-A-5-46242). In this system, a camera is installed at the front of the vehicle to operate the road, and the white line part drawn on the road is detected.

[0003]

However, lines are not drawn on all roads, and a camera that detects a white line is not useful on roads where lines are not drawn.
In addition, when there are obstacles (people, automobiles, etc.) on the white line, it is very difficult to extract the white line.

It is an object of the present invention to provide a vehicle-mounted camera system that reliably and easily displays a display at any place.

[0005]

According to the present invention, an image pickup means for picking up an image of the surroundings of a vehicle and an image for binarizing an image signal obtained by the image pickup means under an arbitrary condition and separately extracting a background and an obstacle. Extraction means, image recognition means for recognizing the extract by calculating a feature amount such as position and size on the screen from the extracted binary image, and the recognized binary image is replaced with a previously stored character. Computer G has a character selection method and a display screen that looks down on the vehicle and its surroundings from directly above.
It is provided with a screen creating means created by raphics (hereinafter abbreviated as CG), an image synthesizing means for synthesizing the CG and the selected character, and a display means for displaying the synthetic image.

[0006]

The vehicle-mounted camera system of the present invention detects an obstacle based on an image captured by the camera and determines the type of the obstacle, thereby avoiding the danger without using a complicated and difficult-to-handle device such as a radar. Can be prevented. Further, not only the captured image is displayed as it is, but also the situation around the own vehicle can be intuitively and quickly understood by making the composition such that the own vehicle is viewed from directly above.

[0007]

FIG. 1 shows an embodiment of the present invention. 10
1 is a car seen from above, 102a to 102f are cameras,
Reference numerals 103a to 103f are camera angles of view, and 104 is a monitor. By arranging the plurality of cameras 102 so that the angles of view thereof overlap with each other as shown in FIG. 1, there is no blind spot from the driver's seat. The monitor 104 is set at a position where it can be easily seen from the driver's seat. The monitor 104 has an image changeover switch, and the camera 102a ...
Any one of the images captured by 102f can be selected and displayed on the monitor 104.

FIG. 2 shows an example in which four cameras are used.
Reference numeral 201 is an automobile viewed from above, 202a to 202d are cameras, 203a to 203d are camera angles of view, 204 is a blind spot of the camera, and 205 is a monitor. The camera 202a is installed in the front bumper of the automobile 201, the camera 202b is installed in the back of the rear seat, the camera 202c is installed in the left door mirror, and the camera 202d is installed in the right door mirror. In this case, the front camera 20
There is a blind spot between the camera 2a and the side cameras 202c and 202d, but this is a portion that can be easily confirmed from the driver's seat and there is no safety problem. With such a camera arrangement, 4
Monitor the surroundings of your vehicle with a small number of units 205
Can be displayed on. The monitor 205 is 203a-2
The image of any camera of 03d can be displayed.

One example of obstacle extraction is shown in FIG.
Shown in Reference numeral 301 is your own car, and 316 is an obstacle such as a human being. An image 303 captured by a camera 302 attached to the vehicle 301 so as to capture the surrounding situation is binarized by an image extraction processing device 304 into an object and a background. At this time, even if there are a plurality of objects in the screen, the size, shape, and
Based on the position on the screen, the arithmetic processing unit 306 individually calculates the azimuth and distance from the own vehicle. The size and distance of the recognized object are coded, and the character selection circuit 307
Further, the azimuth / distance information 308 is sent to the screen creation device 309. The character selection circuit 307 calls each character registered in the memory 310 in advance corresponding to the code of the object and sends it to the image synthesizing device 312 as a picture 311 of the obstacle. On the other hand, the screen creation device 30
In FIG. 9, CG of FIG. 313 in which the own vehicle is viewed from directly above is created by CG, the position viewed from the upper surface of the object is measured based on the sent azimuth / distance information, and the measured position is sent to the image synthesizing device 312. An image 314 is created by combining the picture of the obstacle 311 with the position of the measured obstacle in the top view 313 by the image synthesizer,
The information is displayed on the monitor 315 installed on the console of the own vehicle. A display example on the monitor is shown in FIG. 317 is a screen, 318 is a CG of the own vehicle, 319 is an obstacle replaced by the character, and 320 is a distance between the own vehicle and the obstacle. As described above, the obstacle 316 is the captured image 303.
Is displayed as shown in FIG.

FIG. 4 shows a block diagram of a warning generation system using the present invention. Camera 302, image extraction processing device 3
The functions of 04, the character selection device 307, the memory 310, the screen creation device 309, the image composition device 312, and the monitor 315 are the same as those in the embodiment of FIG. Arithmetic processing device 401
When the type of the extracted object and the azimuth / distance from the own vehicle are calculated in, the relative speed and distance between the own vehicle and the object are sent to the warning generation device 402. If the relative speed of the vehicle and the object is above a certain level or if the distance is below a certain level,
As a warning for the driver to avoid danger, the relative speed and position of the object and the vehicle are displayed on the monitor 315 and the speaker 4
A warning sound is emitted from 04. Further, the extraction is also performed on the road sign, and the arithmetic processing unit 306 extracts the extract and the memory 405.
Compare with the graphic pattern of the sign registered in advance,
If they match, a warning sound is emitted from the warning generation device 402. In this case, a warning is issued with a screen and sound different from the warning for avoiding danger.

Next, an embodiment in which the distance between the vehicle and the obstacle is measured from the image information will be described. FIG. 5 is a view of the positional relationship between the own vehicle and an obstacle as seen directly from the side. 500 is the own vehicle, 501 is a camera, 502 is a shooting angle of view of the camera 501, 503 is another vehicle (obstacle) near, and 504 is Far away other vehicle, A is own vehicle and other vehicle 50
3, B is the distance between the own vehicle and the other vehicle 504, θ1 is the angle between the ground contact portion of the other vehicle 503 and the lower end of the shooting angle of view, and θ2 is the ground contact portion of the other vehicle 504 and the lower end of the shooting angle of view. , H is the distance from the camera 501 to the ground, and θ is the angle perpendicular to the lower end of the angle of view 502. FIG. 6 is an image obtained from a camera mounted in the vehicle, 601 is a screen, 603, 504.
Is the other cars seen from the own vehicle, and 602 is the background.
When this image is divided into a background and an obstacle and binarized, a binary image as shown in FIG. 7 is obtained. 701 is a screen, 702, 70
3 is each binarized other vehicle, a is 70 from the bottom of the screen
2 is the length on the screen, and b is the length from the bottom of the screen to 703 on the screen. In this case, the information the driver wants is the length of A and B in FIG. First, the camera 501
When the position where the vehicle is mounted on the vehicle is determined, it is necessary to measure the length of h and the angle θ in advance. From FIG. 7, the lengths a and b from the bottom of the screen to the other vehicle are measured, and from these lengths, θ1,
Find the angle of θ2. The distances A and B to the other vehicles 603 and 504 are as follows.

[0012]

## EQU1 ## A = h × tan (f (a) + θ) f
(A) = θ1 (Equation 1)

[0013]

## EQU00002 ## B = h.times.tan (f (b) +. Theta.) F
(B) = θ2 (Equation 2) Next, an embodiment using a plurality of cameras will be described with reference to the block diagram of FIG. 8 and the top view of FIG. 9. Camera 8 in FIG.
01a to 801d are installed in front, rear, left and right of the own vehicle as shown in FIG. The front and rear cameras 801a and 801b are installed on the center line of the vehicle. 901,902,903,90
4 is the angle of view of each camera, L is the left and right cameras 8
Length from the installation location of 01c, 801d to the back, w
Is the width from the rear installation location of the camera 801b to the side of the vehicle. The images captured by the cameras 801a to 801d are binarized by the image extraction processing devices 802a to 802d separately for the background and the obstacle. Each binary image is processed by the arithmetic processing unit 803, and the number, size, shape, and position on the screen of obstacles around the vehicle are calculated from a plurality of binary image information. When the obstacles 905, 906, and 907 are present at the locations shown in FIG. 9, the images of the cameras 801a to 801d are as shown by 1001a to 1001d in FIG. The rear camera 80 is shown in FIGS.
The screen imaged in 1b is darkened. Although there is only one obstacle 905, the cameras 801b, 801c, 8
Since the angle of view 901, 902, and 903 of 01d overlap, they are displayed on two screens. Actually, 1101 in FIG.
an obstacle 90 in the overlapping portion such as a to 1101d.
There is 5. The condition that one obstacle is displayed on the two screens is that, when the vehicle is on the left rear side, there is an obstacle within the range of x1 from the left side of the screen on the imaging screen of the camera 801c, and the camera 80
The distance from 1c to the obstacle is larger than L, there is an obstacle within the range of x1 in FIG. 11 from the right side of the image on the image pickup screen of the camera 801b, and the distance from the camera 801b to the obstacle is larger than w. Is. Similarly, the same conditions must be met for the right rear of the vehicle. The above determination is performed in the arithmetic processing unit 803, and one obstacle 905 is displayed even if two screens show obstacles.
And process it. The information obtained as described above is processed in the same manner as in the embodiment shown in FIG. However, unlike the embodiment shown in FIG. 3, the top view creation device 804 displays the situation around the vehicle, so a top view centering on the vehicle is created as in the display screen example shown in FIG. 120
1 is a screen, 1202 is a vehicle displayed in the center, 1203
Is the distance to each obstacle.

Next, an embodiment in which the input image and CG are switched and displayed is shown in FIGS. 13 and 14. Cameras 801a-80
The four images captured in 1d are the extraction process and the selector 1
It becomes the input of 301. The extraction process operates as described in FIG. 3, and the CG created as a result is the selector 130.
This is the fifth image input to 1. Reference numeral 1303 denotes a screen select signal, which generates a signal for selecting one of the five input images by switching a switch attached to the monitor 1302. An example of the monitor 1302 with a selection switch is shown in FIG. The monitor 1302 has five display screen selection switches. Reference numeral 1403 is a CG image, 1402a is a front image, 1402b is a rear image, 1402c is a left side image, and 1402d is a switch for switching to a right side image. By pressing the corresponding switch, the image in that direction can be displayed on the screen 1401. Further, when the CG is displayed, the center vehicle 1405 can be freely moved by the direction keys 1404 so that the desired direction of the screen 1401 can be enlarged. For example, when it is desired to see the front wide, the host vehicle 1405 can be arbitrarily moved rearward, and when the user wants to see the right side widely, the host vehicle 1405 can be arbitrarily moved left. By changing the display position of your vehicle, you can see obstacles that were not displayed on the screen until then. Even when the images directly from the cameras 801a to 801d are displayed or when only one direction is widely displayed, the warning is issued because the obstacles are extracted in four directions inside.

[0015]

According to the present invention, the positional relationship between the obstacle and the vehicle and the relative speed can be intuitively understood. Also, since multiple images can be processed simultaneously, the blind spot from the driver's seat is covered.

[Brief description of drawings]

FIG. 1 is an explanatory diagram in which a blind spot of an automobile is covered by a plurality of cameras.

FIG. 2 is an explanatory diagram in which four cameras are installed in an automobile.

FIG. 3 is an explanatory diagram showing the basic configuration of the embodiment.

FIG. 4 is a block diagram showing an example of a warning device.

FIG. 5 is an explanatory diagram showing a method of calculating a distance to an obstacle.

FIG. 6 is an explanatory diagram of an image pickup screen of the vehicle-mounted camera.

7 is an explanatory diagram of the binary image of FIG. 4.

FIG. 8 is a block diagram when a plurality of cameras are used.

FIG. 9 is an explanatory view showing the installation location of the camera of the own vehicle and the imaging range.

FIG. 10 is an explanatory diagram of a screen imaged by a plurality of cameras.

FIG. 11 is an explanatory diagram showing overlapping images.

FIG. 12 is an explanatory diagram of a display screen when using a plurality of cameras.

FIG. 13 is a block diagram showing a configuration with a CG and live-action switching device.

FIG. 14 is an explanatory diagram of a monitor having a selection switch.

[Explanation of symbols]

301 ... automobile, 302 ... camera, 303 ... captured image,
304 ... Image extraction processing device, 305 ... Binary image, 306
Computation device, 307 ... Character selection device, 308 ... Direction / distance information, 309 ... Screen creation device, 310 ... Memory, 311 ... Obstacle character, 312 ... Image compositing device, 313 ... Creation screen, 314 ... Compositing screen, 315 ... Display device, 316 ... Obstacle, 317 ... Screen, 318 ... Own vehicle CG, 319 ... Obstacle character, 320 ... Distance.

─────────────────────────────────────────────────── ─── Continuation of the front page (51) Int.Cl. ⁶ Identification code Internal reference number FI Technical display location G08G 1/01 G08G 1/04 D 1/04 H04N 7/18 J H04N 7/18 G05D 1/02 K // G05D 1/02 G06F 15/62 380 (72) Inventor Toshiro Kinugasa 292 Yoshida-cho, Totsuka-ku, Yokohama-shi, Kanagawa, Ltd. Inside the Hitachi Media Visual Media Research Laboratory (72) Kenji Sano Totsuka-ku, Yokohama-shi, Kanagawa 292 Yoshidacho Stock Company, Hitachi, Ltd.

Claims (9)

[Claims] 1. An in-vehicle camera system, characterized in that a camera input section is arranged in an automobile so that a blind spot cannot be formed to obtain information on surrounding conditions. 2. The in-vehicle camera system according to claim 1, wherein at least one camera input section is arranged in each of front, rear, left and right directions of the vehicle, and a display device capable of displaying a signal from the camera input section in the vehicle. 3. An in-vehicle camera system comprising: an image pickup means for picking up an image outside the vehicle; and a display means for monitoring an image picked up by the image pickup means inside the vehicle. An image extracting means for calculating a feature amount such as a size, an arithmetic processing means for calculating a distance or an azimuth from the own vehicle based on the characteristics of the obstacle obtained by the image extracting means, and the image extracting means. Specifying means for specifying the type of obstacle from the obtained feature amount; replacement means for replacing the obstacle from the kind of obstacle obtained by the specifying means with an image of the obstacle stored in advance in memory; A screen creating means for creating a screen looking down from directly above the car and a distance or azimuth calculated by the arithmetic processing means on the screen created by the screen creating means. Vehicle camera system characterized by comprising an image synthesizing means for synthesizing the image of the replaced obstacle replacing means in a position. 4. The comparing means for comparing the distance obtained by the arithmetic processing means with a preset setting value, and the comparing means for determining that the distance of the obstacle is smaller than the preset value. A vehicle-mounted camera system having a warning generating unit that generates a warning when the camera is activated. 5. The method according to claim 3, wherein the extraction means also extracts road signs as obstacles and compares the extracted obstacles with a previously stored graphic pattern of the signs, as a result of comparison. An in-vehicle camera system having a sign recognition means for issuing a warning when an obstacle and a graphic pattern of a sign match. 6. The vehicle-mounted camera system according to claim 3, further comprising distance measuring means for measuring the distance between an obstacle and an arbitrary point in the imaged image based on the imaged image obtained by said imager. 7. The vehicle-mounted camera system according to claim 3, wherein there are a plurality of cameras for capturing images outside the vehicle. 8. The in-vehicle camera system according to claim 3, wherein the screen creating means has means for displaying an image of the vehicle as seen from directly above at an arbitrary position on the screen. 9. The vehicle-mounted camera system according to claim 3, wherein the screen displayed by the display means can be switched between the image by the image synthesizing means and the image as it is captured by the camera.