KR100593594B1 - Vehicle driving agent with panorama camera - Google Patents

Abstract

A vehicle driving assistance apparatus capable of providing a driver with a complete panorama image around the vehicle with minimal image processing is disclosed. The apparatus includes a plurality of cameras each having a lens system and an image pickup device and mounted at appropriate positions of the vehicle and photographing a situation around the vehicle, an image combiner for processing images input from the plurality of cameras, and an image combiner And a display means for displaying a photographed image to a driver, wherein the plurality of cameras includes at least two camera modules disposed adjacent to each other for photographing the front and rear of the vehicle, And the adjacent image input lines of the camera modules are arranged so as to be substantially parallel to each other. According to the present invention, it is not necessary to use a complicated image processing technique in displaying a wide range of surveillance images in front or rear, thereby reducing the device cost and the device maintenance cost. It is easy to select and change the vehicle mount position and to easily engage with the operation means such as the tilt of the camera, so that it is not necessary to tune the image composer.

Vehicle, driving, panorama, blind spot, video line, angle of view

Description

VEHICLE DRIVING AGENT WITH PANORAMA CAMERA WITH PANORAMIC CAMERA

FIG. 1A is a view showing an example of a vehicle monitoring apparatus that can be configured to obtain a conventional panorama image. FIG. 1B is a view showing an example of a panorama image obtained by removing one of the optical distortions and overlapping phenomenon appearing in the images I1 and I2 obtained by the apparatus of FIG. 1A FIG. 2 is a diagram schematically illustrating a process of displaying a complete panoramic image of FIG.

FIG. 2A is a diagram showing an example in which a camera group constituting a vehicle driving assistance apparatus of the present invention is disposed around a vehicle, and FIG. 2B is a side view thereof.

3 is a view schematically showing a system configuration of a vehicle driving assistance apparatus of the present invention including a panoramic camera.

4 (a) and 4 (b) are views for explaining a process of combining images photographed from a camera module of the panorama camera by an image composer of a vehicle driving assistance device of the present invention.

5 is a view illustrating a screen displayed on display means provided in a vehicle by the operation of the vehicle driving assistance apparatus of the present invention.

6A to 6F are diagrams illustrating an arrangement of screens that can be displayed by operation of an operation button provided on the display means.

Fig. 7 is a diagram showing another example in which the camera group constituting the vehicle running aiding apparatus of the present invention is disposed around the vehicle. Fig.

BRIEF DESCRIPTION OF THE DRAWINGS FIG.

10A, 10B, 130, 140: camera 20: image processing means

30, 300: display means 110, 110 ', 120: panoramic camera

110A, 110B: camera module 112:

114: Lens system 116: Imaging element

118: camera housing 200: image combiner

310: display screen 320: display control unit

1000: vehicles L1, L2, L3, L4: video incident line

BACKGROUND OF THE INVENTION 1. Field of the Invention [0001] The present invention relates to a vehicle driving assistance system, and more particularly, to a vehicle driving assistance system that assists a driver's vision during driving to enable safer driving.

Driving safety of a vehicle is closely related to the driver's vision. Although means such as a side mirror and a room mirror are mounted on the vehicle for securing the view of the driver, there is still a square in the driver's field of view. These square angles are caused by the driving position of the driver and the arrangement of the surrounding structures. The square caused by the driver's position is especially the rear, side, etc. of the vehicle. The obstacles due to the arrangement of the surrounding structures include the left and right front view limitations caused by the alleyway distance.

In recent years, efforts have been made to install a video surveillance system in a vehicle in order to compensate for such driver's visual limitations.

Korean Patent Laid-Open Publication No. 2003-8582 discloses an automotive rear monitoring device for securing a rear view of a vehicle. The automotive rear monitoring device includes a microprocessor for controlling the operation of the rear monitoring device of an automobile, A camera power supply unit for supplying power to the camera when the backward signal is applied; a camera power supply unit driven by a power supplied from the camera power supply unit, And a monitor for receiving the image pickup signal outputted from the camera and displaying the rear state of the car on the screen.

Similarly, Korean Patent Laid-Open Publication No. 2003-79030 provides a vehicular four-way surveillance system for assisting the side and rear sight of a vehicle. This system is installed on the side of both front fenders of the vehicle, A plurality of digital cameras provided so as to photograph the side and the rear of the display unit, and image data photographed by the digital camera divided into groups by wired or wireless and transmitted to the graphic controller of the display unit, Is divided into a plurality of segmented portions and displayed.

Similarly, Korean Patent Laid-Open Publication No. 2003-22707 discloses a side mirror having a built-in CCD camera. The side mirrors include right and left CCD cameras built in each of the covers and for capturing images of both sides of the front side of the automobile via the front view hole, and image data of the left and right CCD cameras are displayed on the display unit by the image processor. At this time, the left and right CCD cameras of the side mirror can be selectively driven by a selection switch.

Meanwhile, Korean Patent Laid-Open Publication No. 2004-27700 provides a surveillance system that is designed to be able to adjust the angle of a camera mounted in the interior, rear, and left and right rooms of an automobile. This system is able to cope with safety driving and emergency matters quickly, and it can be mounted on various kinds of vehicles, so that the operation radius can be adjusted from the body of the vehicle, so that the detailed movement of all the vehicles can be coordinated. Also, in the above patent, a channel number is assigned to each camera, and the driver can adjust the angle of the camera according to the channel number.

Thus, various camera systems for assisting the driving safety of the vehicle have been proposed. Among these camera systems, it is important that a wide range of images are secured in the case of a camera mounted on the front and rear of the vehicle. For this purpose, a device using a special lens such as a wide-angle lens or a fish-eye lens as a front-and-rear surveillance camera of a vehicle has been proposed.

For example, Korean Utility Laid-Open Publication No. 1999-6873 discloses a structure in which a split prism having a right-angled triangular cross section is attached to the back surface of an outer cover for protecting an image pickup element and a lens, To the image pickup device so that the situation of both sides of the front side of the vehicle can be known through a monitor installed in the inside of the vehicle.

However, this system is disadvantageous in that the camera structure is complicated, and only a part of the front left and right ends are captured by the image due to the optical characteristics of the prism, which is unsuitable as a surveillance camera for monitoring a wide area in front and rear of the vehicle .

On the other hand, a surveillance device for monitoring the front or rear of a car by employing a fisheye lens has been proposed. However, in the case of a fisheye lens, since an area captured on the screen is too wide, the object is displayed too small on the screen, Lt; / RTI >

In recent years, there has been an attempt to use a surveillance camera for a lane departure warning device that emits a warning sound to a driver when the vehicle is separated from the lane by simply moving away from the surroundings of the vehicle. This system can help the safe operation of the vehicle by alerting the driver when the vehicle is leaving due to drowsiness. In addition, there are other attempts to use a car camera as an automatic parking device. This function checks the position and the interval of the parked vehicle with the camera, and carries the function of parking the vehicle at a desired position by automatically running the vehicle. As described above, the camera for mounting a vehicle is not only a visual assistance for the driver but also actively contributes to the safe driving of the vehicle, thereby expanding its application field.

Therefore, there is an urgent need for a vehicle driving assistance system which can monitor a circumstance around the vehicle by ensuring a wider range of front and rear vision, and can simplify the structure and minimize the cost and maintenance cost of the device.

SUMMARY OF THE INVENTION It is an object of the present invention to provide a vehicle driving assistance device capable of providing a driver with a complete panoramic image around the vehicle.

It is another object of the present invention to provide a vehicle driving assistance device capable of displaying a situation around the vehicle to a driver with a minimum of image processing.

According to an aspect of the present invention, there is provided an image pickup apparatus including a plurality of cameras each having a lens system and an image pickup device and mounted at appropriate positions of the vehicle to photograph a situation around the vehicle, an image combiner for processing an image input from the plurality of cameras, And a display unit that is operated by the image combiner and displays a photographed image to a driver, wherein the plurality of cameras include at least two cameras arranged adjacent to each other for photographing the front or rear of the vehicle, 120A and 120F and the adjacent image input lines of the at least two camera modules 110A, 110B, 120E and 120F are arranged substantially parallel to each other to form panoramic cameras 110 and 120 And a driving force control unit that controls driving force applied to the vehicle.

In the apparatus of the present invention, the image combiner 200 divides the screen of the display means 300 into at least two areas, and the image combiner 200 divides the screen of the display unit 300 into at least two areas, And displaying the image in the divided adjacent areas.

In the apparatus of the present invention, the at least two camera modules 110A, 110B, 120E, and 120F may be CCD or CMOS camera modules, respectively. When at least two camera modules 110A, 110B, 120E and 120F are CCD camera modules, the lens system of the at least two camera modules 110A, 110B, 120E, (114) preferably have an optical distortion ratio of 15% or less. The at least two camera modules 110A, 110B, 120E, and 120F may each have an angle of view of 30 to 120 degrees, preferably 30 to 90 degrees.

In the apparatus of the present invention, the display means 300 may include an operation button (not shown) for operating the image combiner 200.

In addition, in the apparatus, the plurality of cameras may further include a camera 130, 140 for photographing the side of the vehicle 1000.

BEST MODE FOR CARRYING OUT THE INVENTION Hereinafter, the present invention will be described in detail with reference to the preferred embodiments of the present invention with reference to the drawings.

FIG. 1A is a diagram showing an example of a vehicle monitoring apparatus that can be configured to obtain two or more conventional panorama images. 1A, a camera 10A, 10B having an image pickup element (not shown) such as a charge coupled device (CCD) or a CMOS (CMOS) and a lens system (not shown) The image is displayed on the display means 30 mounted on the inside of the vehicle, such as an LCD monitor, through the image processing means 20.

The pair of cameras 10A and 10B have an angle of view of approximately 30 ° to 90 ° according to the characteristics of the lens and the image pickup device, respectively, and photograph two adjacent areas. According to the arrangement of the camera shown. Since there is an area b in which the image incident lines overlap each other in the area photographed by the pair of cameras, in order to express the panoramic image, a part of the area (b) overlapped by the image processing unit 20 is removed It is necessary to display.

In addition, in the conventional camera apparatus, a wide-angle lens having a wide angle of view is used in order to obtain a complete image of front and rear sides. A camera having a wide angle lens exhibits a very large optical distortion rate. For example, based on a 1/3-inch CCD camera, a conventional camera exhibits an optical distortion of -80% at a maximum IMAGE HT (intermediate value of IMAGE HEIGHT: CCD diagonal) of 2.99 mm. In the case of the conventional wide-angle lens, the reciprocal of the focal length (-0.19) of the focal length of the front group (the lens group disposed at the front of the aperture) and the focal length of the rear group (the lens group disposed between the aperture and the CCD device) ), There is a large difference in optical distortion.

In this optical distortion, there is about -26.7% monitor distortion (corresponding to 1/3 of the optical distortion rate), which causes a distortion that the image is warped toward the periphery. Therefore, the image processing means 20 needs to correct an optical distortion to display an image.

1B and 1B illustrate a process of displaying one complete panorama image by removing the optical distortion and the overlapping phenomenon appearing in the images I1 and I2 obtained by the respective cameras 10A and 10B Fig.

As described above, when two or more cameras are used to monitor the front or rear of the vehicle, the camera apparatus must perform a very complicated calculation process in order to remove the overlapping region (b) and correct optical distortion. And a high-performance image processing means 20 is required for complicated calculation. In the case where the positions of the cameras 10A and 10B vary from the initially set position, the setting of the image processing algorithm must be changed. Therefore, it is necessary to be fixedly mounted on a predetermined portion of the vehicle.

Thus, a vehicle driving assist system using two or more cameras must employ a complex image processing technique in order to display images obtained from two or more independent cameras. However, such a device has been a stumbling block to the commercialization of the surveillance camera device around the vehicle by raising the device cost and the device maintenance cost.

FIG. 2A is a diagram showing an example in which a camera group constituting a vehicle driving assistance apparatus of the present invention is disposed around a vehicle, and FIG. 2B is a side view thereof.

2A and 2B, a plurality of cameras 110, 120, 130, and 140 are mounted on a roof of a vehicle 1000, a trunk of a vehicle 1000, and both side mirrors.

2A and 2B together show the viewing angles (see arc arrows) covered by the cameras 110, 120, 130, and 140, respectively. As shown in the figure, the roof of the vehicle 1000 and the trunk of the vehicle 1000 are provided with panoramic cameras 110 and 120 each having a view angle of, for example, 180 degrees so as to cover the front and rear images of the vehicle 1000 as much as possible Respectively. Further, general cameras 130 and 140 having an angle of view of, for example, 90 degrees are disposed on the respective side mirrors of the vehicle 1000. 2A and 2B illustrate an example in which the mounted cameras 110, 120, 130, and 140 are arranged so as to observe all situations around the vehicle 1000 without fail, and the vehicle driving assistance The arrangement of the camera group of the apparatus and the angle of view of each camera are not limited thereto.

The vehicle driving assistance apparatus of the present invention includes panoramic cameras 110 and 120 having a wide angle of view disposed at the front and / or rear of the vehicle. This is one feature of the present invention in order to secure a wider angle of view by one camera. The features of the panoramic camera of the present invention will be described below.

3 is a view schematically showing a system configuration of a vehicle driving assistance apparatus of the present invention including the panoramic camera 110. As shown in Fig.

Referring to FIG. 3, the panoramic camera 110 is composed of two camera modules 110A and 110B. In the present invention, each of the camera modules 110A and 110B constituting the panoramic camera 110 may be a CCD camera module or a CMOS camera module.

Each of the camera modules 110A and 110B includes a lens system 114 and an image sensor 116 (image sensor). The lens system 114 of the camera module may have an angle of view of up to 90 degrees when using a wide-angle lens. However, the angle of view of the camera module is preferably 70 ° or less.

The camera modules 110A and 110B are protected by a camera housing 118. The housing 118 is provided with an aperture 112 at a position corresponding to the lens system 114 of the camera modules 110A and 110B. In the present invention, the image input lines L1, L2, L3 and L4 are defined by the camera module 110A and 110B by the aperture element 112 and the lens system 114, respectively. The camera modules 110A and 110B of the present invention are disposed so as to be substantially parallel so that the interval b between the adjacent two image input lines L2 and L3 is constant. This arrangement prevents overlapping of images obtained in each camera module. In the present invention, the fact that the image incident lines are substantially parallel includes a state in which the image incident lines do not overlap with each other even if they are not perfectly parallel. For example, such a condition may be such that the image incidence line is not completely parallel but does not overlap the image at the normal viewing distance, that is, the angle between the adjacent paired two incident image lines is less than 5 degrees ≪ / RTI >

In the present invention, the interval b between the two image incident lines L2 and L3 depends on the size of the lens system 114 and the like, but this gap can be maintained at several millimeters or less at the present technology level. Therefore, in the arrangement of the camera module of the present invention, the portions not captured at the view angle observable by the camera modules 110A and 110B are negligibly small.

Referring back to FIG. 3, the image captured by the camera modules 110A and 110B is transmitted to the image combiner 200. [ In the present invention, the image combiner 200 displays the images A and B taken by the camera modules 110A and 110B on the display unit 300. As shown in the figure, the image combiner 200 divides the screen of the display unit 300 into a predetermined area and displays an image photographed by the camera modules 110A and 110B on a corresponding area of the screen. The image combiner 200 may arrange the images A and B captured by the camera modules 110A and 110B in a predetermined area of the display unit 300 in the horizontal direction. In addition to the panoramic camera 110 described above, the image combiner 200 may also be configured to superimpose images (C, D) photographed from other general cameras (130, 140) installed at the side or a panoramic camera The panoramic camera 120 installed in the rear of the vehicle 1000 plays a role of displaying on a predetermined area of the display means 300 and has the same configuration as the panoramic camera 110 installed in front of the vehicle 1000 .

In the present invention, as the display means 300, various conventional display devices such as a cathode ray tube as well as a flat panel display device such as a liquid crystal display (LCD) and an OLED (Organic Light Emission Display) .

4A and 4B illustrate a process of combining the images A and B photographed from the camera modules 110A and 110B of the panoramic camera 110 by the image composer 200 of the present invention FIG.

FIG. 4A shows images captured by the camera modules 110A and 110B, and FIG. 4B shows images displayed by being combined by the image combiner 200. FIG. As shown in FIG. 1B, the image combiner 200 does not overlap the images as described above with reference to FIG. 1B. Therefore, by simply arranging the respective images on the screen without manipulation such as trimming of the images, The images photographed by the modules 110A and 110B can be configured as one complete screen.

4 (a) and 4 (b), as described above, since the optical distortion of the wide angle lens of the CCD camera reaches -80%, complicated image processing is required to express the panoramic image It was essential. However, in the present invention, when the distortion of the optical lens is limited to ± 15% (5% of monitor distortion), the image of the camera modules 110A and 110B can be displayed as one panorama image have.

Since there are various methods for minimizing optical distortion, various techniques can be used to obtain a desired distortion rate. However, this is outside the scope of the present invention, and therefore, this is not described in detail. However, it is possible to minimize the distortion of the lens by adjusting the power of the front group and the rear group based on the aperture, for example, when the power of the front group is -0.24 and the power of the rear group is +0.183 in the retrofocus type, It is possible to maintain the optical distortion rate at about -10% based on IMAGE HT 2.99 mm of a CCD camera.

Of course, the vehicle travel assistance apparatus of the present invention may solve the optical distortion problem described above in software by adding the image distortion correction function to the image combiner 200, and the present invention may be applied to a case where the conventional image processing means is excluded It does not.

5 is a diagram illustrating a screen displayed on the display means 300 provided in the vehicle by the operation of the vehicle driving assistance apparatus of the present invention. FIG. 5 is a view showing a panoramic camera 110 and 120 described in relation to FIG. 3 on the front and rear sides of the vehicle, a description of the photographed image when ordinary cameras 130 and 140 are mounted on both side mirrors of the vehicle to be. 5, the screen 310 of the display means 300 includes the images A and B photographed by the front panoramic camera 110 and the images C and D obtained by the two side mirror cameras 130 and 140 And the images E and F photographed by the rear panorama camera 120. [ As shown in the figure, the image combiner 200 may be configured to combine the images obtained from the camera modules 110A, 110B, 120E, and 120F of the front and rear panoramic cameras 110 and 120 according to the method described with reference to FIG. (A, B) and (E, F), respectively.

In the present invention, the vehicle driving assistance apparatus operates the image combiner 200 by operating a series of operation buttons provided on the operation unit 320 of the display unit 300, thereby moving the image arrangement on the screen of the display unit 300 Can be changed.

6A to 6F are diagrams illustrating an arrangement of screens that can be displayed by the operation of the operation buttons. In the drawing, A to F correspond to the screens of Fig. 5, respectively. 6A to 6C illustrate an example of screen arrangement in which the screen of the display means 300 can be divided and displayed, and Figs. 6D to 6F show a two-split screen.

The panorama cameras 110 and 120 are configured by two camera modules 110A and 110B and 120E and 120F respectively in the vehicle driving assistance apparatus of the present invention. The number of camera modules constituting the camera module is not limited thereto. For example, in order to configure an angle of view of 180 degrees, panoramic cameras 110 and 120 employing three camera modules having an angle of view of 60 degrees may be used, or four camera modules having an angle of view of 45 degrees may be used. However, at this time, adjacent image input lines of two adjacent camera modules must be arranged substantially parallel as described with reference to FIG.

Fig. 7 is a diagram showing another example in which the camera group constituting the vehicle running aiding apparatus of the present invention is disposed around the vehicle. Fig. 2A except that the front panoramic camera 110 is disposed forwardly of the vehicle. In such an arrangement, when using a camera having the same performance as in Fig. 2A, there is a disadvantage that a blind spot corresponding to the length d _B exists around the vehicle. However, since the camera is arranged at the front, the entry of the unidentified pedestrian M or another vehicle in the sight of the driver D can be confirmed in advance by the obstacle O or the like, It is possible to obtain an effect of preventing a contact accident or the like in advance.

In addition, the panoramic camera 110 installed in front of the vehicle may be mounted in the interior of the vehicle (refer to reference numeral 110 'in FIG. 7). This can prevent the camera from being damaged when an external object collides with the camera during driving.

The vehicle driving assistance apparatus of the present invention can provide a complete panoramic image of the front or rear of the vehicle. Further, since the panoramic image is obtained from the proper arrangement of the camera, there is no fear of increase in the apparatus cost as compared with the conventional monitoring apparatus.

In addition, since the vehicle driving assistance apparatus of the present invention does not need to employ a complicated image processing technique in displaying a wide range of surveillance images in front or rear, it is a burden on the apparatus cost and the apparatus maintenance cost. In addition, the vehicle driving assist system of the present invention minimizes the image processing technique in the display of the screen, so that it is possible to select and change the vehicle mounting position. Further, since the image processing technique of the present invention is simple, it is easy to combine with the operation means such as the tilt of the camera, and the necessity of tuning the image composer according to the operation can be minimized or minimized.

Claims (8)

A plurality of cameras each equipped with a lens system and an image pickup device and mounted at appropriate positions of the vehicle to photograph a situation around the vehicle, an image combiner for processing images input from the plurality of cameras, And a display means for displaying the display means to the driver, The plurality of cameras includes at least two camera modules 110A, 110B, 120E, and 120F disposed adjacent to the front and rear of the vehicle 1000, and the at least two camera modules 110A, 110B, 120B, 120B, 120B, 120B, 120B, 120B, 120B, 120B, 120B, 120B, 120B, 120B, 120B, 120B. The method according to claim 1, The image combiner 200 divides the screen of the display means 300 into at least two areas and outputs an image input from the at least two camera modules 110A, 110B, 120E, and 120F to the divided adjacent Respectively, in a region where the vehicle is traveling. The method according to claim 1, Wherein the at least two camera modules (110A, 110B), (120E, 120F) are CCD or CMOS camera modules, respectively. The method according to claim 1, The at least two camera modules 110A, 110B, 120E and 120F are CCD type camera modules and the lens system 114 of the at least two camera modules 110A, 110B, 120E and 120F is 15 % ≪ / RTI > or less. The method according to claim 1, Wherein the at least two camera modules (110A, 110B), (120E, 120F) each have an angle of view of 30 to 120 degrees. The method according to claim 1, Wherein the at least two camera modules (110A, 110B), (120E, 120F) each have an angle of view of 30 to 90 degrees. The method according to claim 1, Wherein the display means (300) comprises an operation button for operating the image combiner (200). The method according to claim 1, Wherein the plurality of cameras further includes a camera (130, 140) for photographing a side of the vehicle.

Images