KR20100005971A - A vehicle around view monitorring system - Google Patents

Abstract

PURPOSE: A system for monitoring the view around a vehicle is provided to simply inform a driver of the expected track by calculating the expected movement track by using gear stage information. CONSTITUTION: A system for monitoring the view around a vehicle is composed of an image processing unit(20), an image compositing unit(22), an OSD(On Screen Display) display unit(24), and an image output unit(28). The image processing unit converts plural camera images into the aerial-view image about the upward virtual view point of the vehicle. The image compositing unit collects the images around the vehicle from the aerial-view images and generates one all direction view image. The OSD display unit generates the final all direction view image by displaying the image of the vehicle and the expected movement track on the all direction view image. The image output unit outputs the final all direction view image and the camera images on the display.

Description

Vehicle directional monitoring system {A VEHICLE AROUND VIEW MONITORRING SYSTEM}

This document relates to a vehicle omnidirectional monitoring system, and more particularly, to an omnidirectional monitoring system that provides an omnidirectional image added with an indication of a vehicle's expected movement trajectory.

Recently, with the development of camera and image processing technology, the application of an image processing system for the convenience of parking and driving convenience of the driver is increasing. In particular, rear PAS (Parking Assist System) using ultrasonic waves has been applied to most cars to secure the visibility of the left and right side or the rear when the vehicle is parked. However, since the PAS system sounds a warning to the driver only in three levels of warning, information on obstacles around the vehicle is limited.

To compensate for this drawback, a system for increasing the field of view by providing an image of the rear by applying a rear camera may be applied. However, this system is difficult to determine the distance from the obstacle, there is a limit that can not provide information about the rear of the vehicle.

In addition, the system may be equipped with a camera mounted on the left and right side mirrors of the vehicle, the controller may operate the camera, and the system may be applied to secure the field of view by manipulating the camera in a direction to be viewed in the surrounding environment of the vehicle. However, this system has a limited field of view of the camera left and right, and the mechanical system is complicated to implement the motor, limit sensor, and controller (ECU) for manipulating the camera irradiation direction (left, right, up and down). there is a problem.

In the above-described background, an object of the present invention is to provide a vehicle omnidirectional monitoring system that displays and provides a vehicle predicted locus of movement on an omnidirectional image of a vehicle periphery.

As a means for solving the above-mentioned problems, the vehicle omnidirectional monitoring system is an image processing unit for converting a plurality of images obtained from a plurality of cameras mounted on the vehicle into an overhead view image for the upper virtual view of the vehicle, respectively, And an image synthesizing unit generating a single omnidirectional image by combining the plurality of overhead views according to a preset registration position. The display apparatus may include an OSD display unit for generating a final omnidirectional image by adding an indication of the movement trajectory of the vehicle to the omnidirectional image, and an image output unit for outputting the final omnidirectional image on a display.

A plurality of cameras mounted on the vehicle may be mounted to photograph the front, rear, left, and right sides of the vehicle, and may combine images taken from the plurality of cameras to generate an omnidirectional image of the surroundings of the vehicle. In addition, by displaying the predicted locus of the vehicle along with the omnidirectional image, the distance between the vehicle and the obstacle can be intuitively understood, and thus the driver's convenience can be achieved.

The movement prediction trajectory may be determined through at least one of gear stage information, speed information, and steering angle information. When the gear stage information is the reverse R stage, the backward movement predicted locus of the vehicle may be displayed, and when the gear stage information is other than the reverse R stage, the forward movement predicted locus of the vehicle may be displayed. In addition, the OSD display unit may further display the predicted trajectory at the maximum left steering and the predicted trajectory at the maximum right steering together with the movement predicted trajectory.

The OSD display unit may further display an image connection dividing line between the image of the vehicle and the plurality of combined view images on the composite image.

When the camera is a wide-angle camera, the image processor may process an image by considering optical correction.

The image synthesizing unit may generate the synthesized image by collecting an image close to the vehicle from each of the converted plurality of overhead views.

The image output unit may further output individual camera images of at least one of the plurality of cameras as well as the final omnidirectional image on the display. In addition, individual camera images, which are further output to the display, from among a plurality of camera images may be selected through a user interface, for example, a touch screen.

According to the vehicle omnidirectional monitoring system disclosed in this document, it is possible to provide the surrounding information about the front of the vehicle.

In addition, it is possible to maximize the driver's convenience by calculating and providing a moving trajectory of the vehicle and displaying the same on a vehicle omnidirectional image.

In addition, by calculating and providing a movement predicted trajectory using the gear stage information, it is possible to simply inform the driver of the estimated trajectory required.

In addition, the driver may measure the degree of steering by displaying and providing the predicted trajectory at the maximum right steering and / or the predicted trajectory at the maximum left steering on the vehicle omnidirectional image.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, embodiments of the present invention for a vehicle omnidirectional monitoring system in which a predicted movement path of a vehicle is displayed on a composite image composed of omnidirectional images of a vehicle converted to an upper virtual view of a vehicle will be described in detail with reference to the accompanying drawings. do.

 1 is a flowchart illustrating an operation of a vehicle omnidirectional monitoring system according to an embodiment of the present invention, Figure 2 is a structural diagram of a vehicle omnidirectional monitoring system according to an embodiment of the present invention.

First, in step S10, the front image, the rear image, the left-side image of the surrounding environment from each of the four cameras 2a, 2b, 2c, and 2d mounted on the front, rear, left, and right sides of the vehicle in the image processing unit 20. And the right-side image is acquired. In this case, the cameras mounted on the front, rear, left and right sides may be mounted in the front radiator grill, the rear guinish, and the lower side mirror. Then, the front image, the rear image, the left side image, and the right side image acquired by the image processing unit 20 are converted into an overhead view image with respect to the upper virtual view of the vehicle, respectively.

In operation S12, the omnidirectional images converted by the image synthesizing unit 22 are combined according to a preset matching position to generate one omnidirectional image. In operation S14, the OSD display unit 24 adds the display of the vehicle's movement predicted locus to the omnidirectional image to generate the final omnidirectional image.

The image synthesizing unit 22 and / or the OSD display unit 24 receives the speed information 25, the steering angle information 26, and the gear stage information 27 to display the movement trajectory of the vehicle when synthesizing the images. Can be used when

In operation S16, the image output unit 28 outputs, on the display, the final omnidirectional image in which the predicted locus of the vehicle movement is displayed on the omnidirectional image converted to the virtual viewpoint above the vehicle. In this case, the display may be, for example, AV audio mounted in a vehicle.

Hereinafter, each configuration of the vehicle omnidirectional monitoring system will be described in more detail.

3 shows an example of an input and an output image to the image processor 20 of the vehicle omnidirectional monitoring system according to an embodiment of the present invention.

The image processor 20 receives an original image of each camera and converts the original image of each camera into an overhead view image of the upper virtual view of the vehicle as an image as seen from above. The angle 31 between the photographing direction of the vehicle's camera and the vertical direction of the ground may be taken into account when converting the image viewpoint. In this case, when the camera is used with a wide angle camera, not only the viewpoint correction but also a portion distorted by the wide angle lens may be corrected.

For example, the rear view image 30 of the vehicle as shown in FIG. 3A through the viewpoint conversion processing by the image processing unit 20 and the distortion correction by the wide-angle lens according to the present embodiment is illustrated in FIG. 3. As illustrated in (b), the image may be converted into an overhead view 32 of an upper virtual view of the vehicle.

4 is a view for explaining an example of a processing method in the image processing unit 20 of the vehicle omnidirectional monitoring system according to an embodiment of the present invention.

In the present embodiment, a method of performing image processing on a viewpoint transformation or the like using a look up table 40 which sets a mapping relationship before and after a pixel-by-pixel transformation will be described. Here, the lookup table 40 may be configured as a position table of an input image to which image data is to be imported to generate an output image.

The lookup table 40 according to the present exemplary embodiment may be set in consideration of a camera lens model and a camera model mounted on a vehicle, a camera mounting height and an mounting angle, and the like so as to implement viewpoint transformation or distortion correction.

As shown in FIG. 4, the lookup table 40 may be configured with a total of 4 bytes of an address of 3 bytes and an offset of 1 byte. Here, the address may be a memory address value in which image data output to the corresponding pixel is stored. At this time, the image data per pixel for the input image is stored serially in the memory. In addition, the offset is composed of an X-axis offset and a Y-axis offset may be used to correct the lookup table 40.

The image processor 20 may perform pixel operation with reference to the lookup table 40 to perform viewpoint transformation. That is, for each pixel of the output image, the entire output image may be generated by reading image data stored in an address value with reference to the lookup table 40.

5 shows an example of an input and an output image to the image synthesizer 22 of the vehicle omnidirectional monitoring system according to an embodiment of the present invention.

The image synthesizing unit 22 receives an image of each camera converted from a viewpoint and synthesizes one omnidirectional image. In the omnidirectional image, the front image, the rear image, the left-side image, and the right-side image may be determined as positions in the omnidirectional image for each image, and the images of each image may be matched to the determined position to generate one image. This may be corrected when mounted on the actual vehicle.

The position of each image in the omnidirectional image is determined according to which of the omnidirectional images is determined as the front of the vehicle. For example, if the upper portion of the omnidirectional image is determined to be the front of the vehicle, some images of the front image are at the top portion of the omnidirectional image, some images of the rear image are at the bottom portion of the omnidirectional image, and some images of the left side image are In the left portion of the omnidirectional image, some images of the right-side image will be synthesized in the right portion of the omnidirectional image.

When the images of each image are matched and generated as one image, the images may be rotated and matched according to the determined position of each image. That is, in each image, the vehicle side may be rotated to face the center in the omnidirectional image. For example, in the case of the rear image, since the bottom of the image is the side of the vehicle, the bottom image is rotated by 180 ° to face the center in the omnidirectional image.

In addition, a part or the entire image of each image may be matched to the determined position and generated as one image. In particular, when a partial image of each image is matched to generate an omnidirectional image, the entire image region of each image will be a region close to the vehicle.

The rearview-converted rearview image 50 as shown in FIG. 5A has a position determined according to the synthesis process of the image synthesizer 22, that is, within the omnidirectional image as shown in FIG. 5B. The lower portion 51 is rotated 180 degrees to match.

6 shows an example of input and output images of the OSD display unit 24 of the vehicle omnidirectional monitoring system according to an embodiment of the present invention.

The OSD display unit 24 according to the present exemplary embodiment displays and outputs the vehicle movement predicted trajectory 61 on the omnidirectional image 60 illustrated in FIG. 6A as illustrated in FIG. 6B. By displaying the predicted movement trajectory 61 of the vehicle ahead or rearward according to the vehicle traveling direction, it is possible to check in advance whether the vehicle touches an obstacle and moves according to the driver's intention.

At this time, the vehicle movement predicted trajectory 61 may be determined using the vehicle speed information 25 and the steering angle information 26. Here, the steering angle information 26 may be obtained from a steering angle sensor (SAS) for detecting a degree of rotation of the steering wheel. Accurate calculation of the vehicle movement predicted trajectory 61 preferably takes into account various vehicle parameters such as vehicle width, roundabout, wheel width, and the like.

In addition, the OSD display unit 24 may display an image connection dividing line 62 between images matched with the omnidirectional image 60. The part where four images are matched and connected cannot be displayed in the case of an object having a height. Therefore, if there is no specific display, the driver may have an accident caused by an obstacle that does not appear on the screen. Since the image connection dividing line 62 is displayed, the driver may be reminded that an object having a height cannot be detected at the portion where the line is displayed.

In addition, the OSD display unit 24 may display the vehicle image 63 on the omnidirectional image 60. In this case, the vehicle image 63 may be a vehicle picture. In addition, when the upper portion is determined as the front of the vehicle in the omnidirectional image 60, the vehicle image 63 is also displayed so that the front faces the upper end of the omnidirectional image. This allows an image display effect as seen from an actual vehicle.

A vehicle movement prediction trajectory 61, an image connection dividing line 62, a vehicle image 63, etc. are displayed on the image in the synthesized omnidirectional image 60 to help the driver drive safely.

7 illustrates an example of a vehicle movement trajectory display method according to gear setting in the OSD display unit 24 of the vehicle omnidirectional monitoring system according to an embodiment of the present invention.

The OSD display unit 24 according to the present embodiment displays and outputs a vehicle movement predicted locus 61. The vehicle movement predicted trajectory 61 may help the driver determine whether an obstacle collides with the vehicle during parking or whether parking is possible. In this case, the direction of the vehicle movement predicted trajectory 61 displayed according to the gear stage information 27 such as parking P (Park), reverse R (Reverse), neutral N (Neutrality), and driving D (Drive) may be determined.

For example, as shown in FIG. 7A, when the gear is the parking P, the neutral N, and the driving D stage, the forward movement trajectory 61a of the vehicle is displayed. In this case, the display trajectory may be a front left corner predicted trajectory and a front right corner predicted trajectory. In addition, the rear right corner predicted trajectory may be displayed during left steering, and the rear left corner predicted trajectory may be displayed during right steering.

As illustrated in FIG. 7B, when the gear is the reverse R stage, the rear movement predicted trajectory 61b of the vehicle is displayed. In this case, the display trajectory may be a rear left corner predicted trajectory and a rear right corner predicted trajectory. In addition, the front left corner predicted trajectory may be displayed when left steering and the front left corner predicted trajectory may be displayed when right steering.

In addition, the OSD display unit 24 according to the present embodiment displays the predicted trajectories 64a and 64b at the maximum left steering and / or the predicted trajectories 65a and 65b at the maximum right steering when displaying the vehicle movement predicted trajectory 61. Can display more. The predicted trajectories 64a and 64b at maximum left steering and / or the predicted trajectories 65a and 65b at maximum right steering may help determine the degree of steering that the driver should adjust to proceed in the desired direction.

8 illustrates an example of an output image of the image output unit 28 of the vehicle omnidirectional monitoring system according to an embodiment of the present invention.

The image output unit 28 outputs an image which is finally viewed by the driver. The image output unit 28 according to the present exemplary embodiment includes a predicted locus of movement of the vehicle in the omnidirectional image converted into a virtual viewpoint upstream of the vehicle. The displayed final omnidirectional image is output on the display.

Referring to an example of the output screen illustrated in FIG. 8A, the output screen provided to the driver may include only the final omnidirectional image 80 in which the predicted locus of the vehicle is displayed on the omnidirectional image converted into a virtual viewpoint above the vehicle. It may also be configured to show at least one individual camera image 81 at the same time.

The individual camera image 81 may be one of a plurality of camera images, and may be one of a front image, a rear image, a left side image, and a right side image of the surrounding environment of the vehicle. Alternatively, each image may be one of the bird's-eye views converted by the image processor 20.

In this case, an operation message 83 for receiving a warning message 82 or a user input may be inserted into the display. In particular, the user may select an image output as an individual camera image 81 through the operation menu 83, Display settings, etc. can be operated. In this case, the user interface may be implemented by various methods such as a selection button or a touch screen.

Referring to an example of the output screen shown in FIG. 8B, the final omnidirectional image 800 and the individual camera image 810, a warning message 820 of "Please look around for safety!" An operation camera 830 consisting of "setting" and "conversion" and an output camera indication 840 of which camera image the individual camera image 810 is.

When the touch screen function is implemented on the display, when the user touches the front image in the final omnidirectional image 800, an image corresponding to the front camera is output in the individual camera image 810. And, output camera display 840 indicates the front of the vehicle. Although it is not the touch screen, it is obvious that the image output from the individual camera image 810 can be selected by activating the selection button.

Although described above with reference to the drawings and embodiments, those skilled in the art that the present invention can be variously modified and changed within the scope without departing from the spirit of the invention described in the claims below I can understand.

1 is a flow chart for explaining the operation of the vehicle omnidirectional monitoring system according to an embodiment of the present invention.

2 is a structural diagram of a vehicle omnidirectional monitoring system according to an embodiment of the present invention.

3 shows an example of an input and an output image to the image processor 20 of the vehicle omnidirectional monitoring system according to an embodiment of the present invention.

4 is a view for explaining an example of a processing method in the image processing unit 20 of the vehicle omnidirectional monitoring system according to an embodiment of the present invention.

5 shows an example of an input and an output image to the image synthesizer 22 of the vehicle omnidirectional monitoring system according to an embodiment of the present invention.

6 shows an example of input and output images of the OSD display unit 24 of the vehicle omnidirectional monitoring system according to an embodiment of the present invention.

7 illustrates an example of an output image according to gear setting in the OSD display unit 24 of the vehicle omnidirectional monitoring system according to an exemplary embodiment of the present invention.

8 illustrates an example of an output image of the image output unit 28 of the vehicle omnidirectional monitoring system according to an embodiment of the present invention.

<Description of the symbols for the main parts of the drawings>

20: image processing unit 22: image synthesizing unit

24: OSD display unit 28: Image output unit

Claims (5)

An image processing unit for converting a plurality of camera images acquired from a plurality of cameras mounted in a vehicle into an overhead view image of an upper virtual view of the vehicle; An image synthesizer configured to collect an image close to the vehicle from each of the plurality of converted view images and combine the images according to a preset matching position for each image to generate one omnidirectional image; An OSD display unit generating a final omnidirectional image by displaying an image of the vehicle and a predicted movement trajectory determined through at least one of gear stage information, speed information, and steering angle information of the vehicle on the omnidirectional image; And An image output unit configured to output the final omnidirectional image and an individual camera image selected through a user interface among the plurality of cameras on a display; Comprising a vehicle omnidirectional monitoring system. In claim 1, The OSD display unit, if the gear stage information is the reverse R stage, the rear movement predicted locus of the vehicle, and when the other than the reverse R stage, the vehicle characterized in that for displaying the forward movement trajectory of the vehicle Omnidirectional surveillance system. In claim 1, The OSD display unit, the vehicle omnidirectional monitoring system, characterized in that for further displaying the expected trajectory of the maximum left steering and the maximum trajectory of the right steering in the omnidirectional image. In claim 1, And the OSD display unit further displays an image connection dividing line between the plurality of overhead views in combination with the omnidirectional image. In claim 1, And the user interface is implemented as a touch screen.

Images