KR101551697B1 - Around view system displaying outline of the object and methods providing around view of the vehicle - Google Patents

Abstract

There is provided an around view system for displaying an outline of an object, a method for providing an approach view of a vehicle, and a method for providing an approach view of a vehicle. An arousal view system for displaying an outline of the object includes a plurality of image capturing units for capturing the front, rear, left, and right sides of the vehicle, and a distance measuring unit for measuring the distance to an object located at the front, rear, left, An AVM processing unit for generating a surround view image by combining a plurality of images captured by the plurality of image capturing units and a plurality of ultrasonic sensor units, and determining an outline of the object using the distance to the measured object, And an image processor for overlapping an image corresponding to an outline of the object on the view image to regenerate the surround view image.

Description

BACKGROUND OF THE INVENTION Field of the Invention [0001] The present invention relates to an arousal view system for displaying an outline of an object,

The present invention relates to an arousal view system for displaying an outline of an object and a method for providing an arousal view of the vehicle. And more particularly, to an around view system for photographing an appearance of a vehicle through a plurality of image capturing units and providing state information of the vehicle and a method for providing an overview of the vehicle.

In addition to improving the driving performance of automobiles, automobile manufacturers are using various devices required for safety in automobiles to promote convenience and safe driving of the driver. As one of the rear detection systems, a rear camera is adopted to solve the difficulties experienced by the driver in the backward movement of the vehicle, so that the driver can check the rear portion of the vehicle through the rear camera at the time of backward movement.

In recent years, the camera is mounted not only on the rear but also on the front, left, and right sides of the vehicle, allowing the driver to check the front, rear, left and right sides of the vehicle when driving or parking. Generally, a system that provides an integrated video signal to a driver by using a camera mounted on the front, rear, left, and right sides of an automobile is called an Around View Monitor System.

In such an AVM system, if a camera image of high image quality can not be obtained due to noise or the like, it may be difficult to identify an object near the vehicle, and if a part of the camera fails, the object on the shooting region may not be recognized.

Korea Patent No. 01249791

An object of the present invention is to provide an around view system and an approach view providing method capable of grasping an object on an around view image even when a camera image of high image quality can not be obtained due to noise or a part of the camera is broken .

The problems to be solved by the present invention are not limited to the above-mentioned problems, and other matters not mentioned can be clearly understood by those skilled in the art from the following description.

An aspect of an overview system for displaying an outline of an object according to the present invention includes a plurality of image capturing units for capturing a front, a rear, a left, and a right of a vehicle, a front, a rear, An AVM (Around View Monitor) processor for generating an averaged view image by combining the images captured by the plurality of image capturing units, And an image processing unit for determining an outline of the object using the distance of the object, and reproducing the surround view image by overlapping the image corresponding to the outline of the object on the surround view image.

Also, the image processing unit does not regenerate the surround view image when the object does not exist.

In addition, the image processing unit regenerates the surround view image only when a part of the plurality of image capturing units fails.

In addition, the image processing unit overlaps the image on the shooting region of the failed image shooting unit among the surrounding view images to regenerate the surrounding view image.

In addition, the image processor selectively regenerates the surround view image according to a user input.

The display device further includes a display unit for displaying the regenerated surround view image.

According to another aspect of the present invention, there is provided a method for providing an overview of a vehicle that displays an outline of an object, comprising the steps of: receiving respective images photographed from a plurality of image capturing units; And a distance between the object and the object, an AVM processing unit generating the surround view image by combining the received images, determining an outline of the object by using the distance to the measured object in the image processing unit Reproducing the surrounding view image by overlapping the image corresponding to the outline of the object on the surrounding view image, and displaying the reproduced surrounding view image.

The step of regenerating the surround view image may not regenerate the surround view image when the object does not exist.

In addition, the step of regenerating the surround view image regenerates the surround view image only when a part of the plurality of image capturing units is broken down.

The step of regenerating the surround view image selectively regenerates the surround view image according to a user input.

According to the above-described method for providing the surround view system and the vehicle surround view method of the present invention, when a camera image of high image quality can not be obtained due to noise or the like, or an object around the vehicle can not be identified, In this case, it is possible to solve the problem that an object on the shooting region can not be grasped by using the regenerated surround view image, and it is possible to prevent the vehicle from being damaged by colliding with an object.

1 is a view for explaining an overview system for displaying an outline of an object according to an embodiment of the present invention.
FIG. 2A is a view for explaining an overview view image in the case where there is no obstacle around the vehicle in a vehicle provided with an overview system for displaying an outline of the object of FIG. 1. FIG.
FIG. 2B is a view for explaining an overview view image in the case where an obstacle exists around the vehicle in a vehicle provided with an overview system for displaying an outline of the object of FIG.
FIG. 3 is a diagram for explaining how an image corresponding to an outline of an object on the photographing area is overlapped when a part of the image photographing unit is broken in a vehicle provided with an overview system for displaying an outline of the object of FIG. 1;
4 is a flowchart for explaining a method of providing an overview of a vehicle for displaying an outline of an object according to an embodiment of the present invention.

BRIEF DESCRIPTION OF THE DRAWINGS The advantages and features of the present invention and the manner of achieving them will become apparent with reference to the embodiments described in detail below with reference to the accompanying drawings. The present invention may, however, be embodied in many different forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the invention to those skilled in the art. To fully disclose the scope of the invention to those skilled in the art, and the invention is only defined by the scope of the claims. Like reference numerals refer to like elements throughout the specification.

One element is referred to as being "connected to " or" coupled to "another element, either directly connected or coupled to another element, One case. On the other hand, when one element is referred to as being "directly connected to" or "directly coupled to " another element, it does not intervene another element in the middle. Like reference numerals refer to like elements throughout the specification. "And / or" include each and every combination of one or more of the mentioned items.

Although the first, second, etc. are used to describe various elements, components and / or sections, it is needless to say that these elements, components and / or sections are not limited by these terms. These terms are only used to distinguish one element, element or section from another element, element or section. Therefore, it goes without saying that the first element, the first element or the first section mentioned below may be the second element, the second element or the second section within the technical spirit of the present invention.

The terminology used herein is for the purpose of illustrating embodiments and is not intended to be limiting of the present invention. In the present specification, the singular form includes plural forms unless otherwise specified in the specification. It is noted that the terms "comprises" and / or "comprising" used in the specification are intended to be inclusive in a manner similar to the components, steps, operations, and / Or additions.

Unless defined otherwise, all terms (including technical and scientific terms) used herein may be used in a sense commonly understood by one of ordinary skill in the art to which this invention belongs. Also, commonly used predefined terms are not ideally or excessively interpreted unless explicitly defined otherwise.

1 is a view for explaining an overview system for displaying an outline of an object according to an embodiment of the present invention.

Referring to FIG. 1, an overview system 1 for displaying an outline of an object according to an embodiment of the present invention includes an image photographing unit 100, an ultrasonic sensor unit 200, an AVM processing unit 300, an image processing unit 400, and a display unit 500.

The image capturing unit 100 includes a plurality of cameras mounted on a vehicle. A plurality of cameras take images of the front, rear, left, and right sides of the vehicle, respectively, and transmit the photographed images to an AVM (Around View Monitor) processing unit. The image capturing unit 100 may include one or more image capturing units 100 on the front, rear, left, and right sides of the vehicle. The image photographed by the image capturing unit 100 is transmitted to the AVM processing unit 300 by wire or wirelessly. The image capturing unit 100 may include, in addition to a general camera, a wide-angle camera, a thermal imager, an infrared camera, and the like, but is not limited thereto.

The ultrasonic sensor unit 200 includes a plurality of sensors mounted on a vehicle. The plurality of sensors senses the front, rear, left, and right objects of the vehicle, measures the distance between the vehicle and the object, and transmits the measured data to the image processing unit 400. Generally, an ultrasonic sensor is mainly used for an object detecting device of a vehicle. The ultrasonic sensor part is a sensor that measures the position and distance of an object around the vehicle by using a high frequency sound of 20 ~ 30 kHz which can not be heard by the human ear. It detects the ultrasonic wave reflected from the object and returns, Measure the distance. In the ultrasonic sensor part 200, one or more ultrasonic sensor parts 200 may be disposed on the front, rear, left, and right sides of the vehicle. For example, four ultrasonic sensors may be installed on the front, rear, left, and right sides of the vehicle, respectively. However, the present invention is not limited thereto. The distance between the vehicle and the object measured by the ultrasonic sensor unit 200 is transmitted to the image processing unit 400 by wire or wirelessly.

The AVM processing unit 300 combines the images photographed by the plurality of image capturing units 100 to generate an overview image. More specifically, the AVM system includes a plurality of image capturing units 100 and an AVM processing unit 300. When the captured images of the respective cameras are transmitted to the AVM processing unit 300, the AVM processing unit 300 transmits An image distortion correction process is performed to generate a single surround view image. The generated generated surround view image is transmitted to the image processing unit 400 by wire or wirelessly.

The image processing unit 400 receives and regenerates the surrounding view image generated through the AVM processing unit 300. The image processing unit 400 determines an outline of an object by using the distance between the vehicle and the object measured by the plurality of ultrasonic sensor units 200 and overlaps the image corresponding to the outline of the object on the surround view image, The view image is regenerated. That is, based on the data measured by the plurality of ultrasonic sensor units 200, contours of objects in the vicinity of the vehicle can be calculated in 2D or 3D by calculation. Then, the outline of the object is displayed on the surrounding view image so that the object can be identified when the image is blurred to make it difficult to identify the object, or when a part of the image capturing unit 100 is broken.

 Whether or not to reproduce the surround view image is determined through mode selection input by the user interface. That is, the user can decide when to display the overlap of the object on the far view image. The modes that can be selected by the user include a mode (first mode) in which an image corresponding to an outline of an object is overlapped with an allround view image to always reproduce the image, a mode in which only a part of the plurality of image capturing units 100 is reproduced 2 mode), and a mode that is not always reproduced (third mode). However, the present invention is not limited thereto.

The image processing unit 400 is connected to the AVM processing unit 300 and the ultrasonic sensor unit 200, which are disposed at the front, rear, left, and right sides of the vehicle, respectively, in a wired or wireless manner. In addition, the image processing unit 400 receives the surround view image from the AVM processing unit 300 and transmits the reproduced surrounding view image to the display unit 500.

The display unit 500 is installed inside the vehicle, and is generally a device for visually displaying information. The display unit 500 may include various display devices provided in the vehicle. For example, the display unit 500 may be a navigation device, a HUD display device, a display device mounted on an inside mirror or an outside mirror, an AV system, or the like.

The display unit 500 may display an image of a front side of the vehicle photographed by the image photographing unit 100 when a display for a front side image of the vehicle is required. This is not limited to the front direction, and the rear, left, and right images can also be displayed. The display unit 500 may be provided with a plurality of display units for displaying various parts. For example, the display unit 500 may divide and display a screen in one screen, or may display a plurality of display units 500, respectively. The display unit 500 is connected to the image processing unit 400 in a wired or wireless manner. In addition, the display unit 500 receives the reproduced surrounding view image from the image processing unit 400, and displays the reproduced surrounding view image so that the user can view the image.

FIG. 2A is a view for explaining an overview view image in the case where there is no obstacle around the vehicle in a vehicle provided with an overview system for displaying an outline of the object of FIG.

Referring to FIG. 2A, in the surround view system 1 for displaying an outline of an object according to an embodiment of the present invention, the surround view image is an image corresponding to a top view (view from the top of the vehicle) Are divided into a front area A1, a left area A2, a right area A3, and a rear area A4. The image transmitted from the image capturing unit 100 for photographing each area is mapped to each area and the boundary line of each area is aligned. As a result, when the vehicle is viewed from above, it becomes possible to grasp what objects are present around the vehicle. For example, in the center of the surround view image, a virtual vehicle is viewed from the top. On the basis of the vehicle, the image of the front of the vehicle is shown on the upper side, the image of the rear side of the vehicle is shown on the lower side, The right image of the vehicle is mapped and displayed. However, the present invention is not limited thereto, and various embodiments using images captured by a plurality of image capturing units 100 may be implemented. 2A shows an overview image of a vehicle displayed basically when there is no object or obstacle around the vehicle.

FIG. 2B is a view for explaining an overview view image in the case where an obstacle exists around the vehicle in a vehicle provided with an overview system for displaying an outline of the object of FIG.

Referring to FIG. 2B, in the surround view system 1 for displaying an outline of an object according to an embodiment of the present invention, when an object is present around the vehicle, an image of the object is displayed on the surround view image . This allows the user to determine the distance between the object and the vehicle.

However, when there is noise or excessive noise is mixed in the image obtained by the image capturing unit 100, the quality of the image is lowered and it may be difficult to identify the object. In this case, even if the surround view image is generated, it is difficult to accurately grasp the position of the object by only the surrounding view image. If a part of the plurality of image capturing units 100 operates abnormally, that is, if a failure occurs, the front A1, left A2, right A3, rear (A4) Some of the screens may not be displayed. In order to solve this problem, the AVM processing unit 300 generates an overview image by combining the images photographed by the plurality of image capturing units 100, and then transmits the generated surround view image to the image processing unit 400, ) Regenerates the received overview image. Through this process, an outline can be displayed on the surround view image so that the outline of the object can be distinguished. However, only when the object exists on the surround view image, the surround view image including the image corresponding to the outline of the object is regenerated. Otherwise, the surround view image is displayed on the display unit 500 without regenerating.

When an object exists on the surround view image, the surround view image can be regenerated including an image corresponding to the contour of the object according to a setting value pre-stored at the time of shipment or selected by the user. For example, the image corresponding to the contour of the object is overlapped and reproduced (first mode), or only when a part of the plurality of image capturing units 100 is broken (second mode) (Third mode). However, the present invention is not limited to these examples, and various operation modes may be included. Accordingly, the user can select a mode for regenerating the surround view image through the user interface, and can determine whether to reproduce the surround view image including the image corresponding to the outline of the object.

Referring to FIG. 2B, since an object is present on the surround view image generated by the AVM processing unit 300, the surround view image is transmitted to the image processing unit 400. FIG. FIG. 2B is an example of the case where the image is set to be reproduced at all times, and reproduces the surround view image that overlaps the image corresponding to the outline of the object existing in all directions of the vehicle. Therefore, the image corresponding to the outline of the object 01 located on the front side of the vehicle, the object 03 located on the rear side, the object 02 located on the left side, and the object 04 located on the right side overlaps on the surrounding view image, do. The regenerated image is transmitted to the display unit 500, and the display unit 500 displays the received overview image.

The image corresponding to the contour of the object is calculated and generated by the image processing unit 400 on the basis of data corresponding to the distance between the vehicle and the object transmitted from the plurality of ultrasonic sensor units 200. That is, when at least one ultrasonic sensor unit 200 located at the front, rear, left, and right sides of the vehicle sends the distance between the vehicle and the object to the image processing unit 400, And the outline of the object can be displayed and displayed on the surrounding view image.

FIG. 3 is a diagram for explaining how an image corresponding to an outline of an object on the photographing area is overlapped when a part of the image photographing unit is broken in a vehicle provided with an overview system for displaying an outline of the object of FIG. 1;

Referring to FIG. 3, in an overview system 1 for displaying an outline of an object according to an exemplary embodiment of the present invention, FIG. 3 illustrates an example of the surround view system 1 in which an overview image is reproduced only when a part of a plurality of image capturing units 100 fails. The case where the image capturing unit 100 on the left side of the vehicle operates abnormally is shown as an embodiment. Therefore, in this case, the left side portion of the surround view image does not display an image or an image of low quality is displayed. In this case, since the object exists on the surround-view image and a part of the plurality of image-capturing units 100 is broken, the image-processing unit 400 detects the distance between the vehicle and the object sent from the plurality of ultrasonic sensor units 200 And generates an image corresponding to the outline of the object based on the distance. Then, the generated image is overlapped with the surround view image to reproduce the surrounding view image. The reproduced surrounding view image may be displayed on the display unit 500 and may display an image corresponding to an outline of an object existing in the region of the image capturing unit 100 that operates abnormally to the user. However, in this case, the image corresponding to the outline of the object does not overlap the area corresponding to the normal image capturing unit 100. [ However, the present invention is not limited to this, and it is possible to set the outline of the object to appear in all the areas even if a part of the plurality of image capturing units 100 fails.

4 is a flowchart for explaining a method of providing an overview of a vehicle for displaying an outline of an object according to an embodiment of the present invention.

The AVM processing unit 300 receives the images of the front, rear, left, and right sides of the vehicle from the plurality of image capturing units 100 (S500). Then, the image processing unit 400 receives the measured data of distances from the plurality of ultrasonic sensor units 200 to the objects located at the front, rear, left, and right sides of the vehicle (S510). Then, the AVM processing unit 300 combines the images received by the plurality of image capturing units 100 to generate an ambient view image, and transmits the generated surround view image to the image processing unit 400 (S520). Then, the image processing unit 400 determines whether there is an object in the received surrounding view image (S530). If there is no object in the surround view image, the surround view image is displayed on the display unit 500 without being reproduced (S560). On the other hand, if there is an object in the surrounding view image, it is checked whether an overview view image regeneration option is set in the image processing unit 400 (S540). The option allows to selectively reproduce the surround view image according to user input. Therefore, the image corresponding to the outline of the object can always be reproduced by overlapping (first mode), or reproduced only when a part of the plurality of image capturing units 100 fails (second mode), and not always reproduced Third mode). In the first mode or the second mode, the image processing unit 400 determines the outline of the object based on data corresponding to the distance between the vehicle and the object transmitted from the plurality of ultrasonic sensor units 200, The image corresponding to the outline of the object is overlapped with the image of the object to reproduce the surrounding view image (S550). Then, the regenerated surround view image is displayed on the display unit 500 (S560). On the other hand, in the third mode, the surround view image is displayed on the display unit 500 without being reproduced (S560).

The steps of a method or algorithm described in connection with the embodiments of the invention may be embodied directly in hardware, software modules, or a combination of the two, executed by a processor. A software module may reside in RAM memory, flash memory, ROM memory, EPROM memory, EEPROM memory, registers, a hard disk, a removable disk, a CD-ROM, or any form of computer readable recording medium known in the art Lt; / RTI > An exemplary recording medium is coupled to a processor, which is capable of reading information from, and writing information to, the recording medium. Alternatively, the recording medium may be integral with the processor. The processor and the recording medium may reside in an application specific integrated circuit (ASIC). The ASIC may reside within the user terminal. Alternatively, the processor and the recording medium may reside as discrete components in a user terminal.

While the present invention has been described in connection with what is presently considered to be practical exemplary embodiments, it is to be understood that the invention is not limited to the disclosed embodiments, but, on the contrary, You will understand. It is therefore to be understood that the above-described embodiments are illustrative in all aspects and not restrictive.

100: image capturing unit 200: ultrasonic sensor unit
300: AVM processing unit 400:

Claims (10)

A plurality of image capturing units for capturing the front, rear, left, and right sides of the vehicle, respectively;
A plurality of ultrasonic sensor units for measuring distances to objects located at the front, rear, left, and right sides of the vehicle;
An AVM (Around View Monitor) processing unit for combining the images photographed by the plurality of image capturing units and generating an overview image; And
And an image processing unit for determining an outline of the object by using the distance to the measured object and for reproducing an overview view image by overlapping the image corresponding to the outline of the object on the surround view image,
Wherein the image processing unit regenerates the surround view image only when the quality of an image captured by a part of the plurality of image capturing units is lower than a reference value. The method according to claim 1,
Wherein the image processing unit does not regenerate the surround view image when the object does not exist. delete delete The method according to claim 1,
Wherein the image processing unit selectively regenerates the surround view image according to a user input. The method according to claim 1,
And a display unit for displaying the regenerated surround view image. Receiving respective images photographed from a plurality of image capturing units;
Receiving a distance between the vehicle and the object measured from the plurality of ultrasonic sensor units;
Generating an overview view image by combining the received images in an AVM processing unit;
Determining an outline of the object using the distance to the measured object in the image processing unit, and reproducing the surround view image by overlapping the image corresponding to the outline of the object on the surround view image; And
And displaying the regenerated surround view image,
Wherein the regenerating the surround view image regenerates the surround view image only when the quality of the image captured by a part of the plurality of image capturing units is lower than a reference value. 8. The method of claim 7,
Wherein the step of regenerating the surround view image does not regenerate the surround view image when the object does not exist. delete 8. The method of claim 7,
Wherein the step of regenerating the surround view image selectively regenerates the surround view image according to a user input.

Images