KR101822896B1 - Driver assistance apparatus and control method for the same - Google Patents

Abstract

The present invention relates to a driver assistance apparatus and a control method thereof, and more particularly, to a driver assistance apparatus and a control method thereof, which are provided with at least one camera for photographing a periphery of a vehicle and generating a main image, A communication unit for receiving a plurality of sub images generated by at least one other vehicle; And a processor for selecting at least one of the plurality of sub images based on the predetermined condition or user input and generating an extended image using the main image and the selected sub image, / RTI >

Description

[0001] DRIVER ASSISTANCE APPARATUS AND CONTROL METHOD FOR THE SAME [0002]

BACKGROUND OF THE INVENTION 1. Field of the Invention [0002] The present invention relates to a driver assistance apparatus and a control method thereof, and more particularly, to a driver assistance apparatus and a control method thereof for providing an image of a region invisible to a driver of a vehicle.

A vehicle is a device that drives a wheel to transport a person or cargo from one place to another. For example, two-wheeled vehicles such as a motorcycle, a four-wheeled vehicle such as a sedan, as well as a train belong to the vehicle.

In recent years, in order to increase the safety and convenience of a user who uses a vehicle, development of a technique for connecting various sensors and electronic devices to a vehicle has been accelerated. In particular, various devices for the user's driving convenience have been developed.

An AVM (Around View Monitoring) system, which is one of them, is a system in which a plurality of images are generated by photographing the vicinity of 360 degrees of a vehicle using a plurality of cameras, and a plurality of generated images are synthesized, Displays the view screen.

The driver can see the surround view screen and can help to operate the vehicle. However, since the space available on the surround view screen is very limited, there is a limitation in that it is used only in some situations such as parking and slow running (less than about 20 km / h).

Accordingly, there is a need for a technique capable of providing an image of a wider area than a region which can be photographed by a camera mounted on a vehicle, using an image generated by another vehicle.

The present invention relates to a driver assistance device that generates an extended image using a running image generated by different vehicles and provides an image of an area that can not be photographed by a camera mounted on a vehicle that the driver has boarded, Method.

The problems of the present invention are not limited to the above-mentioned problems, and other problems not mentioned can be clearly understood by those skilled in the art from the following description.

According to an aspect of the present invention, there is provided an image processing apparatus including at least one camera for photographing a periphery of a vehicle to generate a main image; A communication unit for receiving a plurality of sub images generated by at least one other vehicle; And a processor for selecting at least one of the plurality of sub images based on the predetermined condition or user input and generating an extended image using the main image and the selected sub image, / RTI > The details of other embodiments are included in the detailed description and drawings.

Effects of the driver assistance device and the control method thereof according to the present invention will be described as follows.

According to at least one of the embodiments of the present invention, by using the image generated by the camera mounted on the vehicle and the image generated by the camera mounted on the other vehicle, It is possible to provide an image (an extended image to be described later).

In addition, according to at least one embodiment of the present invention, when an environment satisfies a predetermined condition, an extended image is automatically generated, thereby improving the convenience of the driver.

In addition, according to at least one embodiment of the present invention, when an extended image is displayed, information related to an obstacle is provided so that safety of the driver can be improved.

In addition, according to at least one embodiment of the present invention, an image of an invisible region is provided to a driver who rides on the vehicle, thereby helping the driver to adjust the vehicle's operation schedule.

According to at least one of the embodiments of the present invention, it is possible to shorten the time required to reach the destination by receiving the information reflecting the real-time road situation by calculating the route that the vehicle can travel on the basis of the extended image.

The effects of the present invention are not limited to the effects mentioned above, and other effects not mentioned can be clearly understood by those skilled in the art from the description of the claims.

1 is a view showing an appearance of a vehicle provided with a driver assistance device according to an embodiment of the present invention.
2A to 2C are views referred to explain a camera attached to the vehicle of FIG. 1 according to an embodiment of the present invention.
Figures 3A-3C illustrate various examples of internal block diagrams of a driver assistance device in accordance with various embodiments of the present invention.
Figures 4A-4B illustrate various examples of internal block diagrams of the processors of Figures 3A-3B.
Figures 5A-5B are views referenced in the operational description of the processors of Figures 4A-4B.
6A and 6B are views referred to in the description of the operation of the driver assistance apparatus of FIGS. 3A to 3C.
Fig. 7 is an example of an internal block diagram of the vehicle of Fig. 1. Fig.
8 is a flowchart illustrating a method of controlling the driver assistance device according to an embodiment of the present invention.
9A to 9D are views for explaining the operation of the driver assistance device according to an embodiment of the present invention.
10A to 10D are views for explaining the operation of the driver assistance device according to an embodiment of the present invention.
11A to 11D are views for explaining the operation of the driver assistance device according to an embodiment of the present invention.
12A to 12D are views for explaining the operation of the driver assistance device according to an embodiment of the present invention.
13A to 13E are views for explaining the operation of the driver assistance device according to an embodiment of the present invention.
14A to 14E are views for explaining the operation of the driver assistance device according to an embodiment of the present invention.
15A to 15C are views for explaining the operation of the driver assistance device according to an embodiment of the present invention.
16A to 16D are views for explaining the operation of the driver assistance device according to an embodiment of the present invention.
17A and 17B are views for explaining the operation of the driver assistance device according to an embodiment of the present invention.
18A and 18B are views for explaining the operation of the driver assistance device according to an embodiment of the present invention.
19 is a view for explaining the operation of the driver assistance device according to an embodiment of the present invention.
20A and 20B are views for explaining the operation of the driver assistance device according to an embodiment of the present invention.
21 is a view for explaining the operation of the driver assistance device according to an embodiment of the present invention.
22A and 22B are views for explaining the operation of the driver assistance device for controlling the display state of the extended image according to an embodiment of the present invention.
23 is a flowchart showing a method of controlling the driver assistance device according to an embodiment of the present invention.

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings, wherein like reference numerals are used to designate identical or similar elements, and redundant description thereof will be omitted. The suffix "module" and " part "for the components used in the following description are given or mixed in consideration of ease of specification, and do not have their own meaning or role. In the following description of the embodiments of the present invention, a detailed description of related arts will be omitted when it is determined that the gist of the embodiments disclosed herein may be blurred. It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory and are intended to provide further explanation of the invention as claimed. , ≪ / RTI > equivalents, and alternatives.

Terms including ordinals, such as first, second, etc., may be used to describe various elements, but the elements are not limited to these terms. The terms are used only for the purpose of distinguishing one component from another.

It is to be understood that when an element is referred to as being "connected" or "connected" to another element, it may be directly connected or connected to the other element, . On the other hand, when an element is referred to as being "directly connected" or "directly connected" to another element, it should be understood that there are no other elements in between. It should also be understood that the term "controlling" one component is meant to encompass not only one component directly controlling the other component, but also controlling through mediation of a third component something to do. It is also to be understood that any element "providing" information or signals to another element is meant to encompass not only providing the element directly to the other element, but also providing it through intermediation of a third element .

The singular expressions include plural expressions unless the context clearly dictates otherwise.

In the present application, the terms "comprises", "having", and the like are used to specify that a feature, a number, a step, an operation, an element, a component, But do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, or combinations thereof.

The vehicle described in the present specification may be a concept including both an internal combustion engine vehicle having an engine as a power source, a hybrid vehicle having an engine and an electric motor as a power source, and an electric vehicle having an electric motor as a power source.

1 is a view showing the appearance of a vehicle 1 according to an embodiment of the present invention. For convenience of explanation, it is assumed that the vehicle 1 is a four-wheeled vehicle.

Referring to the drawings, a vehicle 1 includes a tire 11a-11d rotated by a power source, a steering wheel 12 for adjusting the traveling direction of the vehicle 1, head lamps 13a and 13b, wipers 14a , 14b, a driver assistance device 100 to be described later, and the like.

The driver assistance device 100 according to the embodiment of the present invention generates a peripheral image of a vehicle, detects information in the generated peripheral image, and controls the traveling direction of the vehicle 1 or the like based on the detected information A signal can be output. At this time, the control signal may be provided to the control unit (770 of FIG. 7), and the control unit (770 of FIG. 7) may control the steering unit and the like based on the control signal.

The driver assistance device 100 may include at least one camera, and the image obtained by the at least one camera may be signal processed within the processor (170 of Figures 3A-B). For example, as shown, the camera 195 may be mounted on the windshield top of the vehicle 1 and take a picture of the front of the vehicle.

On the other hand, the distance between the lowest point of the vehicle body of the vehicle 1 and the road surface can be separated by the minimum ground clearance G. [ Thus, the vehicle body can be prevented from being damaged by an object having a height lower than the minimum ground clearance G.

It is also assumed that the distance between the front left and right tires 11a and 11b of the vehicle 1 and the distance between the rear left and right tires 11c and 11d are the same. It is assumed that the distance between the inside of the front wheel left tire 11a and the inside of the right tire 11b and the distance between the inside of the rear left tire 11c and the inside of the right tire 11d are the same value T do.

The total width O of the vehicle 1 is defined as the maximum distance between the left end of the vehicle 1 and the right end of the vehicle 1 except for the side mirrors.

Meanwhile, the vehicle 1 shown in Fig. 1 may include a driver assistance device 100 to be described later.

2A to 2C are views referred to explain a camera attached to the vehicle 1 of Fig. 1 according to an embodiment of the present invention.

Referring to Fig. 2A, the driver assistance device 100 including the cameras 195a and 195b for acquiring images of the front of the vehicle 1 will be described.

Although FIG. 2A illustrates the driver assistance device 100 as including two cameras 195a and 195b, the present invention specifies that the present invention is not limited to the number of cameras.

Referring to the drawings, the driver assistance apparatus 100 may include a first camera 195a having a first lens 193a, and a second camera 195b having a second lens 193b. In this case, the camera 195 may be referred to as a stereo camera.

The driver assistance apparatus 100 includes a first light shield 192a and a second light shield 192b for shielding light incident on the first lens 193a and the second lens 193b, And a portion 192b.

The driver assist device 100 in the drawing may be a structure that can be detachably attached to the interior or exterior of the vehicle 1 (e.g., ceiling or windshield).

Such driver assistance device 100 may obtain a stereo image of the vehicle ahead from the first and second cameras 195a and 195b. Also, based on the stereo image, disparity detection may be performed and object detection may be performed on the at least one stereo image based on the disparity information. After the object is detected, the movement of the object can be continuously tracked.

The driver assistance apparatus 100 including the cameras 195, 196, 197 and 198 for acquiring the vehicle surroundings image will be described with reference to FIGS. 2B and 2C. FIG.

Although the driver assistance device 100 is shown as including four cameras in Figures 2B and 2C, the present invention is not limited to the number of cameras.

Referring to the drawings, the driver assistance device 100 may include a plurality of cameras 195, 196, 197, and 198. In this case, the plurality of cameras 195, 196, 197, and 198 may be referred to as an arousal view camera.

A plurality of cameras 195, 196, 197, and 198 may be disposed at the front, left, right, and rear of the vehicle 1, respectively.

The left camera 196 may be disposed in a case surrounding the left side mirror. Alternatively, the left camera 196 may be disposed outside the case surrounding the left side mirror. Alternatively, the left camera 196 may be disposed in a region outside the left front door, the left rear door, or the left fender.

The right camera 197 may be disposed in a case surrounding the right side mirror. Or the right camera 197 may be disposed outside the case surrounding the right side mirror. Alternatively, the right camera 197 may be disposed in one area outside the right front door, the right rear door, or the right fender.

On the other hand, the rear camera 198 may be disposed in the vicinity of a rear license plate or a trunk switch.

The front camera 195 may be disposed near the windshield, near the ambulance, or near the radiator grill.

Each image photographed by the plurality of cameras 195-198 is transmitted to the processor 170, and the processor 170 may synthesize the respective images to generate a vehicle periphery image.

2C shows an example of a vehicle surroundings image. The vehicle surroundings image 201 includes a first image area 196i photographed by the left camera 196, a second image area 198i photographed by the rear camera 198, A third image region 197i and a fourth image region 195i imaged by the front camera 195. [

On the other hand, when a surround view image is generated from a plurality of cameras, a boundary portion between each image area occurs. These boundary portions can be naturally displayed by image blending processing.

On the other hand, the boundaries 202a, 202b, 202c, and 202d may be displayed at the boundaries of the plurality of images. In addition, a vehicle image may be included in the center of the vehicle periphery image 201. [ Where the vehicle image may be an image generated by the processor 170. In addition, the vehicle peripheral image 201 can be displayed through the display portion 741 of the vehicle 1 or the display portion 180 of the driver assistance device.

Figures 3A-3C illustrate various examples of a driver assistance device 100 block diagram in accordance with various embodiments of the present invention.

The driver assistance device 100 of FIGS. 3A and 3B can generate vehicle-related information by signal processing an image received from the camera 195 based on computer vision. The vehicle-related information may include vehicle-control information for direct control of the vehicle, or vehicle-driving assistance information for a driver's guide to the vehicle driver.

Here, the camera 195 may be a monaural camera. Alternatively, the camera 195 may be a stereo camera 195a, 195b that photographs the vehicle front image. Alternatively, the camera 195 may be included in an ambient view camera 195-198 for photographing the surroundings of the vehicle.

3A is an internal block diagram of the driver assistance apparatus 100 according to the embodiment of the present invention.

3A, the driver assistance apparatus 100 of FIG. 3A includes an input unit 110, a communication unit 120, an interface unit 130, a memory 140, a processor 170, a power supply unit 190, a camera 195, a display unit 180, an audio output unit 185, and the like.

The input unit 110 receives various inputs from the driver. For example, the input unit 110 may include a plurality of buttons or touch screens attached to the cameras 195-198. It is possible for the driver to turn on and operate the driver's auxiliary apparatus 100 via a plurality of buttons or a touch screen. In addition, it is also possible to perform various input operations.

The communication unit 120 can exchange data with external devices such as the mobile terminal 600, the server 500, and other vehicles in a wireless manner. In particular, the communication unit 120 can exchange data wirelessly with the mobile terminal 600 of the vehicle driver. Various data communication methods such as Bluetooth, WiFi Direct, WiFi, APiX, and NFC are available for wireless data communication.

The communication unit 120 can receive weather information and traffic situation information of the road, for example, TPEG (Transport Protocol Expert Group) information from the mobile terminal 600 or the server 500. Meanwhile, the driver assistance device 100 may transmit the detected real time information to the mobile terminal 600 or the server 500.

On the other hand, when the user is boarding the vehicle, the user's mobile terminal 600 and the driver assistance device 100 can perform pairing with each other automatically or by executing the application of the user.

The communication unit 120 can receive the traffic light change information from the external server 510. [ Here, the external server 510 may be a server located in a traffic control station that controls traffic.

The interface unit 130 may receive the vehicle-related data or may transmit the processed or generated signal to the outside by the processor 170. To this end, the interface unit 130 can perform data communication with a control unit 770, an AVN (Audio Video Navigation) device 400, a sensing unit 760, and the like in the vehicle by a wired communication or a wireless communication method have.

The interface unit 130 can receive the navigation information by the data communication with the control unit 770, the AVN apparatus 400, or another navigation apparatus. Here, the navigation information may include set destination information, route information according to the destination, map information related to driving the vehicle, and current position information of the vehicle. On the other hand, the navigation information may include position information of the vehicle on the road.

Meanwhile, the interface unit 130 may receive the sensor information from the control unit 770 or the sensing unit 760.

Here, the sensor information includes at least one of vehicle direction information, vehicle position information (GPS information), vehicle angle information, vehicle speed information, vehicle acceleration information, vehicle tilt information, vehicle forward / backward information, battery information, fuel information, , Vehicle internal temperature information, vehicle internal humidity information, and object information.

Such sensor information may include a heading sensor, a yaw sensor, a gyro sensor, a position module, a vehicle forward / backward sensor, a wheel sensor, a vehicle speed sensor, (E.g., radar, lidar, ultrasonic sensor, etc.), such as a vehicle body inclination sensor, a battery sensor, a fuel sensor, a tire sensor, a steering sensor by steering wheel rotation, have. On the other hand, the position module may include a GPS module for receiving GPS information.

On the other hand, among the sensor information, the vehicle direction information, the vehicle position information, the vehicle angle information, the vehicle speed information, the vehicle tilt information, and the like relating to the vehicle running can be referred to as vehicle running information.

The interface unit 130 may receive the turn signal information. Here, the turn signal information may be a turn-on signal of the turn signal lamp for the left turn or the turn right turn inputted by the user. When the left or right turn signal turn-on input is received through the user input part (724 in FIG. 7) of the vehicle, the interface part 130 may receive left turn signal information or right turn signal information.

The interface unit 130 may receive vehicle speed information, rotation angle information of the steering wheel, or gear shift information. The interface unit 130 may receive the sensed vehicle speed information, the steering wheel rotation angle information, or the gear shift information through the sensing unit 760 of the vehicle. Alternatively, the interface unit 130 may receive vehicle speed information, steering wheel rotation angle information, or gear shift information from the control unit 770 of the vehicle. Here, the gear shift information may be information on which state the shift lever of the vehicle is in. For example, the gear shift information may be information on which state the shift lever is in the parking (P), reverse (R), neutral (N), running (D) .

The interface unit 130 may receive a user input received through the user input unit 724 of the vehicle 1. [ The interface unit 130 may receive the user input from the input unit 720 of the vehicle 1 or via the control unit 770.

The interface unit 130 may receive the information obtained from the external server 510. [ The external server 510 may be a server located in a traffic control station that controls traffic. For example, when the traffic light change information is received from the external server 510 through the communication unit 710 of the vehicle, the interface unit 130 may receive the traffic light change information from the control unit (770 in FIG. 7). The memory 140 may store various data for operation of the entire driver assistance device 100, such as a program for processing or controlling the processor 170. [

The memory 140 may store data for object identification. For example, the memory 140 may store data for confirming what the object corresponds to, by a predetermined algorithm, when a predetermined object is detected in the image obtained through the camera 195. For example,

The memory 140 may store data on traffic information. For example, when predetermined traffic information is detected in an image obtained through the camera 195, the memory 140 may store data for checking what the traffic information corresponds to according to a predetermined algorithm .

Meanwhile, the memory 140 may be various storage devices such as a ROM, a RAM, an EPROM, a flash drive, a hard drive, and the like in hardware.

The processor 170 controls the overall operation of each unit in the driver assistance device 100.

The processor 170 may process the vehicle front image or the vehicle periphery image obtained by the camera 195. [ In particular, the processor 170 performs signal processing based on computer vision. Accordingly, the processor 170 can acquire images from the camera 195 in front of or around the vehicle, and can perform object detection and object tracking based on the images. Particularly, when detecting an object, the processor 170 may detect lane detection (LD), vehicle detection (VD), pedestrian detection (PD), light detection (Brightspot Detection) Traffic sign recognition (TSR), road surface detection, and the like.

On the other hand, the traffic signal may mean predetermined information that can be transmitted to the driver of the vehicle 1. Traffic signals can be delivered to the driver through a traffic light, traffic sign, or road surface. For example, the traffic signal may be a Go or Stop signal of a vehicle or pedestrian output from a traffic light. For example, the traffic signal may be various designs or texts displayed on a traffic sign. For example, traffic signals may be various designs or texts displayed on the road surface.

The processor 170 may detect information in the vehicle surroundings image generated by the camera 195. [

The information may be information on the driving situation of the vehicle. For example, the information may be a concept including road information, traffic regulation information, surrounding vehicle information, vehicle or pedestrian signal information, construction information, traffic situation information, parking lot information, lane information, etc., which the vehicle travels.

The information may be traffic information. The processor 170 may detect traffic information from any one of a traffic light, a traffic sign, and a road surface included in the image obtained by the camera 195. [ For example, the processor 170 may detect a Go or a Stop signal of a vehicle or a pedestrian from a signal light included in the image. For example, the processor 170 may detect various patterns or texts from traffic signs included in the image. For example, the processor 170 may detect various patterns or texts from the road surface included in the image.

The processor 170 may compare the detected information with the information stored in the memory 140 to verify the information.

For example, the processor 170 detects a graphic or text indicating a rampway in an object included in the acquired image. Here, the object may be a traffic sign or a road surface. Pattern or text. The processor 170 may compare the traffic information stored in the memory 140 with the detected pattern or text to confirm the lampway information.

For example, the processor 170 detects a graphic or text indicating a vehicle or a pedestrian stop in an object included in the acquired image. Here, the object may be a traffic sign or a road surface. The processor 170 may compare the traffic information stored in the memory 140 with the detected pattern or text to check the stop information. Alternatively, the processor 170 detects a stop line from the road surface included in the acquired image. The processor 170 may compare the traffic information stored in the memory 140 with the stop line to confirm the stop information.

For example, the processor 170 can detect the presence or absence of a lane in an object included in the acquired image. Here, the object may be a road surface. The processor 170 can check the color of the detected lane. The processor 170 can confirm whether the detected lane is a driving lane or a waiting lane.

For example, the processor 170 may detect the Go or Stop information of the vehicle from the object included in the acquired image. Here, the object may be a vehicle traffic light. Here, the Go information of the vehicle may be a signal instructing the vehicle to go straight, turn left or right. The stop information of the vehicle may be a signal instructing the vehicle to stop. The Go information of the vehicle may be displayed in green, and the Stop information of the vehicle may be displayed in red.

For example, the processor 170 may detect the Go or Stop information of the pedestrian from the object included in the acquired image. Here, the object may be a pedestrian signal or the like. Here, the Go information of the pedestrian may be a signal instructing the pedestrian to cross the lane in the pedestrian crossing. The stop information of the pedestrian may be a signal instructing the pedestrian to stop in the pedestrian crossing.

On the other hand, the processor 170 can control the zoom of the camera 195. For example, the processor 170 may control the zoom of the camera 195 according to the object detection result. For example, if a traffic sign is detected but the contents displayed on the traffic sign are not detected, the processor 170 may control the camera 195 to zoom in.

Meanwhile, the processor 170 can receive weather information, traffic situation information on the road, and TPEG (Transport Protocol Expert Group) information, for example, through the communication unit 120.

Meanwhile, the processor 170 may grasp, in real time, the traffic situation information about the vehicle, which is based on the stereo image, in the driver assistance device 100. [

The processor 170 may receive navigation information or the like from the AVN apparatus 400 or a separate navigation apparatus (not shown) through the interface unit 130. [

The processor 170 may receive the sensor information from the control unit 770 or the sensing unit 760 through the interface unit 130. [ Here, the sensor information includes at least one of vehicle direction information, vehicle position information (GPS information), vehicle angle information, vehicle speed information, vehicle acceleration information, vehicle tilt information, vehicle forward / backward information, battery information, fuel information, Lamp information, vehicle interior temperature information, vehicle interior humidity information, and steering wheel rotation information.

Meanwhile, the processor 170 may receive navigation information from the control unit 770, the AVN apparatus 400, or a separate navigation device (not shown) via the interface unit 130. [

The processor 170 may be implemented as an application specific integrated circuit (ASIC), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs) May be implemented using at least one of controllers, micro-controllers, microprocessors, and electrical units for performing other functions.

The processor 170 may be under the control of the controller 770.

The display unit 180 may display various kinds of information processed by the processor 170. The display unit 180 may display an image related to the operation of the driver assistance device 100. For this image display, the display unit 180 may include a cluster or an HUD (Head Up Display) on the inside of the vehicle interior. On the other hand, when the display unit 180 is the HUD, it may include a projection module that projects an image on the windshield of the vehicle 1. [

The audio output unit 185 can output the sound to the outside based on the audio signal processed by the processor 170. [ To this end, the audio output unit 185 may include at least one speaker.

An audio input unit (not shown) can receive a user voice. For this purpose, a microphone may be provided. The received voice may be converted to an electrical signal by the audio input and transmitted to the processor 170.

The power supply unit 190 can supply power necessary for operation of each component under the control of the processor 170. [ Particularly, the power supply unit 190 can receive power from a battery or the like inside the vehicle.

The camera 195 acquires the vehicle front image or the vehicle periphery image. The camera 195 may be a mono camera or a stereo camera 195a, 195b for shooting the vehicle front image. Alternatively, the camera 195 may be a plurality of cameras 196, 197, 195f, and 195 that photograph the surroundings of the vehicle.

The camera 195 may include an image sensor (e.g., CMOS or CCD) and an image processing module.

The camera 144 may process still images or moving images obtained by the image sensor. The image processing module can process the still image or moving image obtained through the image sensor. Meanwhile, according to the embodiment, the image processing module may be separately configured or integrated with the processor 170. [

The camera 195 may acquire an image of at least one of a traffic light, a traffic sign, and a road surface.

The camera 195 may be set to a zoom according to the control of the processor 170. [ For example, in accordance with the control of the processor 170, a zoom barrel (not shown) included in the camera 195 moves, and zoom can be set.

The camera 195 can be set to focus under the control of the processor 170. [ For example, according to the control of the processor 170, the focus barrel (not shown) included in the camera 195 may be moved and the focus may be set. The focus can be automatically set based on the zoom setting.

On the other hand, the processor 170 can automatically control the focus in response to the zoom control of the camera 195. [

3B is an internal block diagram of the driver assistance apparatus 100 according to another embodiment of the present invention.

Referring to FIG. 3B, the driver assistance device 100 of FIG. 3B differs from the driver assistance device 100 of FIG. 3A in that it includes stereo cameras 195a and 195b. Hereinafter, the differences will be mainly described.

The driver assistance device 100 may include first and second cameras 195a and 195b. Here, the first and second cameras 195a and 195b may be referred to as a stereo camera.

The stereo cameras 195a and 195b may be detachably attached to the ceiling or the windshield of the vehicle 1. [ The stereo cameras 195a and 195b may include a first lens 193a and a second lens 193b.

The stereo cameras 195a and 195b respectively include a first light shield 192a and a second light shield 192b for shielding light incident on the first lens 193a and the second lens 193b, Shielding portion 192b.

The first camera 195a acquires the first image in front of the vehicle. The second camera 195b acquires a second image in front of the vehicle. The second camera 195b is disposed at a predetermined distance from the first camera 195a. By disposing the first and second cameras 195a and 195b apart from each other by a predetermined distance, disparity occurs, and the distance to the object can be detected according to the time difference.

On the other hand, when the driver assistance device 100 includes the stereo cameras 195a and 195b, the processor 170 performs signal processing based on computer vision. Accordingly, the processor 170 acquires a stereo image for the vehicle front from the stereo cameras 195a and 195b, performs a disparity calculation for the front of the vehicle based on the stereo image, and based on the calculated disparity information To perform object detection for at least one of the stereo images, and to continuously track the motion of the object after object detection. Here, the stereo image is based on the first image received from the first camera 195a and the second image received from the second camera 195b.

Particularly, when the object is detected, the processor 170 may detect a lane detection (LD), a vehicle detection (VD), a pedestrian detection (PD), a brilliant spot detection (Traffic Sign Recognition, TSR), road surface detection, and the like.

In addition, the processor 170 may perform a distance calculation to the detected nearby vehicle, a speed calculation of the detected nearby vehicle, a speed difference calculation with respect to the detected nearby vehicle, and the like.

The processor 170 may separately control the zoom of the first and second cameras 195a, 195b. The processor 170 may periodically change the zoom magnification of the second camera 195b while the zoom of the first camera 195a is fixed. The processor 170 may periodically change the zoom magnification of the first camera 195a while the zoom of the second camera 195b is fixed.

The processor 170 may control the first or second camera 195a or 195b to zoom in or out in a predetermined cycle.

The processor 170 may set the zoom of the first camera 195a to a high magnification so as to be advantageous for object detection at a long distance. In addition, the processor 170 may set the zoom of the second camera 195b at a low magnification so as to be advantageous for object detection at a short distance. At this time, the processor 170 controls the first camera 195a to zoom in and controls the second camera 195b to zoom out.

Conversely, the processor 170 may set the zoom of the first camera 195a at a low magnification to favor object detection in the short range. Further, the processor 170 can set the zoom of the second camera 195b at a high magnification so as to be advantageous for object detection at a long distance. At this time, the processor 170 controls the first camera 195a to zoom in and the second camera 195b to zoom in.

For example, the processor 170 may control the zoom of the first camera 195a or the second camera 195b according to the object detection result. For example, if the traffic sign is detected but the contents displayed on the traffic sign are not detected, the processor 170 may control the first camera 195a or the second camera 195b to zoom in.

On the other hand, the processor 170 can automatically control the focus in response to the zoom control of the camera 195. [

3C is an internal block diagram of the driver assistance device 100 according to another embodiment of the present invention.

The driver assistance device 100 of FIG. 3C differs from the driver assistance device 100 of FIG. 3A in that it includes an ambient view camera 195-198. Hereinafter, the differences will be mainly described.

The driver assistance device 100 may include an ambient view camera 195-198.

The surround view cameras 195-198 may each include a lens and a light shield for shielding light incident on the lens.

The surround view camera may include a left camera 196, a rear camera 198, a right camera 197, and a front camera 195.

The front camera 195 acquires the vehicle front image. The left camera 196 acquires the vehicle left side image. The right camera 197 acquires the right-side room image of the vehicle. The rear camera 198 acquires the vehicle rear image.

The respective images obtained in the surrounding view cameras 195-198 are transmitted to the processor 170. [

The processor 170 may synthesize a left side image, a rear image, a right image, and a forward image of the vehicle to generate a vehicle periphery image. At this time, the vehicle surroundings image may be a top view or a bird eye view image. The processor 170 may receive a left side image, a rear image, a right image, and a forward image of the vehicle, synthesize the received images, and convert the synthesized image into a top view image to generate a vehicle periphery image.

On the other hand, the processor 170 can detect the object based on the vehicle surroundings image. Particularly, when the object is detected, the processor 170 may detect a lane detection (LD), a vehicle detection (VD), a pedestrian detection (PD), a brilliant spot detection (Traffic Sign Recognition, TSR), road surface detection, and the like.

On the other hand, the processor 170 can individually control the zoom of the surrounding view cameras 195-198. The zoom control of the processor 170 can be operated in the same manner as in the case of the stereo camera described with reference to Fig. 3B.

However, some of the components shown in Figs. 3A to 3C may not be essential in implementing the driver assistance device 100. [ Thus, the driver assistance device 100 described herein may have more or fewer components than those listed above. For example, the driver assistance device 100 may include only the processor 170 and the camera 195.

Figures 4A-4B illustrate various examples of internal block diagrams of the processors of Figures 3A-3B, and Figures 5A-B are views referenced in the operational description of the processors of Figures 4A-4B.

4A is an internal block diagram of the processor 170. The processor 170 in the driver assistance apparatus 100 includes an image preprocessing unit 410, a disparity computing unit 420, An object detecting unit 434, an object tracking unit 440, and an application unit 450.

An image preprocessor 410 may receive an image from the camera 195 and perform preprocessing.

In detail, the image preprocessing unit 410 may perform a noise reduction, a rectification, a calibration, a color enhancement, a color space conversion (CSC) Interpolation, camera gain control, and the like. Thus, a clearer image can be obtained than the stereo image photographed by the camera 195.

The disparity calculator 420 receives an image signal processed by the image preprocessing unit 410, performs stereo matching on the received images, and performs disparity calculation based on stereo matching, A disparty map can be obtained. That is, it is possible to obtain the disparity information about the stereo image with respect to the front of the vehicle.

At this time, the stereo matching may be performed on a pixel-by-pixel basis of stereo images or on a predetermined block basis. On the other hand, the disparity map may mean a map in which binaural parallax information of stereo images, i.e., left and right images, is numerically expressed.

The segmentation unit 432 may perform segmentation and clustering on at least one of the images based on the dispetity information from the disparity calculation unit 420.

Specifically, the segmentation unit 432 can separate the background and the foreground for at least one of the stereo images based on the disparity information.

For example, an area having dispaly information within a disparity map of a predetermined value or less can be calculated as a background, and the corresponding part can be excluded. Thereby, the foreground can be relatively separated.

As another example, an area in which the dispetity information is equal to or greater than a predetermined value in the disparity map can be calculated with the foreground, and the corresponding part can be extracted. Thereby, the foreground can be separated.

Thus, by separating the foreground and the background based on the disparity information information extracted based on the stereo image, it becomes possible to shorten the signal processing speed, signal processing amount, and the like at the time of object detection thereafter.

Next, the object detector 434 can detect the object based on the image segment from the segmentation unit 432. [

That is, the object detecting unit 434 can detect the object for at least one of the images based on the disparity information.

Specifically, the object detection unit 434 can detect an object for at least one of the images. For example, an object can be detected from a foreground separated by an image segment.

Next, the object verification unit 436 classifies and verifies the separated object.

For this purpose, the object identification unit 436 identifies the object using a neural network identification method, a SVM (Support Vector Machine) method, an AdaBoost identification method using a Haar-like feature, or a Histograms of Oriented Gradients Etc. may be used.

On the other hand, the object checking unit 436 can check the objects by comparing the objects stored in the memory 140 with the detected objects.

For example, the object checking unit 436 can identify nearby vehicles, lanes, roads, signs, hazardous areas, tunnels, etc. located around the vehicle.

The object tracking unit 440 may perform tracking on the identified object. For example, it sequentially identifies an object in the acquired stereo images, calculates a motion or a motion vector of the identified object, and tracks movement of the object based on the calculated motion or motion vector . Accordingly, it is possible to track nearby vehicles, lanes, roads, signs, dangerous areas, tunnels, etc., located in the vicinity of the vehicle.

Next, the application unit 450 can calculate the risk and the like of the vehicle 1 based on various objects located in the vicinity of the vehicle, for example, other vehicles, lanes, roads, signs and the like. It is also possible to calculate the possibility of a collision with a preceding vehicle, whether the vehicle is slipping or the like.

Then, the application unit 450 can output a message or the like for notifying the user to the user as vehicle driving assistance information based on the calculated risk, possibility of collision, sleep, or the like. Alternatively, a control signal for attitude control or running control of the vehicle 1 may be generated as the vehicle control information.

7 (b), the object preprocessing unit 410, the segmentation unit 432, the object detection unit 434, the object verification unit 436, the object tracking unit 440, and the application unit 450, And may be the internal configuration of the image processing unit 810 in the processor 170 below.

The processor 170 includes an image preprocessing unit 410, a disparity computing unit 420, a segmentation unit 432, an object detection unit 434, an object verification unit 436, an object tracking unit 440, and an application unit 450. [0040] For example, when the camera 195 is configured as a mono camera or an arousal view camera, the disparity calculating unit 420 may be omitted. Also, according to the embodiment, the segmentation section 432 may be omitted.

4B is another example of an internal block diagram of the processor.

Referring to the drawings, the processor 170 of FIG. 4B has the same internal structure as the processor 170 of FIG. 4A, but differs in signal processing order. Only the difference will be described below.

The object detection unit 434 receives the stereo image and can detect the object for at least one of the stereo images. 4A, it is possible to detect an object directly from the stereo image, instead of detecting the object, on the segmented image, based on the disparity information.

Next, the object verification unit 436 classifies the detected and separated objects based on the image segment from the segmentation unit 432 and the object detected by the object detection unit 434, Verify.

For this purpose, the object identification unit 436 identifies the object using a neural network identification method, a SVM (Support Vector Machine) method, an AdaBoost identification method using a Haar-like feature, or a Histograms of Oriented Gradients Etc. may be used.

FIGS. 5A and 5B are views for explaining the operation method of the processor 170 of FIG. 4A based on the stereo images obtained respectively in the first and second frame periods.

First, referring to FIG. 5A, during the first frame period, the stereo camera 195 acquires a stereo image.

The disparity calculating unit 420 in the processor 170 receives the stereo images FR1a and FR1b signal-processed by the image preprocessing unit 410 and performs stereo matching on the received stereo images FR1a and FR1b , And obtains a disparity map (520).

The disparity map 520 is obtained by leveling the parallax between the stereo images FR1a and FR1b. The larger the disparity level is, the closer the distance is from the vehicle, and the smaller the disparity level is, The distance can be calculated to be far.

On the other hand, when such a disparity map is displayed, it may be displayed so as to have a higher luminance as the disparity level becomes larger, and a lower luminance as the disparity level becomes smaller.

In the figure, first to fourth lanes 528a, 528b, 528c, and 528d have corresponding disparity levels in the disparity map 520, and the construction area 522, the first front vehicle 524 ) And the second front vehicle 526 have corresponding disparity levels, respectively.

The segmentation unit 432, the object detection unit 434 and the object identification unit 436 determine whether or not a segment, an object detection, and an object of at least one of the stereo images FR1a and FR1b based on the disparity map 520 Perform verification.

In the figure, using the disparity map 520, object detection and confirmation for the second stereo image FRlb is performed.

That is, the first to fourth lanes 538a, 538b, 538c, and 538d, the construction area 532, the first forward vehicle 534, and the second forward vehicle 536 are included in the image 530, And verification may be performed.

Next, referring to FIG. 5B, during the second frame period, the stereo camera 195 acquires a stereo image.

The disparity calculating unit 420 in the processor 170 receives the stereo images FR2a and FR2b signal-processed by the image preprocessing unit 410 and performs stereo matching on the received stereo images FR2a and FR2b , And a disparity map (540).

In the figure, the first to fourth lanes 548a, 548b, 548c, and 548d have corresponding disparity levels in the disparity map 540, and the construction area 542, the first forward vehicle 544 ) And the second front vehicle 546 have corresponding disparity levels, respectively.

The segmentation unit 432, the object detection unit 434 and the object identification unit 436 determine whether or not a segment, an object detection, and an object of at least one of the stereo images FR2a and FR2b, based on the disparity map 540, Perform verification.

In the figure, using the disparity map 540, object detection and confirmation for the second stereo image FR2b is performed.

That is, the first to fourth lanes 558a, 558b, 558c, and 558d, the construction area 552, the first forward vehicle 554, and the second forward vehicle 556 in the image 550, Verification can be performed.

On the other hand, the object tracking unit 440 can compare the FIG. 5A and FIG. 5B and perform tracking on the identified object.

Specifically, the object tracking unit 440 can track the movement of the corresponding object based on the motion or motion vector of each object identified in FIGS. 5A and 5B. Accordingly, it is possible to perform tracking on the lane, the construction area, the first forward vehicle, the second forward vehicle, and the like, which are located in the vicinity of the vehicle.

6A and 6B are views referred to in the description of the operation of the driver assistance apparatus of FIGS. 3A to 3C.

First, FIG. 6A is a view illustrating a vehicle frontal situation photographed by a stereo camera 195 provided inside a vehicle. In particular, the vehicle front view is indicated by a bird eye view.

Referring to the drawing, a first lane 642a, a second lane 644a, a third lane 646a, and a fourth lane 648a are located from the left to the right, and the first lane 642a and the second The construction area 610a is positioned between the lanes 644a and the first front vehicle 620a is positioned between the second lane 644a and the third lane 646a and the third lane 646a and the fourth It can be seen that the second forward vehicle 630a is disposed between the lane lines 648a.

Next, Fig. 6B illustrates display of the vehicle front situation, which is grasped by the driver assistance device, together with various kinds of information. In particular, the image as shown in FIG. 6B may be displayed on the display unit 180, the AVN apparatus 400, or the display unit 741 provided in the driver assistance apparatus.

6B illustrates that information is displayed on the basis of an image photographed by the stereo camera 195, unlike FIG. 6A.

A first lane 642b, a second lane 644b, a third lane 646b and a fourth lane 648b are located from the left to the right and the first lane 642b and the second The construction area 610b is located between the lanes 644b and the first front vehicle 620b is located between the second lane 644b and the third lane 646b and the third lane 646b and the fourth It can be seen that the second forward vehicle 630b is disposed between the lane 648b.

The driver assistance device 100 performs signal processing on the basis of the stereo image photographed by the stereo cameras 195a and 195b and outputs it to the construction area 610b, the first front vehicle 620b, the second front vehicle 630b You can see the object for. In addition, the first lane 642b, the second lane 644b, the third lane 646b, and the fourth lane 648b can be confirmed.

On the other hand, in the drawing, it is exemplified that each of them is highlighted by a frame to indicate object identification for the construction area 610b, the first forward vehicle 620b, and the second forward vehicle 630b.

On the other hand, the driver assistance device 100 calculates distance information on the construction area 610b, the first forward vehicle 620b, and the second forward vehicle 630b based on the stereo image taken by the stereo camera 195 .

In the figure, calculated first distance information 611b, second distance information 621b, and third distance information 621b corresponding to the construction area 610b, the first forward vehicle 620b, and the second forward vehicle 630b, respectively, Information 631b is displayed.

On the other hand, the driver assistance device 100 can receive the sensor information about the vehicle from the control unit 770 or the sensing unit 760. Particularly, it is possible to receive and display the vehicle speed information, the gear information, the yaw rate indicating the speed at which the vehicle's rotational angle (yaw angle) changes, and the angle information of the vehicle.

The figure illustrates that the vehicle speed information 672, the gear information 671 and the yaw rate information 673 are displayed on the vehicle front image upper portion 670. In the vehicle front image lower portion 680, Information 682 is displayed, but various examples are possible. Besides, the width information 683 of the vehicle and the curvature information 681 of the road can be displayed together with the angle information 682 of the vehicle.

On the other hand, the driver assistance device 100 can receive speed limit information and the like for the road running on the vehicle through the communication unit 120 or the interface unit 130. In the figure, it is exemplified that the speed limitation information 640b is displayed.

The driver assistance device 100 may display various information shown in FIG. 6B through the display unit 180 or the like, but may store various information without a separate indication. And, by using such information, it can be utilized for various applications.

7 is an example of an internal block diagram of the vehicle 1 in Fig.

The vehicle 1 includes a communication unit 710, an input unit 720, a sensing unit 760, an output unit 740, a vehicle driving unit 750, a memory 730, an interface unit 780, a control unit 770, A driver assistance device 100, and an AVN device 400. [0033]

The communication unit 710 includes one or more modules that enable wireless communication between the vehicle 1 and the mobile terminal 600, between the vehicle 1 and the external server 510, or between the vehicle 1 and another vehicle . In addition, the communication unit 710 may include one or more modules for connecting the vehicle 1 to one or more networks.

The communication unit 710 may include a broadcast receiving module 711, a wireless Internet module 712, a local area communication module 713, a location information module 714, and an optical communication module 715.

The broadcast receiving module 711 receives broadcast signals or broadcast-related information from an external broadcast management server through a broadcast channel. Here, the broadcast includes a radio broadcast or a TV broadcast.

The wireless Internet module 712 refers to a module for wireless Internet access, and may be embedded in the vehicle 1 or externally. The wireless Internet module 712 is configured to transmit and receive wireless signals in a communication network according to wireless Internet technologies.

Wireless Internet technologies include, for example, WLAN (Wireless LAN), Wi-Fi (Wireless Fidelity), Wi-Fi (Wireless Fidelity) Direct, DLNA, WiBro Interoperability for Microwave Access, High Speed Downlink Packet Access (HSDPA), High Speed Uplink Packet Access (HSUPA), Long Term Evolution (LTE) and Long Term Evolution-Advanced (LTE-A) 712 send and receive data according to at least one wireless Internet technology, including Internet technologies not listed above. For example, the wireless Internet module 712 can exchange data with the external server 510 wirelessly. The wireless Internet module 712 can receive weather information and road traffic situation information (for example, TPEG (Transport Protocol Expert Group)) information from the external server 510. [

The short-range communication module 713 is for short-range communication and may be a Bluetooth ™, a Radio Frequency Identification (RFID), an Infrared Data Association (IrDA), a UWB (Ultra Wideband), a ZigBee, (Near Field Communication), Wi-Fi (Wireless-Fidelity), Wi-Fi Direct, and Wireless USB (Wireless Universal Serial Bus) technology.

The short-range communication module 713 may form short-range wireless communication networks to perform short-range communication between the vehicle 1 and at least one external device. For example, the short-range communication module 713 can exchange data with the mobile terminal 600 wirelessly. The short distance communication module 713 can receive weather information and traffic situation information of the road (for example, TPEG (Transport Protocol Expert Group)) from the mobile terminal 600. For example, when the user has boarded the vehicle 1, the user's mobile terminal 600 and the vehicle 1 can perform pairing with each other automatically or by executing the user's application.

The position information module 714 is a module for obtaining the position of the vehicle 1, and a representative example thereof is a Global Positioning System (GPS) module. For example, when the vehicle utilizes a GPS module, it can acquire the position of the vehicle using a signal sent from the GPS satellite.

The optical communication module 715 may include a light emitting portion and a light receiving portion.

The light receiving section can convert the light signal into an electric signal and receive the information. The light receiving unit may include a photodiode (PD) for receiving light. Photodiodes can convert light into electrical signals. For example, the light receiving section can receive information of the front vehicle through light emitted from the light source included in the front vehicle.

The light emitting unit may include at least one light emitting element for converting an electric signal into an optical signal. Here, the light emitting element is preferably an LED (Light Emitting Diode). The optical transmitter converts the electrical signal into an optical signal and transmits it to the outside. For example, the optical transmitter can emit the optical signal to the outside through the blinking of the light emitting element corresponding to the predetermined frequency. According to an embodiment, the light emitting portion may include a plurality of light emitting element arrays. According to the embodiment, the light emitting portion can be integrated with the lamp provided in the vehicle 1. [ For example, the light emitting portion may be at least one of a headlight, a tail light, a brake light, a turn signal lamp, and a car light. For example, the optical communication module 715 can exchange data with another vehicle through optical communication.

The input unit 720 may include a driving operation unit 721, a camera 195, a microphone 723, and a user input unit 724.

The driving operation means 721 receives a user input for driving the vehicle 1. The driving operation means 721 may include a steering input means 721a, a shift input means 721b, an acceleration input means 721c, and a brake input means 721d.

The steering input means 721a receives a forward direction input of the vehicle 1 from the user. The steering input means 721a may include a steering wheel 12 as shown in FIG. According to an embodiment, the steering input means 721a may be formed of a touch screen, a touch pad, or a button.

The shift input means 721b receives the input of parking (P), forward (D), neutral (N), and reverse (R) of the vehicle 1 from the user. The shift input means 721b is preferably formed in a lever shape. According to the embodiment, the shift input means 721b may be formed of a touch screen, a touch pad, or a button.

The acceleration input means 721c receives an input for acceleration of the vehicle 1 from the user. The brake input means 721d receives an input for decelerating the vehicle 1 from the user. The acceleration input means 721c and the brake input means 721d are preferably formed in a pedal shape. According to the embodiment, the acceleration input means 721c or the brake input means 721d may be formed of a touch screen, a touch pad, or a button.

The camera 195 may include an image sensor and an image processing module. The camera 195 may process still images or moving images obtained by an image sensor (e.g., CMOS or CCD). The image processing module processes the still image or moving image obtained through the image sensor, extracts necessary information, and transmits the extracted information to the control unit 770. On the other hand, the vehicle 1 may include a camera 195 for photographing a vehicle front image or a vehicle periphery image, and an internal camera 199 for photographing an in-vehicle image.

The internal camera 199 may obtain an image of the occupant. The internal camera 199 can acquire an image for biometrics of the occupant.

7, the camera 195 is included in the input unit 720. However, as described with reference to FIGS. 2 to 6, the camera 195 may include the camera 195 in a configuration included in the driver assistance device 100 .

The microphone 723 can process an external acoustic signal into electrical data. The processed data can be variously utilized depending on the function being performed in the vehicle 1. [ The microphone 723 can convert the voice command of the user into electrical data. The converted electrical data can be transmitted to the control unit 770.

The camera 722 or the microphone 723 may be a component included in the sensing unit 760 rather than a component included in the input unit 720. [

The user input unit 724 is for receiving information from a user. When information is inputted through the user input unit 724, the control unit 770 can control the operation of the vehicle 1 so as to correspond to the inputted information. The user input unit 724 may include touch input means or mechanical input means. According to an embodiment, the user input 724 may be located in one area of the steering wheel. In this case, the driver can operate the user input portion 724 with his / her finger while holding the steering wheel.

The sensing unit 760 senses a signal related to the running of the vehicle 1 or the like. To this end, the sensing unit 760 may include a sensor, a steering sensor, a speed sensor, a tilt sensor, a weight sensor, a heading sensor, a yaw sensor, a gyro sensor, Position sensor, vehicle forward / backward sensor, battery sensor, fuel sensor, tire sensor, steering sensor by steering wheel rotation, vehicle internal temperature sensor, internal humidity sensor, ultrasonic sensor, infrared sensor, radar, . ≪ / RTI >

Thus, the sensing unit 760 can detect the vehicle collision information, vehicle direction information, vehicle position information (GPS information), vehicle angle information, vehicle speed information, vehicle acceleration information, vehicle tilt information, Fuel information, tire information, vehicle lamp information, vehicle interior temperature information, vehicle interior humidity information, steering wheel rotation angle, and the like. The driver assistance device 100 to be described later is a device for driving the vehicle 1 on the basis of the surrounding information obtained by at least one of the camera, the ultrasonic sensor, the infrared sensor, the radar, It is possible to generate control signals for acceleration, deceleration, direction switching and the like. Here, the peripheral environment information may be information related to various objects located within a predetermined distance range from the vehicle 1 while driving. For example, the surrounding information may include information on the number of obstacles located within a distance of 100 m from the vehicle 1, the distance to the obstacle, the size of the obstacle, the type of the obstacle, and the like.

In addition, the sensing unit 760 may include an acceleration pedal sensor, a pressure sensor, an engine speed sensor, an air flow sensor AFS, an intake air temperature sensor ATS, a water temperature sensor WTS, A sensor (TPS), a TDC sensor, a crank angle sensor (CAS), and the like.

The sensing unit 760 may include a biometric information sensing unit. The biometric information sensing unit senses and acquires the biometric information of the passenger. The biometric information may include fingerprint information, iris-scan information, retina-scan information, hand geo-metry information, facial recognition information, Voice recognition information. The biometric information sensing unit may include a sensor that senses the passenger's biometric information. Here, the internal camera 199 and the microphone 723 can operate as sensors. The biometric information sensing unit can acquire the hand shape information and the face recognition information through the internal camera 199.

The output unit 740 is for outputting information processed by the control unit 770 and may include a display unit 741, an acoustic output unit 742, and a haptic output unit 743. [

The display unit 741 can display information processed in the control unit 770. For example, the display unit 741 can display the vehicle-related information. Here, the vehicle-related information may include vehicle control information for direct control of the vehicle, or vehicle driving assistance information for a driving guide to the vehicle driver. Further, the vehicle-related information may include vehicle state information indicating the current state of the vehicle or vehicle driving information related to the driving of the vehicle.

The display unit 741 may be a liquid crystal display (LCD), a thin film transistor-liquid crystal display (TFT LCD), an organic light-emitting diode (OLED) display, a 3D display, and an e-ink display.

The display unit 741 may have a mutual layer structure with the touch sensor or may be integrally formed to realize a touch screen. Such a touch screen may function as a user input 724 that provides an input interface between the vehicle 1 and the user and may provide an output interface between the vehicle 1 and the user. In this case, the display unit 741 may include a touch sensor that senses a touch with respect to the display unit 741 so that a control command can be received by a touch method. When a touch is made to the display unit 741, the touch sensor senses the touch, and the control unit 770 generates a control command corresponding to the touch based on the touch. The content input by the touch method may be a letter or a number, an instruction in various modes, a menu item which can be designated, and the like.

Meanwhile, the display unit 741 may include a cluster so that the driver can check the vehicle state information or the vehicle driving information while driving. Clusters can be located on the dashboard. In this case, the driver can confirm the information displayed in the cluster while keeping the gaze ahead of the vehicle.

Meanwhile, according to the embodiment, the display unit 741 may be implemented as a Head Up Display (HUD). When the display unit 741 is implemented as a HUD, information can be output through a transparent display provided in the windshield. Alternatively, the display unit 741 may include a projection module to output information through an image projected on the windshield.

The sound output unit 742 converts an electric signal from the control unit 770 into an audio signal and outputs the audio signal. For this purpose, the sound output unit 742 may include a speaker or the like. It is also possible for the sound output section 742 to output a sound corresponding to the operation of the user input section 724. [

The haptic output unit 743 generates a tactile output. For example, the haptic output section 743 may vibrate the steering wheel, the seat belt, and the seat so that the user can operate to recognize the output.

The vehicle drive unit 750 can control the operation of various devices of the vehicle. The vehicle driving unit 750 includes a power source driving unit 751, a steering driving unit 752, a brake driving unit 753, a lamp driving unit 754, an air conditioning driving unit 755, a window driving unit 756, an airbag driving unit 757, A driving unit 758 and a wiper driving unit 759. [

The power source drive unit 751 can perform electronic control of the power source in the vehicle 1. [ The power source drive section 751 may include an accelerator for increasing the speed of the vehicle 1 and a decelerator for decreasing the speed of the vehicle 1. [

For example, when the fossil fuel-based engine (not shown) is a power source, the power source driving unit 751 can perform electronic control on the engine. Thus, the output torque of the engine and the like can be controlled. When the power source drive unit 751 is an engine, the speed of the vehicle can be limited by limiting the engine output torque under the control of the control unit 770. [

As another example, when the electric motor (not shown) is a power source, the power source driving unit 751 can perform control on the motor. Thus, the rotation speed, torque, etc. of the motor can be controlled.

The steering driver 752 may include a steering apparatus. Thus, the steering driver 752 can perform electronic control of the steering apparatus in the vehicle 1. [ For example, the steering driver 752 may be provided with a steering torque sensor, a steering angle sensor, and a steering motor, and the steering torque applied by the driver to the steering wheel 12 may be sensed by the steering torque sensor. The steering driver 752 can control the steering force and the steering angle by changing the magnitude and direction of the current applied to the steering motor based on the speed of the vehicle 1 and the steering torque. In addition, the steering driver 752 can determine whether the running direction of the vehicle 1 is properly adjusted based on the steering angle information obtained by the steering angle sensor. Thereby, the running direction of the vehicle can be changed. In addition, the steering driver 752 reduces the weight of the steering wheel 12 by increasing the steering force of the steering motor when the vehicle 1 is running at a low speed, and reduces the steering force of the steering motor when the vehicle 1 is running at a high speed The weight of the steering wheel 12 can be increased. When the autonomous running function of the vehicle 1 is executed, the steering driver 752 can control the steering unit 760 even when the driver operates the steering wheel 12 (e.g., a situation in which the steering torque is not detected) Based on a sensing signal output by the steering angle sensor 170 or a control signal provided by the processor 170, or the like.

The brake driver 753 can perform electronic control of a brake apparatus (not shown) in the vehicle 1. [ For example, it is possible to reduce the speed of the vehicle 1 by controlling the operation of the brakes disposed on the wheels. As another example, it is possible to adjust the traveling direction of the vehicle 1 to the left or right by differently operating the brakes respectively disposed on the left wheel and the right wheel.

The lamp driver 754 may control the turn-on / turn-off of at least one or more lamps disposed inside or outside the vehicle. The lamp driver 754 may include a lighting device. Further, the lamp driving unit 754 can control intensity, direction, etc. of light output from each of the lamps included in the illuminating device. For example, it is possible to perform control for a direction indicating lamp, a head lamp, a brake lamp, and the like.

The air conditioning driving unit 755 may perform electronic control on an air conditioner (not shown) in the vehicle 1. [ For example, when the temperature inside the vehicle is high, the air conditioner can be operated to control the cool air to be supplied to the inside of the vehicle.

The window driving unit 756 may perform electronic control of the window apparatus in the vehicle 1. [ For example, it is possible to control the opening or closing of the side of the vehicle with respect to the left and right windows.

The airbag driving unit 757 can perform electronic control of the airbag apparatus in the vehicle 1. [ For example, in case of danger, the airbag can be controlled to fire.

The sunroof driving unit 758 may perform electronic control of a sunroof apparatus (not shown) in the vehicle 1. [ For example, the opening or closing of the sunroof can be controlled.

The wiper driving unit 759 can perform control on the wipers 14a and 14b provided in the vehicle 1. [ For example, the wiper driver 759 may provide an electronic control for the number of drives, drive speeds, etc. of the wipers 14a, 14b in response to user input upon receipt of a user input instructing to drive the wiper via the user input 724. [ Can be performed. For example, the wiper driving unit 759 may determine the amount or intensity of the rainwater based on the sensing signal of the rain sensor included in the sensing unit 760 so that the wipers 14a and 14b may be used without user input, Can be automatically driven.

Meanwhile, the vehicle driving unit 750 may further include a suspension driving unit (not shown). The suspension driving unit may perform electronic control of a suspension apparatus (not shown) in the vehicle 1. [ For example, when there is a curvature on the road surface, it is possible to control the suspension device so as to reduce the vibration of the vehicle 1. [

The memory 730 is electrically connected to the control unit 770. The memory 770 may store basic data for the unit, control data for controlling the operation of the unit, and input / output data. The memory 790 may be, in hardware, various storage devices such as ROM, RAM, EPROM, flash drive, hard drive, and the like. The memory 730 may store various data for operation of the entire vehicle 1, such as a program for processing or controlling the control unit 770. [

The interface unit 780 may serve as a pathway to various kinds of external devices connected to the vehicle 1. [ For example, the interface unit 780 may include a port that can be connected to the mobile terminal 600, and may be connected to the mobile terminal 600 through the port. In this case, the interface unit 780 can exchange data with the mobile terminal 600.

Meanwhile, the interface unit 780 may serve as a channel for supplying electrical energy to the connected mobile terminal 600. The interface unit 780 provides electric energy supplied from the power supply unit 790 to the mobile terminal 600 under the control of the control unit 770 when the mobile terminal 600 is electrically connected to the interface unit 780 do.

The control unit 770 can control the overall operation of each unit in the vehicle 1. [ The control unit 770 may be referred to as an ECU (Electronic Control Unit).

The controller 770 may be implemented in hardware as application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs) ), Controllers, micro-controllers, microprocessors, and other electronic units for performing other functions.

The power supply unit 790 can supply power necessary for the operation of each component under the control of the control unit 770. Particularly, the power supply unit 770 can receive power from a battery (not shown) in the vehicle.

The driver assistance device 100 can exchange data with the control unit 770. [ The control signal generated by the driver assistance device 100 may be output to the controller 770. The control unit 770 can control the traveling direction of the vehicle 1 based on the control signal received from the driver's auxiliary device 100. [

The AVN (Audio Video Navigation) device 400 can exchange data with the control unit 770. The control unit 770 can receive navigation information from the AVN apparatus 400 or a separate navigation device (not shown). Here, the navigation information may include set destination information, route information according to the destination, map information about the vehicle driving, or vehicle location information.

On the other hand, some of the components shown in Fig. 7 may not be essential for realizing the vehicle 1. Fig. Thus, the vehicle 1 described herein may have more or fewer components than those listed above.

Hereinafter, for convenience of explanation, it is assumed that the driver assistance device 100 according to the embodiment of the present invention is provided in the vehicle 1 shown in Fig.

8 is a flowchart showing a control method of the driver assistance device 100 according to an embodiment of the present invention.

Referring to FIG. 8, the processor 170 enters the image enlargement mode (S800). In the present invention, the image expansion mode means a mode of generating an extended image by using at least a part of the peripheral image of the vehicle 1 and at least a part of the peripheral image of the other vehicle. Hereinafter, a peripheral image of the vehicle 1 will be referred to as a 'main image', and a peripheral image of another vehicle will be referred to as a 'sub image'. On the other hand, the main image and the sub image are intended to distinguish whether the subject generating each image is the vehicle 1 or another vehicle, and do not limit the scope of the present invention.

When the processor 170 satisfies the predetermined condition, the processor 170 can enter the image enlargement mode.

For example, when the input unit 110 receives a user input that commands entry into the video enhancement mode, the processor 170 may enter the video enhancement mode. In this case, the user input may be at least one of various forms such as touch, voice, pushing, gesture, and the like.

For example, the processor 170 may calculate the risk based on the environment information acquired by the sensing unit 760 of the vehicle 1, and if the calculated risk exceeds the reference value, can do. When the processor 170 judges that various objects such as another vehicle approach the vehicle 1 within a reference distance (for example, 2 m) based on the surrounding environment information, the risk of an accident is great, .

As another example, the processor 170 may enter the image enlargement mode when the speed of the vehicle 1 slows down to less than a reference speed (e.g., 10 km).

As another example, the processor 170 may enter the image expansion mode when the vehicle 1 enters the parking mode.

Next, the processor 170 generates a main image using at least one camera 195-198 (S805). For example, when the processor 170 enters the image enlargement mode, the processor 170 may turn on at least one of the plurality of cameras 195-198 shown in FIG. 2B to generate a main image. That is, the main image may include at least one of a front image, a left image, a right image, and a rear image of the vehicle 1.

At this time, the shape of the main image may vary. For example, the main image may be a still image or a moving image. For example, the main image may be a forward image, a left image, a right image, and a left image of the vehicle 1, as shown in FIG. 2C. It is assumed that the image is an all-view image including all the backward images.

Subsequently, the processor 170 receives the sub-image generated by the other vehicle using the communication unit 120 or 710 (S810). That is, the communication unit 120 or 710 receives a sub-image from another vehicle under the control of the processor 170. [ Here, the sub video is an image generated by a camera provided in another vehicle.

The communication unit 120 or 710 can receive the sub video directly from the other vehicle. For example, the communication unit 120 or 710 may receive a sub-image directly from another vehicle based on a vehicle-to-vehicle communication network.

Alternatively, the communication unit 120 or 710 may receive a sub-image generated by another vehicle through intermediation of at least one external device. For example, the other vehicle transmits the sub-image to the external server, and the external server can transmit the sub-image transmitted by the other vehicle to the driver assistance device 100 provided in the vehicle.

At this time, the shape of the sub image may be various. For example, the sub-image may be a still image or a moving image. For example, the sub-image may include a forward image, a left-side image, and a left-eye image of the other vehicle, similar to the main image. It is assumed that the image is an all-view image including both the right-side image and the backward image, and the description will be continued.

Further, the communication unit 120 or 710 can receive the sub-image generated by each of the plurality of vehicles. That is, the communication unit 120 or 710 can receive a plurality of different sub-images.

In this case, the processor 170 may select some of the plurality of sub-images based on predetermined conditions or user inputs. For example, the processor 170 may select a sub-image in which an obstacle appears among a plurality of sub-images. The selected sub-image can be used in step S815 to be described later.

In addition, the processor 170 can receive only the sub-images generated by the specific vehicle among the plurality of vehicles using the communication unit 120 or 710. [

For example, the processor 170 controls the communication unit 120 or 710 to receive only the sub-image generated by the other vehicle positioned in the front of the other vehicle positioned in front of the vehicle 1 and the other vehicle positioned in the rear of the vehicle 1, Can be controlled.

Alternatively, the processor 170 may be configured to communicate with the communication unit 120 or 710 to receive only sub-images generated by other vehicles located at points corresponding to user inputs or predetermined conditions (e.g., traffic conditions, weather, etc.) Can be controlled.

Specifically, under the control of the processor 170, the communication unit 120 or 710 transmits a video request signal to a specific other vehicle, and receives a sub video transmitted in response to the video request signal have.

On the other hand, the communication unit 120 or 710 can further receive the location information of the vehicle 1. [ Accordingly, the processor 170 can determine whether there is an attention interval within a predetermined distance from the vehicle 1, based on the positional information received by the communication unit 120 or 710. The attention section may include sections requiring attention of the driver, such as an intersection, an uphill, a downhill, a crosswalk, a parking lot, a tunnel, a narrow road, or a curve road. The type of attention interval can be changed according to user input. In this case, the processor 170 may control the communication unit 120 or 710 to transmit a video request signal to the other vehicle positioned within the attention period. Accordingly, the communication unit 120 or 710 receives the sub-image generated by the other vehicle positioned within the attention zone, and the processor 170 can generate the extended image using the sub-image, There is an advantage in that the driver of the vehicle can provide a real image of a section of the road which is not directly visible.

On the other hand, in Fig. 8, although step S805 is shown as preceding to step S810, this is an example, and steps S805 and S810 may be performed simultaneously, or step S810 may precede step S805.

Next, the processor 170 generates an extended image using the main image and the sub-image (S815). That is, the processor 170 may generate an extended image, which is an image having a wider field of view than the main image, by summing at least a portion of the main image and at least a portion of the sub-image.

In this case, the main image and the sub image included in the extended image may have portions overlapping each other. For example, when there is a common range between the photographing range of the camera 195-198 mounted on the vehicle 1 and the photographing range of the camera mounted on the other vehicle, there may exist overlapping portions between the main image and the sub image have.

When there is a portion overlapped with the main image and the sub image, the processor 170 can generate an extended image by combining the main image and the sub image based on the overlap area.

Alternatively, the main image and the sub image included in the extended image may be spaced apart from each other without overlapping portions. For example, in a case where the camera is located outside the shooting range of the camera 195-198 mounted on the vehicle 1, there are no overlapped portions between the main image and the sub image. Therefore, Sub images appear apart from each other.

In this case, the processor 170 can generate the extended image by combining the main image and the sub image based on the position information of the vehicle 1 and the position information of the other vehicle received by the communication unit 120 or 710. In addition, when the main image and the sub image which do not overlap each other are combined, the processor 170 determines the main image and the sub image based on the vehicle direction information of the vehicle 1 and the vehicle direction information of the other vehicle It is possible to generate an extended image. The vehicle body direction information of the vehicle 1 is obtained by the sensing unit 760 of the vehicle 1 and the vehicle body direction information of the other vehicle may be received by the communication unit 120 or 710. [

Meanwhile, the processor 170 may generate an extended image in real time or periodically. In the case where the extended image is periodically generated, the processor 170 may change the period of generating the extended image according to the speed of the vehicle 1. [ For example, if the speed of the vehicle 1 is the first speed, the extended image is updated every three seconds. If the speed of the vehicle 1 is a second speed higher than the first speed, You can update it.

Conventionally, there has been a limitation that only an image (i.e., a main image) of an area belonging to a photographing range of a camera 195-198 mounted on the vehicle 1 can be provided to a user. However, 100) can additionally provide the user with an image (i.e., a sub-image) for an area belonging to the shooting range of the camera mounted on the other vehicle. As a result, the safety and convenience of the driver on board the vehicle 1 can be improved at the same time.

Subsequently, the processor 170 executes at least one of the predetermined operations related to the vehicle 1 based on the extended image (S820).

For example, the processor 170 may generate information on an obstacle appearing only in the sub-image, among the main image and the sub-image included in the extended image. Here, an obstacle appearing only in the sub-image may mean an obstacle belonging to the field of view of another vehicle that does not belong to the field of view of the vehicle 1 but provides the sub-image. The information on the obstacle may include various information related to the obstacle such as the position, size, color, shape, type, and motion of the obstacle.

In another example, the processor 170 may generate a control signal that commands to change at least one of the speed and direction of the vehicle 1 based on information about the obstacle. For example, the control unit 770 may control the steering driver 752, the power source driver 751, the brake driver 753, and the like based on the control signal provided from the processor 170. [ Thus, the risk of an accident caused by an obstacle can be reduced compared with a case where only a main image is provided.

For another example, the processor 170 may generate a path through which the vehicle 1 can travel based on the extended image. Specifically, the extended image has a wider field of view than that of the main image. Therefore, the processor 170 determines whether or not the vehicle 1 is moving from the current position of the vehicle 1 based on the positional relationship between the vehicle 1 and the other vehicle, ) To a position outside the visual range of the camera.

In this case, the path that the vehicle 1 generated by the processor 170 can travel is plural, and the processor 170 can select at least any one of a plurality of paths according to a user input or a predetermined priority have. Alternatively, the processor 170 may generate a control signal to instruct the control unit 770 of the vehicle 1 to enter the autonomous mode for the selected route.

In another example, the processor 170 may generate a control signal that commands to display at least one of the extended image and the information associated with the extended image. That is, at least one of the display unit 180 of the driver assistance apparatus 100, the display unit 741 of the vehicle 1, and the AVN apparatus 400 is displayed on the screen in accordance with the control signal provided from the processor 170 Only the extended image, only the information associated with the extended image, or both can be displayed.

At this time, the processor 170 divides the screen of at least one of the display unit 180 of the driver assistance apparatus 100, the display unit 741 of the vehicle 1, and the AVN apparatus 400 into a plurality of sub-screens , Different information can be displayed for each divided sub-screen. For example, the extended image may be displayed on any one of the plurality of sub-screens, and information related to the extended image may be displayed on any of the remaining sub-screens.

The processor 170 also controls the display 180 of the driver assistance unit 100 and the display unit 741 of the vehicle 1 to give a predetermined visual effect to the area corresponding to the actual position of the obstacle, ) And the AVN apparatus 400. [0064] FIG. For example, it is possible to control the display unit 180 to periodically blink the area corresponding to the actual position of the obstacle in red, of the extended image.

The processor 170 also controls the display 180 of the driver assistance device 100, the display portion 741 of the vehicle 1, and the AVN device 400 (not shown) to rotate and display the extended image by an angle corresponding to the user input. ) Can be controlled. Accordingly, the driver can help recognize the surrounding environment of the vehicle 1 while rotating the extended image clockwise or counterclockwise according to his / her preference.

The processor 170 also controls the display 180 of the driver assistance device 100, the display portion 741 of the vehicle 1 and the AVN device (not shown) of the vehicle 1, 400 can be controlled. For example, the vehicle image of the main image displayed on the display unit 180 may be displayed in red, and the other vehicle image of the sub image may be displayed in blue.

9A to 22B, it is assumed that the driver assistance device 100 has entered the image expansion mode. It is also assumed that the extended image generated by the driver auxiliary apparatus 100 entering the image enlargement mode and information about the extended image are displayed on the screen of the AVN apparatus 400. [

9A to 9D are views for explaining the operation of the driver assistance device 100 according to an embodiment of the present invention.

First, FIG. 9A illustrates a top view in a situation where one other vehicle 2 is in the front of the vehicle 1. It is assumed that both the vehicle 1 and the other vehicle 2 are in the same lane or in a stopped state or in a traveling state at the same speed.

9A, the camera 195-198 mounted on the vehicle 1 photographs the first range 911, and the camera mounted on the other vehicle 2 photographs the second range 921. As shown in FIG. In this case, as shown in the figure, an overlap region 931 may exist in the first range 911 and the second range 921. That is, the overlap region 931 is a region commonly belonging to the first range 911 and the second range 921, and includes a camera 195-198 mounted on the vehicle 1 and a camera 195-198 mounted on the other vehicle 2 Quot; means an area photographed by all.

FIG. 9B illustrates a main image 941 corresponding to the first range 911 shown in FIG. 9A. It is assumed that the main image 941 is an ambient view image, as shown in the figure. The processor 170 determines whether or not the traveling images (for example, the front image, the left side image, the right side image, and the rear image) generated by photographing the first range 911 by the camera 195-198 mounted on the vehicle 1 As a basis, the main image 941 can be generated.

The processor 170 may generate a main image 941 in which an image 942 corresponding to the vehicle 1 is centered. The image 942 corresponding to the vehicle 1 may be generated directly by the processor 170 or stored in the memory 140 of the driver assistance device 100 or the memory 730 of the vehicle 1 . 9A, since the overlap area 931 is located in front of the first range 911, a part of the rear vehicle body of the other vehicle 2 may appear in front of the main image 941. FIG.

FIG. 9C illustrates a sub-image 951 corresponding to the second range 921 shown in FIG. 9A. It is assumed that the sub-image 951 is the same as the main image 941, as shown in FIG. The other vehicle 2 can generate the sub image 951 based on the running image (e.g., the forward image, the left image, the right image, and the back image) generated by photographing the second range 921 .

In addition, an image 952 corresponding to the other vehicle 2 may be disposed at the center of the sub image 951. 9A, since the overlap area 931 is located behind the second range 921, a part of the front body of the vehicle 1 may appear behind the sub image 951. [

FIG. 9D is a view showing an indoor state of the vehicle 1 in the situation shown in FIG. 9A. Referring to Fig. 9D, the driver of the vehicle 1 can watch the other vehicle 2 positioned ahead through the windshield.

The processor 170 can generate the extended image 961 by using the main image 941 shown in FIG. 9B and the sub image 951 shown in FIG. 9C while entering the image expansion mode. For example, the processor 170 determines whether the image portion corresponding to the overlapping region 931 of the main image 941 shown in FIG. 9A and the image portion corresponding to the overlapping region 931 of the sub- It is possible to generate the extended image 961 by changing the direction, size, etc. of at least one of the main image 941 and the sub image 951 and then combining them with each other. That is, the driver assistance device 100 can provide the driver with an image that extends the range of the main image 941 forward. In this case, the extended image 961 may be an overview image such as looking down on the vehicle 1 and the other vehicle 2 from above.

The processor 170 also generates a control signal for displaying the extended image 961 and transmits the control signal to the AVN apparatus 400 provided in the vehicle 1, the display section 741 of the vehicle 1, And the display unit 180 of the driver assistance apparatus 100. [0064]

9D, the AVN apparatus 400 can display the extended image 961 on the screen based on the control signal provided by the processor 170. [ Although not shown, the display unit 741 of the vehicle 1 and the display unit 180 of the driver assistance device 100 also display the extended image 961 on the screen based on the control signal provided by the processor 170, Can be displayed. At this time, based on the screen size and the ratio of each of the display section 741 of the vehicle 1, the display section 180 of the driver assistance device 100, and the AVN device 400, the processor 170 selects the extended image 961 ) Are displayed differently from each other.

The driver visually confirms the surroundings of the vehicle 1 through the extended image 961 in which the main image 941 and the sub image 951 are displayed together, It is possible to secure a wide field of view.

Meanwhile, the extended image generated by the processor 170 can provide useful information about an invisible area to the driver of the vehicle 1, which will be described below.

10A to 10D are views for explaining the operation of the driver assistance device 100 according to an embodiment of the present invention.

First, FIG. 10A illustrates a top view in a situation where one other vehicle 3 is close to the front of the vehicle 1. FIG. It is assumed that both the vehicle 1 and the other vehicle 3 are in a stopped state or traveling at the same speed in the same lane and the obstacle 31 exists in front of the other vehicle 3. [

According to Fig. 10A, the camera 195-198 mounted on the vehicle 1 photographs the first range 1011, and the camera mounted on the other vehicle 3 photographs the second range 1021. Fig. In this case, as shown in the figure, the overlap region 1031 may exist in the first range 1011 and the second range 1021. That is, the overlap area 1031 is a region commonly belonging to the first range 1011 and the second range 1021, and includes a camera 195-198 mounted on the vehicle 1 and a camera 195-198 mounted on the other vehicle 3 It may mean an area photographed by all.

FIG. 10B illustrates a main image 1041 corresponding to the first range 1011 shown in FIG. 10A. It is assumed that the main image 1041 is an ambient view image, as shown in the figure. The processor 170 determines whether or not the traveling images (for example, front image, left side image, right side image, and rear image) generated by photographing the first range 1011 of the camera 195-198 mounted on the vehicle 1 As a basis, the main image 1041 can be generated.

The processor 170 may generate the main image 1041 in which the image 1042 corresponding to the vehicle 1 is arranged in the center. The image 1042 corresponding to the vehicle 1 may be generated directly by the processor 170 or stored in the memory 140 of the driver assistance apparatus 100 or the memory 730 of the vehicle 1 . 10A, since the overlap area 1031 is located in front of the first range 1011, a part of the rear vehicle body of the other vehicle 3 may appear in front of the main image 1041. [

FIG. 10C illustrates a sub image 1051 corresponding to the second range 1021 shown in FIG. 10A. It is assumed that the sub-image 1051 is the same as the main image 1041, as shown in FIG. The other vehicle 3 can generate the sub image 1051 based on the running image (e.g., the forward image, the left image, the right image, and the back image) generated by photographing the second range 1021 .

An image 1052 corresponding to the other vehicle 3 may be disposed at the center of the sub image 1051. [ 10A, since the overlap area 1031 is located behind the second range 1021, a part of the vehicle body ahead of the vehicle 1 may appear behind the sub image 1051. [

On the other hand, unlike the situation shown in Fig. 9A, according to Fig. 10A, the obstacle 31 exists in front of the other vehicle 3, and the obstacle 31 is in the second range (1021). As a result, the obstacle 31 appears in front of the sub image 1051 generated by the other vehicle 3.

Fig. 10D is a view showing an indoor state of the vehicle 1 in the situation shown in Fig. 10A. Referring to Fig. 10D, the driver of the vehicle 1 can watch the other vehicle 3 positioned ahead through the windshield. However, since the obstacle 31 is located in front of the other vehicle 3, the driver of the vehicle 1 can only confirm the other vehicle 3 through the windshield, Do.

The processor 170 can generate the extended image 1061 by using the main image 1041 shown in FIG. 10B and the sub image 1051 shown in FIG. 10C in the image enlargement mode. For example, the processor 170 determines whether the video portion corresponding to the overlapping region 1031 of the main video 1041 shown in FIG. 10A and the video portion corresponding to the overlapping region 1031 among the sub- It is possible to generate the extended image 1061 by changing the direction, size, etc. of at least one of the main image 1041 and the sub image 1051 and combining them afterwards. That is, the driver assistance device 100 can provide the driver with an image in which the range of the main image is extended in the forward and backward directions. In this case, the extended image 1061 may be an all-view image such as looking down on the vehicle 1 and the other vehicle 3 from above.

The processor 170 generates a control signal for displaying the extended image 1061 and transmits the control signal to the AVN apparatus 400, the display unit 741, and the driver assistance apparatus 100 The display unit 180 may be provided with at least one of them.

10D, the AVN apparatus 400 can display the extended image 1061 on the screen based on the control signal provided by the processor 170. [ Since the extended image 1061 includes at least a part of the sub image 1051 as well as the main image 1041, the driver of the vehicle 1 can recognize the obstacle 31 existing at a position that can not be confirmed by the main image 1041 alone And the extended image 1061 to which the sub-image 1051 is added. As a result, the driver of the vehicle 1 is caught by the other vehicle 3, recognizes the obstacle 31, which can not be confirmed through the windshield, through the extended image 1061 in advance, The risk of accident can be reduced by operating the deceleration, braking, direction, and the like of the vehicle.

Although not shown, the display unit 741 of the vehicle 1 and the display unit 180 of the driver assistance device 100 are also connected to the extended image 1061 on the screen based on the control signal provided by the processor 170, Can be displayed. At this time, based on the screen size and the ratio of each of the display section 741 of the vehicle 1, the display section 180 of the driver assistance device 100, and the AVN device 400, the processor 170 extracts the extended image 1061 ) Are displayed differently from each other.

11A to 11D are diagrams for explaining the operation of the driver assistance device 100 according to an embodiment of the present invention.

First, Fig. 11A illustrates a top view in a situation in which one other vehicle 4 is close to the side of the vehicle 1. Fig. For convenience of explanation, the other vehicle 4 is located on the right side of the vehicle 1, and the vehicle 1 and the other vehicle 4 are both stopped in the two lanes close to each other, And it is assumed that there is no object in front of the vehicle 1.

According to Fig. 11A, the camera 195-198 mounted on the vehicle 1 photographs the first range 1111, and the camera mounted on the other vehicle 4 photographs the second range 1121. Fig. In this case, as shown in the figure, the overlap region 1131 may exist in the first range 1111 and the second range 1121. That is, the overlap area 1131 is a region commonly belonging to the first range 1111 and the second range 1121, and includes a camera 195-198 mounted on the vehicle 1 and a camera 195-198 mounted on the other vehicle 4 Quot; means an area photographed by all.

FIG. 11B illustrates a main image 1141 corresponding to the first range 1111 shown in FIG. 11A. It is assumed that the main image 1141 is an ambient view image, as shown in the figure. The processor 170 determines whether the camera 195-198 mounted on the vehicle 1 has taken a traveling image (e.g., a front image, a left side image, a right side image, and a rear image) As a basis, the main image 1141 can be generated.

The processor 170 may generate a main image 1141 in which an image 1142 corresponding to the vehicle 1 is centered. The image 1142 corresponding to the vehicle 1 may be created directly by the processor 170 or stored in the memory 140 of the driver assistance device 100 or the memory 730 of the vehicle 1 . 11A, since the overlap region 1131 is located on the right side of the first range 1111, a part of the left vehicle body of the other vehicle 4 may appear on the right side of the main image 1141. [

FIG. 11C illustrates a sub image 1151 corresponding to the second range 1121 shown in FIG. 11A. It is assumed that the sub image 1151 is the same as the main image 1141, as shown in FIG. The other vehicle 4 can generate the sub image 1151 based on the running image (e.g., the forward image, the left image, the right image, and the back image) generated by photographing the second range 1121 .

An image 1152 corresponding to the other vehicle 4 may be disposed at the center of the sub image 1151. 11A, since the overlap area 1131 is located on the left side of the second range 1121, a part of the right side of the vehicle 1 may appear on the left side of the sub image 1151. [

Fig. 11D is a view showing an indoor state of the vehicle 1 in the situation shown in Fig. 11A. Referring to FIG. 11D, the driver of the vehicle 1 can directly observe the forward situation through the windshield. 11A, since the other vehicle 4 is located on the right side of the vehicle 1, the driver can not identify the other vehicle 4 through the windshield of the vehicle 1. [

The processor 170 can generate the extended image 1161 by using the main image 1141 shown in FIG. 11B and the sub image 1151 shown in FIG. For example, the processor 170 determines whether the video portion corresponding to the overlapping region 1131 of the main video 1141 shown in Fig. 11A and the video portion corresponding to the overlapping region 1131 among the sub video 1151 It is possible to generate the extended image 1161 by changing the directions, sizes, etc. of at least one of the main image 1141 and the sub image 1151 and then synthesizing them. That is, the driver assistance device 100 can provide the driver with an image in which the range of the main image is extended in the left and right directions. In this case, the extended image 1161 may be an overview image such as looking down on the vehicle 1 and the other vehicle 4 from above.

The processor 170 generates a control signal for displaying the extended image 1161 and transmits the control signal to the AVN apparatus 400, the display unit 741, and the driver assistance apparatus 100 The display unit 180 may be provided with at least one of them. 11D, the AVN apparatus 400 may display the extended image 1161 on the screen based on the control signal provided by the processor 170. [ Although not shown, the display unit 741 of the vehicle 1 and the display unit 180 of the driver assistance device 100 may also display the extended image 1161 on the screen based on the control signal provided by the processor 170, Can be displayed. At this time, based on the screen size and ratio of each of the display section 741 of the vehicle 1, the display section 180 of the driver assistance device 100, and the AVN device 400, the processor 170 displays the extended image 1161 ) Are displayed differently from each other.

The driver visually confirms the peripheral situation of the vehicle 1 through the extended image 1161 in which the main image 1141 and the sub image 1151 are displayed together, It is possible to secure a wide field of view to the room.

Meanwhile, the extended image generated by the processor 170 can provide useful information about an invisible area to the driver of the vehicle 1, which will be described below.

12A to 12D are views for explaining the operation of the driver assistance device 100 according to an embodiment of the present invention.

First, Fig. 12A illustrates a top view in a situation where one other vehicle 5 is close to the side of the vehicle 1. Fig. For convenience of explanation, the other vehicle 5 is located on the right side of the vehicle 1, and the vehicle 1 and the other vehicle 5 are both stopped in the two lanes close to each other, And it is assumed that there is no object in front of the vehicle 1.

On the other hand, unlike the situation shown in FIG. 11A, there is a difference in that a pedestrian 41 exists on the right side of the other vehicle 5. In such a situation, it is considerably difficult for the driver of the vehicle 1 to identify the pedestrian 41 covered by the other vehicle 5. [

According to Fig. 12A, the camera 195-198 mounted on the vehicle 1 photographs the first range 1211, and the camera mounted on the other vehicle 5 photographs the second range 1221. Fig. In this case, as shown in the figure, an overlap region 1231 may exist in the first range 1211 and the second range 1221. [ That is, the overlap area 1231 is a region commonly belonging to the first range 1211 and the second range 1221, and the cameras 195-198 mounted on the vehicle 1 and the camera mounted on the other vehicle 5 Quot; means an area photographed by all.

FIG. 12B illustrates a main image 1241 corresponding to the first range 1211 shown in FIG. 12A. It is assumed that the main image 1241 is an ambient view image, as shown in the figure. The processor 170 determines whether or not the traveling images (for example, the front image, the left side image, the right side image, and the rear image) generated by photographing the first range 1211 by the camera 195-198 mounted on the vehicle 1 As a basis, the main image 1241 can be generated.

The processor 170 may generate a main image 1241 in which the image 1242 corresponding to the vehicle 1 is centered. The image 1242 corresponding to the vehicle 1 may be generated directly by the processor 170 or stored in the memory 140 of the driver assistance apparatus 100 or the memory 730 of the vehicle 1 . 12A, since the overlap area 1231 is located on the right side of the first range 1211, a part of the left vehicle body of the other vehicle 5 may appear on the right side of the main image 1241. [

12C illustrates a sub image 1251 corresponding to the second range 1221 shown in FIG. 12A. It is assumed that the sub-image 1251 is the same as the main image 1241, as shown in FIG. The other vehicle 5 can generate the sub image 1251 based on the running image (e.g., front image, left side image, right side image, and rear image) generated by photographing the second range 1221 .

An image 1252 corresponding to the other vehicle 5 may be disposed at the center of the sub image 1251. [ 12A, since the overlap area 1231 is located on the left side of the second range 1221, a part of the right side of the vehicle 1 may appear on the left side of the sub image 1251. [

12A, the pedestrian 41 is present on the right side of the other vehicle 5, and the pedestrian 41 is located on the right side of the other vehicle 5 in the second range (1221). Thus, the pedestrian 41 appears on the right side of the sub image 1251 generated by the other vehicle 5.

12D is a view showing an indoor state of the vehicle 1 in the situation shown in FIG. 12A. 12D, since the pedestrian 41 is located on the right side of the other vehicle 5, it is difficult for the driver of the vehicle 1 to confirm the pedestrian 41 as well as the other vehicle 5 through the windshield .

The processor 170 can generate the extended image 1261 by using the main image 1241 shown in FIG. 12B and the sub image 1251 shown in FIG. For example, the processor 170 determines whether or not the image portion corresponding to the overlapping region 1231 in the main image 1241 shown in Fig. 12A and the image portion corresponding to the overlapping region 1231 in the sub- It is possible to generate the extended image 1261 by changing the direction, size, etc. of at least one of the main image 1241 and the sub image 1251 and then synthesizing them. That is, the driver assistance device 100 can provide the driver with an image in which the range of the main image is extended in the forward and backward directions. In this case, the extended image 1261 may be an all-view image such as looking down on the vehicle 1 and the other vehicle 5 from above.

The processor 170 generates a control signal for displaying the extended image 1261 and transmits the control signal to the AVN apparatus 400, the display unit 741, and the driver assistance apparatus 100 The display unit 180 may be provided with at least one of them.

12D, the AVN apparatus 400 can display the extended image 1261 on the screen based on the control signal provided by the processor 170. [ The extended image 1261 includes at least a part of the main image 1241 as well as the sub image 1251 so that the driver of the vehicle 1 can recognize the pedestrian 41 existing at a position that can not be confirmed only by the main image 1241 And an extended image 1261 to which a sub image 1251 is added. As a result, the driver of the vehicle 1 recognizes the pedestrian 41, which is hidden by the other vehicle 5 and can not be confirmed by only the main video 1241, through the extended video 1261, The risk of accident can be reduced by operating the deceleration, braking, direction, etc. of the vehicle 1.

Although not shown, the display unit 741 of the vehicle 1 and the display unit 180 of the driver assistance device 100 are also connected to the extended image 1261 on the screen based on the control signal provided by the processor 170. [ Can be displayed. At this time, based on the screen size and ratio of each of the display section 741 of the vehicle 1, the display section 180 of the driver assistance device 100, and the AVN device 400, the processor 170 displays the extended image 1261 ) Are displayed differently from each other.

9A to 12D, the case where one vehicle close to the vehicle 1 (that is, another vehicle that generates a sub image having a portion common to the main image of the vehicle 1) is described as a center, But is not limited thereto. That is, the driver assistance device 100 according to the embodiment of the present invention can generate an extended image even when a plurality of other vehicles close to the vehicle 1 are present. This will be described in detail below.

13A to 13E are views for explaining the operation of the driver assistance device 100 according to an embodiment of the present invention.

First, FIG. 13A illustrates a top view of the parking section 1300. FIG. As shown in the figure, it can be assumed that the vehicle 1, the other vehicle 6 on the left side of the vehicle 1, and the other vehicle 7 on the right side of the vehicle 1 are located in the parking space 1300.

13A, the camera 195-198 mounted on the vehicle 1 photographs the first range 1311, the camera mounted on the other vehicle 6 photographs the second range 1321, The camera mounted on the vehicle 7 photographs the third range 1331. [

In this case, as shown in the figure, a first overlap region 1341 may exist in the first range 1311 and the second range 1321. In addition, a second overlapping region 1342 may exist in the first range 1311 and the third range 1331. [ That is, the first overlapping area 1341 is a region commonly included in the first range 1311 and the second range 1321 and is attached to the cameras 195-198 and the other vehicles 6 mounted on the vehicle 1 Which is photographed by all of the cameras. The second overlapping region 1342 is a region commonly included in the first range 1311 and the third range 1331 and is attached to the camera 195-198 and the other vehicle 7 mounted on the vehicle 1 Which is photographed by all of the cameras.

13B illustrates a main image 1351 corresponding to the first range 1311 shown in FIG. 13A. It is assumed that the main image 1351 is an ambient view image, as shown in the figure. The processor 170 determines whether or not the traveling images (e.g., front image, left side image, right side image, and rear image) generated by photographing the first range 1311 of the camera 195-198 mounted on the vehicle 1 As a basis, the main image 1351 can be generated.

The processor 170 may generate a main image 1351 in which an image 1352 corresponding to the vehicle 1 is centered. The image 1352 corresponding to the vehicle 1 may be generated directly by the processor 170 or stored in the memory 140 of the driver assistance device 100 or the memory 730 of the vehicle 1 .

13A, the first overlap region 1341 is located on the left side of the first range 1311. [ Therefore, as shown in FIG. 13B, a part of the right side of the other vehicle 6 may appear on the left side of the main image 1351. Since the second overlapping area 1342 is located on the right side of the first range 1311, a part of the left vehicle body of the other vehicle 7 may appear on the right side of the main image 1351.

13C illustrates a first sub-image 1361 corresponding to the second range 1321 shown in FIG. 13A. It is assumed that the first sub-image 1361 is the same as the main image 1351, as shown in FIG. The other vehicle 6 generates the first sub-image 1361 based on the running image (e.g., the forward image, the left-side image, the right-side image, and the rear image) generated by photographing the second range 1321 .

Also, an image 1362 corresponding to the other vehicle 6 may be disposed in the center of the first sub-image 1361. 13A, since the first overlapping area 1341 is located on the right side of the second range 1321, a part of the left side of the vehicle 1 may appear on the right side of the first sub-image 1361.

FIG. 13D illustrates a second sub-image 1371 corresponding to the third range 1331 shown in FIG. 13A. It is assumed that the second sub-image 1371 is the same as the main image 1351, as shown in FIG. The other vehicle 7 generates the second sub-image 1371 based on the traveling image (e.g., the forward image, the left-side image, the right-side image, and the rear image) generated by photographing the third range 1331 .

In addition, an image 1372 corresponding to the other vehicle 7 may be disposed in the center of the second sub-image 1371. 13A, since the second overlapping area 1342 is located on the left side of the third range 1331, a part of the right side of the vehicle 1 may appear on the left side of the second sub-image 1371. [

13E is a view showing an indoor state of the vehicle 1 in the situation shown in Fig. 13A. Fig. The processor 170 enters the image enlargement mode. The main image 1351 shown in FIG. 13B, the first sub-image 1361 shown in FIG. 13C, and the second sub-image 1371 shown in FIG. The extended image 1381 can be generated.

For example, the processor 170 may convert the image portion corresponding to the first overlapping region 1341 of the main image 1351 shown in FIG. 13A and the image portion corresponding to the first overlapping region 1341 of the first sub- The sizes and the like of at least one of the main image 1351 and the first sub image 1361 may be changed so that the image portions corresponding to the main image 1351 and the first sub image 1361 may be changed and combined. The processor 170 also stores the image portion corresponding to the second overlapping region 1342 of the main image 1351 shown in Fig. 13A and the image portion corresponding to the second overlapping region 1342 of the second sub- The size and the like of at least one of the main image 1351 and the second sub image 1371 may be changed and combined so that the image portions of the main image 1351 and the second sub image 1371 are matched with each other.

For example, assuming that the width of the parking line constituting the parking section 1300 is constant, the processor 170 determines that the width of the parking line shown in the main image 1351 is equal to the width of the parking line shown in the first sub- The size of the main image 1351 is reduced to 1/2 or the size of the first sub image 1361 is doubled and then the main image 1351 and the first sub image 1361 are combined can do. In addition, the processor 170 can combine the second sub-image 1371 with the main image 1351 in the same manner. Accordingly, the images generated by the different vehicles can be combined without any distortion to generate the extended image 1381.

 As a result, the driver assisting apparatus 100 can provide the driver with an image in which the range of the main image 1351 is extended in both the left and right directions. In this case, the extended image 1381 may be an overview image such as looking down on the vehicle 1 and two other vehicles 6, 7 at the same time.

The processor 170 generates a control signal for displaying the extended image 1381 and transmits the control signal to the AVN apparatus 400, the display unit 741, and the driver assistance apparatus 100 The display unit 180 may be provided with at least one of them. 13E, the AVN apparatus 400 can display the extended image 1381 on the screen based on the control signal provided by the processor 170. [ Although not shown, the display unit 741 of the vehicle 1 and the display unit 180 of the driver assistance device 100 also display the extended image 1381 on the screen based on the control signal provided by the processor 170, Can be displayed. At this time, based on the screen size and the ratio of each of the display section 741 of the vehicle 1, the display section 180 of the driver assistance device 100, and the AVN device 400, the processor 170 displays the extended image 1381 ) Are displayed differently from each other.

The driver visually confirms the surroundings of the vehicle 1 through the extended image 1381 in which the main image 1351, the first sub-image 1361 and the second sub-image 1371 are displayed together, 1351 are provided, it is possible to secure a wide field of view to the left and right sides.

13A to 13E, the case where two vehicles 6 and 7 are located on the right and left sides of the vehicle 1 has been described. However, the present invention is not limited thereto. For example, even when two vehicles 6 and 7 are located before and after the vehicle 1, an extended image can be generated using the above-described method. In another example, even when one riding vehicle 6 is located on the left side of the vehicle 1 and the remaining other vehicles 7 are located on the rear side of the vehicle 1, Can be generated.

Meanwhile, the extended image generated by the processor 170 can provide useful information about an invisible area to the driver of the vehicle 1, which will be described below.

14A to 14E are views for explaining the operation of the driver assistance device 100 according to an embodiment of the present invention.

First, FIG. 14A illustrates a top view of the parking zone 1400. FIG. As shown in the figure, it can be assumed that the vehicle 1, the other vehicle 8 on the left side of the vehicle 1, and the other vehicle 9 on the right side of the vehicle 1 are located in the parking space 1400. 14A differs from FIG. 13A in that the pedestrian 51 is moving toward the vehicle 1 in the vicinity of the other vehicle 9. FIG.

14A, the camera 195-198 mounted on the vehicle 1 photographs the first range 1411, the camera mounted on the other vehicle 8 photographs the second range 1421, The camera mounted on the vehicle 9 photographs the third range 1431.

In this case, as shown in the figure, a first overlap region 1441 may exist in the first range 1411 and the second range 1421. In addition, a second overlapping region 1442 may exist in the first range 1411 and the third range 1431. That is, the first overlapping area 1441 is a part commonly belonging to the first range 1411 and the second range 1421 and is attached to the cameras 195-198 and the other vehicles 8 mounted on the vehicle 1 Which is photographed by all of the cameras. The second overlap region 1442 is a region commonly included in the first range 1411 and the third range 1431 and is attached to the camera 195-198 and the other vehicle 9 mounted on the vehicle 1 Which is photographed by all of the cameras.

FIG. 14B illustrates a main image 1451 corresponding to the first range 1411 shown in FIG. 14A. It is assumed that the main image 1451 is an ambient view image, as shown in the figure. The processor 170 determines whether or not the traveling image (e.g., the front image, the left side image, the right side image, and the rear image) generated by photographing the first range 1411 of the camera 195-198 mounted on the vehicle 1 As a basis, the main image 1451 can be generated.

The processor 170 may generate a main image 1451 in which an image 1452 corresponding to the vehicle 1 is centered. The image 1452 corresponding to the vehicle 1 may be generated directly by the processor 170 or stored in the memory 140 of the driver assistance device 100 or the memory 730 of the vehicle 1 .

14A, the first overlap region 1441 is located on the left side of the first range 1411. [ Accordingly, as shown in FIG. 14B, a part of the right side of the other vehicle 8 may appear on the left side of the main image 1451. FIG. Since the second overlap region 1442 is located on the right side of the first range 1411, a portion of the left vehicle body of the other vehicle 9 may appear on the right side of the main image 1451.

However, the pedestrian 51 does not appear in the main image 1451, and the pedestrian 51 is hidden by the other vehicle 9. Therefore, when the driver of the vehicle 1 travels in order to leave the parking zone 1400 depending on only the main image 1451, there is a risk of collision with the pedestrian 51 moving toward the vehicle 1 do.

FIG. 14C illustrates a first sub-image 1461 corresponding to the second range 1421 shown in FIG. 14A. It is assumed that the first sub-image 1461 is an arousal view image such as the main image 1451, as shown in FIG. The other vehicle 8 generates the first sub-image 1461 based on the running image (e.g., the forward image, the left-side image, the right-side image, and the rear image) generated by photographing the second range 1421 .

In addition, an image 1462 corresponding to the other vehicle 8 may be disposed at the center of the first sub-image 1461. 14A, since the first overlap area 1441 is located on the right side of the second range 1421, a part of the left side of the vehicle 1 may appear on the right side of the first sub video 1461.

FIG. 14D illustrates a second sub-image 1471 corresponding to the third range 1431 shown in FIG. 14A. It is assumed that the second sub-image 1471 is the same as the main image 1451 as shown in FIG. The other vehicle 9 generates the second sub image 1471 based on the running image (e.g., the forward image, the left image, the right image, and the back image) photographed in the third range 1431 .

In addition, an image 1472 corresponding to the other vehicle 9 may be disposed at the center of the second sub-image 1471. 14A, since the second overlap region 1442 is located on the left side of the third range 1431, a portion of the right side of the vehicle 1 may appear on the left side of the second sub image 1471. [

14E is a view showing an indoor state of the vehicle 1 in the situation shown in Fig. 14A. Fig. The processor 170 enters the image enlargement mode. The main image 1451 shown in Fig. 14B, the first sub-image 1461 shown in Fig. 14C, and the second sub-image 1471 shown in Fig. The extended image 1481 can be generated.

For example, the processor 170 may convert the image portion corresponding to the first overlapping region 1441 of the main image 1451 shown in FIG. 14A and the image portion corresponding to the first overlapping region 1441 of the first sub- The sizes and the like of at least one of the main image 1451 and the first sub image 1461 may be changed so that the image portions corresponding to the main image 1451 and the first sub image 1461 are merged. In addition, the processor 170 corresponds to the image portion corresponding to the second overlapping region 1442 of the main image 1451 shown in Fig. 14A and the second overlapping region 1442 corresponding to the second overlapping region 1442 of the second sub- The size, and the like of at least one of the main image 1451 and the second sub image 1471 may be changed so that the image portions corresponding to the main image 1451 and the second sub image 1471 are matched with each other.

For example, assuming that the width of the parking line constituting the parking section 1400 is constant, the processor 170 determines that the width of the parking line shown in the main image 1451 is equal to the width of the parking line shown in the first sub- The size of the main image 1451 is reduced to 1/2 or the size of the first sub image 1461 is doubled and then the main image 1451 and the first sub image 1461 are combined can do. In addition, the processor 170 may combine the second sub-image 1471 with the main image 1451 through the same method. Accordingly, the images generated by the different vehicles can be combined with no distortion to generate the extended image 1481.

 As a result, the driver assistance device 100 can provide the driver with an image in which the range of the main image 1451 is expanded in both the left and right directions. In this case, the extended image 1481 can be an overview image such as looking down on the vehicle 1 and two other vehicles 8, 9 from the top.

The processor 170 generates a control signal for displaying the extended image 1481 and transmits the control signal to the AVN apparatus 400, the display unit 741, and the driver assistance apparatus 100 The display unit 180 may be provided with at least one of them. 14E, the AVN apparatus 400 can display the extended image 1481 on the screen based on the control signal provided by the processor 170. [

Although not shown, the display unit 741 of the vehicle 1 and the display unit 180 of the driver assistance device 100 also display the extended image 1481 on the screen based on the control signal provided by the processor 170, Can be displayed. At this time, based on the screen sizes and ratios of the display section 741 of the vehicle 1, the display section 180 of the driver assistance device 100, and the AVN device 400, the processor 170 displays the extended image 1481 ) Are displayed differently from each other.

The driver visually confirms the surrounding situation of the vehicle 1 through the extended image 1481 in which the main image 1451, the first sub image 1461 and the second sub image 1471 are displayed together, 1451) are provided, it is possible to secure a wide field of view to the left and right sides. Particularly, when an object such as the pedestrian 51 existing in a position where it is difficult to obtain a view is displayed only on the basis of the driver's vision or the main image 1451 boarded on the vehicle 1, the extended image 1481 displayed on the AVN apparatus 400 or the like And can operate the vehicle 1 with care.

13A to 14E illustrate the situation in which there are two vehicles close to the vehicle 1, the driver assistance apparatus 100 can generate an extended image through the same method as described above even when three or more vehicles are present Will be apparent to those skilled in the art.

Meanwhile, the driver assistance device 100 according to an embodiment of the present invention can generate an extended image using a sub-image generated by another vehicle not adjacent to the vehicle 1, Let's take a look.

15A to 15C are views for explaining the operation of the driver assistance device 100 according to an embodiment of the present invention.

First, FIG. 15A illustrates a top view of an intersection 1500. As shown, it is assumed that the vehicle 1 and the three different vehicles 10, 11, 12 are running in the vicinity of the intersection 1500.

15A, the camera 195-198 mounted on the vehicle 1 photographs the first range 1511, the camera mounted on the other vehicle 10 photographs the second range 1521, The camera mounted on the vehicle 11 photographs the third range 1531 and the camera mounted on the other vehicle 12 photographs the fourth range 1541. [

In this case, as shown in the figure, the first range 1411 may not have a region overlapping any of the second to fourth ranges 1521, 1531, and 1541. On the other hand, the other vehicle 10 and the other vehicle 11 are close to each other, and a region 1571 overlapping the second range 1521 and the third range 1531 may be formed.

Fig. 15B illustrates a screen of the AVN apparatus 400 provided in the vehicle 1 in the situation shown in Fig. 15A and Fig. 15A. It is assumed that a screen of the AVN apparatus 400 is provided with a touch sensor and can receive a touch input of a driver.

The navigation image 1510 can be displayed on the screen of the AVN apparatus 400 in a section corresponding to the current position of the vehicle 1. [ For example, the processor 170 transmits the position information of the vehicle 1 received by the communication unit 120 or 710 of the vehicle 1 to the memory 140 of the driver assistance device 100 or the memory of the vehicle 1 It is possible to provide the AVN apparatus 400 with a control signal for displaying the navigation image 1510 for the matched portion, in correspondence with the electronic map stored in the electronic map 730. [ At this time, an indicator 1511 for guiding the current position of the vehicle 1 may be displayed on the navigation image 1510.

In contrast, in the intersection 1500, there is no other vehicle close to the vehicle 1, and thus the driver assistance device 100 is provided with the main image 1551 corresponding to the first range 1411, The image can not be received.

In this case, the driver assisting apparatus 100 photographs the second range to the fourth range 1521, 1531, 1541 in which the overlapping area with the other vehicle, that is, the first range 1411 is not formed, 11, and 12, and receive the sub-image generated by the selected other vehicle.

The driver assistance device 100 can select another vehicle located at a point remote from the vehicle 1 based on the user input.

Referring back to FIG. 15B, the driver can touch one point P1 of the entire area of the navigation image 1510 displayed on the screen of the AVN apparatus 400. [ At this time, a predetermined visual effect may occur on the navigation image 1510 at the touched point P1. The processor 170 can identify the other vehicles 10 and 11 running in the portion of the intersection 1500 corresponding to the area including the touched point P1 among the entire area of the navigation image 1510. [

For example, the processor 170 may obtain a GPS coordinate value that matches the touched point P1 of the electronic map. The processor 170 may then transmit the video request signal to the other vehicles 10 and 11 running at the actual position of the intersection 1500 corresponding to the GPS coordinates using the communication unit 120 or 710. In addition, the processor 170 may receive the sub-image transmitted by the other vehicles 10 and 11 in response to the video request signal, using the communication unit 120 or 710. [

15C illustrates a screen of the AVN apparatus 400 in which the extended video 1520 is displayed. The processor 170 may divide the screen of the AVN apparatus 400 into two or more sub-screens and display different information for each sub-screen. The processor 170 divides the screen of the AVN apparatus 400 into the first sub screen S1 and the second sub screen S2 and displays the navigation image 1510 on the first sub screen S1, And a control signal for displaying the extended image 1520 on the second sub screen S2 may be generated. The display unit 180 of the driver assistance device 100 and the display unit 741 of the vehicle are also displayed on the screen of the AVN apparatus 400 based on the control signal provided by the processor 170, It is apparent to those skilled in the art that the sub-screen can be divided into two sub-screens.

On the other hand, since the overlapping areas 1571 exist in the shooting ranges 1521 and 1531 of the two other vehicles 10 and 11, the sub image 1561 included in the extended image 1520 corresponds to the other vehicle 10 And the image 1572 corresponding to the other vehicle 11 may be displayed together.

Since the second range 1521 and the third range 1531 which are the imaging ranges of the two vehicles 10 and 11 do not overlap the first range 1511, the extended image 1520 includes the main image 1551 ) And the sub image 1561 may be displayed in a separated state.

Specifically, based on the positional information of the vehicle 1 and the positional information of the two vehicles 10 and 11, the processor 170 determines how far the two other vehicles 10 and 11 are moving in the vehicle 1 . The processor 170 may generate an extended image 1520 in which the main image 1551 and the sub image 1561 are spaced apart by a predetermined distance in a predetermined direction based on the determined result.

16A to 16D are views for explaining the operation of the driver assistance device 100 according to an embodiment of the present invention.

The driver assistance device 100 according to the present invention determines whether there is a warning section within a predetermined distance (e.g., 300 meters) from the vehicle 1, An extended image can be generated using an image.

Specifically, when the communication unit 120 or 710 receives the positional information (e.g., GPS coordinate value) of the vehicle 1, the processor 170 calculates the positional information of the vehicle 1 based on the positional information and the electronic map stored in the memory 140 or 730 , It is possible to judge whether or not there is an attention period within a predetermined distance from the vehicle 1. [ For example, the electronic map may include various kinds of information about a route on which the vehicle 1 can travel, such as road information (e.g., bump, road width, traffic sign), facility information (e.g., hospital, golf course, park) have. Therefore, the processor 170 can identify the interval of the predetermined distance from the GPS coordinate value of the vehicle 1 among various types of information included in the electronic map.

Here, the attention period may mean a section in which the driver of the vehicle 1 must pay more attention than the ordinary running situation. For example, an intersection, an uphill, a downhill, a crosswalk, a parking lot, a tunnel, a narrow road, a curve road, and a porthole may be included in the section of the state. The processor 170 may change the number, type, priority, etc. of the attention interval according to user input.

The processor 170 can guide only a predetermined number of consecutive attention periods in the order of higher priority when there are a plurality of attention intervals within a predetermined distance from the vehicle 1. [

In addition, the processor 170 can control so that, when there are a plurality of attention intervals within a predetermined distance from the vehicle 1, the plurality of attention intervals are visually displayed differently on the navigation screen or the like in accordance with the priority order. For example, the state of the highest priority may be displayed in red, and the state of the lowest priority may be displayed in blue.

In addition, the processor 170 can automatically enter the image enlargement mode when at least one attention interval is present within a predetermined distance from the vehicle 1. [

FIG. 16A illustrates a crosswalk 1601, which is one of the attention sections. The pedestrian crossing 1601 is a road area where pedestrians cross. 16A, the vehicle 1 may generate a main image corresponding to the first range 1611, and the other vehicle 13 may generate a sub image corresponding to the second range 1621. [

Since the crosswalk 1601 does not belong to the first range 1611, the driver can not confirm the real-time situation of the crosswalk 1601 with only the main image corresponding to the first range 1611. [ On the other hand, in the second range 1621, the pedestrian 61 near the pedestrian crosswalk 1601 belongs to at least a part of the pedestrian crosswalk 1601, so that the subpicture corresponding to the second range 1622 includes the pedestrian crosswalk 1601 A pedestrian 61 waiting to be crossed appears.

The processor 170 generates an extended image including the main image corresponding to the first range 1611 and the sub image corresponding to the second range 1621 so that the vehicle 1 is moved away from the crosswalk 1601 The driver of the vehicle 1 can confirm the pedestrian 61 in advance.

FIG. 16B illustrates a curve path 1602, which is one of the attention intervals. The curve path 1602 may be a road area having a curvature of a predetermined value or more. The curve road 1602 may cause a blind spot that is invisible to the driver of the vehicle 1, depending on the degree of curvature. 16B, the vehicle 1 may generate a main image corresponding to the first range 1612, and the other vehicle 13 may generate a sub image corresponding to the second range 1622. [

Since the curve range 1602 does not belong to the first range 1612, the driver can not confirm the real-time situation of the curve length 1602 only with the main image corresponding to the first range 1612. [ On the other hand, since the second range 1622 belongs to at least a part of the curve road 1602 and the obstacle 62 in the curve road 1602, the sub-image corresponding to the second range 1622 includes collision with the vehicle 1 A potential obstacle 62 appears.

The processor 170 generates an extended image including the main image corresponding to the first range 1612 and the sub image corresponding to the second range 1622 so that the vehicle 1 is moved away from the curve road 1602 The driver of the vehicle 1 can confirm the obstacle 62 in advance.

FIG. 16C illustrates a downhill road 1603, which is one of the attention zones. Downhill Road (1603) is a sloping road from high to low. According to Fig. 16C, the vehicle 1 may generate a main image corresponding to the first range 1613, and the other vehicle 13 may generate a sub image corresponding to the second range 1623. Fig.

Since the downhill road 1603 does not belong to the first range 1611, the driver can not confirm the real time situation of the downhill road 1603 only by the main image corresponding to the first range 1613. [ On the other hand, since the second range 1621 belongs to at least a part of the downhill road 1603 and the obstacle 63 in the downhill road 1603, the sub-image corresponding to the second range 1623 has collision with the vehicle 1 A possible obstacle 63 appears.

The processor 170 generates an extended image including the main image corresponding to the first range 1613 and the sub image corresponding to the second range 1623 so that the vehicle 1 is moved away from the downhill road 1603 The driver of the vehicle 1 can confirm the obstacle 63 in advance.

16D illustrates a tunnel 1604, which is one of the attention zones. According to Fig. 16D, the vehicle 1 may generate a main image corresponding to the first range 1614, and the other vehicle 14 may generate a sub image corresponding to the second range 1624. Fig.

Since the tunnel 1604 does not belong to the first range 1611, the driver can not check the real-time situation in the tunnel 1604 only with the main image corresponding to the first range 1614. [ On the other hand, since the second range 1621 belongs to at least a portion of the tunnel 1604 and the porthole 64 in the tunnel 1604, the sub-image corresponding to the second range 1624 may damage the vehicle 1 A port hole 64 is formed.

The processor 170 generates an extended image including the main image corresponding to the first range 1614 and the sub image corresponding to the second range 1624 so that the vehicle 1 is located at a position distant from the tunnel 1604 The driver of the vehicle 1 can confirm the port hole 64 in advance.

17A and 17B are views for explaining the operation of the driver assistance device 100 according to an embodiment of the present invention.

The driver assistance apparatus 100 according to the present invention determines whether or not a congestion section 1700 exists within a predetermined distance (for example, 300 meters) or an area selected by the driver from the vehicle 1, An extended image can be generated by using a sub-image generated by at least one other vehicle located at the position.

Specifically, when the communication unit 120 or 710 receives the traffic information for the current position of the vehicle 1, the processor 170 determines whether the vehicle 1 is within the predetermined distance (for example, 300 m) Or whether there is a congestion period 1700 in the area selected by the driver. Traffic information may include information on various factors affecting traffic such as traffic conditions, accidents, and construction by road section. For example, the processor 170 may set the congestion period 1700 as a period in which the vehicle can not travel at a speed of 60 km / h or more.

In addition, the processor 170 can automatically enter the image enlargement mode when there is a congestion section 1700 within a predetermined distance (e.g., 300 m) or an area selected by the driver.

Fig. 17A illustrates a top view of one way secondary. 17A, before the vehicle 1 enters the current congestion section 1700, a plurality of other vehicles 17-21 are located in the congestion section 1700. [

The vehicle 1 generates a main image corresponding to the first range 1711. [ The camera mounted on the other vehicle 17 photographs the second range 1721 and the camera mounted on the other vehicle 18 photographs the third range 1731 and the camera mounted on the other vehicle 17 The camera photographs the fourth range 1741 and the camera mounted on the other vehicle 20 photographs the fifth range 1751 and the camera mounted on the other vehicle 21 photographs the sixth range 1761 do.

Accordingly, the other vehicle 17-21 in the congestion section 1700 can generate the first to sixth sub images corresponding to the second to sixth ranges 1721, 1731, 1741, 1751, and 1761 . The driver assistance device 100 transmits a video request signal to the other vehicle 17-21 and the other vehicle 17-21 transmits the first to sixth sub video images to the driver assistance device 100 ).

FIG. 17B illustrates a screen of the AVN apparatus 400 provided in the vehicle 1 in the situation shown in FIG. 17A.

The processor 170 may generate a main image 1771 corresponding to the first range 1711. [ The main image 1771 may include an image 1772 corresponding to the vehicle 1. [ In addition, the processor 170 may generate the new sub-image 1772 by combining the first through sixth sub-images based on the overlapping portion between the first through sixth sub-images. In this case, the sub image 1772 may include images 1722, 1732, 1742, 1752, and 1762 corresponding to the other vehicles 17-21 in the congestion section 1700. [

The driver visually confirms the real-time situation of the congestion section 1700 through the extended image 1770 in which the main image 1771 and the sub image 1772 are displayed together, , It is possible to obtain help such as shortening the traveling time to the destination.

18A and 18B are views for explaining the operation of the driver assistance device 100 according to an embodiment of the present invention.

18A, the driver assisting apparatus 100 is provided with an overlapped portion between the main image corresponding to the first range 1811 of the vehicle 1 and the sub image corresponding to the second range 1812 of the other vehicle 22 The positional relationship between the main image and the sub image can be determined based on the information obtained by the sensing unit 760. [

More specifically, the sensing unit 760 acquires information on the relative position of the other vehicle 22 with respect to the vehicle 1 using a predetermined signal 1813 (e.g., ultrasonic waves, infrared rays, laser, etc.) .

The processor 170 determines the distance between the vehicle 1 and the other vehicle 22 based on the information obtained by the sensing unit 760 and the distance between the center line 1814 of the vehicle 1 and the center line 1814 of the other vehicle 22 The positional relationship between the vehicle 1 and the other vehicle 22 such as the angle?

The processor 170 changes at least one of the main image and the sub image based on the positional relationship between the vehicle 1 and the other vehicle 22 when the extended image is generated using the main image and the sub image, Rotation, and movement). For example, the processor 170 may rotate the sub image in the clockwise direction by an angle? Based on the main image, and then generate the extended image using the rotated sub image.

18B illustrates a situation in which the other vehicle 23 is located at a position where the sensing distance of the sensing unit 760 of the vehicle 1 is exceeded.

In this case, the processor 170 can determine the positional relationship between the main image and the sub-image based on the positional information of the vehicle 1 and the other vehicle 23 received by the communication unit 120 or 710. Also, based on the positional relationship between the main image and the sub image, an extended image can be generated by combining the main image and the sub image.

When generating the extended image, the processor 170 may combine the main image and the sub image based on the vehicle direction information of the vehicle 1 and the vehicle direction information of the other vehicle 23. [ The vehicle direction information V1 of the vehicle 1 may be acquired by the sensing unit 760 and the vehicle direction information V2 of the other vehicle 23 may be received by the communication unit 120 or 710. [ For example, the vehicle 1 and the other vehicle 23 may be provided with at least one sensor for sensing which direction the vehicle body is pointing, such as a geomagnetic sensor, an azimuth sensor, or the like.

18B, the processor 170 compares the GPS coordinates P11 of the vehicle 1 and the GPS coordinates P12 of the other vehicle 23 to determine whether the other vehicle 23 is at a distance from the vehicle 1 And direction can be determined. That is, the processor 170 can determine that the other vehicle 23 is located at a distance of (X2-X1) from the vehicle 1 in the X-axis and (Y2-Y1) in the Y-axis.

The processor 170 compares the vehicle body direction information V1 of the vehicle 1 and the vehicle body direction information V2 of the other vehicle 23 to compare the centerline of the vehicle 1 with the centerline of the other vehicle 23 It is possible to determine the angle formed.

The processor 170 calculates the distance between the center line of the vehicle 1 and the center line of the other vehicle 23 by reflecting the distance and direction of the other vehicle 23 away from the vehicle 1, And the sub-image, thereby making it possible to generate an extended image more matched to the actual situation.

19 is a view for explaining the operation of the driver assistance apparatus 100 according to an embodiment of the present invention.

Specifically, Fig. 19 is an example for explaining the operation of the driver assistance device 100 for controlling the movement of the vehicle 1 based on the extended image. For convenience of explanation, it is assumed that the vehicle 1 corresponds to the situation shown in Fig. 16B.

19, the driver assistance device 100 generates a control signal for instructing to change at least one of the speed and the direction of the vehicle 1 based on the information about the obstacle 62 appearing in the extended image . The information on the obstacle 62 may include various information such as the size of the obstacle 62, the distance to the obstacle 62, and the like.

For example, the vehicle 1 may enter the curve road 1602 from the current position P30 and then sequentially pass through the first to third points P31 to P31. In this case, the processor 170 may generate a control signal that instructs the vehicle 1 to decelerate the speed to below the first value before passing the first point P31. The processor 170 then sends a control command to command the vehicle 1 to decelerate the speed below a second value less than the first value before passing through the second point P32 after passing the first point P31. Signal can be generated. The processor 170 then sends a control command to command the vehicle 1 to decelerate the speed below a third value less than the second value before passing through the third point P33 after passing the second point P32. Signal can be generated. The driving unit 750 of the vehicle 1 can block the possibility of collision between the vehicle 1 and the obstacle 62 by decelerating the vehicle 1 based on the control signal provided from the processor 170. [

20A and 20B are views for explaining the operation of the driver assistance apparatus 100 according to an embodiment of the present invention.

20A illustrates a top view of the parking section 2000 into which the vehicle 1 has entered. 20A, three other vehicles 24-26 are located on the left side of the parking section 2000 and three other vehicles 27-29 are located on the right side of the parking section 2000. As shown in FIG. For the sake of convenience of explanation, it is assumed that the entire area of the parking section 2000 is within the range of the photographing range of the vehicle 1 and the photographing range of the six other vehicles 24-29.

On the other hand, there is a pedestrian 71 on the left side of the other vehicle 24. In this case, the driver of the vehicle 1 is hard to visually confirm the pedestrian 71 covered by the other vehicle 24. [

However, when it is assumed that the pedestrian 71 belongs to the photographing range of the other vehicle 24, the pedestrian 71 appears on the sub-image corresponding to the photographing range of the other vehicle 24. The processor 170 may generate an extended image using the sub-image generated by the other vehicle 24.

20B exemplifies an extended video 2010 displayed on the screen of the AVN apparatus 400 of the vehicle 1 when entering the parking section 2000 of the vehicle 1. Fig. For convenience of explanation, it is assumed that the entire area of the parking section 2000 shown in Fig. 20A appears in the extended image 2010. [ As shown in the figure, the extended image 2010 includes the first image 2011 corresponding to the vehicle 1, and the second through seventh images 2021 and 2021 corresponding to the second through seventh others 24-29, 2031, 2041, 2051, 2061, 2071).

In addition, since the pedestrian 71 is displayed on the sub-image generated by the other vehicle 24, the driver can confirm the pedestrian 71 hidden by the other vehicle 24 through the extended image 2010. [

On the other hand, the processor 170 may generate at least one path through which the vehicle 1 can travel based on the extended image 2010. [ For example, the processor 170 may analyze the extended image 2010 to determine whether there is a contiguous area in the parking space 2000 with the full width of the vehicle 1 secured.

20B, the processor 170 calculates the distance between the left side of the fifth image 2051 in the extended image 2010 and the right side of the sixth image 2061, converts the calculated distance into an actual distance, The actual distance can be compared with the full width of the vehicle 1. That is, the processor 170 may determine whether the vehicle 1 can enter between the fifth vehicle 27 and the sixth vehicle 28. [

The processor 170 detects the parking line of the parking section 2000 and judges that the present vehicle 1 has entered the parking section 2000 and judges that the fifth vehicle 27 and the sixth vehicle 28 It is possible to generate the information 2101 for guiding the parking of the inter-vehicle 1.

Accordingly, an indicator 2101 indicating that the parking space between the fifth vehicle 27 and the sixth vehicle 28 is vacant may be displayed on the screen of the AVN apparatus 400. [

The processor 170 also generates a control signal for instructing to display an indicator 2102 for guiding the generated route when generating at least one route in which the vehicle 1 can travel based on the extended image 2010 Can be generated. For example, as shown, the indicator 2102 may guide the path leading the vehicle 1 to the parking space between the fifth vehicle 27 and the sixth vehicle 28. [ At this time, the indicators 2101 and 2102 may be displayed in a manner of being overlaid on the extended image 2010.

The processor 170 also receives the first image 2011 corresponding to the vehicle 1 included in the extended image 2010 from the second to seventh images 2021, 2031, 2041, 2051, 2061 and 2071 It is possible to generate a control signal for instructing to distinguish and display. For example, the AVN apparatus 400 may generate the first image 2011 from the second to the seventh images 2021, 2031, 2041, 2041, and 2041 based on the control signal provided from the processor 170, 2051, 2061, and 2071).

FIG. 21 is a view for explaining the operation of the driver assistance device 100 according to an embodiment of the present invention.

The processor 170 may generate a control signal that instructs to apply a visual effect to an area of the extended image where an obstacle exists. At least one of the display section 741 of the vehicle 1, the display section 180 of the driver assistance device 100 and the AVN device 400 may be connected to the control signal provided from the processor 170 Accordingly, a visual effect can be given to an area where an obstacle exists in the extended image. For example, the processor 170 may provide AVN device 400 with a control signal to cause the effect of color, style, blinking, highlight, etc., of the entire area of the extended image to be close to the obstacle or obstacle.

Referring to FIG. 21, an indicator 2103 indicating the presence of the pedestrian 71 may be displayed in an area of the extended image 2010 shown in FIG. 20B where the pedestrian 71 appears. If the range of the extended image 2010 is wide, the driver may not recognize the pedestrian 71 appearing in the extended image 2010. Therefore, the pedestrian 71 can be emphasized and displayed by using the indicator 2103 compared to other parts of the extended image 2010, so that it is possible to help the driver of the vehicle 1 to recognize the pedestrian 71 quickly have.

22A and 22B are views for explaining the operation of the driver assistance device 100 for controlling the display state of the extended image according to an embodiment of the present invention. For convenience of explanation, the case where the extended video 2010 is displayed on the screen of the AVN apparatus 400 as shown in FIG. 20 will be exemplified.

Referring to FIG. 22A, the processor 170 may divide the screen of the AVN apparatus 400 into a plurality of sub-screens S11, S12, and S13. Further, the AVN apparatus 400 can display different information for each of the plurality of sub-screens S11, S12, and S13 under the control of the processor 170. [

For example, the AVN apparatus 400 displays the extended image 2010 on the first sub screen S11 and the obstacle such as the pedestrian 71 appearing on the extended image 2010 on the second sub screen S12 And icons 2201 to 2205 corresponding to various functions can be displayed on the third sub screen S13.

The information on the information on the obstacle displayed on the second sub screen S12 includes, for example, a warning image or message (e.g., "pedestrian detected") for guiding the detection of the obstacle, (E.g., " forward 5m "), a message (e.g.," slow down ") that guides the driver to perform an action to prevent collision with an obstacle.

Further, the driver can touch at least one of the icons 2201-2205 displayed on the screen of the AVN apparatus 400 to execute the function corresponding to the touched icons 2201-2205.

For example, the first icon 2201 corresponds to the enlarging function of the extended image 2010, the second icon 2202 corresponds to the reducing function of the extended image 2010, and the third icon 2203 corresponds to the enlarged image 2010), the fourth icon 2204 corresponds to the autonomous navigation function for the path generated based on the extended image 2010, the fifth icon 2205 corresponds to the rotation function of the extended image 2010 And can respond to the function of moving to the home screen.

Next, FIG. 22B shows an example in which the driver assistance device 100 changes the display state of the extended image based on the user input.

22B, when the driver touches the third icon 2203 of the third sub screen S13 shown in FIG. 22A, the processor 170 may rotate the extended image 2010 by a predetermined angle. For example, each time the driver touches the third icon 2203 once, the processor 170 may generate a control signal instructing to display the extended image 2010 rotated by 90 degrees in the clockwise direction. 22B illustrates a state in which the third icon 2203 is touched once and rotated 90 degrees in the clockwise direction. When the driver again touches the third icon 2203, the AVN apparatus 400 displays 180 The rotated extended image 2010 can be displayed. Of course, it is apparent to those skilled in the art that when the driver touches the third icon 2203 four times in total, the extended image 2010 is rotated 360 degrees and displayed as shown in FIG. 22A.

8 to 22B, the driver assistance device 100 receives the sub-image generated by the other vehicle and generates an extended image. However, the driver assistance device 100 may be configured to transmit the main image, that is, ) May be provided to other vehicles. Hereinafter, the operation of the driver assisting apparatus 100 for providing the main image to another vehicle will be described in more detail.

23 is a flowchart showing a control method of the driver assistance device 100 according to an embodiment of the present invention.

Referring to FIG. 23, the processor 170 enters the image providing mode (S2300). In the present invention, the image providing mode means a mode for providing at least one of at least a part of the main image of the vehicle 1 and information related to the main image to another vehicle.

The processor 170 can enter the image providing mode when the predetermined condition is satisfied.

For example, when the input unit 110 receives a user input that commands entry into the image providing mode, the processor 170 may enter the image providing mode. In this case, the user input may be at least one of various forms such as touch, voice, pushing, gesture, and the like.

For example, when the communication unit 120 or 710 receives a video request signal from another vehicle, the processor 170 may enter an image providing mode.

Next, the processor 170 generates a main image using at least one camera 195-198 (S2305). For example, when the processor 170 enters the image providing mode, at least one of the plurality of cameras 195-198 shown in FIG. 2B may be turned on to generate a main image. That is, the main image may include at least one of a front image, a left image, a right image, and a rear image of the vehicle 1.

At this time, the shape of the main image may vary. For example, the main image may be a still image or a moving image. For example, the main image may be a forward image, a left image, a right image, and a left image of the vehicle 1, as shown in FIG. 2C. It is assumed that the image is an all-view image including all the backward images.

Then, the processor 170 determines whether an obstacle exists in the main image generated in step S2305 (S2310). For example, the processor 170 performs object detection on the main image of the vehicle 1, and detects, based on the object detection result, a reference distance from the vehicle 1 (for example, 2m) to determine whether there is an obstacle whose accident risk exceeds the reference value.

Next, if it is determined in step S2310 that an obstacle exists in the main image, the processor 170 selects another vehicle to which the main image is to be transmitted (S2315), and transmits the main image to the selected other vehicle (S2320). At this time, the processor 170 may select only a part of a plurality of other vehicles located in the periphery of the vehicle 1 according to a predetermined criterion and transmit the main image.

Specifically, the processor 170 can select another vehicle to which the main image is to be transmitted based on the positional relationship between the vehicle 1 and the obstacle. For example, when one vehicle is running ahead of and behind the vehicle 1, and an obstacle present in the main image is located behind the vehicle 1, the processor 170 moves the rearward of the vehicle 1 It is possible to provide the main image only to the other vehicles in the running.

The processor 170 may select another vehicle to which the main image is to be transmitted based on the risk of an obstacle appearing in the main image. For example, when there are ten other vehicles in the vicinity of the vehicle 1, the processor 170 determines that the risk of the obstacle detected from the main image is the first value, If the risk of the obstacle is a second value larger than the first value, the main image can be provided to all of the ten other vehicles. That is, the higher the risk of the obstacle detected from the main image is, the more the processor 170 can increase the number of other vehicles to provide the main image. Here, the risk may be a value calculated on the basis of the distance between the vehicle 1 and the obstacle, the size of the obstacle, the type of the obstacle, and the like.

In addition, the processor 170 may select another vehicle to which the main image is to be transmitted, based on the video request signal received by the communication unit 120 or 710. For example, the processor 170 may control the communication unit 120 or 710 to transmit the main image only to another vehicle that has transmitted the video request signal.

At this time, the communication unit 120 or 710 can directly transmit the sub-image to another vehicle. That is, the communication unit 120 or 710 can directly transmit a sub-image to another vehicle based on a vehicle-to-vehicle communication network.

Alternatively, the communication unit 120 or 710 may transmit the main image to another vehicle through intermediation of at least one external device. For example, the driver assistance device 100 may transmit the main image to the external server, and the external server may transmit the sub image transmitted from the driver assistance device 100 to another vehicle.

8 and FIG. 23, the image expansion mode and the image providing mode have been separately described, but these are exemplary and do not limit the scope of the present invention. The driver assistance device 100 according to the embodiment of the present invention may simultaneously enter the image expansion mode and the image providing mode. That is, the driver assistance device 100 may provide the main image to the other vehicle in the image enlargement mode, and may generate the extended image in the image providing mode. Alternatively, the driver assistance device 100 can always generate an extended image or provide the main image to another vehicle when the vehicle 1 is in operation.

The embodiments of the present invention described above are not only implemented by the apparatus and method but may be implemented through a program for realizing the function corresponding to the configuration of the embodiment of the present invention or a recording medium on which the program is recorded, The embodiments can be easily implemented by those skilled in the art from the description of the embodiments described above.

It is to be understood that both the foregoing general description and the following detailed description of the present invention are exemplary and explanatory and are intended to be illustrative, The present invention is not limited to the drawings, but all or some of the embodiments may be selectively combined so that various modifications may be made.

1: vehicle 100: driver assistance device

Claims (10)

A display unit;
At least one camera for photographing the periphery of the vehicle and generating a main image;
A communication unit for receiving a plurality of sub images generated by a plurality of other vehicles; And
Selecting at least one of the plurality of sub-images based on a user input for selecting at least one of the plurality of vehicles,
And a processor for generating an extended image using the main image and the selected sub-image,
The processor comprising:
Wherein the controller selects a first vehicle traveling in an area corresponding to an area including a touched point out of the electronic maps displayed on the display unit,
Displays the extended image on the display and displays an image corresponding to the first vehicle on the extended image. The method according to claim 1,
The processor comprising:
And generates information on an obstacle based on the extended image. 3. The method of claim 2,
The processor comprising:
And generates a control signal instructing to change at least one of a speed and a direction of the vehicle based on the information on the obstacle. 3. The method of claim 2,
The processor comprising:
And controls the display unit to give a predetermined visual effect to an area corresponding to the obstacle among the extended images. 5. The method of claim 4,
The processor comprising:
And controls the display unit to give a visual effect to an area corresponding to an obstacle located on a road on which the vehicle is to be moved, out of the extended images. The method according to claim 1,
The processor comprising:
A screen of the display unit is divided into a first sub screen and a second sub screen,
A navigation image is displayed on the first sub screen,
And displays the extended image on the second sub-screen. The method according to claim 1,
The processor comprising:
Based on the positional information of the vehicle, whether or not there is an attention zone within a predetermined distance from the vehicle,
The control unit controls the communication unit to transmit the video request signal to another vehicle located within the predetermined interval,
The above-
At least one of an intersection, an uphill, a downhill, a crosswalk, a parking lot, a tunnel, a narrow road, a porthole, and a curve road. The method according to claim 1,
The processor comprising:
When there is no overlapping area between the main image and the selected sub-image,
And generates the extended image by combining the main image and the sub image based on the position information of the vehicle, the vehicle body direction information of the vehicle, the position information of the other vehicle, and the vehicle body direction information of the other vehicle. The method according to claim 1,
The processor comprising:
Generates a control signal for instructing the vehicle to enter an autonomous driving mode for the path based on the extended image,
The path includes:
Wherein the vehicle is a route connecting from a current position of the vehicle to a position outside the vehicle's field of view based on a positional relationship between the vehicle and the other vehicle. The method according to claim 1,
The processor comprising:
And generates the extended image based on a period that is changed in accordance with the speed of the vehicle.

Images