WO2015093853A1

WO2015093853A1 - Vehicle driving auxiliary device and vehicle having same

Info

Publication number: WO2015093853A1
Application number: PCT/KR2014/012500
Authority: WO
Inventors: 한종우
Original assignee: 엘지전자 주식회사
Priority date: 2013-12-17
Filing date: 2014-12-17
Publication date: 2015-06-25
Also published as: KR20150070832A

Abstract

The present invention relates to a vehicle driving auxiliary device and a vehicle having the same. A vehicle driving auxiliary device according to an embodiment of the present invention comprises a stereo camera and a processor which detects a lane on the basis of a stereo image received from the stereo camera, receives parking and stopping-related information related to the detected lane information, and controls to display the parking and stopping-related information. Accordingly, the parking and stopping-related information can be displayed on the basis of a photographed image.

Description

Vehicle driving assistance device and vehicle having same

The present invention relates to a vehicle driving assistance apparatus and a vehicle having the same, and more particularly, to a vehicle driving assistance apparatus capable of displaying parking-related information based on a photographed image and a vehicle having the same.

The vehicle is a device for moving in the direction desired by the user on board. An example is a car.

On the other hand, for the convenience of the user using the vehicle, various types of sensors, electronic devices, etc. are provided. In particular, various devices for driving convenience of the user have been developed.

It is an object of the present invention to provide a vehicle driving assistance apparatus capable of displaying parking-related information based on a photographed image and a vehicle having the same.

The vehicle driving assistance apparatus according to the embodiment of the present invention for achieving the above object, based on the stereo camera and the stereo image received from the stereo camera, detects the lane, and receives the parking-related information associated with the detected lane information And a processor for controlling to display parking related information.

On the other hand, the vehicle driving assistance apparatus according to an embodiment of the present invention for achieving the above object, based on the mono camera, the radar, and the image received from the mono camera, detects the lane, the parking stop associated with the detected lane information And a processor that receives the related information and controls to display the parking related information.

On the other hand, the vehicle driving assistance apparatus according to an embodiment of the present invention for achieving the above object, based on the rider performing scanning on the external object, and the scanned image received from the rider, detecting the lane, And a processor configured to receive parking related information related to the information and to display the parking related information.

On the other hand, the vehicle according to an embodiment of the present invention for achieving the above object, the sensor unit for sensing the vehicle state, the steering drive unit for driving the steering device, the brake drive unit for driving the brake device, the engine drive unit for driving the engine And a lane based on a suspension driver for driving a suspension device, a steering driver, a brake driver, a power source driver, a suspension driver, a stereo camera, a display, and a stereo image received from the stereo camera. And a vehicle driving assistance apparatus including a processor configured to receive parking related information related to the detected lane information and to display the parking related information on a display.

The vehicle driving assistance apparatus according to an embodiment of the present disclosure controls to detect a lane, receive parking-related information related to the detected lane information, and display parking-related information based on a stereo image received from a stereo camera. do. Accordingly, parking stop related information can be displayed on the display based on the stereo image.

In particular, as in the exemplary embodiment of the present invention, when the lane information collected based on the stereo image and the location information are used, the current vehicle location and the surrounding situation can be grasped to display accurate parking related information.

On the other hand, in the image photographed by the stereo camera, it is possible to display the parking availability message only for the lane to be checked, whereby the user can easily check whether parking is possible for the roadside while driving the vehicle. .

On the other hand, based on a stereo image, when detecting an object, the disparity can be calculated using the stereo image, and the object can be detected based on the disparity information, thereby reducing the data processing speed and the like.

On the other hand, the vehicle driving assistance apparatus according to another embodiment of the present invention, based on the image received from the mono camera, to detect the lane, to receive the parking-related information associated with the detected lane information, to display the parking-related information To control. Accordingly, it is possible to display the parking-related information on the display based on the mono image.

On the other hand, the vehicle driving assistance apparatus according to another embodiment of the present invention, based on the scan image received from the rider, detects the lane, receives the parking-related information associated with the detected lane information, and receives the parking-related information Control to display. Accordingly, the parking-related information can be displayed on the display based on the scanned image.

1 is a view showing the appearance of a vehicle having a stereo camera according to an embodiment of the present invention.

FIG. 2 is a view illustrating an appearance of a stereo camera attached to the vehicle of FIG. 1.

3A to 3B illustrate various examples of an internal block diagram of a vehicle driving assistance apparatus according to an embodiment of the present invention.

4A-4B illustrate various examples of internal block diagrams of the processor of FIGS. 3A-3B.

5A through 5B are views referred to for describing an operation of the processor of FIGS. 4A through 4B.

6A to 6B are views referred to for describing the operation of the vehicle driving assistance apparatus of FIGS. 3A to 3B.

7 is an example of an internal block diagram of the electronic control apparatus inside the vehicle of FIG. 1.

8 is a flowchart illustrating a method of operating a vehicle driving assistance apparatus according to an embodiment of the present invention.

9 to 11C are views referred to for describing the operating method of FIG. 8.

12A to 14B are views referred to for describing another embodiment of the present invention.

FIG. 15 is a view illustrating an exterior of a vehicle including a mono camera according to another embodiment of the present invention.

16A illustrates an example of an internal block diagram of a vehicle driving assistance apparatus according to another embodiment of the present invention.

16B illustrates an example of an internal block diagram of a vehicle driving assistance apparatus according to another embodiment of the present invention.

17A-17B illustrate various examples of internal block diagrams of the processor of FIG. 16A.

18A through 18B are views for explaining an operation of the processor of FIG. 17A.

Hereinafter, with reference to the drawings will be described the present invention in more detail.

The suffixes "module" and "unit" for components used in the following description are merely given in consideration of ease of preparation of the present specification, and do not impart any particular meaning or role by themselves. Therefore, the "module" and "unit" may be used interchangeably.

The vehicle described herein may be a concept including an automobile and a motorcycle. In the following, a vehicle is mainly described for a vehicle.

Meanwhile, the vehicle described herein may be a concept including a vehicle having an engine, a hybrid vehicle having an engine and an electric motor, an electric vehicle having an electric motor, and the like. Hereinafter, the description will be given mainly on a vehicle provided with an engine.

Meanwhile, the vehicle driving assistance apparatus described in the present specification may be referred to as an Advanced Driver Assistance Systems (ADAS) or an Advanced Driver Assistance Apparatus (ADAA). Hereinafter, a vehicle driving assistance apparatus for a vehicle and a vehicle having the same according to various embodiments of the present disclosure will be described.

Referring to the drawings, the vehicle 200 includes wheels 103FR, 103FL, 103RL,... Rotated by a power source, a steering wheel 150 for adjusting a traveling direction of the vehicle 200, and an interior of the vehicle 200. It may be provided with a stereo camera 195 provided in.

The stereo camera 195 may include a plurality of cameras, and the stereo image obtained by the plurality of cameras may be signal processed in the vehicle driving assistance apparatus (100 of FIG. 3).

Meanwhile, the drawing illustrates that the stereo camera 195 includes two cameras.

Referring to the drawings, the stereo camera module 195 may include a first camera 195a having a first lens 193a and a second camera 195b having a second lens 193b.

Meanwhile, the stereo camera module 195 includes a first light shield 192a and a second light for shielding light incident on the first lens 193a and the second lens 193b, respectively. The shield 192b may be provided.

The stereo camera module 195 of the drawing may be a structure that can be attached to or detached from the ceiling or the windshield of the vehicle 200.

A vehicle driving assistance device (100 in FIG. 3) having such a stereo camera module 195 obtains a stereo image of the front of the vehicle from the stereo camera module 195 and based on the stereo image, a disparity. ) Detect, perform object detection on the at least one stereo image based on the disparity information, and continuously track the movement of the object after object detection.

The vehicle driving assistance apparatus 100 of FIGS. 3A to 3B may process the stereo image received from the stereo camera 195 based on computer vision to generate vehicle related information. The vehicle related information may include vehicle control information for direct control of the vehicle, or vehicle driving assistance information for driving guide to the vehicle driver.

First, referring to FIG. 3A, the vehicle driving assistance apparatus 100 of FIG. 3A includes a communication unit 120, an interface unit 130, a memory 140, a processor 170, a power supply unit 190, and a stereo camera. 195. In addition, an audio input unit (not shown) and an audio output unit (not shown) may be provided.

The communication unit 120 may exchange data with the mobile terminal 600 or the server 500 in a wireless manner. In particular, the communication unit 120 may exchange data wirelessly with a mobile terminal of a vehicle driver. As a wireless data communication method, various data communication methods such as Bluetooth, WiFi Direct, WiFi, and APiX are possible.

The communication unit 120 may receive weather information, road traffic condition information, for example, TPEG (Transport Protocol Expert Group) information from the mobile terminal 600 or the server 500. In the meantime, the vehicle driving assistance apparatus 100 may transmit the real-time traffic information grasped based on the stereo image to the mobile terminal 600 or the server 500.

On the other hand, when the user rides in the vehicle, the user's mobile terminal 600 and the vehicle driving assistance device 100 may perform pairing with each other automatically or by executing the user's application.

The interface unit 130 may receive vehicle-related data or transmit a signal processed or generated by the processor 170 to the outside. To this end, the interface unit 130 may perform data communication with the ECU 770, the AVN (Audio Video Navigation) device 400, the sensor unit 760, etc. in the vehicle by wired communication or wireless communication. have.

The interface unit 130 may receive map information related to vehicle driving through data communication with the AVN device 400.

The interface unit 130 may receive sensor information from the ECU 770 or the sensor unit 760.

Here, the sensor information includes vehicle direction information, vehicle position information (GPS information), vehicle angle information, vehicle speed information, vehicle acceleration information, vehicle tilt information, vehicle forward / reverse information, battery information, fuel information, tire information, vehicle It may include at least one of lamp information, vehicle interior temperature information, vehicle interior humidity information.

Such sensor information may include heading sensors, yaw sensors, gyro sensors, position modules, vehicle forward / reverse sensors, wheel sensors, vehicle speed sensors, It may be obtained from a vehicle body tilt sensor, a battery sensor, a fuel sensor, a tire sensor, a steering sensor by steering wheel rotation, a vehicle interior temperature sensor, a vehicle interior humidity sensor, and the like. Meanwhile, the position module may include a GPS module for receiving GPS information.

Meanwhile, among the sensor information, the vehicle direction information, the vehicle position information, the vehicle angle information, the vehicle speed information, the vehicle tilt information, and the like related to the vehicle driving may be referred to as vehicle driving information.

The memory 140 may store various data for operations of the overall vehicle driving assistance apparatus 100, such as a program for processing or controlling the processor 170.

The audio output unit (not shown) converts an electrical signal from the processor 170 into an audio signal and outputs the audio signal. To this end, a speaker or the like may be provided. The audio output unit (not shown) may output sound corresponding to the operation of the input unit 110, that is, the button.

The audio input unit (not shown) may receive a user voice. To this end, a microphone may be provided. The received voice may be converted into an electrical signal and transmitted to the processor 170.

The processor 170 controls the overall operation of each unit in the vehicle driving assistance apparatus 100.

In particular, the processor 170 performs computer vision-based signal processing. Accordingly, the processor 170 obtains a stereo image of the front of the vehicle from the stereo camera 195, performs a disparity operation on the front of the vehicle based on the stereo image, and based on the calculated disparity information. , Object detection may be performed on at least one of the stereo images, and after the object detection, the movement of the object may be continuously tracked.

In particular, when detecting an object, the processor 170 may perform lane detection, vehicle detection, pedestrian detection, traffic sign detection, road surface detection, and the like. Can be.

The processor 170 may perform a distance calculation on the detected surrounding vehicle, a speed calculation of the detected surrounding vehicle, a speed difference calculation with the detected surrounding vehicle, and the like.

Meanwhile, the processor 170 may receive weather information, road traffic information, for example, TPEG (Transport Protocol Expert Group) information through the communication unit 120.

On the other hand, the processor 170, the vehicle driving assistance apparatus 100 may grasp, in real time, the traffic situation information around the vehicle, which is determined based on the stereo image.

The processor 170 may receive map information or the like from the AVN device 400 through the interface unit 130.

Meanwhile, the processor 170 may receive sensor information from the ECU 770 or the sensor unit 760 through the interface unit 130. Here, the sensor information includes vehicle direction information, vehicle position information (GPS information), vehicle angle information, vehicle speed information, vehicle acceleration information, vehicle tilt information, vehicle forward / reverse information, battery information, fuel information, tire information, vehicle It may include at least one of lamp information, vehicle interior temperature information, vehicle interior humidity information.

The power supply unit 190 may supply power required for the operation of each component under the control of the processor 170. In particular, the power supply unit 190 may receive power from a battery inside the vehicle.

The stereo camera 195 may include a plurality of cameras. Hereinafter, as described in FIG. 2 and the like, two cameras are provided.

The stereo camera 195 may be detachable from the ceiling or the windshield of the vehicle 200, and may include a first camera 195a having a first lens 193a and a second camera having a second lens 193b. 195b.

On the other hand, the stereo camera 195 has a first light shield 192a and a second light shield for shielding light incident on the first lens 193a and the second lens 193b, respectively. The part 192b may be provided.

Next, referring to FIG. 3B, the vehicle driving assistance apparatus 100 of FIG. 3B may further include an input unit 110 and a display 180 as compared to the vehicle driving assistance apparatus 100 of FIG. 3A. Hereinafter, only the description of the input unit 110 and the display 180 will be described.

The input unit 110 may include a plurality of buttons or a touch screen attached to the vehicle driving assistance apparatus 100, particularly, the stereo camera 195. Through a plurality of buttons or a touch screen, the vehicle driving assistance apparatus 100 may be turned on and operated. In addition, various input operations may be performed.

The display 180 may display an image related to the operation of the vehicle driving assistance apparatus. For displaying such an image, the display 180 may include a cluster or a head up display (HUD) on the front surface of the vehicle. On the other hand, when the display 180 is a HUD, it may include a projection module for projecting an image on the windshield of the vehicle 200.

4A-4B illustrate various examples of internal block diagrams of the processor of FIGS. 3A-3B, and FIGS. 5A-5B are diagrams referred to in describing the operation of the processor of FIGS. 4A-4B.

First, referring to FIG. 4A, FIG. 4A is an example of an internal block diagram of the processor 170. The processor 170 in the vehicle driving assistance apparatus 100 includes an image preprocessor 410 and a disparity calculator 420. ), An object detector 434, an object tracking unit 440, and an application unit 450 may be provided.

The image preprocessor 410 receives a stereo image from the stereo camera 195 and performs preprocessing.

In detail, the image preprocessing unit 410 may include noise reduction, rectification, calibration, color enhancement, and color space conversion for a stereo image. CSC), interpolation, camera gain control, and the like. Accordingly, a stereo image that is clearer than a stereo image captured by the stereo camera 195 may be obtained.

The disparity calculator 420 receives a stereo image signal-processed by the image preprocessor 410, performs stereo matching on the received stereo images, and performs stereo matching. Obtain a disparity map. That is, disparity information about the stereo image of the front of the vehicle may be obtained.

In this case, the stereo matching may be performed in units of pixels of stereo images or in units of predetermined blocks. The disparity map may refer to a map in which stereo parallax information of stereo images, that is, left and right images, is numerically represented.

The segmentation unit 432 may perform segmentation and clustering on at least one of the stereo images based on the disparity information from the disparity calculator 420.

In detail, the segmentation unit 432 may separate the background and the foreground from at least one of the stereo images based on the disparity information.

For example, an area in which the disparity information is less than or equal to a predetermined value in the disparity map may be calculated in the background, and the corresponding part may be excluded. Thereby, the foreground can be relatively separated.

As another example, an area in which the disparity information is greater than or equal to a predetermined value in the disparity map may be calculated in the foreground and a corresponding portion may be extracted. Thereby, the foreground can be separated.

As described above, by separating the foreground and the background based on the disparity information information extracted based on the stereo image, the signal processing speed, the signal processing amount, and the like can be shortened in the subsequent object detection.

Next, the object detector 434 may detect the object based on the image segment from the segmentation unit 432.

That is, the object detector 434 may detect an object with respect to at least one of the stereo images based on the disparity information information.

In detail, the object detector 434 may detect an object with respect to at least one of the stereo images. For example, an object can be detected from the foreground separated by image segments.

Next, an object verification unit 436 classifies and verifies the separated object.

To this end, the object verification unit 436 may include an identification method using a neural network, a support vector machine (SVM) method, a method of identifying by AdaBoost using a haar-like feature, or a histograms of oriented gradients (HOG). Techniques can be used.

The object checking unit 436 may check the objects by comparing the objects stored in the memory 140 with the detected objects.

For example, the object checking unit 436 may check surrounding vehicles, lanes, road surfaces, signs, dangerous areas, tunnels, and the like, which are positioned around the vehicle.

The object tracking unit 440 performs tracking on the identified object. For example, in order to sequentially identify the object in the obtained stereo images, calculate the motion or motion vector of the identified object, track the movement of the object, etc. based on the calculated motion or motion vector. Can be. Accordingly, it is possible to track surrounding vehicles, lanes, road surfaces, signs, dangerous areas, tunnels, and the like, which are located around the vehicle.

Next, the application unit 450 may calculate a risk of the vehicle 200 based on various objects located around the vehicle, for example, another vehicle, a lane, a road surface, a sign, and the like. In addition, it is possible to calculate the possibility of colliding with the vehicle ahead, whether the vehicle slips.

The application unit 450 may output, as vehicle driving assistance information, a message for informing the user of such information, based on the calculated risk, the possibility of colliding or the slip. Alternatively, a control signal for attitude control or driving control of the vehicle 200 may be generated as vehicle control information.

4B is another example of an internal block diagram of a processor.

Referring to the drawings, the processor 170 of FIG. 4B has the same internal configuration unit as the processor 170 of FIG. 4A, but the signal processing order is different. Only the differences are described below.

The object detector 434 may receive a stereo image and detect an object with respect to at least one of the stereo images. Unlike FIG. 4A, based on the disparity information, for the segmented image, the object may be detected directly from the stereo image, instead of detecting the object.

Next, the object verification unit 436 classifies the detected and separated objects based on the image segments from the segmentation unit 432 and the objects detected by the object detection unit 434. , Verify.

5A and 5B are views referred to for describing a method of operating the processor 170 of FIG. 4A based on stereo images obtained in the first and second frame sections, respectively.

First, referring to FIG. 5A, during the first frame period, the stereo camera 195 acquires a stereo image.

The disparity calculator 420 in the processor 170 receives the stereo images FR1a and FR1b signal-processed by the image preprocessor 410 and performs stereo matching on the received stereo images FR1a and FR1b. To obtain a disparity map 520.

The disparity map 520 is a leveling disparity between the stereo images FR1a and FR1b. The greater the disparity level is, the closer the vehicle is to the distance, and the smaller the disparity level is, the lower the disparity map is. We can calculate that distance of is far.

On the other hand, when displaying such a disparity map, the disparity map may be displayed such that the larger the disparity level, the higher the luminance, and the smaller the disparity level, the lower the luminance.

In the drawing, in the disparity map 520, the first to

fourth lanes

528a, 528b, 528c, 528d and the like have corresponding disparity levels, respectively, the construction area 522, the first front vehicle 524. For example, each of the second front vehicles 526 has a corresponding disparity level.

The segmentation unit 432, the object detection unit 434, and the object confirmation unit 436 may perform segment, object detection, and detection on at least one of the stereo images FR1a and FR1b based on the disparity map 520. Perform object verification.

In the figure, using the disparity map 520, it illustrates that object detection, and verification for the second stereo image FR1b is performed.

That is, in the image 530, the first to

fourth lanes

538a, 538b, 538c, 538d, the construction area 532, the first front vehicle 534, and the second front vehicle 536 detect an object. And confirmation can be performed.

Next, referring to FIG. 5B, during the second frame period, the stereo camera 195 acquires a stereo image.

The disparity calculator 420 in the processor 170 receives the stereo images FR2a and FR2b signal-processed by the image preprocessor 410 and performs stereo matching on the received stereo images FR2a and FR2b. To obtain a disparity map 540.

In the drawing, in the disparity map 540, the first to

fourth lanes

548a, 548b, 548c, 548d and the like have corresponding disparity levels, respectively, the construction area 542, the first front vehicle 544. For example, each of the second front vehicles 546 has a corresponding disparity level.

The segmentation unit 432, the object detection unit 434, and the object confirmation unit 436 may perform segment, object detection, and detection on at least one of the stereo images FR2a and FR2b based on the disparity map 520. Perform object verification.

In the figure, using the disparity map 540, it illustrates that object detection, and verification is performed on the second stereo image FR2b.

That is, in the image 550, the first to

fourth lanes

558a, 558b, 558c and 558d, the construction area 552, the first front vehicle 554, and the second front vehicle 556 detect the object. And confirmation can be performed.

Meanwhile, the object tracking unit 440 may perform tracking on the identified object by comparing FIGS. 5A and 5B.

In detail, the object tracking unit 440 may track the movement of the corresponding object based on the motion or the motion vector of each object identified in FIGS. 5A and 5B. Accordingly, tracking of a lane, a construction area, a first front vehicle, a second front vehicle, and the like located around the vehicle can be performed.

6A to 6B are views referred to for describing the operation of the vehicle driving assistance apparatus of FIG. 3.

First, FIG. 6A is a diagram illustrating a situation in front of a vehicle captured by the stereo camera 195 provided in a vehicle. In particular, the vehicle front situation is displayed in a bird eye view.

Referring to the drawings, from the left to the right, the first lane 642a, the second lane 644a, the third lane 646a, the fourth lane 648a is located, the first lane 642a and the second A construction area 610a is located between the lanes 644a, a first front vehicle 620a is located between the second lane 644a and the third lane 646a, and the third lane 646a and the fourth lane. It can be seen that the second front vehicle 630a is disposed between the lanes 648a.

Next, FIG. 6B illustrates displaying the vehicle front situation detected by the vehicle driving assistance apparatus together with various types of information. In particular, the image as shown in FIG. 6B may be displayed on the display 180 or the AVN device 400 provided in the vehicle driving assistance apparatus.

6B illustrates that information display is performed based on an image captured by the stereo camera 195, unlike FIG. 6A.

Referring to the drawings, from the left to the right, the first lane 642b, the second lane 644b, the third lane 646b, the fourth lane 648b, the first lane 642b and the second lane A construction area 610b is located between the lanes 644b, a first front vehicle 620b is located between the second lane 644b and the third lane 646b, and the third lane 646b and the fourth lane. It can be seen that the second front vehicle 630b is disposed between the lanes 648b.

The vehicle driving assistance apparatus 100 performs signal processing based on the stereo image captured by the stereo camera 195 to provide the construction area 610b, the first front vehicle 620b, and the second front vehicle 630b. You can check the object. In addition, the first lane 642b, the second lane 644b, the third lane 646b, and the fourth lane 648b may be identified.

On the other hand, in the drawing, to illustrate the object confirmation for the construction area 610b, the first front vehicle 620b, the second front vehicle 630b, it is illustrated that each is highlighted with a border.

On the other hand, the vehicle driving assistance apparatus 100 is based on the stereo image captured by the stereo camera 195, the distance to the construction area 610b, the first front vehicle 620b, the second front vehicle 630b. Information can be calculated.

In the drawing, the calculated first distance information 611b, second distance information 621b, and third distance corresponding to each of the construction area 610b, the first front vehicle 620b, and the second front vehicle 630b. It illustrates that information 631b is displayed.

Meanwhile, the vehicle driving assistance apparatus 100 may receive sensor information about the vehicle from the ECU 770 or the sensor unit 760. In particular, the vehicle speed information, the gear information, the yaw rate information (yaw rate) indicating the speed at which the rotation angle (concave angle) of the vehicle changes, and the angle information of the vehicle can be received, and such information can be displayed.

In the drawing, the vehicle speed information 672, the gear information 671, and the yaw rate information 673 are displayed on the vehicle front image upper portion 670, and the angle of the vehicle is displayed on the vehicle front image lower portion 680. While the information 682 is illustrated, various examples are possible. In addition, the width information 683 of the vehicle and the curvature information 681 of the road may be displayed together with the angle information 682 of the vehicle.

On the other hand, the vehicle driving assistance apparatus 100 may receive speed limit information, etc. for the road on which the vehicle is traveling, through the communication unit 120 or the interface unit 130. In the figure, it illustrates that the speed limit information 640b is displayed.

The vehicle driving assistance apparatus 100 may display various pieces of information illustrated in FIG. 6B through the display 180. Alternatively, the vehicle driving assistance device 100 may store various pieces of information without additional display. In addition, the information may be used for various applications.

FIG. 7 is an example of an internal block diagram of the electronic control apparatus in the vehicle of FIG. 1.

Referring to the drawings, the vehicle 200 may include an electronic control apparatus 700 for controlling the vehicle. The electronic control apparatus 700 can exchange data with the vehicle driving assistance apparatus 100 and the AVN apparatus 400 described above.

The electronic control apparatus 700 includes an input unit 710, a communication unit 720, a memory 740, a lamp driver 751, a steering driver 752, a brake driver 753, a power source driver 754, and a sunroof driver. 755, suspension driver 756, air conditioning driver 757, window driver 758, airbag driver 759, sensor unit 760, ECU 770, display unit 780, audio output unit 785. The power supply unit 790 may be provided.

The input unit 710 may include a plurality of buttons or a touch screen disposed in the vehicle 200. Through a plurality of buttons or touch screens, it is possible to perform various input operations.

The communication unit 720 may exchange data with the mobile terminal 600 or the server 500 in a wireless manner. In particular, the communication unit 720 may exchange data wirelessly with the mobile terminal of the vehicle driver. As a wireless data communication method, various data communication methods such as Bluetooth, WiFi Direct, WiFi, and APiX are possible.

The communication unit 720 may receive weather information, road traffic information, for example, TPEG (Transport Protocol Expert Group) information from the mobile terminal 600 or the server 500.

On the other hand, when the user is in the vehicle, the mobile terminal 600 and the electronic control apparatus 700 of the user can perform pairing with each other automatically or by executing the user's application.

The memory 740 may store various data for operating the entire electronic control apparatus 700, such as a program for processing or controlling the ECU 770.

The lamp driver 751 may control turn on / off of lamps disposed inside and outside the vehicle. In addition, it is possible to control the intensity, direction, etc. of the light of the lamp. For example, control of a direction indicator lamp, a brake lamp, and the like can be performed.

The steering driver 752 may perform electronic control of a steering apparatus in the vehicle 200. As a result, the traveling direction of the vehicle can be changed.

The brake driver 753 may perform electronic control of a brake apparatus in the vehicle 200. For example, the speed of the vehicle 200 may be reduced by controlling the operation of the brake disposed on the vehicle. As another example, by varying the operation of the brakes disposed on the left wheels and the right wheels, the traveling direction of the vehicle 200 may be adjusted to the left or the right.

The power source driver 754 may perform electronic control of the power source in the vehicle 200.

For example, when the fossil fuel-based engine is a power source, the power source driver 754 may perform electronic control of the engine. Thereby, the output torque of an engine, etc. can be controlled.

As another example, when the electric-based motor is a power source, the power source driver 754 may perform control on the motor. Thereby, the rotation speed, torque, etc. of a motor can be controlled.

The sunroof driver 755 may perform electronic control of a sunroof apparatus in the vehicle 200. For example, the opening or closing of the sunroof can be controlled.

The suspension driver 756 may perform electronic control of a suspension apparatus in the vehicle 200. For example, when the road surface is curved, the suspension device may be controlled to control the vibration of the vehicle 200 to be reduced.

The air conditioning driver 757 may perform electronic control of an air cinditioner in the vehicle 200. For example, when the temperature inside the vehicle is high, the air conditioner may operate to control the cool air to be supplied into the vehicle.

The window driver 758 may perform electronic control of a suspension apparatus in the vehicle 200. For example, the opening or closing of the left and right windows of the side of the vehicle can be controlled.

The airbag driver 759 may perform electronic control of an airbag apparatus in the vehicle 200. For example, in case of danger, the airbag can be controlled to burst.

The sensor unit 760 senses a signal related to traveling of the vehicle 100. To this end, the sensor unit 760 may include a heading sensor, a yaw sensor, a gyro sensor, a position module, a vehicle forward / reverse sensor, and a wheel sensor. , A vehicle speed sensor, a vehicle body tilt sensor, a battery sensor, a fuel sensor, a tire sensor, a steering sensor by steering wheel rotation, a vehicle internal temperature sensor, a vehicle internal humidity sensor, and the like.

Accordingly, the sensor unit 760 includes vehicle direction information, vehicle position information (GPS information), vehicle angle information, vehicle speed information, vehicle acceleration information, vehicle tilt information, vehicle forward / reverse information, battery information, fuel information, A sensing signal may be acquired for tire information, vehicle lamp information, vehicle interior temperature information, vehicle interior humidity information, and the like.

On the other hand, the sensor unit 760 may include an accelerator pedal sensor, a pressure sensor, an engine speed sensor, an air flow sensor (AFS), an intake temperature sensor (ATS), a water temperature sensor (WTS), and a throttle. Position sensor (TPS), TDC sensor, crank angle sensor (CAS), etc. may be further provided.

The ECU 770 may control the overall operation of each unit in the electronic control apparatus 700.

By the input by the input unit 710, a specific operation may be performed, or a signal sensed by the sensor unit 760 may be received and transmitted to the vehicle driving assistance apparatus 100, and the map information may be transmitted from the AVN device 400. It can receive, and can control the operation of the driving unit (751, 752, 753, 754, 756).

In addition, the ECU 770 may receive weather information, road traffic condition information, for example, TPEG (Transport Protocol Expert Group) information from the communication unit 720.

The display unit 780 may display an image related to the operation of the vehicle driving assistance apparatus. For displaying such an image, the display unit 780 may include a cluster or a head up display (HUD) on the front surface of the vehicle. On the other hand, when the display unit 780 is a HUD, it may include a projection module for projecting an image on the windshield of the vehicle 200. Meanwhile, the display unit 780 may include a touch screen that can be input.

The audio output unit 785 converts the electrical signal from the ECU 770 into an audio signal and outputs the audio signal. To this end, a speaker or the like may be provided. The audio output unit 785 may output a sound corresponding to the operation of the input unit 710, that is, the button.

The power supply unit 790 may supply power required for the operation of each component under the control of the ECU 770. In particular, the power supply unit 790 may receive power from a battery (not shown) in the vehicle.

8 is a flowchart illustrating a method of operating a vehicle driving assistance apparatus according to an exemplary embodiment of the present invention, and FIGS. 9 to 11C are views for explaining the operation method of FIG. 8.

First, referring to FIG. 8, the processor 170 of the vehicle driving assistance apparatus 100 receives a stereo image from a stereo camera (S810). The processor 170 of the vehicle driving assistance apparatus 100 detects a lane based on the received stereo image (S820). Meanwhile, FIG. 9 illustrates that operation S810 and operation S820 (S820) are performed in the vehicle driving assistance apparatus 100, respectively.

As described with reference to FIG. 2, the processor 170 of the vehicle driving assistance apparatus 100 receives the first image through the first camera 195a through the stereo camera 195 disposed inside the vehicle. Through the two cameras 195b, a second image is received.

Since there is a distance between the first camera 195a and the second camera 195b, disparity occurs in the first image and the second image.

The processor 170 of the vehicle driving assistance apparatus 100 stereo-matches the first image and the second image, and obtains a disparity map according to stereo matching by stereo matching. That is, disparity information about the stereo image of the front of the vehicle may be obtained.

The disparity map is a leveling disparity between stereo images FR1a and FR1b. The greater the disparity level is, the closer the vehicle is to the disparity level, and the smaller the disparity level is, the closer the vehicle is to the disparity map. It can mean far.

The processor 170 of the vehicle driving assistance apparatus 100 may perform segment, object detection, and object verification on at least one of the first image and the second image by using the disparity information.

The processor 170 of the vehicle driving assistance apparatus 100 detects lanes, detects lanes, detects vehicles, detects pedestrians, detects traffic signs, and detects roads. Surface detection and the like.

In particular, in accordance with an embodiment of the present invention, the processor 170 of the vehicle driving assistance apparatus 100 uses the detected lane.

Next, the processor 170 of the vehicle driving assistance apparatus 100 transmits the detected lane information and the location information to the outside (S830). Then, the vehicle-related parking information is received from the outside (S840). Meanwhile, FIG. 9 illustrates that operation S810 and operation S840 and S840 are performed by the vehicle driving assistance apparatus 100 and the server 500.

The processor 170 of the vehicle driving assistance apparatus 100 may include lane information detected based on the stereo image, and sensor information received from the ECU 720 or the sensor unit 760 through the interface unit 130. The location information (GPS information) may be controlled to be transmitted to the outside through the communication unit 120.

In particular, the communication unit 120 may transmit lane information and location information (GPS information) to the mobile terminal 600 or the server 500. In the following description, the lane information and the location information (GPS information) are transmitted to the server 500 in order to collect related information.

The server 500 may collect parking stop related information based on the received lane information and location information (GPS information). In addition, the collected parking-related information may be transmitted to the vehicle driving assistance apparatus 100. Accordingly, the communication unit 120 of the vehicle driving assistance apparatus 100 may receive the parking stop related information from the server 500.

The parking-related information may include at least one of parking availability information, parking available time information, parking available information, and intermittent camera information according to the type of roadside lane.

Next, the processor 170 of the vehicle driving assistance apparatus 100 controls to display the received parking-related information on the display 180 (S850). Meanwhile, FIG. 9 illustrates that operation 850 is performed by the vehicle driving assistance apparatus 100.

The normal parking-related information providing service cannot provide accurate parking-related information when the current vehicle is at the boundary between the parking allowable area and the impossible area or the like based on the location information only.

However, as in the embodiment of the present invention, when the lane information collected based on the stereo image and the location information (GPS information) are used, the current vehicle location and the surrounding situation can be grasped to display accurate parking related information. It becomes possible.

On the other hand, the provision of the parking stop related information may be provided during vehicle driving, vehicle temporary stopping, and vehicle parking.

That is, the processor 170 of the vehicle driving assistance apparatus 100 transmits the outermost lane information and the location information to the server 500 when the vehicle travels the outermost lane of the road through lane detection. The vehicle-related parking information may be received and the received parking-related information may be displayed on the display 180.

In particular, the processor 170 of the vehicle driving assistance apparatus 100 transmits the location information to the server 500 when the speed of the vehicle is less than or equal to a predetermined speed while the vehicle travels on the outermost lane of the road, and the parking vehicle related information. And control the display to display the received parking-related information on the display 180. Thereby, the user can automatically display the parking-related information desired by the user without a separate input operation, so that the user's convenience can be increased.

On the other hand, the speed of the vehicle is calculated based on the position information (GPS information) received from the AVN device 400 through the interface unit 130, or via the interface unit 130, ECU 770 or the sensor unit It may be calculated based on the vehicle speed information received from 760.

10 illustrates an internal block diagram of a processor for displaying parking related information.

Referring to the drawings, the processor 170 of the vehicle driving assistance apparatus 100 may include a lane detecting unit 1010 and a parking stop determining unit 1020.

The processor 170 of the vehicle driving assistance apparatus 100 receives map information from the AVN device 400 through the interface unit 130, and via the interface unit 130, the ECU 720 or the sensor unit ( The sensor information may be received from 760, and a stereo image Sim may be received from the stereo camera 195.

Among the sensor information, the location information (GPS information) Slo and the vehicle driving information Scar may be included.

Here, the vehicle driving information Scar may include vehicle direction information, vehicle position information, vehicle angle information, vehicle speed information, vehicle tilt information, and the like.

The lane detection unit 1010 performs lane detection based on the stereo image Sim. In particular, it is possible to detect whether the bus is a dedicated lane, a variable lane, or a roadside lane.

The lane detection unit 1010 may transmit information Sin about the roadside lane, which is the outermost lane of the road, to the parking stop determination unit 1020.

The parking stop determination unit 1020 may receive the location information (GPS information) Slo, the vehicle driving information Scar, and the information Sin about the road lane to perform the parking stop determination.

To this end, the parking stop determination unit 1020 transmits the location information (GPS information) Slo and the roadside information Sin to the server 500 so as to be transmitted to the outside, and from outside, Information (Sinf) can be received.

In addition, the stop determination unit 1020 determines whether or not parking is possible by using the current vehicle position, the current vehicle speed, and parking related information (Sinf), and displays a message Spa on whether parking is possible. The output may be 180.

In addition, when parking is possible, the parking guide message Sg about the available parking time, the parking fee, and the like is output to the display 180.

Accordingly, the display 180 may display a message Spa or a parking guide message Sg regarding whether parking is possible.

11A illustrates that the vehicle travels on the outermost lane 1100 of the road while the road is running. The stereo camera 195 in the vehicle 200 may acquire a stereo image.

In this case, the stereo camera 195 in the vehicle 200 may acquire an image including the outermost lane of the road, and the processor 170 of the vehicle driving assistance apparatus 100 detects the lane of the outermost lane. Can be performed.

In the figure, four types of lanes are illustrated as outermost lanes. The first outermost lane 1110a is a white solid line, indicating that parking is possible, and the second outermost lane 1120a is a yellow dotted line, indicating that parking is prohibited and only stops within a predetermined time period, and the third outermost lane. The lane 1130a is a yellow solid line and indicates that the parking vehicle may vary according to time and day of the week, and the fourth outermost lane 1140a is a double yellow solid line and may indicate that parking is prohibited.

The processor 170 of the vehicle driving assistance apparatus 100 may transmit four types of lane information and location information to the server 500, and receive parking information for each lane type from the server 500.

FIG. 11B illustrates that when the vehicle travels on the outermost lane 1100 of the road, the parking vehicle related information is output through the display in the vehicle.

In particular, (a) of FIG. 11B illustrates that when the vehicle 200 travels below a predetermined speed while driving the outermost lane 1100 of the road, or the vehicle 200 runs the outermost lane 1100 of the road. In the meantime, a case in which the right direction indicating lamp or the like is turned on is illustrated.

The processor 170 of the vehicle driving assistance apparatus 100 may receive the parking information for each lane type according to the lane type, and control the display 180 to display a message indicating whether parking is possible in consideration of the vehicle driving. have.

FIG. 11B (b) illustrates that the

parking availability messages

1110b, 1120b, and 1130b for each lane type are displayed on the display area 1109 on the windshield of the vehicle through the HUD.

For example, when the vehicle travels at a predetermined speed or less while driving the outermost lane 1100 of the road, or the vehicle 200 drives the outermost lane 1100 of the road, the right direction indicator lamp or the like turns on. In this case, as illustrated in (b) of FIG. 11B,

parking availability messages

1110b, 1120b, and 1130b may be displayed.

Alternatively, when the vehicle travels and enters the outermost lane 1100 of the road, the lane crossing is detected and based on the lane crossing, the parking availability message 1110b as shown in FIG. 11B (b). 1120b and 1130b may be displayed.

On the other hand, it is preferable to display a parking stop availability message only for the identified lane in the image photographed by the stereo camera.

In the drawing, the first outermost lane 1110b, the second outermost lane 1120b, and the third outermost lane 1130b are illustrated as visible lanes on the front of the vehicle, and correspondingly, the display area ( In operation 1109, the

parking availability messages

1110b, 1120b, and 1130b for the first to third

outermost lanes

1110b, 1120b, and 1130b may be displayed. Thereby, while driving a vehicle, the user can easily confirm whether or not parking on the roadside is possible.

11C illustrates that when the vehicle stops, the parking-related information is output through the display in the vehicle.

In particular, (a) of FIG. 11C illustrates that the vehicle 200 is parked near the third outermost lane 1130a.

Accordingly, the processor 170 of the vehicle driving assistance apparatus 100 may control to display the parking stop guide message associated with the third outermost lane 1130a.

FIG. 11C (b) illustrates that the parking guide message 1150c is displayed on the display area 1109 on the windshield of the vehicle via the HUD.

In detail, the parking guide message 1150c may include a type of lane, parking time available information, parking fee information, and the like. Accordingly, when the vehicle is stopped, the user can easily check the parking guide message for the roadside.

Meanwhile, when one of the

parking availability messages

1110b, 1120b, and 1130b is selected by the user input while the vehicle

parking availability messages

1110b, 1120b, and 1130b are displayed while driving the vehicle in FIG. It is also possible to output a parking guide message for the lane.

For example, when the user selects the parking availability message 1130b corresponding to the third outermost lane, the parking guide message 1150c may be displayed as illustrated in (b) of FIG. 11C.

On the other hand, although not shown in the drawing, when the parking time is exceeded or is imminent after the vehicle is parked, the processor 170 of the vehicle driving assistance apparatus 100 through the communication unit 120, the user's mobile terminal 600 ) May be controlled to transmit a message such as parking timeout information or impending information. As a result, the user's ease of use may be increased.

Alternatively, even after the vehicle is parked, while the stereo camera is activated and the stereo image of the front is obtained, the processor 170 of the vehicle driving assistance apparatus 100 detects the object of the vehicle tow by the object detection. In this case, it is also possible to carry out a control for starting the vehicle. Alternatively, the mobile terminal 600 may be controlled to transmit a message such as towing impending information. As a result, the user's ease of use may be increased.

Alternatively, even after the vehicle is parked, when the stereo camera is activated and all the vehicles parked in front of the vehicle are moved while the stereo image of the front is obtained, a notification message is transmitted to the user's mobile terminal 600. It can also be controlled. As a result, the user's ease of use may be increased.

Meanwhile, the vehicle parking lot related information display corresponding to the outermost lane detection and the outermost lane can be variously modified based on the stereo image from the stereo camera described in the present invention.

For example, it is possible to detect bus-only lanes and display information related to bus-only lanes corresponding to bus-only lanes. As another example, detection of variable lanes and display of variable lane-related information corresponding to variable lanes is possible. It is also possible to detect the front vehicle and display overtaking information corresponding to the front vehicle. This will be described with reference to FIGS. 12A to 14B below.

First, FIG. 12A is a diagram illustrating a vehicle front situation photographed by the stereo camera 195 provided in the vehicle. In particular, the vehicle front situation is displayed in a bird eye view.

Referring to the drawings, from the left to the right, the first lane 1211a, the second lane 1210a is a bus-only lane, it is exemplified to be distinguished from other lanes. In addition, the bus 1200a travels in a bus-only lane between the first lane 1211a and the second lane 1210a, and

other vehicles

1203a and 1205a travel.

The processor 170 of the vehicle driving assistance apparatus 100 obtains a stereo image of the front of the vehicle from the stereo camera 195 provided inside the vehicle, and based on the stereo image, lanes and buses shown in FIG. 12A. The object detection may be performed on the vehicles.

In addition, the processor 170 of the vehicle driving assistance apparatus 100 may transmit location information and detected bus dedicated lane information to the server 500, and receive bus related lane related information from the server 500.

12B illustrates an example of an image 1250 that includes bus dedicated lane related information.

The image 1250 including bus-only lane related information may be displayed on the display 180, for example, the HUD. As described above, the image 1250 may be displayed in the display area 1109 on the windshield.

The image 1250 of FIG. 12B includes a bus 1200b, distance information 1611b with the bus,

front vehicles

1203b and 1205b, distance information with each

front vehicle

621b and 631b, and a bus-only lane 1210b. 1211b) and bus-only lane related information 1210c are displayed.

In particular, the bus dedicated lane related information 1210c may include bus dedicated lane operating time information. As a result, the user can easily grasp information on the bus-only lane.

Next, FIG. 13A is a diagram illustrating a vehicle front situation photographed by the stereo camera 195 provided in the vehicle. In particular, the vehicle front situation is displayed in a bird eye view.

Referring to the drawings, the outermost lane 1320a on the right is a variable lane, which exemplifies a different lane. In addition, the bus 1200a travels in a bus-only lane between the first lane 1211a and the second lane 1210a, and

other vehicles

1203a and 1205a travel.

The processor 170 of the vehicle driving assistance apparatus 100 obtains a stereo image of the front of the vehicle from the stereo camera 195 provided in the vehicle, and displays an object for the lanes, buses, and vehicles shown in FIG. 13A. Detection can be performed.

The processor 170 of the vehicle driving assistance apparatus 100 may transmit the location information and the detected variable lane information to the server 500, and receive the variable lane related information from the server 500.

13B illustrates an example of an image 1370 that includes variable lane related information.

The image 1370 including bus-specific lane-related information may be displayed on the display 180, for example, the HUD. As described above, the image 1370 may be displayed in the display area 1109 on the windshield.

Similarly to FIG. 12B, the image 1370 of FIG. 13B displays

front vehicles

1200b, 1203b, 1205b, and

distance information

1611b, 621b, 631b with each front vehicle, and other variable lane related information. Illustrates that 1320c is displayed.

In particular, the variable lane related information 1320c may include variable lane operating time information. As a result, the user can easily grasp information on the variable lane.

Next, FIG. 14A is a diagram illustrating a vehicle front situation photographed by the stereo camera 195 provided in the vehicle. Since FIG. 14A is the same as FIG. 12A, the difference is described below.

The stereo camera 195 provided in the vehicle may acquire a stereo image of the front of the vehicle and perform object detection on the lanes, buses, and vehicles shown in FIG. 14A.

In particular, the processor 170 of the vehicle driving assistance apparatus 100 may determine whether passing of the vehicle ahead is possible based on the detected lanes, buses, and vehicles.

14B illustrates an example of an image 1470 that includes passable information.

The image 1470 including bus-specific lane-related information may be displayed on the display 180, for example, the HUD. As described above, the image 1470 may be displayed in the display area 1109 on the windshield.

Similarly to FIG. 14B, the image 1470 of FIG. 14B displays

front vehicles

1200b, 1203b, and 1205b, and

distance information

1611b, 621b, and 631b with each front vehicle, and other information that can be passed. 1420c is displayed. As a result, the user can easily grasp the information that can be transferred.

Referring to the drawings, the vehicle 200 according to another embodiment of the present invention, the wheels (150FR, 135FL, 135RL, ...) rotating by the power source, the handle 150 for adjusting the traveling direction of the vehicle 200 And a mono camera 193 and a radar 194 provided in the vehicle 200.

The mono image through the mono camera 193 and the distance information through the radar 194 may be signal processed in the vehicle driving assistance apparatus 2100 of FIG. 16A.

Referring to the drawings, an internal block of the vehicle driving assistance device 2100 of FIG. 16A is similar to an internal block of the vehicle driving assistance device 100 of FIG. 3, but based on a stereo image from the stereo camera 195. The difference is that the signal processing is performed based on the mono image through the mono camera 193 and the distance information through the radar 194, rather than the processing. That is, the vehicle attitude control signal can be generated. In the following, only the differences are described.

The mono camera 193 may be detachable from the ceiling or the windshield of the vehicle 200 and may include a single camera having a lens.

The radar 194 may be detachable from the ceiling or the windshield of the vehicle 200, transmit radio waves of a predetermined frequency in front of the vehicle, and receive radio waves reflected from an object in front of the vehicle.

The processor 2170 may calculate distance information based on the difference between the transmission wave and the reception wave in the radar 194. In addition, the object may be separated, detected, and recognized by matching the mono image and distance information through the mono camera 193.

Next, FIG. 16B illustrates an example of an internal block diagram of a vehicle driving assistance apparatus according to another embodiment of the present invention.

Referring to the drawings, the inner block of the vehicle driving assistance device 2150 of FIG. 16B is similar to the inner block of the vehicle driving assistance device 2100 of FIG. 16A, but not the mono camera 193 and the radar 194. The difference lies in having a rider (Lidar) 2101 that performs scanning on an external object.

In describing the difference, the rider 2101 may acquire a scan image of a forward situation by a laser scanning method, and the processor 2270 performs signal processing based on the scan image received from the rider. . That is, the vehicle attitude control signal can be generated.

17A-17B illustrate various examples of internal block diagrams of the processor of FIG. 16A, and FIGS. 18A-18B are diagrams for reference to an operation description of the processor of FIG. 17A.

The processor internal block of FIGS. 17A to 17B is similar to the processor of FIGS. 4A to 4B except that the processor internal block does not include a disparity calculator.

Instead, there is a difference in receiving the distance information Sd corresponding to the disparity information from the external radar 194. The distance information Sd is input to the segmentation unit 432 and may be used when segmenting an image.

18A to 18B are views referred to for describing a method of operating the processor 2170 of FIG. 17A based on a mono image obtained in each of the first and second frame periods.

Referring to FIGS. 18A to 18B, the method is similar to FIGS. 5A to 5B except that since a mono image and radar based distance information are used, there is no need to generate a disparity map.

First, referring to FIG. 18A, during a first frame period, the mono camera 193 acquires a mono image FR1, and the radar obtains distance information Sd1.

Accordingly, the segmentation unit 432, the object detection unit 434, and the object confirmation unit 436 in the processor 2170 may be configured to include the first segment in the mono image FR1 based on the radar based distance information Sd1. The lanes to the

fourth lanes

538a, 538b, 538c, 538d, the construction area 532, the first front vehicle 534, and the second front vehicle 536 can be detected and confirmed.

Next, referring to FIG. 18B, during the second frame period, the mono camera 193 obtains a mono image FR2 and the radar obtains distance information Sd2.

Accordingly, the segmentation unit 432, the object detection unit 434, and the object confirmation unit 436 in the processor 2170 may be configured to include the first segment in the mono image FR2 based on the radar based distance information Sd2. The lanes to the

fourth lanes

558a, 558b, 558c, and 558d, the construction area 552, the first front vehicle 554, and the second front vehicle 556 can be detected and confirmed.

Meanwhile, the object tracking unit 440 may perform tracking on the identified object by comparing FIGS. 18A and 18B.

In detail, the object tracking unit 440 may track the movement of the corresponding object based on the motion or the motion vector of each object identified in FIGS. 18A and 18B. Accordingly, tracking of a lane, a construction area, a first front vehicle, a second front vehicle, and the like located around the vehicle can be performed.

Meanwhile, the vehicle driving assistance device 2100 including the mono camera 193 and the radar 194 or the vehicle driving assistance device 2150 including the rider 2101 described in FIGS. 15 to 18 are illustrated in FIGS. 8 to 18. As described with reference to FIG. 14B, it is possible to control to detect a lane, to receive parking related information related to the detected lane information, and to display the parking related information on a display.

Specifically, the processor 2170 of the vehicle driving assistance device 2100 or the processor 2270 of the vehicle driving assistance device 2150, when the vehicle travels near the outer lane of the road and the speed of the vehicle is below a predetermined speed, The vehicle may be controlled to display parking related information related to the outer lane on the display.

The parking stop related information may include at least one of parking availability information, parking available time information, parking available information, and intermittent camera information obtained corresponding to the location and time of the vehicle.

The vehicle driving assistance apparatus and the vehicle having the same according to an embodiment of the present invention are not limited to the configuration and method of the embodiments described above, but the embodiments may be modified in various ways so that various modifications can be made. All or part of the examples may be optionally combined.

On the other hand, the vehicle driving assistance apparatus or the method of operating the vehicle of the present invention can be implemented as code that the processor can read on a recording medium that can be read by the processor provided in the vehicle driving assistance apparatus or the vehicle. The processor-readable recording medium includes all kinds of recording devices that store data that can be read by the processor. Examples of the processor-readable recording medium include ROM, RAM, CD-ROM, magnetic tape, floppy disk, optical data storage device, and the like, and may also be implemented in the form of a carrier wave such as transmission over the Internet. . The processor-readable recording medium can also be distributed over network coupled computer systems so that the processor-readable code is stored and executed in a distributed fashion.

In addition, although the preferred embodiment of the present invention has been shown and described above, the present invention is not limited to the specific embodiments described above, but the technical field to which the invention belongs without departing from the spirit of the invention claimed in the claims. Of course, various modifications can be made by those skilled in the art, and these modifications should not be individually understood from the technical spirit or the prospect of the present invention.

The present invention is applicable to a vehicle driving assistance apparatus and a vehicle having the same, and in particular, to a vehicle driving assistance apparatus capable of displaying parking-related information based on a photographed image and a vehicle having the same.

Claims

Stereo cameras;

And a processor configured to detect a lane based on the stereo image received from the stereo camera, receive a parking stop related information related to the detected lane information, and display the parking stop related information. Auxiliary devices.
The method of claim 1,

Communication unit for exchanging data with a mobile terminal or a server; further comprising,

The communication unit,

And transmitting the detected lane information to the mobile terminal or server, and receiving the parking-related information from the mobile terminal or server.
The method of claim 2,

An interface unit for exchanging data with at least one in-vehicle device;

The interface unit receives the sensor information and the map information of the vehicle,

The communication unit,

And transmitting location information of the detected lane information and the sensor information of the vehicle to the mobile terminal or the server.
The method of claim 1,

The display further includes;

The processor,

And when the vehicle travels near the outer lane of a road and the speed of the vehicle is less than or equal to a predetermined speed, controlling to display the parking vehicle related information related to the outer lane on the display.
The method of claim 1,

The parking-related information,

And at least one of parking availability information, parking availability time information, parking availability information, and intermittent camera information obtained corresponding to the location and time of the vehicle.
The method of claim 1,

The display further includes;

The processor,

And when the vehicle stops or parks in an outer lane of the road, controlling the vehicle to display parking time information and parking fee information on the display.
The method of claim 1,

The display further includes;

The display,

And a projection module for projecting an image onto the windshield of the vehicle.
The method of claim 1,

The processor,

A disparity calculator configured to perform a disparity operation of the stereo image; And

And an object detector configured to detect an object of at least one of the stereo images based on the disparity information of the stereo image.

And the object detector is configured to perform the lane detection.
The method of claim 8,

The processor,

And an object tracking unit which performs tracking on the detected object.
The method of claim 8,

The processor,

A segmentation unit segmenting an object in the stereo image based on disparity information of the stereo image; And

And an object checking unit to classify the detected object.

The object detector,

And based on the segmented object, perform object detection on at least one of the stereo images.
Mono camera;

Radar;

And a processor configured to detect a lane based on the image received from the mono camera, to receive parking related information related to the detected lane information, and to display the parking related information. Device.
A rider performing scanning on an external object;

And a processor configured to detect a lane based on the scanned image received from the rider, receive parking related information related to the detected lane information, and display the parking related information. Device. A vehicle driving assistance device, characterized in that for generating a control signal.
A sensor unit for sensing a vehicle state;

A steering driver for driving a steering device;

A brake drive unit for driving a brake device,

A power source driver for driving a power source;

A suspension driver for driving the suspension device;

A controller for controlling the steering driver, the brake driver, the power source driver, and the suspension driver; And

And a stereo camera, and a processor configured to detect a lane based on the stereo image received from the stereo camera, receive a parking related information related to the detected lane information, and control to display the parking related information. Vehicle; including.
The method of claim 13,

The vehicle driving assistance device,

An interface unit for exchanging data with at least one in-vehicle device; And

Communication unit for exchanging data with a mobile terminal or a server; further comprising,

The interface unit receives the sensor information and the map information of the vehicle,

The communication unit,

And transmitting location information of the detected lane information and the sensor information to the mobile terminal or the server.
The method of claim 13,

The display further includes;

The processor,

And when the vehicle travels near the outer lane of a road and the speed of the vehicle is less than or equal to a predetermined speed, controlling to display the parking vehicle related information related to the outer lane on the display.
The method of claim 13,

The display further includes;

The processor,

And when the vehicle stops or parks in an outer lane of the road, controlling the vehicle to display parking time information and parking fee information on the display.
The method of claim 13,

The processor,

A disparity calculator configured to perform a disparity operation of the stereo image; And

And an object detector configured to detect an object of at least one of the stereo images based on the disparity information of the stereo image.

And the object detecting unit performs the lane detection.