KR101613849B1 - Driving Assist System for the Vehicles - Google Patents

Abstract

The present invention relates to a vehicle driving assist system, including: a broadband camera to photograph and input images around a vehicle; a map generation part to generate a Depth Map on the basis of the images inputted from the broadband camera; an image generation part to generate a GNF image on the basis of the images inputted from the broadband camera; and a broadband image generation part to generate a three-dimensional map template by receiving data of the map generation part and the image generation part. The broadband camera inputs an image including information of four channels having different wavelengths, and generates a three-dimensional map, and matches a location of the vehicle to the three-dimensional map. Thus, the broadband camera can recognize a road and an obstacle easily during vehicle driving using a smaller number of cameras. Thus, recognition ability as to a driving possible region is improved highly, and positioning of a vehicle navigation is easy, and bio-recognition ability of a driver monitoring camera is improved and thus driver convenience due to improvement of performance of a driving assist apparatus is improved highly.

Description

[0001] The present invention relates to a driving assist system,

BACKGROUND OF THE INVENTION 1. Field of the Invention [0001] The present invention relates to a driving assistance system for an automobile, and more particularly, to a driving assistance system for a vehicle that recognizes a space when driving the vehicle and recognizes the driving course accurately.

Recently, there is a tendency that a vehicle is mounted on a system that assists the running of the vehicle and recognizes the surrounding situation for the convenience of the driver and provides the information.

Techniques for recognizing roads in accordance with vehicle driving have been implemented in various ways.

Generally, a photodiode is used for image acquisition, a narrow band image data collection device, and a device for extracting a boundary line.

However, these devices have a limited range of processing, and their utilization has been limited. For example, bandwidths other than visible light (< 400 nm, > 700 nm) are not processed and thus bandwidth utilization is limited, and resource limitation for color space narrowing, boundary extraction and object recognition due to bandwidth limitation has been a problem.

In addition, resources for boundary extraction and object recognition are limited, and even when an excellent post-processing algorithm is applied, there is a limit in recognition performance due to limitation of the processed color space data, which particularly adversely affects the road driving environment. This is because the driving environment of the road is mostly achromatic and limited to the visible ray region, and the spatial recognition ability is limited.

When the road image is divided, there is a problem that the accuracy of asphalt and cement part division is very low.

However, in the case of a conventional three-dimensional spatial information acquiring device, a radar sensor, a TOF camera, a stereo camera, an infrared camera, an ultrasound Sensors, and lasers mounted on a vehicle. However, there is a problem in that the cost is increased due to the use of an excessive amount of equipment.

In addition, since the pixel information between the depth map and the RGB camera is different in the process of converging the RGB camera and the depth map, the complexity of the signal / image processing is improved, the software load due to the hardware due to the advanced algorithm is increased, And various problems are caused.

If the camera, sensor, or camera and camera are not fixed in position, such as a vehicle, the matching difficulty is further increased. Therefore, a very complicated algorithm is required, resulting in an increase in processing time and a significant reduction in accuracy do.

Therefore, a system that is simpler and improves accuracy is required.

It is an object of the present invention to provide a traveling assistance system for a vehicle that improves the spatial recognition function of a road-running video camera and accurately extracts a contour line so as to assist a vehicle running.

A driving assistance system for a vehicle according to the present invention includes a broadband camera for capturing and inputting images of the surroundings of a vehicle, a map generator for generating a Depth Map based on the image input from the wideband camera, And a wideband image generator for receiving the data of the map generator and the image generator and generating a three-dimensional map template, wherein the wideband camera comprises four channels having different wavelengths And an image including the information of the image data.

The wideband camera is characterized by inputting images including information of four channels of Green, NIR1 of 800 nm band, NIR2 of 900 nm band, and FIR.

According to another aspect of the present invention, there is provided a method of generating a GNF image, the method comprising: inputting an image including four channels of information from a wideband camera; generating a depth map based on channel- And generating a three-dimensional map information by combining the Depth Map and the GNF image by a broadband image generating unit.

Estimating a position of the vehicle and matching the three-dimensional map.

The driving assistance system of the automobile according to the present invention can easily recognize roads and obstacles when driving the vehicle using a small number of cameras, greatly improve the recognition performance of the travelable area, And the driver's surveillance camera's biometric recognition capability is improved. Therefore, the driver's convenience due to the performance enhancement of the driving assistance device is greatly improved.

1 is a block diagram showing a configuration of a vehicle driving assistance system according to the present invention.
FIG. 2 is a block diagram of a camera of a vehicle driving assistance system according to the present invention.
3 is a view illustrating a flow of data in extracting a wideband image map according to image processing of a vehicle driving assistance system according to the present invention.
FIG. 4 is a diagram for explaining a method of extracting the wideband image map of FIG. 3; FIG.
3 is a view illustrating a flow of data in extracting a wideband image map according to image processing of a vehicle driving assistance system according to the present invention.
FIG. 4 is a diagram for explaining a method of extracting the wideband image map of FIG. 3; FIG.
FIG. 5 is an exemplary diagram that is referenced to illustrate image generation using signals of different wavelengths in FIG. 3;
FIG. 6 is a diagram illustrating an embodiment of the obstacle detection by the vehicle travel assistance system according to the present invention.
FIG. 7 is an exemplary diagram illustrating an embodiment of pedestrian detection in a vehicle driving assistance system according to the present invention.
8 is a view illustrating the configuration of a vehicle driving assistance system according to the present invention.
FIG. 9 is a flowchart illustrating a three-dimensional global map generation method according to the present invention.
FIGS. 10 to 12 illustrate an image and a three-dimensional map to be photographed when the vehicle is traveling.

BRIEF DESCRIPTION OF THE DRAWINGS The advantages and features of the present invention, and the manner of achieving them, will be apparent from and elucidated with reference to the embodiments described hereinafter in conjunction with the accompanying drawings. The present invention may, however, be embodied in many different forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the invention to those skilled in the art. To fully disclose the scope of the invention to those skilled in the art, and the invention is only defined by the scope of the claims. Like reference numerals refer to like elements throughout the specification.

1 is a block diagram showing a configuration of a vehicle driving assistance system according to the present invention.

1, an automobile includes a broadband camera 120, a map generation unit 130, an image generation unit 140, a broadband image generation unit 150, a data processing unit 160, an obstacle detection unit 170, A communication unit 190, a data unit 195, and a control unit 110 for controlling overall operation of the vehicle.

The vehicle includes an engine for driving the vehicle, a motor, and a mission, and may further include a plurality of sensors, but a description thereof will be omitted below.

The vehicle includes input means (not shown) including a plurality of switches for inputting a predetermined signal by an operation of a driver, output means (not shown) for outputting information during a current operation of the electric vehicle, a steering wheel, And an operation means for operation such as an accelerator, a brake, and the like.

At this time, the input means includes a plurality of switches, buttons, and the like for operation of a turn signal lamp, a tail lamp, a head lamp, a brush, etc.,

The output means includes a display unit for displaying information, a speaker for outputting music, an effect sound and a warning sound, an instrument panel of the vehicle, and various states. The output means outputs status information on the overall operation of the current vehicle, such as speed information, lamp lighting status, and the like. Particularly, the output means outputs a warning corresponding to the occurrence of a vehicle abnormality, and can output a predetermined image through the display unit. At this time, the warning of the vehicle can be outputted in the form of at least one of a warning sound of an acoustic or voice, a warning light, a warning message, and a warning image.

In the data unit 195, operational data according to the vehicle operation, reference data for determining whether the vehicle is abnormal or not, and data generated during vehicle operation are stored. In addition, the data unit 195 stores information on the image input by the wideband camera 120, data generated in the course of processing the image, three-dimensional image data, and obstacles extracted from the image.

The driving unit 180 controls the respective components of the vehicle to operate according to the input by the operating unit and the control command of the control unit 110 so that the vehicle moves and follows the operation of the driver. The driving unit 130 directly controls driving of the vehicle configuration such as an engine, a mission, and a brake in accordance with a control command.

The communication unit 190 includes a plurality of communication modules to allow the vehicle to transmit and receive data to and from the outside. In particular, the communication unit 190 communicates with a server (not shown) that provides information to the vehicle in autonomous driving using a three-dimensional map.

The control unit 110 controls the lamps to be turned on or turn on and off according to the switch operation of the input means, and controls the vehicle speed to accelerate or decelerate the vehicle in response to the operation of the excel or brake. Also, the controller 110 detects an abnormality of the vehicle and outputs a warning through the output means.

The broadband camera 120 captures and inputs an image. At this time, the wide-angle camera 120 includes a lens having a wide angle of view. In addition, the broadband camera 120 inputs an image including color information of four channels.

The map generator 130 analyzes a video input from the wideband camera 120 to generate a depth map.

The image generator 140 analyzes the image input from the broadband camera 120 to generate a GNF image.

The broadband image generator 150 analyzes the data of the map generator 130 and the data of the image generator 140 to generate a three-dimensional map template.

The data processor 160 analyzes the wideband camera 120 to generate a 2.5-dimensional structure. At this time, the data processing unit 160 determines the position of the vehicle based on the 2.5-dimensional structure and matches the map with the 3-dimensional map.

The obstacle detecting unit 170 analyzes an image input from the wideband camera 120 to detect an obstacle.

The control unit 110 controls the driving unit 180 to drive the vehicle based on the generated three-dimensional map. The control unit 110 transmits the generated data to the server through the communication unit 190, receives the positional information or the three-dimensional map data from the server, and transmits the control command to the driving unit 180 to drive the vehicle.

The description of the three-dimensional map generation will be described in more detail with reference to the drawings.

FIG. 2 is a diagram illustrating a configuration of a broadband camera of a vehicle driving assistance system according to the present invention.

2, the broadband camera 120 provided in the vehicle includes a CCD (Charge Coupling Device) 124 and an image processing unit 125. [ The image sensor 124 is composed of a plurality of pixels 121 and 122. The number of pixels determines the pixels of the image sensor.

In each of the pixels 121 and 122, as shown in FIG. 2 (a), when a light having a predetermined frequency or higher is incident, a photoelectric effect occurs in which photoelectrons inside the pixel are emitted.

The photoelectrons generated in each pixel are converted into signals having a voltage of a predetermined magnitude by the conversion unit 123 inside the image sensor, as shown in Fig. 2B.

At this time, the color of the image sensor 124 is determined by applying a filter of a transmission band. The image sensor 124 determines the color by the signal processing method of Green, NIR1 of 800 nm band, NIR2 of 900 nm band, and GN1N2F of FIR instead of the conventional RGB filter method.

The color signal of the image sensor 124 and the voltage signal of each pixel are input to the image processing unit 125.

The image processing unit 125 processes signals for each pixel to form image frames for each time, and outputs image data A composed of a plurality of image frames. The output image data has a frequency band of 300 to 1100 nm.

3 is a view illustrating a flow of data in extracting a wideband image map according to image processing of a vehicle driving assistance system according to the present invention.

3, the image signal 11 input through the image sensor 124 is converted to green (21), NIR1 (NIR1), and green (NIR1) by the color filters applied to the image sensor 124, 22), NIR2 (23), and FIR (24) channels, image data including color information of four channels is generated. That is, the image data obtained by the wide-band camera 120 includes color information of four channels.

These separated signals are used to generate Depth MAP 31 and GNF image 32, respectively.

At this time, the map generator 130 determines the spatial information using the 800 nm band NIR1 channel from the image data. In addition, the map generating unit 130 distinguishes objects in an image by using color information of Green, NIR2, and FIR channels in distinguishing objects based on color signals of image data. For example, it is possible to distinguish between asphalt and curb using such color information per channel. In addition, due to the use of the green, NIR2, and FIR channels, there is little influence due to fog and light reflection, and the area can be easily distinguished.

FIG. 4 is a diagram for explaining a method of extracting the wideband image map of FIG. 3; FIG.

In generating the Depth MAP 31, the map generating unit 130 determines the spatial information using the NIR1 channel of the 800-nm band and the NIR projector among the image data.

The NIR projector emits infrared light in the 800 nm band on a pixel-by-pixel basis (S1). At this time, when the infrared ray reflected by the object is incident, the broadband camera 120 receives the infrared ray from the NIR1 channel (S2). The Depth MAP can be configured by analyzing the pattern of the infrared light received through the NIR1 channel (S3).

The map generating unit 130 extracts the infrared pattern projected from the reference scene in advance and the controller 110 stores the extracted infrared pattern in the data unit 195.

The map generation unit 130 calculates depth information based on the infrared pattern and the parallax change reflected from the object area (Object Plane) based on the stored infrared pattern. The depth information is calculated based on the trigonometric function and is calculated by the following equation.

Z is the depth value in the reference area, b is the distance between the NIR projector and the broadband camera, d is the parallax recorded in the image plane, D is the distance from the broadband camera at point k, Lt; RTI ID = 0.0 &gt; k &lt; / RTI &gt; When the D value is added to the first equation (1) in the second equation (2), the distance value Zk of the object is expressed as the following equation (3) and output as depth information.

At this time, in generating the Depth MAP, the Depth MAP and the image of the image coincide with each other using the color information of the plurality of channels described above, so that a three-dimensional image is generated on the same optical axis. Accordingly, the map generating unit 130 may generate a three-dimensional image by matching the calculated depth information, the image, and the depth map, while considering the motion of the wide-angle camera 120.

In the case of using the conventional RGB filter, since the information of the depth map and the RGB filter do not coincide with each other, a complicated algorithm is applied in order to generate a three-dimensional image, and the processing amount is also delayed, .

However, since the image data obtained through the wideband camera 120 includes information of four channels of Green (21), NIR1 (22), NIR2 (23), and FIR (24) as described above, So that a three-dimensional image can be generated.

FIG. 5 is an exemplary diagram that is referenced to illustrate image generation using signals of different wavelengths in FIG. 3; In this case, a and d in FIG. 5 are visible light images, and b and e in FIG. 5 are far infrared ray images, and c and f in FIG. 5 are combined images of a visible light image and a far infrared ray image.

As shown in FIG. 5, since the visible light image, the near-infrared light, and the far-infrared light have different wavelengths, the information of the displayed image is different from each other. Due to the difference of the wavelengths, it is possible to express thermal information which can not be expressed in the visible light image through the infrared image. At this time, the reflectance of the object, for example, asphalt may change depending on its lifetime, and in the case of concrete, reflectance differs depending on the material to be mixed. This difference makes it possible to easily distinguish the object according to the wavelength.

Especially, in case of far infrared ray image, the object can be detected more accurately even in the case of fog or cloudy weather, and the object of long distance can be taken clearly. In the case of near-infrared rays, it is possible to more accurately distinguish areas which are difficult to distinguish asphalt and visible light such as a curb.

The image generation unit 140 generates the GNF image by fusing the images of the three wavelength ranges described above from the image data of the wideband camera 120.

The broadband image generator 150 combines the depth map input from the map generator 130 and the image generator 140 with the GNF image to generate a 3D template for the feature information. Thus, a three-dimensional map is generated.

At this time, the broadband image generator 150 highlights the contour of the GNF image using a Canny edge detection method. This is to make it easier to distinguish areas where the depth information is the same, such as curb and asphalt.

FIG. 6 is a diagram illustrating an embodiment of the obstacle detection by the vehicle travel assistance system according to the present invention.

As described above, when the broadband image is generated, the broadband image includes not only the appearance of the object but also distance information on the object, so that the obstacle detecting unit 170 recognizes the obstacle from the wideband image based on the type and distance information of the object.

As shown in FIG. 6A, in the case of a wide-band image, the road and the obstacle can be distinguished according to the distance information.

At this time, the obstacle detecting unit 170 models the object of the image data based on the Cubic B-Spline function for the wideband image, and identifies obstacles accordingly. At this time, obstacles are classified using distance information.

The obstacle detecting unit 170 sets the other area except the road area as an obstacle area, and distinguishes the obstacle only to the obstacle area. That is, since the obstacle classification for the road area is not performed, the computation amount is decreased.

Also, as shown in FIG. 6B, the obstacle detecting unit 170 may classify the obstacle area and set the obstacle area as an area of interest, among the obstacle areas.

Based on the results of estimating obstacles and roads in particular, it is possible to narrow down the area of interest to be applied when the vehicle is hunted, making it easier to determine obstacles to the area of interest. Also, since the amount of computation can be reduced as the area of interest decreases, the erroneous detection rate is reduced.

It is determined whether or not it is an obstacle approaching the vehicle in response to the movement in the area of interest.

FIG. 7 is an exemplary diagram illustrating an embodiment of pedestrian detection in a vehicle driving assistance system according to the present invention.

As shown in FIG. 7, the obstacle detecting unit 170 detects an obstacle, in particular, a pedestrian by using the brightness, the heat, and the depth information of the image with respect to the wideband image including the information of four channels.

As shown in FIG. 7A, the obstacle detecting unit 170 can distinguish pedestrians based on the SVM according to the body temperature of a pedestrian from thermal images using thermal information. Also, as shown in FIG. 7B, the obstacle detecting unit 170 can detect the pedestrian based on the Haar classifier using the brightness information of the broadband image. As shown in FIG. 7C, it is also possible to detect the pedestrian using the Depth information.

The obstacle detecting unit 170 combines the pedestrian information detected based on each piece of information as described above to finally set the pedestrian area. Thereby not only detecting the pedestrian easily but also improving its accuracy.

8 is a view illustrating the configuration of a vehicle driving assistance system according to the present invention.

As shown in Fig. 8, the vehicle 1 transmits a template of the three-dimensional map generated by the broadband image generation unit 150 to the server 2, and the server 2 receives and accumulates the template.

The server 2 provides three-dimensional map information to a plurality of vehicles based on accumulated data, determines the position of the vehicle, matches the three-dimensional map, and provides the information.

At this time, the control unit 110 of the vehicle 1, based on the three-dimensional map of the wide-band image including both the distance information and the traveling environment information generated by the broadband image generating unit 150, So that the vehicle can be autonomously driven.

The data processor 160 estimates the motion of the camera relative to the continuous image frame photographed from the wideband camera 120 and continuously outputs the motion of the camera relative to the data unit 195, .

The data processing unit 160 uses the 2.5-dimensional distance image generated from the previous image at time t-1 and the current image at time t, as an input value for estimating the motion of the broadband camera 120. [ Here, the 2.5-dimensional distance image is a data structure that combines the 2-dimensional array of the GNF image with the 3-dimensional distance information acquired by the N1 channel.

Given a 2.5-dimensional distance image, the relative motion T <t-1, t> of a broadband camera to successive stereo image frames is calculated as follows.

When the two-dimensional image is input, the data processing unit 160 extracts two-dimensional feature points of the images obtained at time t-1 and time t, i.e., the previous image and the current image, using the SURF algorithm. The data processing unit 160 generates matching point groups between t-1 and t frames by matching the two-dimensional minutiae points to each other. Since the initial matching result may include errors due to erroneous matching, the data processing unit 160 applies the epipolar geometry constraint and the RANSAC algorithm to remove outliers.

The data processing unit 160 calculates a 3D-3D matching relationship between t-1 and t using a 2.5-dimensional distance image having depth information when a matching point group from which the external points are removed is calculated, And calculates a transformation matrix between the time points. In this case, given a three-dimensional-3D matching relationship, a three-dimensional relative motion T <t-1, t> can be calculated using a least squre fitting method. When the transformation matrix T <t-1, t> between t-1 and t is determined through the motion estimation process, the transformation matrix Tcw <t> for moving from the camera coordinate system to the world coordinate system at time t is calculated at the previous time And the conversion matrix are all accumulated.

Accordingly, the data processing unit 160 recognizes the position of the vehicle on the three-dimensional map based on the wide-band camera 120. In some cases this can be performed by the server 2.

As described above, the wideband image generator 150 generates a three-dimensional global map based on the data of the map generator 130 and the image generator 140.

At this time, the broadband image generator 150 stores and manages the information acquired in the motion estimation process of the image frame according to the graph theory to generate a three-dimensional global map. When the data processing unit 160 estimates the camera motion with respect to the continuous images, the estimated motion information and the distance information are DBized and stored every time. Whereby one node is allocated per one stereo frame. Therefore, the node consists of one two-dimensional image scene and three-dimensional information about the image scene.

When the Tcw < t > is determined by the data processing unit 160 on the basis of the wideband camera, a three-dimensional world corresponding to the SURF feature points Ft, k and Ft, The coordinate point Xwt, k is stored. The three-dimensional world coordinate Xwt, k is calculated by equation (5).

In Equation (4), Ft, k, σt, k, and dt, k represent the rotation angles of the feature points of the two-dimensional image coordinates of the SURF feature point Ft, k corresponding to the kth time of the time, the size of the feature points, , Xct, k is the 3D point of the camera coordinate system for Ft, k. Xct, k is obtained from the distance image ITt of the two-dimensional image coordinates xt, k and t time as shown in Equation (6).

 The 3D map can be constructed in the same manner as described above, and the image and GPS information input from the broadband camera can be fused in real time for autonomous driving.

FIG. 9 is a flowchart illustrating a three-dimensional global map generation method according to the present invention.

9, when the broadband camera 120 starts shooting and an image is input 310, the map generator 130 generates a Depth Map (S320), and the image generator 140 generates a GNF An image is generated (S330).

The broadband image generator 150 combines the Depth Map and the GNF image to generate three-dimensional map information (S340).

On the other hand, for the image input from the wideband camera 120, the data processing unit 160 generates a 2.5-dimensional structure based on the change of the image with time (S350). The data processing unit 160 matches the SURF feature points (S360), and removes the outliers using the RANSAC (S370). The data processing unit 160 calculates a transformation matrix (S380), and recognizes the position of the vehicle on the three-dimensional map based on the image of the wideband camera 120 (S390).

A three-dimensional global map is generated by recognizing the position of the vehicle from the current input image on the three-dimensional map stored as described above.

The generated 3D global map is used for autonomous driving of the vehicle.

FIGS. 10 to 12 illustrate an image and a three-dimensional map to be photographed when the vehicle is traveling.

As shown in FIG. 10A, during the autonomous vehicle running, a contour line is extracted on the basis of the color of the broadband image as shown in FIG. 10B for the input image, the image is divided as described above, A database can be constructed using a template for a three-dimensional map.

As a result, information on a three-dimensional map is stored as shown in FIG.

At this time, the three-dimensional data is stored in the server 2 and operated in a cloud manner, thereby providing information to a plurality of vehicles. Asphalt and cement are used. In rural areas and off-roads where buildings are not available, images of surrounding large trees and hills are acquired and applied according to season, time, and climate.

The controller 110 controls the operation of the server 2 through the communication unit 190 when the information about the image input from the broadband camera 120 is generated by the broadband image generation unit 150 and the data processing unit 160, In real time.

At this time, the server 2 measures the position of the vehicle by combining the received data with the previously stored three-dimensional wideband pumplet data, and performs map matching. And may be performed by the data processing unit 160 of the vehicle 1 as occasion demands.

The server 2 transmits the matched information to the vehicle. This process is repeated and the vehicle 1 travels by reflecting the received information. Also, the vehicle 1 can run while avoiding obstacles detected during running.

Accordingly, as shown in Fig. 12, the vehicle can autonomously travel through three-dimensional map information and vehicle position recognition.

The present invention can acquire four-dimensional color information from a broadband camera, and can recognize roads and detect obstacles without adding other devices. In addition, a 3-dimensional map can be generated based on the image of the wideband camera, and the position of the vehicle can be matched and applied to autonomous driving.

It is to be understood that the terms "comprises", "comprising", or "having" as used in the foregoing description mean that the constituent element can be implanted unless specifically stated to the contrary, But should be construed as further including other elements.

While the present invention has been particularly shown and described with reference to exemplary embodiments thereof, it is to be understood that the invention is not limited to the disclosed exemplary embodiments, but, on the contrary, It should be understood that various modifications may be made by those skilled in the art without departing from the spirit and scope of the present invention.

1: vehicle 2: server
110: control unit 120: broadband camera
130: map generating unit 140:
150: Broadband image generator 160: Data processor
170: obstacle detection unit 180:
190: communication unit 195: data unit

Claims (20)

A broadband camera for photographing and inputting images of the surroundings of the vehicle;
A map generator for generating a depth map based on an image input from the wideband camera;
An image generation unit for generating a GNF image based on an image input from the wideband camera; And
A broadband image generation unit receiving the data of the map generation unit and the image generation unit and generating a three-dimensional map template;
A data processing unit for estimating a motion of a camera relative to a continuous image frame, which is photographed from the wideband camera, to continuously update the posture of the vehicle while driving;
Lt; / RTI &gt;
The data processing unit extracts and matches feature points from a previous image at time t-1 and a current image at time t in an image frame, calculates a matching relationship in three dimensions based on the depth information, and calculates a transformation matrix between the two points Thereby estimating the position of the vehicle. The method according to claim 1,
Wherein the broadband camera inputs images including information of four channels of Green, NIR1 of 800 nm band, NIR2 of 900 nm band, and FIR. The method according to claim 1,
Wherein the map generator generates a depth map by determining spatial information using NIR1 channel and NIR projector from the image data input from the wideband camera. The method according to claim 1,
Wherein the map generating unit distinguishes an object based on a difference in wavelength for each channel by using color information of Green, NIR2, and FIR channels in the image data input from the wideband camera, . The method of claim 3,
Wherein the map generator is configured to determine spatial information using NIR1 channel of the 800-nm band among 4-channel color information from the image data input from the broadband camera. 6. The method of claim 5,
Wherein the map generating unit receives the infrared light emitted from the NIR projector into the wideband camera and extracts a pattern of infrared light received through the NIR1 channel of the wideband camera to determine the spatial information system. The method according to claim 6,
Wherein the map generating unit calculates depth information based on the pattern of the infrared rays and the parallax change of the image data, and matches the depth information with the depth map to generate a three-dimensional image. The method according to claim 1,
Wherein the image generator generates the GNF image based on information of Green, NIR2, and FIR channels in the image data of the wideband camera. The method according to claim 1,
Wherein the broadband image generator combines the depth map of the map generator and the GNF image of the image generator to generate a 3D map template for the feature information. 10. The method of claim 9,
Wherein the broadband image generator divides the GNF image of the image generator by using a Canny edge detection method to highlight an outline of the GNF image. The method according to claim 1,
Further comprising an obstacle detecting unit for detecting an obstacle from the image data,
Wherein the obstacle detection unit models an object included in the wideband image of the broadband image generator and distinguishes obstacles based on the distance information. 12. The method of claim 11,
Wherein the obstacle detection unit sets an obstacle area other than the road area as an obstacle area from the wideband image and sets the obstacle area as an area of interest and detects an obstacle only to the area of interest. 13. The method of claim 12,
Wherein the obstacle detection unit tracks movement of an object in the ROI to determine whether the obstacle obstructs the traveling of the vehicle. delete delete 13. The method of claim 12,
A communication unit for transmitting the three-dimensional map template generated from the wideband image generation unit to a server; And
And a control unit for transmitting and receiving data to and from the server through the communication unit and controlling driving of the vehicle by combining the three-dimensional map and GPS information. Inputting an image including information of four channels from a wideband camera;
Generating a depth map based on the channel-by-channel information from the image;
Generating an GNF image from the image generator; And
Generating a three-dimensional map information by combining the Depth Map and the GNF image;
Estimating a position of the vehicle and matching the three-dimensional map information;
Extracting a feature point of a previous image and a current image according to a change in time from the image input from the wideband camera; And
Calculating a matching relation for the feature points based on the depth information and calculating a transformation matrix between the previous image and the current image to estimate a position of the vehicle;
Wherein the vehicle driving assistance system comprises: 18. The method of claim 17,
And detecting an obstacle from the broadband image calculated from the wideband image generator. delete delete

Images