KR20140006462A - Apparatus and method for assisting safe driving - Google Patents

Abstract

The present invention relates to a device and a method for assisting safe driving to enable users to accurately recognize obstacles which may put the drivers in danger while driving a vehicle using a three-dimensional flash lidar and controlling the vehicle. The device for assisting safe driving using the vehicle and lidar of a vehicle may comprises a camera information receiving part for receiving camera information collected by the camera; a lidar information receiving part for receiving lidar information collected by the lidar; an obstacle detecting part for detecting obstacles using the camera and lidar information; and a control part for controlling the vehicle when obstacles are detected. [Reference numerals] (100) Device for assisting safe driving; (110) Camera information receiving part; (120) Lidar information receiving part; (130) Image signal processing part; (140,170) 3D pre-processing part; (150) 2D/3/matching part; (160) Obstacle detecting part; (180) Control part

Description

Apparatus and Method for Assisting Safe Driving}

The present invention relates to a safe driving support apparatus and method. More specifically, the present invention relates to a safe driving support apparatus and method for accurately grasping an obstacle that may be dangerous for safe driving while driving a vehicle using a 3D flash rider and controlling the vehicle based on the grasping.

Conventional riders (Lidar) using the vehicle, using the rider to obtain the distance information with the other vehicle, using the obtained distance information and the image received from the camera of the vehicle is different from the presence of other vehicles There is a system that recognizes the distance to the vehicle to prevent the collision between the vehicles.

However, a system for preventing collision between vehicles using such a rider does not have a function of recognizing pedestrians, and thus, when the recognized item is a pedestrian rather than a vehicle, there is a problem in that an accident with a pedestrian cannot be prepared.

In addition, an obstacle recognition system using a conventional rider and a collision avoidance system using a vehicle do not distinguish a vehicle existing in a driving lane from a vehicle existing in a lane other than a driving lane.

In addition, there is a problem that the obstacle recognition using the existing rider does not accurately grasp the type of obstacle.

The present invention has been made to solve the above problems, using a three-dimensional flash rider accurately grasps obstacles that may be dangerous to drive safely, and safe driving support apparatus and method for controlling the vehicle based on the grasp The purpose is to provide.

A safe driving support apparatus for solving the above problems includes a camera supporting a safe driving using a camera and a rider (Lidar), the camera information receiving unit for receiving the camera information collected by the camera of the vehicle; A rider information receiver configured to receive rider information collected by the rider; An obstacle detecting unit which determines whether an obstacle exists using the camera information and the rider information; If it is determined that the obstacle exists, it may include a control unit for controlling the vehicle.

Preferably, the safe driving support apparatus generates intensity information and depth information by using the information received from the rider information receiving unit, and generates a 3D image by using the generated information. The apparatus may further include a 3D preprocessor, and the obstacle detecting unit may determine whether the obstacle exists by using the camera information and the generated 3D image.

Preferably, when it is determined that an obstacle exists, the obstacle detecting unit may determine the type of the obstacle using the intensity information and the depth information included in the 3D image.

Preferably, the safe driving support apparatus further includes a database unit for storing a pedestrian, a vehicle, a speed bump, a sign or a manhole image, and information associated with the image, wherein the obstacle detecting unit detects the type of the obstacle. The information of the obstacle may be determined using the information associated with the image having the highest matching rate by matching with the image stored in the.

A safe driving support method for solving the above problems includes: a method of supporting safe driving using a camera and a rider of a vehicle, the method comprising: receiving camera information collected by the camera of the vehicle; A rider information receiving step of receiving rider information collected by the rider; An obstacle detecting step of identifying whether an obstacle exists using the camera information and the rider information; If it is determined that the obstacle exists, it may include a control step of controlling the vehicle.

Preferably, in the safe driving support method, intensity information and depth information are generated using the information received in the rider information receiving step, and a 3D image is generated using the generated information. The method may further include a 3D preprocessing step, wherein the obstacle detecting step may determine whether the obstacle exists by using the camera information and the generated 3D image.

Preferably, when determining that the obstacle exists, the type of the obstacle may be determined using the intensity information and the depth information included in the 3D image.

Preferably, the safe driving support method further comprises a database step of storing the image associated with the image of the pedestrian, vehicle, speed bumps, signs or manhole and the image, the obstacle identifying step is to identify the type of obstacle identified; By matching the stored image, the information of the obstacle may be determined using the information associated with the image having the highest matching rate.

The present invention can accurately grasp the type of obstacle and control the vehicle to suit the situation. Accordingly, the driver can drive a more comfortable and safe driving.

1 is a block diagram of a safe driving support apparatus according to a preferred embodiment of the present invention.
2 is a block diagram illustrating an example of an obstacle detecting unit of the safe driving support apparatus according to the present invention.
3 is a diagram illustrating an example of a result of applying Equation 1 when the obstacle type determining unit 164 determines whether an obstacle exists using image information.
4 is a flowchart illustrating a safe driving support method according to an exemplary embodiment of the present invention.

Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the drawings. In the following description and the accompanying drawings, substantially the same components are denoted by the same reference numerals, respectively, and redundant description will be omitted. In the following description of the present invention, a detailed description of known functions and configurations incorporated herein will be omitted when it may make the subject matter of the present invention rather unclear.

It is to be understood that when an element is referred to as being "connected" or "connected" to another element, it may be directly connected or connected to the other element, . On the other hand, when an element is referred to as being "directly connected" or "directly connected" to another element, it should be understood that there are no other elements in between.

In this specification, a singular form may include plural forms unless specifically stated in the phrase. It is noted that the terms "comprises" and / or "comprising" used in the specification are intended to be inclusive in a manner similar to the components, steps, operations, and / Or additions.

The obstacle classification recognition apparatus and the sign recognition apparatus according to the present invention use a rider. Preferably, a 3D Flash Lidar may be used.

1 is a block diagram of a safe driving support apparatus according to a preferred embodiment of the present invention.

Referring to FIG. 1, the safe driving support apparatus 100 includes a camera information receiver 110, a rider information receiver 120, an image signal processor 130, a 2D / 3D matcher 150, and a 3D preprocessor 140. , Obstacle detecting unit 160, database unit 170, and / or controller 180.

The camera information receiver 110 receives information collected by a camera installed in a vehicle. The information collected by the camera may include image or image information.

The rider information receiver 120 receives information collected by a rider.

The camera and the rider are preferably installed to easily collect information in front of the vehicle. For example, the camera and the rider may be installed directly behind the windshield of the vehicle and between the room mirror of the ceiling.

The camera information received by the camera information receiver 110 is transmitted to the image signal processor 130. The camera information received by the camera information receiver 110 may include information obtained by photographing a field of view (FOV) area of the rider as a 2D image.

The image signal processor 130 receives the camera information, changes the size and format to improve the processing speed of the image or the image included in the camera information, and performs an image quality improvement operation. The image signal processor 130 changes the size and format and transmits the information on the image quality improvement to the 2D / 3D matching unit 150. The information transmitted from the image signal processor 130 to the 2D / 3D matching unit 150 may be 2D image information.

The 2D / 3D matching unit 150 extracts an image of the ROI by using the information transmitted from the image signal processor 130 and the information transmitted from the 3D preprocessor 140. A more detailed description will be described again after explaining the 3D preprocessor 140.

The 3D preprocessor 140 receives the information collected by the rider from the rider information receiver 120 and generates intensity information using the intensity of the position and the wavelength received from the light receiving array unit of the rider. do. In addition, the 3D preprocessor 140 generates depth information according to the time received by the light receiving array unit of the rider.

The 3D preprocessor 140 may generate a 3D volume image using the generated intensity information and the depth information.

The 2D / 3D matching unit 150 may extract an image of the ROI by using 2D image information received from the image signal processor 130 and a 3D volume image generated by the 3D preprocessor 140.

The image extracted by the 2D / 3D matching unit 150 may be transmitted to the obstacle detecting unit 160 and used.

The obstacle detecting unit 160 receives the information from the image signal processor 130, the 3D preprocessor 140, and the 2D / 3D matching unit 150 to determine the existence of the obstacle, to identify the type of the obstacle, and also to determine the type of obstacle. I can grasp the information.

Here, the obstacles generally refer to elements that prevent driving or safe driving of a vehicle, such as another vehicle, a pedestrian, a manhole or a speed bump, and may include an object thrown on a road, a traffic accident fragment, and the like.

2 is a block diagram illustrating an example of an obstacle detecting unit of the safe driving support apparatus according to the present invention.

Referring to FIG. 2, the obstacle detecting unit 160 may include an obstacle existence determining unit 162, an obstacle type determining unit 164, and an obstacle information identifying unit 166.

The obstacle presence detecting unit 162 detects the existence of an obstacle using the 3D volume image generated by the 3D preprocessor.

In detail, the obstacle existence determiner 162 detects the existence of an obstacle by using intensity information included in the 3D volume image. Since the intensity value is different for each object, if it is out of the range of the intensity value of the preset road, it can be understood that an object other than the road exists on the road.

When the obstacle existence identifying unit 162 determines whether an obstacle exists, only the image of the ROI may be cut from the image to determine whether the obstacle exists, and the ROI is designated by the 2D / 3D matching unit 150. Can be. The region of interest may be the front of the vehicle, the lane in which the vehicle travels, or the lanes in which the vehicle travels and both side lanes.

The obstacle presence determining unit 162 may also obtain information on the location of the obstacle and the distance between the obstacle and the vehicle by using the depth information included in the 3D volume image.

When it is determined that the obstacle exists in the obstacle existence detecting unit 162, the obstacle type determining unit 164 may generate an image generated by the image signal processor 130, the 3D preprocessor 140, or the 2D / 3D matching unit 150. Use the information to identify the type of obstacle.

The obstacle type determiner 164 may extract edge information of the image pixel from the image information to be used. Extracting the edge information of the image pixel by the obstacle existence determining unit 162 may use a gradient calculation formula such as Equation (1).

By accumulating G (x, y), which is a size value based on angle information for each pixel, and dividing it by the total pixel value, a gradient histogram (HOG) value can be obtained.

3 is a diagram illustrating an example of a result of applying Equation 1 when the obstacle type determining unit 164 determines whether an obstacle exists using image information.

Referring to FIG. 3, when Equation 1 is applied to an image, it can be seen that contour information of a specific object is displayed in the image.

That is, (b) represents the contour information obtained by applying the equation (1) in the image (a). (D) represents the contour information obtained by applying the equation (1) in the image (c).

The obstacle type determiner 164 may infer the type of obstacle based on the contour information.

In detail, the obstacle type determiner 164 may determine the type of obstacle by matching the image stored in the database unit 170 with the image containing the contour information. The database unit 170 stores numerous images for each type of various obstacles such as other vehicles, pedestrians, manholes, speed bumps, and the like. If the operation for matching is not overloaded, the more images stored in the database unit 170, the better.

The database unit 170 may store information associated with each image.

For example, when the image stored in the database unit 170 is a manhole, information on the diameter, size, etc. of the manhole may be associated and stored. When the stored image is a speed bump, the height, width, or width of the speed bump may be stored. Related information may be associated and stored. Alternatively, when the image stored in the database unit 170 is a vehicle, information on the type, acceleration force, etc. of the vehicle may be associated and stored.

The obstacle information identifying unit 166 may obtain information related to the image of the database unit 170 that best matches the contour information determined by the obstacle type identifying unit 164 to determine the information of the corresponding obstacle. The grasped information may be used by the controller 180.

The controller 180 may control the vehicle based on the detected obstacle information.

In detail, the controller 180 controls the vehicle's craxon, the vehicle's brake, the vehicle's accelerator pedal, the vehicle's seat belt, the vehicle's lamp, or the vehicle's seat to support the driver's safe driving. .

For example, when the identified obstacle is a pedestrian, the controller 180 may control the operation of the vehicle to notify the pedestrian and the driver of the danger.

When the identified obstacle is a vehicle, the controller 180 may control the brake of the vehicle to operate to secure the inter-vehicle distance.

If the identified obstacle is a manhole, the controller 180 may give attention to the driver by vibrating the seat belt of the vehicle or the seat of the vehicle, and display information of the manhole detected by the obstacle information identifying unit 166 of the display or navigation. The sound can be output by using a voice or the like.

If the identified obstacle is the speed bump, it may be controlled to operate the brake of the vehicle.

However, such an example is only one embodiment, and various modifications, changes, and substitutions may be made by those skilled in the art without departing from the essential characteristics of the present invention. For example, when the identified obstacle is a pedestrian, the controller 180 may control the vehicle to cope with a sudden situation by operating the vehicle's crack and the lamp of the vehicle to give attention to the pedestrian and to control the brake.

The safe driving support apparatus according to the present invention can accurately grasp the type of obstacle and control the vehicle to be appropriate to the situation. Accordingly, the driver can drive a more comfortable and safe driving.

4 is a flowchart illustrating a safe driving support method according to an exemplary embodiment of the present invention.

Referring to FIG. 4, the camera information receiver 110 receives camera information that is 2D image information (step S410).

The camera information received from the camera information receiving unit 110 may be changed in size and format by the image signal processing unit 130, and image quality improvement may also be performed (step S420).

The rider information receiver 120 receives rider information collected from the rider (S430).

The 3D preprocessor 140 receives the information collected by the rider from the rider information receiver 120, generates intensity information and depth information, and generates a 3D volume image using the generated intensity information and depth information (S440). ).

The 2D / 3D matching unit 150 extracts the ROI image by using the 3D volume image and the camera information, which is the 2D image information processed by the image signal processing unit 130 (S450).

The obstacle presence detecting unit 162 of the obstacle detecting unit 160 determines whether an obstacle exists in the ROI image or the 3D volume image (step S460).

If it is determined that the obstacle exists, the obstacle type determining unit 164 of the obstacle detecting unit 160 determines the type of the obstacle (step S470).

In addition, the obstacle information grasping unit 166 may grasp the information of the obstacle using the database unit 170 (step S480).

The controller 180 may support the driver's safe driving by controlling the vehicle using the identified type of obstacle or information on the obstacle (S490).

It is to be understood that the block diagram of the safe driving support apparatus 100 according to the preferred embodiment of the present invention represents an exemplary conceptual aspect embodying the principles of the invention. Similarly, all of the flowcharts should be understood to represent various processes that may be substantially represented on a computer-readable medium and executed by a computer or processor, whether the computer or processor is explicitly shown.

The functionality of the various elements shown in the figures, including functional blocks represented by a processor or similar concept, can be provided by the use of dedicated hardware as well as hardware capable of executing software in association with appropriate software. When provided by a processor, the functions may be provided by a single dedicated processor, a single shared processor, or a plurality of individual processors, some of which may be shared.

Also, the explicit use of terms such as processor, control, or similar concepts should not be interpreted exclusively as hardware capable of running software, and may be used without limitation as a digital signal processor (DSP) (ROM), random access memory (RAM), and non-volatile memory. Other hardware may also be included.

It will be apparent to those skilled in the art that various modifications, substitutions and substitutions are possible, without departing from the scope and spirit of the invention as disclosed in the accompanying claims. will be. Accordingly, the embodiments disclosed in the present invention and the accompanying drawings are not intended to limit the technical spirit of the present invention but to describe the present invention, and the scope of the technical idea of the present invention is not limited by the embodiments and the accompanying drawings. . The scope of protection of the present invention should be construed according to the following claims, and all technical ideas within the scope of equivalents should be construed as falling within the scope of the present invention.

Claims (8)

In a device that supports safe driving using a camera and a rider (Lidar) of the vehicle,
A camera information receiver configured to receive camera information collected by the camera of the vehicle;
A rider information receiver configured to receive rider information collected by the rider;
An obstacle detecting unit which determines whether an obstacle exists using the camera information and the rider information;
If it is determined that the obstacle exists, the safe driving support device, characterized in that it comprises a control unit for controlling the vehicle.
The method of claim 1,
The safe driving support device,
Further comprising a 3D pre-processing unit for generating intensity information and depth information using the information received from the rider information receiver, and generates a three-dimensional image by using the generated information,
The obstacle detecting unit, using the camera information and the generated three-dimensional image to determine whether the obstacle exists safe driving support apparatus.
3. The method of claim 2,
The obstacle detecting unit, if it is determined that the obstacle exists, safe driving support device, characterized in that for identifying the type of the obstacle, using the intensity information and the depth information included in the three-dimensional image.
The method of claim 3, wherein
Safe driving support device,
Further comprising a database unit for storing the image associated with the pedestrian, vehicle, speed bumps, signs or manhole and the image,
The obstacle detecting unit matches the obstacle of the type identified with the image stored in the database unit, and secure driving assistance apparatus, wherein the information of the obstacle is determined using the information associated with the image having the highest matching rate.
In a method of supporting a safe driving using a camera and a rider (Lidar) of the vehicle,
A camera information receiving step of receiving camera information collected by the camera of the vehicle;
A rider information receiving step of receiving rider information collected by the rider;
An obstacle detecting step of identifying whether an obstacle exists using the camera information and the rider information;
And if it is determined that the obstacle exists, a safe driving support method comprising controlling the vehicle.
The method of claim 5, wherein
The safe driving support method,
And a 3D preprocessing step of generating intensity information and depth information by using the information received in the rider information receiving step, and generating a 3D image using the generated information.
The obstacle identifying step, the safe driving support method, characterized in that to determine whether the obstacle exists using the camera information and the generated three-dimensional image.
The method according to claim 6,
The obstacle identifying step, if it is determined that the obstacle exists, safe driving support method characterized in that by using the intensity information and the depth information included in the three-dimensional image to determine the type of the obstacle.
The method of claim 7, wherein
Safe driving support method,
A database step of storing images of pedestrians, vehicles, bumps, signs or manholes and information associated with the images,
The obstacle identifying step may include identifying the type of obstacle by using the information associated with the image having the highest matching rate by matching the identified obstacle with the stored image.

Images