KR101351875B1 - Multi-Functional Sensor for Vehicle and Method thereof - Google Patents

Abstract

In the present specification, a composite sensor and a method thereof are disclosed. More specifically, a vehicle composite sensor and a method thereof are disclosed.
The composite sensor for a vehicle disclosed in the present specification includes a camera module and an image separator for separating a camera image photographed by the camera module into a lane detection image for lane detection and a rainwater detection image for rainwater detection, and a lane based on a lane detection image. And a lane departure controller to determine whether the vehicle deviates and generate a lane departure control signal, and a wiper controller to determine the amount of rain based on the rainwater detection image and to generate a wiper control signal.

Description

Multi-Functional Sensor for Vehicle and Method

In the present specification, a composite sensor and a method thereof are disclosed. More specifically, a vehicle composite sensor and a method thereof are disclosed.

Recently, the composition ratio of electronic equipment is continuously increasing in the manufacture of vehicles, and functions for driver convenience of the vehicle are continuously increasing.

In the past, the device was operated manually according to the information input through the driver's cognitive process. However, with the increase of such electronic devices, various signals input to the sensors are processed, thereby enabling proper device operation control according to the situation.

In particular, as the Intelligent Transportation Systems (ITS) is applied, the number of sensors mounted on the vehicle is continuously increasing as the number of devices mounted on the vehicle and performing automatic control increases.

The rain sensor is one of the sensors for automatically controlling the wiper. Depending on the amount of rain detected by the rain sensor and the speed of the vehicle automatically controls whether the wiper is driven and the drive speed.

A typical rain sensor depends on the windshield of a vehicle to detect the wavelength between the LEDs emitting light and passing through the glass and back to the sensor. In the rain glass, the light flow is normal, but when the rain water forms a certain amount of diffuse reflection, rain reflection occurs, and the rain sensor uses the principle of detecting this, thereby driving the wiper.

However, such a rain sensor is expensive, in particular, in order to install the rain sensor has to replace the entire windshield, there is a problem that the additional cost of expensive.

Lane Departure Warning System (LDWS) is a driver assistance system that prevents lane departures and collisions. With the development of electronic control technology of vehicles, we are pursuing improved safety and convenience than the concept of simple means of transportation.In particular, we utilize various sensor and equipment development to prevent traffic accidents that may occur due to driver's mistake while driving. Algorithm development research is active.

One of these lane departure detection systems is to acquire images of the road ahead using the vision sensor and extract the road information necessary for autonomous driving through the lane recognition process. A technique using an algorithm that characterizes only white lines and recognizes lines is being developed.

In the present specification, it is an object of the present invention to provide a composite sensor and a composite sensing method for a vehicle capable of controlling the wiper of the vehicle and determining whether the lane is separated using the image photographed by the camera.

The composite sensor for a vehicle disclosed in the present specification includes a camera module and an image separator for separating a camera image photographed by the camera module into a lane detection image for lane detection and a rainwater detection image for rainwater detection, and a lane based on a lane detection image. And a lane departure controller to determine whether the vehicle deviates and generate a lane departure control signal, and a wiper controller to determine the amount of rain based on the rainwater detection image and to generate a wiper control signal.

In the vehicle composite sensing method disclosed herein, an image separation step of separating a camera image photographed by a camera module into a lane detection image for lane detection and a rainwater detection image for rainwater detection, and whether the lane deviates based on the lane detection image And a lane departure control step of generating a lane departure control signal and a wiper control step of determining a rain amount and generating a wiper control signal based on the rainwater detection image.

According to the inventions disclosed herein, it is possible to determine whether the lane is separated using an image photographed by the camera and to control the wiper operation of the vehicle.

1 is a schematic diagram for explaining an embodiment of a composite sensor for a vehicle disclosed in the present specification.
2A and 2B are diagrams for describing an embodiment of separating an image.
3 is a flow chart for explaining an embodiment of a composite sensing method for a vehicle disclosed herein.
4 is a view for explaining a specific embodiment of the vehicle composite sensing method described in FIG.

The following merely illustrates the principles of the invention. Therefore, those skilled in the art will be able to devise various apparatuses which, although not explicitly described or shown herein, embody the principles of the invention and are included in the concept and scope of the invention. It is also to be understood that all conditional terms and examples recited in this specification are, in principle, expressly intended for the purpose of enabling the inventive concept to be understood, and are not intended to be limiting as to such specifically recited embodiments and conditions .

It is also to be understood that the detailed description, as well as the principles, aspects and embodiments of the invention, as well as specific embodiments thereof, are intended to cover structural and functional equivalents thereof. It is also to be understood that such equivalents include all elements contemplated to perform the same function irrespective of the currently known equivalents as well as the equivalents to be developed in the future, i.e., the structure.

Thus, for example, it should be understood that the block diagrams herein illustrate exemplary conceptual aspects embodying the principles of the invention. Similarly, all flowcharts, state transition diagrams, pseudo code, and the like are representative of various processes that may be substantially represented on a computer-readable medium and executed by a computer or processor, whether or not the computer or processor is explicitly shown .

The functions of the various elements shown in the figures, including the functional blocks depicted in the processor or similar concept, may be provided by use of dedicated hardware as well as hardware capable of executing software in connection with appropriate software. When provided by a processor, the functions may be provided by a single dedicated processor, a single shared processor, or a plurality of individual processors, some of which may be shared.

Also, the explicit use of terms such as processor, control, or similar concepts should not be interpreted exclusively as hardware capable of running software, and may be used without limitation as a digital signal processor (DSP) (ROM), random access memory (RAM), and non-volatile memory. Other hardware may also be included.

In the claims of the present specification, the elements represented as means for performing the functions described in the detailed description may include, for example, a combination of circuit elements performing the function or any type of software including firmware / And is coupled with appropriate circuitry for executing the software to perform the function. It is to be understood that the invention as defined by the appended claims is not to be interpreted as limiting the scope of the invention as defined by the appended claims, as the functions provided by the various enumerated means are combined and combined with the manner in which the claims require. .

The above objects, features, and advantages will become more apparent from the following detailed description taken in conjunction with the accompanying drawings, and thus, it will be possible to easily implement the technical idea of self-invention having ordinary skill in the art. . In the following description, a detailed description of known technologies related to the present invention will be omitted when it is determined that the gist of the present invention may be unnecessarily blurred. Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the accompanying drawings.

In the present specification, a vehicle composite sensor and a composite sensing method for determining a lane departure using an image photographed by a camera and controlling a vehicle wiper are disclosed. That is, a lane using a video captured by a camera using a camera module that performs a function of a lane departure warning system (LDWS), which is one of active safety devices, without adding a conventional rain sensor. The function of the departure detection system and the rain sensor can be performed simultaneously.

The general rain sensor communicates with the control unit of the vehicle and performs a function of automatically operating the wiper during the 'rainwater detection mode'. The rain sensor uses an optical component that emits infrared light into the windshield, the coupler is attached to the windshield by the windshield producer using adhesive SOTEFA, and the rain sensor module is mechanically attached to the optical coupler using a spring clip. . However, such a rain sensor uses an omnidirectional scattering method, causing errors in the wiper operation due to errors in the calculation of raindrops, high cost (coupler application), increased production costs due to increased assembly processes, failure of the wiper and undetected rain sensing. There are drawbacks to this.

However, the invention disclosed herein can solve this problem, and performs rain sensor detection based on the vision sensor. In case of rain sensor detection based on vision sensor, the image captured by camera installed in the vehicle is recognized through image processing process, so it is possible to sense the windshield of the same area as the driver's field of view. By increasing, the occurrence rate of the existing wiper operation error can be significantly reduced.

The lane departure detection system is realized through one camera module installed in the vehicle, and the blurring of the input image can be processed through software, so there is no need to install an additional expensive rain sensor. It can bring savings.

Meanwhile, the lane departure prevention system obtains an image of the road ahead using a vision sensor and extracts road information necessary for autonomous driving through a lane recognition process. Concentration projection, Hough transform, Sobel mask, and matching methods using stored patterns are used to recognize such lanes.

Since the methods of the conventional lane departure prevention system require a large amount of computation, it may be subject to limitations in applying to a real-time autonomous driving system. Therefore, in order to solve this problem, a real-time autonomous driving system can be implemented by using a thinning area algorithm using Kalman Filter.

Hereinafter, the above-described invention will be described in detail with reference to the accompanying drawings.

1 is a schematic diagram for explaining an embodiment of a composite sensor for a vehicle disclosed in the present specification.

Referring to FIG. 1, the composite sensor for a vehicle may include an image separator configured to separate a camera image captured by the camera module 101 and the camera module into a lane detection image for lane detection and a rainwater detection image for rainwater detection ( 102, the lane departure control unit 104 determines whether the lane departure based on the lane detection image and generates a lane departure control signal; and the wiper control unit 103 which determines the amount of rain based on the rainwater detection image and generates the wiper control signal. ).

The camera module 101 is for acquiring an image, and the method of acquiring an image is not limited. Typically can be implemented by a digital sensor such as an image sensor (eg CMOS image sensor). The image captured by the camera module 101 may be, but is not limited to, a forward-looking image of the vehicle, and the location of the captured image is not a problem as long as the lane detection and rainwater detection can be performed simultaneously.

The image separation unit 102 performs image processing on an image input from the camera module 101, and separates the lane detection image for lane detection and the rainwater detection image for rainwater detection. Image separation can use a variety of methods. For example, 1) a single image input from the camera module 101 may be separated into a lane detection image and a rainwater detection image through filtering, or 2) the image captured by different focusing points of the camera module 101 may be lane Various methods may be used, such as separating the detected image and the rainwater detection image, or 3) dividing one image input from the camera module 101 according to a region into a lane detection image and a rainwater detection image. Here, dividing a single image according to an area may set a rainwater detection area at an optimal position to detect rainwater among input images, and the size and position of the area may be changed according to the state and condition of the input image. Can be.

Therefore, the image separation unit 102 may be an image for determining lane detection and rainwater detection using one or a plurality of images photographed from one camera mounted on a vehicle, regardless of a method of separation. It performs the function of separating.

The wiper controller 104 may determine the amount of raindrops based on the sharpness of the edge detected from the rainwater detection image. That is, based on the sharpness of the edge extracted by image processing the rain detection image, the degree of blurring (Blurring) is determined to perform rain sensing. The wiper controller 104 outputs a control signal for operating the wiper when the blurring is greater than or equal to the reference value. The wiper drive module operates the wiper according to the control signal.

The lane departure control unit 103 determines whether the lane is departed based on the lane detection image, and generates a lane departure control signal. The lane departure detection module notifies the driver of a lane departure alarm according to a lane departure control signal.

Here, the wiper controller 104 and the lane departure controller 103 may perform image processing on the rainwater detection image and the lane detection image to determine the amount of raindrops and lane departure, respectively. The image separation unit 102 may be integrated into one image processing unit.

Meanwhile, the wiper control unit 104 and the lane departure control unit 103 may include a rainwater detection analysis for determining raindrops and a lane detection analysis for determining lane departure, or rainwater detection. The processors performing the analysis and the lane detection analysis can also be shared.

In addition, image processing and rainwater detection analysis or image processing and lane detection analysis may be integrated in one processor, and each function may be implemented as a plurality of processors by sharing each function.

Therefore, according to the implementation method, the image separation unit 102, the wiper control unit 104, and the lane departure control unit 103 may be integrated into one processor, each may be implemented as a separate processor, or some may be one processor. It can be integrated.

2A and 2B are diagrams for describing an embodiment of separating an image.

Referring to FIG. 2A, an image 200 captured by a camera module is divided into an area 201 of an image for lane detection and an area 202 of an image for rainwater detection. The area 201 of the image for lane detection is an image obtained by dividing an area including the lane image 211, and the area 202 of the image for rain detection is divided into an area not including the lane image 211. It is a video.

In the example of FIG. 2A, an image 200 obtained by photographing a front driving area by a single camera module is divided into a lane detecting area 201 and a rainwater detecting area 202 to perform image processing on each area. do. That is, the division of the area in FIG. 2 primarily determines the area where the driving lane 211 is detected as the lane detection area 201, and the area where the lane is not visible (in FIG. 2A, the bottom of the vehicle detection area 201). The part including the vehicle bonnet 212 is divided into the rainwater detection region 202 and the determination is performed.

In FIG. 2A, the rainwater detecting region 202 is located at the bottom of the lane detecting region 201. FIG. 2B is another embodiment of dividing an input image, and may set the rainwater detection region 221 between the driving lanes 211 of the lane detection region 201. Accordingly, as shown in FIG. 2B, a predetermined area may be set as the rainwater detection area 221 in the lane detection area 201, and in some cases, the rainwater detection area and the lane detection area may overlap.

The lane detection area 201 detects the departure of the white driving line 211 of the driving lane and determines whether the vehicle departs. The rain detection area 202 extracts the edge of the rain water and then blurs the image based on the sharpness of the extracted edge. Judge the ring.

Hereinafter, a hybrid sensing method for a vehicle will be described.

3 is a flow chart for explaining an embodiment of a composite sensing method for a vehicle disclosed herein.

Referring to FIG. 3, the hybrid sensing method for a vehicle is based on an image separation step of separating a camera image photographed by a camera module into a lane detection image for lane detection and a rainwater detection image for rainwater detection (S301), based on a lane detection image. Determining whether the lane is separated (S302), the lane departure control step of generating a lane departure control signal (S303) and the rain quantity determination based on the rainwater detection image (S304) and the wiper control step of generating a wiper control signal (S305) ).

The lane detection image may include an image obtained by dividing an area including a lane image from a camera image, and the rainwater detection image may include an image obtained by dividing an area not including a lane image from a camera image.

The camera image may be a forward-looking image of the vehicle, and in the wiper control steps S304 and S305, an image processed by using a Kalman filter on the lane detection image may be used for determining the amount of rain.

The vehicle composite sensing method may be implemented by a computer program or may be mounted on the vehicle composite sensor in the form of a system on chip or the like, or may be mounted on a vehicle controller such as a central controller mounted on the vehicle.

Detailed description of the other vehicle composite sensing method is the same as the description of the above-described vehicle composite sensor will be omitted here.

4 is a view for explaining a specific embodiment of the vehicle composite sensing method described in FIG.

Referring to FIG. 4, the front image of the vehicle is photographed using the camera module (S401). The image photographed by the camera module is scaled into an image region for determining a driving lane and an image region for determining rainwater detection (S402).

The image area for lane determination and the rainwater detection image area serve as the basis for determination of lane departure detection and rainwater detection, respectively.

First, the lane departure detection method will be described. Based on the captured image, it is determined whether the lane is separated (S412), and when it is determined that the lane is separated, a control signal for operating the alarm device is generated (S413). The alarm device operates according to the control signal (S414).

Next, when the rainwater detection method is described, the blurring degree of the image for rainwater detection is determined (S422), and when the degree exceeds the reference value, a control signal for operating the wiper is generated (S423). The wiper is operated according to the generated control signal (S424).

The composite sensor for a vehicle disclosed in the present specification can be implemented by integrating various types of sensors interspersed in a vehicle. In this case, there is an effect of reducing the cost of the components by integrating or unifying the electronic components that must be applied in common for each sensor.

The above-described vehicle composite sensor integrates a lane departure warning system and a rainwater detection system. However, the above-described vehicle composite sensor may also be integrated with a defog sensor or illuminance solar sensor (ALS / SUN).

While the present invention has been particularly shown and described with reference to exemplary embodiments thereof, it is to be understood that the invention is not limited to the disclosed exemplary embodiments, but, on the contrary, is intended to cover various modifications and equivalent arrangements included within the spirit and scope of the appended claims. It can be understood that the modification and application of the present invention are possible. For example, each component specifically shown in the embodiments can be modified and implemented. It is to be understood that the present invention may be embodied in many other specific forms without departing from the spirit or essential characteristics thereof.

101: camera module
102: video separation unit
103: wiper control unit
104: lane departure control

Claims (10)

Camera module;
An image separator for dividing the camera image photographed by the camera module into a lane detection image for lane detection and a rainwater detection image for rainwater detection;
A lane departure control unit determining whether lane departure is performed based on the lane detection image and generating a lane departure control signal; And
A wiper control unit for determining a raindrop amount and generating a wiper control signal based on the rainwater detection image;
The lane detection image includes an image obtained by dividing an area including a lane image from the camera image.
The rainwater detection image includes an image obtained by dividing an area in which the lane image is not included in the camera image.
delete The method of claim 1,
The camera image is a front perspective image of the vehicle, the composite sensor for a vehicle.
The method of claim 1,
And the wiper control unit determines the amount of raindrops based on the sharpness of an edge detected from the rainwater detection image.
The method of claim 1,
The lane detection image includes an image processed using a Kalman filter.
An image separation step of separating the camera image photographed by the camera module into a lane detection image for lane detection and a rainwater detection image for rainwater detection;
A lane departure control step of determining whether lane departure is based on the lane detection image and generating a lane departure control signal; And
A wiper control step of determining a raindrop amount and generating a wiper control signal based on the rainwater detection image;
The lane detection image includes an image obtained by dividing an area including a lane image from the camera image.
The rainwater detection image comprises an image obtained by dividing an area in which the lane image is not included in the camera image. delete delete delete delete

Images