KR101750419B1 - Integated sensor for vehicle - Google Patents

Abstract

An image sensing unit sequentially outputting normal images by successively acquiring images of a windshield of a vehicle in a predetermined cycle and sequentially outputting a differential image obtained by subtracting the images at the first and second points of view from the normal image sequentially; And a controller for outputting signals for controlling operations of a wiper, an air conditioner, and a lighting device installed in a vehicle using the normal image and the differential image output from the image sensing unit, respectively, wherein the image sensing unit includes: A first and a second output line which are branched at an output terminal of each pixel of the light receiving element; and a differential circuit which is provided on the second output line and outputs a differential value of an output value of the pixel, And a switch for switching the output signals of the first and second output lines at regular or arbitrarily set intervals so that the controller alternately acquires the normal image and the differential image at an output terminal of the image sensing unit And an integrated sensor for a vehicle is provided.

Description

Integrated sensor for vehicle {INTEGATED SENSOR FOR VEHICLE}

The present invention relates to a vehicle integrated sensor in which the functions of a vehicle rain sensor for detecting raindrops of a windshield of a vehicle, a sunroof sensor for detecting the position of the sun, and an auto light sensor for sensing the brightness of light are incorporated in one sensor .

In recent years, vehicle automation / intelligent technologies such as a driver assistance system, a collision warning system, and a fully automatic temperature control system have been rapidly developed. These automation / intelligent technologies are used to control the operation of the wiper Technology, a technique for controlling the air conditioner of a vehicle by sensing the position of the sun on the vehicle, and a technique for controlling the operation of a vehicle lighting device (head lamp, taillight, etc.) by sensing the brightness of light around the vehicle.

In order to realize these technologies, various sensors are installed in the vehicle. The sensors include a rain sensor for detecting the amount of raindrops, a sun-load sensor for detecting the position of the sun on the vehicle ), An auto light sensor for detecting the brightness of the surroundings of the vehicle, and the like. In the case of a rain sensor, a light type structure is generally used in which a signal light such as an LED is incident on a windshield at an angle and a light detector installed on the opposite side detects the light. A light load sensor or an auto light sensor A structure using a light receiving sensor whose output voltage varies according to the amount is generally used. Each of these sensors is generally installed separately at various positions of the vehicle. Such separate structures increase the manufacturing cost and hinder efficient space utilization of the vehicle.

In order to solve such a problem, an integrated sensor that performs the functions of a plurality of sensors on one sensor using an image sensing means such as a camera has been proposed (see Patent Document 1 and Patent Document 2 below) It is structured to detect raindrops, brightness, etc. by using a captured image of the image sensing means.

However, in the case of the conventional technique, the image information obtained by the image sensing means is used as it is in the image sensing means. In order to cope with the changing environment in real time, the amount of image data to be calculated increases, There is a problem that a high-performance computing means is required.

Patent Registration No. 10-1510010 (Apr. 201, 2015) Japanese Patent Application Laid-Open No. 2010-210374 (Sep. 24, 2010)

SUMMARY OF THE INVENTION The present invention has been made in view of the above points, and it is an object of the present invention to integrate the functions of a rain sensor, a linear load sensor and an auto light sensor into a single sensor, And to provide an integrated vehicle sensor capable of realizing such a vehicle.

It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory and are not intended to limit the invention to the precise forms disclosed. Other objects, which will be apparent to those skilled in the art, It will be possible.

In order to achieve the above-described object, the present invention provides a windshield of a vehicle, comprising: sequentially acquiring images of a windshield of a vehicle in a predetermined cycle to output normal images sequentially; and calculating a difference between a first viewpoint and a second viewpoint An image sensing unit sequentially outputting a differential image in which only pixels having a change in pixel data are displayed, separately from the normal image; And a controller for outputting signals for controlling operations of a wiper, an air conditioner, and a lighting device installed in a vehicle using the normal image and the differential image output from the image sensing unit, respectively, wherein the image sensing unit includes: A first and a second output line which are branched at the output end of each pixel of the light receiving element and a second output line which is provided on the second output line and outputs the pixel at the first and second points of time And a differential amplifier for outputting the difference value for each pixel and outputting the difference image by combining the signals of the difference values of the output pixels, wherein the output terminal of the image sensing unit is connected to the normal image and the difference When the output signals of the first and second output lines are alternately or periodically set to obtain images alternately And the key is provided with a switch.

According to the vehicle integrated sensor of the present invention, the light receiving element can be configured to focus the focus on the surface of the windshield.

According to the vehicle integrated sensor of the present invention, a wide dynamic range image sensor can be used as the light receiving element.

According to the vehicle integrated sensor of the present invention, the differential value of each pixel corresponds to the voltage difference at the second time point and the first time point, and the controller controls the wiper control based on the number of differential values output from the differential amplifier A signal can be output.

According to the vehicle integrated sensor of the present invention, the controller calculates a value obtained by dividing the number of pixels for outputting the difference value by the total number of pixels and stores the value in a separate storage space, and when the stored value stored in the storage space exceeds a predetermined threshold value , And accumulates the output signal in the storage space so as to be cumulatively added to the operation value of the next period if the stored value is smaller than the threshold value.

According to the vehicle integrated sensor of the present invention, the controller outputs a signal relating to the coordinate information of the sun for controlling the operation of the air conditioner based on the voltage peak value information of the normal image, and based on the voltage average information of the normal image To output a signal relating to brightness information for controlling the operation of the lighting apparatus.

According to the present invention having the above-described structure, it is possible to reduce the size of the sensor device and the manufacturing cost by integrating the three function sensors into a single sensor, and it is possible to solve the restriction of the sensor installation space.

Particularly, a differential image between successively acquired images is used for the rain sensing. Since this differential image does not include an image other than a detection target (i.e., a rain, an eye, a foreign object, etc.) It is effective. In addition, the detection result of the amount of raindrops can be obtained by simple calculation using such small-sized data.

In addition, a differential amplifier can be applied to the image sensing unit, so that a difference image can be obtained only by the circuit configuration without processing the controller, thereby reducing the calculation load of the controller. By using an image sensing unit equipped with Wide Dynamic Range function, it is not necessary to use a time lag to switch the exposure and to adjust the light level separately. It is necessary to use separate illumination and diffusion plate There is no advantage.

BRIEF DESCRIPTION OF THE DRAWINGS FIG. 1 is a block diagram illustrating components of an integrated vehicle sensor and associated vehicle according to an embodiment of the present invention; FIG.
2 is a sectional view showing an example of the internal structure of an integrated sensor for a vehicle related to the present invention.
3 is a view showing an installation position and a form of an integrated vehicle sensor related to the present invention.
4 is a diagram showing the concept of a differential image related to the present invention;
5 is a photographed image illustrating a differential image related to the present invention.
Fig. 6 is a schematic view schematically showing the configuration of the image sensing unit shown in Fig. 1; Fig.
Fig. 7 is a schematic diagram showing the configuration and operation of one of the differential amplifiers of Fig. 6; Fig.
FIG. 8 is a flowchart showing a rain sensing, line load sensing, and auto light sensing process according to the present invention. FIG.
9 is a detailed flowchart of the rain sensing process of FIG.
10 shows an image receiving area associated with the pre-load sensing process of Fig.
11 shows voltage distributions of a normal image associated with the linear load sensing and auto light sensing processes of FIG. 8;

Hereinafter, an integrated vehicle sensor related to the present invention will be described in detail with reference to the drawings.

1 is a block diagram illustrating components of an integrated vehicle sensor and associated vehicle according to an embodiment of the present invention.

1, an integrated vehicle sensor 100 according to an exemplary embodiment of the present invention is a sensor integrated with functions of a rain sensor, a road rod sensor, and an auto light sensor. The sensors include an image of the surface state of the windshield 10 (Rain sensing information), sun coordinate information (sun load sensing information), sunlight incident through the windshield 10, and incident light amount based on the sensed light intensity information Information) to be provided to the vehicle controller 200.

The integrated vehicle sensor 100 includes an image sensing unit 110 and a controller 120.

The image sensing unit 110 sequentially acquires images of the windshield 10 of the vehicle at predetermined intervals and outputs the images. Specifically, the image sensing unit 110 is configured to sequentially output the image of the image of the windshield 10, sequentially output the difference image obtained by subtracting the image at the first and second points of view from the normal image separately .

The controller 120 controls the operation of the devices installed in the vehicle, that is, the wiper 210, the air conditioner 220, and the lighting device 230 using the normal image and the differential image output from the image sensing unit 110 And outputs a signal.

The vehicle controller 200 is for controlling the operation of the electronic components of the vehicle. In this embodiment, these components include a wiper 210, an air conditioner 220, a lighting device 230 (for example, a head lamp, .

The vehicle controller 200 controls the operation of the wiper 210, for example, the operation period, the speed, and the like in accordance with the wiper control information provided by the integrated vehicle sensor 100. For example, when the detected amount of raindrops is large, the operation period or speed of the wiper 210 is increased.

The vehicle controller 200 controls the operation of the air conditioner 220 according to the coordinate information of the sun (i.e., the relative position of the sun relative to the vehicle) provided by the integrated vehicle 100 for a vehicle. For example, according to the left and right position of the sun, the temperature of the driver's seat and the passenger's seat can be controlled differently, or the air volume can be controlled according to the position of the sun. Also, the vehicle controller 200 controls ON / OFF, brightness, etc. of the illumination device 230 according to the brightness information of the light provided by the integrated sensor 100 for a vehicle.

FIG. 2 is a sectional view showing an example of an internal structure of an integrated sensor for a vehicle according to the present invention, and FIG. 3 is a view showing an installed position and a configuration of an integrated sensor for a vehicle according to the present invention.

2, the vehicle integrated sensor 100 includes a housing 101 for receiving an image sensing unit 110, a controller 120, and the like. A circuit board 102 on which electronic elements such as an image sensing unit 110 and a controller 120 are mounted is installed in the housing 101 and a condenser lens 103 is mounted on an opening of the housing 101, And the light receiving portion of the light source 100 may be installed.

3 (a), the vehicle integrated sensor 100 may be installed in the dashboard 20 of the vehicle so that the light receiving portion thereof is oriented in the upper direction of the vehicle. This is to obtain a more accurate image by reducing the influence of the headlight light of the opposite vehicle or the image change due to the driving of the vehicle. The influence of sunlight, road illumination, etc. can be compensated by applying a wide dynamic range function to the image sensing unit 100. Further, since the focus is on the windshield 10, the background image is not clear and it is easy to distinguish it from a sensing material such as a raindrop.

Although not shown in the drawings, the integrated vehicle sensor 100 may be installed in the room mirror as well as the dashboard 20. In such a case, the light receiving unit may be oriented upward.

On the other hand, it is also possible to dispose the vehicle integrated sensor 100 such that the light receiving portion of the integrated sensor 100 for a vehicle is oriented in an inclined direction at a certain angle with respect to the horizontal direction (or vertical direction) will be.

4 is a diagram showing the concept of a difference image related to the present invention.

The controller 120 outputs a signal for controlling the operation of the wiper 210 by using a difference image (differential image, difference image) obtained by subtracting the image at the first and second points of time. The controller 120 alternately acquires the normal image, that is, the image of the windshield 10 as the differential image, and outputs signals for controlling the operation of the air conditioner 220 and the lighting device 230, respectively.

One rectangle in Fig. 4 represents a pixel, which illustrates an image frame having nine (3 x 3) pixels.

The image sensing unit 110 sequentially acquires images in a predetermined cycle, and this cycle is arbitrarily settable. For example, it is possible to set to sequentially acquire one image frame per second.

4 illustrates an image F1 at a first time point and an image F2 at a second time point after one cycle, respectively. According to this, it is exemplified that a change occurs in one pixel when compared with the image F2 at the second time point and the image F1 at the first time point.

The image sensing unit 110 outputs a difference image F2-F1 obtained by subtracting the image F1 at the first time point and the image F2 at the second time point. According to Fig. 4, only the pixel in which the pixel data has changed remains in the difference image F2-F1, and the pixel in which the change has not occurred is not displayed in the difference image F2-F1. That is, the data related to the pixels that have not changed are removed in the differential processing, and the amount of data transferred from the image sensing unit 110 to the controller 120 can be reduced accordingly.

5 is a photograph image illustrating a difference image related to the present invention.

FIG. 5 is a diagram showing an example in which the image sensing unit 110 calculates the difference between the image F1 actually photographed at the first time point, the image F2 actually photographed at the second time point of the next cycle and the image of the first and second time points, Image F2-F1. The photographed image indicates that the foreign object has been added at the second time point, and it can be confirmed that the remaining portion (background building) excluding the portion where the change has occurred in the difference image F2-F1 is removed as described above.

The image sensing unit 110 sequentially outputs the differential image as described above. That is, the image sensing unit 110 includes first, second, third, fourth, ... Sequentially obtain differential images F2-F1, F3-F2, F4-F3, ... between the respective viewpoints by successively acquiring images F1, F2, F3, F4,

Fig. 6 is a schematic view schematically showing the configuration of the image sensing unit shown in Fig. 1, and Fig. 7 is a schematic diagram showing the configuration and operation of one of the differential amplifiers of Fig.

As shown in FIG. 6, the image sensing unit 110 includes a light receiving element 111, first and second output lines 113 and 114, and a differential amplifier 112.

The light receiving element 111 is an electric element that receives light and converts the image of the windshield 30 into an electric signal and is used as an element such as a CCD (Charge Coupled Device) or a CMOS (Complementary Metal-Oxide Semiconductor) . As the light receiving element 111, an image sensor having a wide dynamic range function may be used. This makes it possible to improve the robustness against environmental changes such as nighttime or strong light incidence. In addition, it is not necessary to use the illumination or diffusion plate used in the existing technologies (Patent Documents 1 and 2). For reference, FIG. 6 illustrates a light receiving element 111 having nine (3 × 3) pixels.

The light receiving element 111 is configured to focus the focus on the surface of the windshield 10. Accordingly, the shape of the raindrop on the surface of the windshield 10 can be accurately obtained, and the data on the background image deviated from the focus can be easily excluded from the processed data.

The first output line 113 and the second output line 114 are connected to the output terminal of each pixel of the light receiving element 111, specifically to the photodiode PD of each pixel. The first output line 113 and the second output line 114 are branched from the output terminal of each pixel of the light receiving element 111, respectively. According to the present embodiment, an amplifier 118 is provided at an output terminal of a photodiode PD of each pixel, and a first output line 113 and a second output line 114 are connected to an output terminal thereof. .

The differential amplifier 112 is provided on the second output line 114 and outputs a differential value of the output value (for example, voltage value) of the pixel.

7, an electric signal N at a first point in time of each pixel is input to one input terminal of the differential amplifier 112, and an electric signal N at a second point in time of each pixel is input to the other input terminal +1) is input. (N + 1) - (N)] corresponding to the difference value between the output values of the electric signals at the second time point and the first time point, for example, the voltage difference, is outputted at the output terminal of the differential amplifier 112 . The differential image in which the signals output from the output terminals of the differential amplifiers 112 are combined is outputted through the second output line 114.

According to the above configuration, the first output line 113 outputs an electric signal output from each photodiode PD to output a normal image. The second output line 114 causes the differential amplifier 112 to output a differential image in which the image between the first and second viewpoints is differentiated.

According to this configuration, the image sensing unit 110 outputs the normal image and the differential image through the first and second output lines 113 and 114, respectively.

A switch 130 for switching the output signals of the first and second output lines at regular intervals or at arbitrarily set intervals is installed at the output terminal of the image sensing unit 110 so that the controller 120 alternately acquires the normal image and the differential image do. The switch 115 is operated to alternately connect the first and second output lines 113 and 114 to the output line 116. The switching operation of the switch 130 is configurable to be performed under the control of the controller 120. [

The normal image or difference image from the switch 130 is input to the controller 120 through the output line 140.

FIG. 8 is a flowchart illustrating a rain sensing, pre-load sensing, and auto light sensing processes according to the present invention.

The controller 120 repeats and acquires the difference image and the normal image at regular intervals (S10, S20).

The controller 120 performs a rain sensing process based on the difference image. The controller 120 may output a wiper control signal based on the number of difference values output from the image sensing unit 110 (S30, S40).

9 is a detailed flowchart of the rain sensing process of FIG. As shown in FIG. 9, the controller 120 calculates a value obtained by dividing the number of pixels for which the difference value is output by the total number of pixels (S30). The calculated values are stored in a separate storage space (for example, storage means such as a memory) (S41).

Next, the controller 120 compares the stored value stored in the storage space with a predetermined threshold value (S42). As a result, if the storage value stored in the storage space is equal to or greater than a preset threshold value (for example, 30%), an output signal is generated (S43). That is, when the changed area of the entire light receiving area is equal to or larger than a predetermined ratio, it is determined that the raindrop covers the entire area of the windshield 10 at a predetermined ratio or more, thereby generating a signal for operating the wiper 30.

The controller 120 accumulates the stored value in the storage space so that the stored value is cumulatively added to the calculated value in the next cycle if the stored value of the comparison result of the stored value and the threshold is less than the threshold value (S44). In this case, the accumulated values accumulated in the storage space are cumulatively added and updated until the value exceeds the threshold value. The controller 120 generates an output signal when the cumulatively accumulated value becomes equal to or greater than a threshold value. The output signal may have a pulsed wave form and the vehicle controller 200 may have a configuration for controlling the driver of the wiper 210 to reciprocate the wiper 210 a predetermined number of times per pulse.

The controller 120 performs a pre-load sensing process and an auto light sensing process based on the voltage distribution of the normal image. In other words, the controller 120 outputs a signal relating to the coordinate information of the sun for the operation of the air conditioner 220 and a signal relating to brightness information for controlling the operation of the lighting apparatus 230, based on the voltage distribution of the normal image . Since the focus of the image sensing unit 110 is adjusted to the windshield 110, a clear image can be obtained, but it is sufficient to calculate the position of the sun and the brightness of the light.

Fig. 10 is a view showing an image receiving area associated with the pre-load sensing process of Fig. 8, and Fig. 11 is a diagram showing a voltage distribution of a normal image related to the pre-load sensing and the auto light sensing process of Fig.

10 shows a divided screen of the light receiving element 111 obtained by dividing the normal image into a plurality of mesh-shaped areas, and the number of divided areas can be variously set up to the maximum number of pixels. 11 shows the distribution of the voltage signal level of the normal image obtained by the controller 120 in the X direction (or the Y direction).

In connection with the pre-load sensing process, the controller 120 detects the area information in which the voltage peak value is detected in the split screen of the normal image (i.e., the portion indicated by red in Fig. 10 and the portion indicated by the peak in Fig. 11 S50), and outputs a signal relating to the coordinate information of the sun based on the detected voltage peak value information (S60). The controller 120 determines that the voltage level of the peak value is equal to or greater than a predetermined threshold value and calculates the highest level value (center value in the X-Y direction) as the sun's coordinates.

Regarding the auto light sensing process, the controller 120 calculates a voltage average value (see FIG. 11) from the normal image (S70) and outputs a signal relating to the brightness information (S80). The controller 120 may divide the voltage level into a plurality of sections and generate a brightness level corresponding to the section as an output signal. That is, the controller 120 may generate the output level of the brightness level corresponding to the corresponding interval after calculating the interval corresponding to the calculated voltage average value.

According to this method, since the line load sensing and the auto light sensing can be simultaneously performed through the process of calculating the voltage distribution, there is an advantage that the calculation load can be reduced accordingly.

The vehicle integrated sensor described above is not limited to the configuration and the method of the embodiments described above, but various modifications can be made by those skilled in the art within the scope of the technical idea of the present invention.

10: Windshield 20: Dashboard
30: Wiper 100: Rain sensor for vehicle
110: Image sensing unit 111: Light receiving element
112: differential amplifier 120: controller
130: switch 140: output line
200: vehicle controller 210: wiper
220: air conditioner 230: lighting device

Claims (6)

The image of the windshield of the vehicle is successively acquired at a predetermined cycle to sequentially output the normal images, and only the pixels at which the pixel data at the first and second points of time are changed are obtained by subtracting the images at the first and second points An image sensing unit sequentially outputting a displayed difference image separately from the normal image; And
And a controller for outputting a signal for controlling the operation of the wiper, the air conditioner, and the lighting device installed in the vehicle using the normal image and the differential image output from the image sensing unit,
The image sensing unit includes:
A light receiving element for generating the normal image;
First and second output lines branched at the output terminal of each pixel of the light receiving element; And
Outputting a difference value between the output values of the pixels at the first view point and the second view point for each pixel and combining the signals of the difference values of the output pixels to output the difference image; A differential amplifier,
Wherein a switch is provided at an output terminal of the image sensing unit to switch the output signals of the first and second output lines at arbitrary set intervals so that the controller can alternately acquire the normal image and the differential image. . The method according to claim 1,
Wherein the light receiving element is configured to focus the focus on the surface of the windshield. The method according to claim 1,
Wherein the light receiving element is a wide dynamic range image sensor. The method according to claim 1,
Wherein a difference value of each pixel corresponds to a voltage difference between the second and first time points,
Wherein the controller outputs a wiper control signal based on the number of differential values output from the differential amplifier. 5. The apparatus of claim 4,
A value obtained by dividing the number of pixels for which the difference value is output by the total number of pixels is calculated and stored in a separate storage space,
Generates an output signal when the stored value stored in the storage space is equal to or greater than a predetermined threshold value,
If the stored value is smaller than the threshold value, accumulates the accumulated value in the storage space so as to be cumulatively added to the calculated value of the next cycle. 2. The apparatus of claim 1,
And outputs a signal relating to the coordinate information of the sun for controlling the operation of the air conditioner based on the voltage peak value information of the normal image,
And outputs a signal relating to brightness information for controlling the operation of the lighting device based on the voltage average value information of the normal image.

Images