KR20170030816A - Smart cruise control apparatus and method with traffic sign detecting function - Google Patents

Abstract

The present invention relates to an automatic cruise control apparatus having a traffic light detecting function and a control method thereof. The apparatus of the present invention comprises: a front camera which shoots an image of the front side of a vehicle; a stop line recognition unit which detects a stop line in the image shot by the front camera; a traffic signal recognition unit which detects the color of a traffic light in the image shot by the front camera; and a control module which, when the stop line is detected, determines whether the vehicle must stop at the stop line in accordance with the color of the traffic light and a preset condition, and which controls the vehicle to stop or drive while keeping a certain distance from the preceding vehicle. Accordingly, the present invention can prevent an accident at a crosswalk or a crossing, and prevent an accident or a traffic jam from being caused by a change in traffic light after a vehicle enters a crossing in congested traffic.

Description

TECHNICAL FIELD [0001] The present invention relates to an automatic cruise control system and a control method thereof,

The present invention relates to an automatic cruise control device and a control method having a signal light sensing function, and more particularly, to an automatic cruise control device and a control method thereof that detect a traffic light during an automatic cruise control and control the operation of the vehicle according to a traffic signal, The present invention relates to an automatic cruise control device and a control method thereof.

With the rapid increase in the use of cars in modern society, the number of fatalities and casualties due to car accidents has reached tens of millions of people every year. Accordingly, a variety of automotive technologies such as ADAS (Advanced Driver Assistance System), which prevents accidents by using advanced sensing sensors and intelligent video equipment, are being developed in order to reduce personal injury and economic loss caused by traffic accidents.

Such an intelligent driving assist system includes a forward collision warning (FCW) technique, an automatic cruise control (ACC) technique, a lane change assistance technique, a lane departure warning technique , And Parking Assistance technology.

Among these, the automatic cruise control technology is also called a smart cruise control technique. It automatically detects the preceding vehicle in the same direction in front of the driving lane while maintaining the driving lane according to the driver's setting conditions And is a technology for automatically running at a target speed while maintaining the safety distance from the preceding vehicle by automatically accelerating or decelerating according to the speed of the preceding vehicle.

On the other hand, in the case of urban areas, frequent stops are required such as stop lines and crosswalks due to high traffic volume. Therefore, when the vehicle travels in the automatic running mode using the automatic cruise control technology that keeps the safety distance from the preceding vehicle, even if the traffic light is a stop signal, if the preceding vehicle does not stop at the stop line or the crosswalk, As with the preceding vehicles, they do not stop at stop lines or pedestrian crossings. Therefore, a penalty due to a traffic signal violation may be imposed, or if a driver can not find a pedestrian using a pedestrian crossing, it may lead to a personal accident. In addition, when entering an intersection due to a stop line violation of an intersection, serious traffic accidents may occur, and traffic jam due to intersection tail may occur in the delay section.

Thus, by enabling traffic signals to be detected even while using the automatic cruise control technology, it is necessary to prevent the vehicle from crossing a stop line or a crosswalk in a stop signal or entering an intersection in a lagging section.

The present invention proposes an automatic cruise control device and a control method having a signal light detection function for detecting a traffic light during automatic cruise control and controlling the operation of the vehicle according to a traffic signal to prevent traffic congestion at an accident or an intersection do.

The object is achieved by a front camera for photographing the front of a vehicle; A stop line recognizing unit for detecting a stop line in the image photographed by the front camera; A traffic signal recognition unit for sensing a color of a traffic light in an image photographed by the front camera; And a control module for determining whether the vehicle should stop at the stop line according to a hue of the traffic light and a preset condition when the stop line is detected and controlling the vehicle to stop or run while maintaining a certain distance from the preceding vehicle And an automatic cruise control device having a signal light detection function.

The above object can be accomplished by a vehicle comprising: a front image of a vehicle taken to form a forward image; Detecting a stop line in the forward image; Sensing a color of a traffic light in the forward image; Whether or not the vehicle should stop at the stop line in accordance with the color of the traffic light, and whether or not the vehicle should stop at the stop line in accordance with whether or not the traffic light is green, ; And controlling the vehicle to stop at a certain distance from the preceding vehicle when the vehicle is determined to stop at the stop line. .

According to the automatic cruise control device capable of recognizing a signal light, according to the present invention, by recognizing the stop line and the traffic light during the operation of the automatic cruise function, it is possible not only to prevent an accident at a crosswalk or at an intersection, It is possible to prevent accidents and traffic congestion due to the change of traffic lights after entering.

1 is a block diagram of a configuration of an automatic cruise control device capable of recognizing a traffic light according to the present invention,
Fig. 2 is a block diagram of the camera module of Fig. 1,
Figure 3 is a block diagram of the arithmetic module of Figure 1,
4A is a diagram illustrating a method for calculating a distance from a vehicle to a stop line using a pinhole camera model,
FIG. 4B is a diagram simply showing an image taken by a front camera,
Fig. 5 is a block diagram of the control module of Fig. 1,
FIG. 6 is a flowchart illustrating a process of controlling the driving of a vehicle according to recognition of a traffic light in an automatic cruise control device capable of recognizing a traffic light according to the present invention.

Hereinafter, some embodiments of the present invention will be described in detail with reference to exemplary drawings. It should be noted that, in adding reference numerals to the constituent elements of the drawings, the same constituent elements are denoted by the same reference symbols as possible even if they are shown in different drawings. In the following description of the present invention, a detailed description of known functions and configurations incorporated herein will be omitted when it may make the subject matter of the present invention rather unclear.

In describing the components of the present invention, terms such as first, second, A, B, (a), and (b) may be used. These terms are intended to distinguish the constituent elements from other constituent elements, and the terms do not limit the nature, order or order of the constituent elements. When a component is described as being "connected", "coupled", or "connected" to another component, the component may be directly connected to or connected to the other component, It should be understood that an element may be "connected," "coupled," or "connected."

1 is a block diagram of a configuration of an automatic cruise control apparatus capable of recognizing a traffic light according to the present invention.

The automatic cruise control device (1) detects a traffic light and a stop line during use of the automatic cruise function, and when the vehicle must stop at the stop line, the vehicle can stop at the stop line while maintaining the inter- .

The automatic cruise control device 1 includes a camera module 10 that processes an image generated by photographing the front of the vehicle and recognizes a stop line and a traffic signal in the image, a distance between the preceding vehicle and a stop line, A control module 50 for determining whether the vehicle is stopped and controlling the speed of the vehicle based on the information recognized by the camera module 10 and the information calculated by the arithmetic module 30 .

The camera module 10 may include a front camera 11, a region of interest setting section 13, a stop line recognition section 15 and a traffic signal recognition section 17, as shown in FIG.

The front camera 11 is mounted on the front windshield of the vehicle, and can capture the front of the vehicle to generate a forward image. The front camera 11 may be mounted at a position other than the upper portion of the front windshield, for example, a front windshield lower portion or a side portion, a front grill, a bumper, or the like.

The ROI setting unit 13 may set a region of interest (ROI) for recognizing a traffic light and a stop line in the image photographed by the front camera 11.

The region-of-interest setting unit 13 can project a position where stop lines and signal lights can be detected on a forward image using a homography matrix (H-matrix) in a three-dimensional actual coordinate system. Here, the homography matrix represents a function for mapping a three-dimensional coordinate system to a two-dimensional forward image coordinate system.

Accordingly, the ROI setting unit 13 can generate a signal light ROI for recognizing the traffic light and a stop line ROI for recognizing the stop line. In this case, the signal light region of interest may be generated in the upper region of the front image, which is generally the region where the traffic light is located, and the stop line region of interest may be generated in a predetermined width in the vertical direction centering on the central horizontal line of the forward image.

The traffic signal recognizing unit 17 can recognize the color of the traffic lights in the region of interest of the traffic lights of the forward image set by the region of interest setting unit 13. [ The traffic signal recognition unit 17 can recognize the color of the traffic lights using one of the RGB model, the HSV model, and the YCbCr model.

When the RGB model is used, a range of each color region of R (red), G (green), and B (blue) is set in advance, and the traffic signal recognizing unit 17 recognizes the R, By comparing the G and B values with preset R, G, and B values, it is possible to determine whether the color of the traffic light is red, green, or orange.

The stop line recognition unit 15 can detect the stop line in the stop line interest area of the forward image set by the ROI setting unit 13. [ Here, the stop line is used as a term including a stop line formed for stopping the vehicle before the crosswalk, a stop line indicated before entering the intersection or a three-way intersection, a crosswalk, and the like.

Generally, stop lines are formed by using a white light-emitting paint, and accordingly, the brightness and the saturation are higher than those of other roads because they reflect light. Therefore, the stop line recognition unit 15 can detect the stop line using the luminance and the saturation in the stop line interest area. In other words, the stop line recognition unit 15 recognizes an area where the accumulated value of luminance and chroma in the stop line interest area is equal to or greater than a preset constant value as a stop line.

On the other hand, the stop line recognition unit 15 may detect the stop line by detecting the edge in the forward image. The stop line recognition unit 15 performs a differential operation to detect an edge, and can calculate a horizontal slope and a vertical slope of a forward image using, for example, a Sobel differential operator. Then, the stop line recognition unit 15 binarizes the horizontal slope image and the vertical slope image to 0 and 1, respectively. That is, the stop line recognition unit 15 binarizes the horizontal slope image and the vertical slope image by setting a point having a slope value equal to or greater than a preset threshold value in a horizontal slope image and a vertical slope image as 1 and setting a point having a slope value equal to or less than a threshold value as 0 Thereby generating a horizontal binarized image and a vertical binarized image. Then, the stop line recognition unit 15 can detect the edge in the stop line interest area in the horizontal tilt image and recognize it as a stop line.

3, the arithmetic module 30 includes a stop line distance calculating section 31 for calculating the distance to the stop line, a preceding vehicle distance calculating section 33 for calculating the distance to the preceding vehicle, And an average speed calculating section 35 for calculating the speed.

The stop line distance calculating unit 31 can calculate the distance from the vehicle to the stop line using the forward image. The stop line distance calculating unit 31 may calculate the distance using various models. In this embodiment, a method of calculating the distance from the vehicle to the stop line using a pin-hole camera model is described do.

FIG. 4A is a diagram illustrating a method for calculating a distance from a vehicle to a stop line using a pinhole camera model, and FIG. 4B is a diagram simply showing an image captured by the front camera 11. FIG.

In FIG. 4A, the distance from the vehicle to the stop line is Dx, the height of the camera is hc, the focal length of the camera is f, and the height of the image formed on the forward image is represented by y.

The stop line distance calculating unit 31 calculates the ratio of the height hc of the camera to the distance Dx from the vehicle to the stop line in this pinhole camera model is proportional to the ratio of the focal length f of the camera to the height y of the image formed on the forward image The distance Dx from the vehicle to the stop line can be calculated. That is, the stop line distance calculating unit 31 can calculate the distance Dx from the vehicle to the stop line using the following equation (1).

Here, the height of the camera is hc and the focal length of the camera is f, and the stop line distance calculating unit 31 already has information on these values.

4B shows that the height y of the image formed on the forward image is equal to the height between the stop line and the horizontal line. That is, the y value can be measured by measuring the height between the stop line and the horizontal line in the front image.

Accordingly, the stop line distance calculating unit 31 can measure the height y between the stop line and the horizontal line using the forward image. In this case, since the horizontal line is a horizontal line formed at the center of the forward image, the position is determined in advance, and the stop line can use the stop line position sensed by the stop line recognizing part 15.

The stop line distance calculating unit 31 measures the height y between the stop line and the horizontal line and then calculates the distance Dx from the vehicle to the stop line by substituting the known height hc of the camera and the focal length f of the camera into the equation .

The preceding vehicle distance calculating section 33 can calculate the distance to the preceding vehicle by using a laser sensor, an ultrasonic sensor, or the like. The laser sensor and the ultrasonic sensor are installed in front of the vehicle and emit a laser or an ultrasonic wave toward the preceding vehicle so that the distance from the preceding vehicle can be calculated using the laser or ultrasonic waves reflected from the preceding vehicle.

In the present embodiment, the case where the laser sensor is used is described as an example, but it goes without saying that the preceding vehicle distance calculating unit 33 may use another sensor. The laser sensor has a light emitting unit for emitting the laser beam and a light receiving unit for receiving the laser beam reflected from the preceding vehicle. The time difference between the time when the laser beam is emitted from the light emitting unit and the time difference , The distance traveled by the laser can be calculated. Accordingly, the preceding vehicle distance calculating section 33 can calculate the distance to the preceding vehicle.

The average speed calculating section 35 can calculate the average speed of the submersible traveling for a predetermined time. The average speed calculating section 35 can be calculated using the following equation (2).

Here, the gear ratio is a ratio of the number of revolutions of the input shaft to the output shaft of the transmission. When the gear ratio is 5: 1, it means that the tire rotates once while the engine rotates 5 times. The longitudinal reduction ratio is an additional reduction ratio for calculating a value as close as possible to the actual gear ratio and is used to increase the power and acceleration of the engine as well as to make the variable gear of each gear stage small by increasing the reduction ratio at low speed and high speed . The negative speed ratio refers to the gear ratio of the gear selected by the driver in the first stage gear requiring the driving force and the second gear advantageous for the constant speed driving.

5, the control module 50 includes a travel control determining section 51 for determining whether the vehicle is stopped, a target speed calculating section 53 for calculating a real time target speed for stopping the vehicle, And a speed control unit 55 for controlling the speed of the vehicle according to the target speed.

The operation control determining unit 51 can determine whether the vehicle is stopped based on the information recognized by the camera module 10 and the information calculated by the arithmetic module 30. [ That is, the driving control determining unit 51 can determine whether to stop the vehicle using the color of the traffic lights, the distance to the stop line, the distance to the preceding vehicle, and the average speed of the vehicle.

If the color of the traffic light detected by the traffic signal recognition unit 17 is red, the operation control determination unit 51 determines that the vehicle is unconditionally stopped. At this time, the operation control determination unit 51 can determine that the vehicle should be stopped at a position where the distance from the preceding vehicle calculated by the preceding vehicle distance calculation unit 33 is a reference distance, for example, 5 m.

When the traffic light is red and the vehicle is stopped, the traffic control determination unit 51 changes the color of the traffic light detected by the traffic signal recognition unit 17 to green. If the distance to the preceding vehicle exceeds 5 meters, It is necessary to resume the running of the vehicle.

On the other hand, if the color of the traffic light detected by the traffic signal recognition unit 17 is green, the travel control determining unit 51 determines whether to stop the vehicle according to the distance to the stop line, the distance to the preceding vehicle, .

When the distance between the vehicle and the stop line is between 10 and 20 m, even if the color of the traffic light is green, the operation control determining unit 51 determines that the vehicle is driving the vehicle It can be determined that it should be stopped.

When the traffic light determination unit 51 determines that the vehicle should stop the vehicle even though the traffic light is green, the distance between the average speed of the vehicle and the preceding vehicle is continuously received and the distance to the preceding vehicle exceeds 5 m, If it is green, it can be judged that the vehicle should be driven again.

The target speed calculating section 53 can calculate the real time target speed so that the vehicle can travel at a distance of 5 m from the preceding vehicle. If it is determined that the vehicle should stop, the target speed calculating unit 53 calculates the target speed of the vehicle so that the vehicle can stop at a position spaced apart from the preceding vehicle by a predetermined distance, for example, It is possible to calculate the real time target speed for decelerating the speed of the vehicle at the speed of 0 km / h.

The target speed calculating unit 53 divides the distance remaining until the stopping point by the distance from the preceding vehicle to the current speed of the vehicle sensed by the speed sensor and obtains the braking time until the stopping point. Then, using the preset negative acceleration C, a real time target speed for making the vehicle speed zero during the braking time can be calculated. For this purpose, the target speed calculating unit 53 may calculate the real time target speed using the following equation (3).

Here, Vx (t) is the real-time target speed, Vx (t-1) is the vehicle speed with the lapse of time, C is the negative acceleration, and Cx? T is the deceleration.

The speed control unit 55 is an electronic control unit for engine control that controls the engine. The electronic control unit includes an engine control unit (Engine Control Unit) for controlling the fuel injection in accordance with the engine condition and the running condition, And a control unit (Transmission Control Unit).

The speed control section 55 can control the speed of the vehicle by controlling the engine in accordance with the real time target speed calculated by the target speed calculating section 53. [ The speed control unit 55 controls the engine in accordance with the real time target speed calculated by the target speed calculating unit 53 when it is determined that the vehicle is not required to be stopped by the driving control determining unit 51 during running of the vehicle. On the other hand, if it is determined that the vehicle should stop, the operation control determining unit 51 controls the engine to stop the vehicle according to the real-time target speed so that the vehicle stops at a position separated from the preceding vehicle by 5 m.

Referring to FIG. 6, a process of controlling the driving of the vehicle in accordance with the recognition of the traffic lights in the automatic cruise control device 1 capable of recognizing the traffic lights by such a configuration will now be described.

When the vehicle is started and the automatic cruise control device 1 is operated, the front camera 11 photographs the front of the vehicle to generate a forward image (S600), and the forward image is provided to the ROI setting unit 13, The ROI and the ROI RO are set (S605).

In step S610, the stop line recognition unit 15 detects the stop line by detecting the saturation, the luminance, or the edge of the stop line interest area in the forward image, and the traffic signal recognition unit 17 detects the color of the traffic light in the traffic light interest area.

When the stop line is recognized and the traffic light recognizing unit 17 determines that the color of the traffic light is red (S615), the operation control determining unit 51 determines that the vehicle should be stopped at a distance of 5 m from the preceding vehicle.

Then, the target speed calculating unit 53 calculates the real time target speed using the distance from the preceding vehicle calculated in the preceding vehicle distance calculating unit 33, the current speed of the vehicle sensed by the speed sensor, and the negative acceleration ( S620). Then, the speed control unit 55 controls the engine according to the real-time target speed so that the vehicle stops at a distance of 5m from the preceding vehicle (S625).

The color of the traffic light recognized by the traffic signal recognition unit 17 is changed to green (S630), and when the distance to the preceding vehicle exceeds 5 meters (S635), the driving control determining unit 51 resumes the driving of the vehicle . The target speed calculating unit 53 calculates the real time target speed of the vehicle so as to maintain the distance from the preceding vehicle to 5 m and provides it to the speed control unit 55 (S640). The speed control unit 55 controls the engine in accordance with the real-time target speed so as to cause the vehicle to run again (S645).

When the stop line is recognized and the traffic light recognizing unit 17 determines that the color of the traffic light is green (S615), the travel control determining unit 51 calculates the stop line distance calculating unit 31, the preceding vehicle distance calculating unit 33, , And determines whether or not the vehicle is to be stopped based on the result calculated by the average speed calculating unit 35.

When the distance between the vehicle and the stop line is between 10 and 20 m, the operation control determining unit 51 determines that the vehicle should be stopped if the average speed of the vehicle is 35 km / h or less and the distance from the preceding vehicle is 5 m or less (S650).

When the vehicle stop control unit 51 determines that the vehicle is stopped, the target speed calculating unit 53 calculates the real time target speed (S655). The speed control unit 55 controls the engine according to the real- And stops at a distance of 5 m from the preceding vehicle (S660).

When the distance to the preceding vehicle exceeds 5 m, the driving control determining unit 51 determines to stop the operation when the color of the traffic light is green, in a state where the traffic light is green and the vehicle is stopped. (S665).

Then, the target speed calculating unit 53 calculates the real time target speed (S640), and the speed control unit 55 controls the engine in accordance with the real time target speed so as to drive the vehicle (S645).

As described above, the automatic cruise control device 1 capable of recognizing the signal lamp according to the present invention recognizes the stop line and the traffic light during the operation of the automatic cruise function so that when the traffic light is red, the vehicle is stopped at a certain distance from the preceding vehicle, When the signal lamp is green, the vehicle is stopped when the average speed of the vehicle is less than a certain level and the distance from the preceding vehicle is less than a certain level, that is, when the flow of the vehicle is slow. Accordingly, it is possible to stop the vehicle by recognizing the traffic light during the automatic cruise control, so that it is possible not only to prevent an accident at the crosswalk or at the intersection, but also to prevent the accident from occurring at the intersection, It can prevent accidents and traffic congestion.

The standard content or standard documents referred to in the above-mentioned embodiments constitute a part of this specification, for the sake of simplicity of description of the specification. Therefore, it is to be understood that the content of the above standard content and portions of the standard documents are added to or contained in the scope of the present invention.

The foregoing description is merely illustrative of the technical idea of the present invention, and various changes and modifications may be made by those skilled in the art without departing from the essential characteristics of the present invention. Therefore, the embodiments disclosed in the present invention are intended to illustrate rather than limit the scope of the present invention, and the scope of the technical idea of the present invention is not limited by these embodiments. The scope of protection of the present invention should be construed according to the following claims, and all technical ideas within the scope of equivalents should be construed as falling within the scope of the present invention.

10: camera module 11: front camera
13: ROI setting unit 15: Stop line recognition unit
17: Traffic signal recognition unit 30: Arithmetic module
31: stop line distance calculating unit 33: preceding vehicle distance calculating unit
35: average speed calculating unit 50: control module
51: Operation Control Decision Unit 53: Target Speed Calculator
55: Speed control section

Claims (8)

A front camera for photographing the front of the vehicle;
A stop line recognizing unit for detecting a stop line in the image photographed by the front camera;
A traffic signal recognition unit for sensing a color of a traffic light in an image photographed by the front camera; And
A control module for determining whether the vehicle should stop at the stop line in accordance with a hue of the traffic light and a preset condition when the stop line is detected and controlling the vehicle to stop or travel while maintaining a certain distance from the preceding vehicle; And an automatic cruise control device having a signal light detection function. The method according to claim 1,
The control module includes:
Whether or not the vehicle should stop at the stop line in accordance with the color of the traffic light, and whether or not the vehicle should stop at the stop line in accordance with whether or not the traffic light is green, An operation control unit
A target speed computing unit for computing a real time target speed for causing the vehicle to stop or travel while maintaining a predetermined distance from the preceding vehicle;
And a speed controller for controlling the speed of the vehicle according to the real time target speed. 3. The method of claim 2,
The vehicle speed control unit determines that the vehicle should stop if the traffic light is red, and the target speed calculation unit calculates a real time target speed so that the vehicle can stop in a state in which the vehicle maintains a certain distance from the preceding vehicle Wherein the automatic cruise control device has a signal light detection function. The method of claim 3,
The traffic control determining unit determines that the vehicle is stopped when the traffic light is red, the traffic light is changed to green, and when the distance from the preceding vehicle exceeds a certain distance, Wherein the automatic cruise control device has a signal light detection function. 3. The method of claim 2,
When the traffic light is green and the average speed of the preceding vehicle is equal to or less than a preset reference speed and the distance between the vehicle and the preceding vehicle is less than a preset reference distance, Wherein the automatic cruise control device has a signal light detection function. 6. The method of claim 5,
Wherein the driving control determining unit determines that the vehicle should be resumed when the distance from the preceding vehicle exceeds the reference distance in a state where the traffic light is maintained in green. Cruise control device. The method according to claim 1,
A preceding vehicle distance calculating unit for calculating a distance between the vehicle and the preceding vehicle;
A stop line distance calculating unit for calculating a distance from the vehicle to the stop line;
Further comprising an arithmetic module including an average speed calculating unit for calculating an average speed of the vehicle. Capturing an image of the front of the vehicle to form a forward image;
Detecting a stop line in the forward image;
Sensing a color of a traffic light in the forward image;
Whether or not the vehicle should stop at the stop line in accordance with the color of the traffic light, and whether or not the vehicle should stop at the stop line in accordance with whether or not the traffic light is green, ; And
And controlling the vehicle to stop at a predetermined distance from the preceding vehicle when the vehicle is determined to stop at the stop line.

Images