RU2779773C1 - Traffic light recognition method and traffic light recognition device - Google Patents

Abstract

FIELD: traffic light recognition.

SUBSTANCE: invention relates to a traffic light recognition method and a traffic light recognition device. A traffic light recognition method comprising the steps of capturing an image of a vehicle's direction of travel using an imaging unit, acquiring the location of a stop line corresponding to a traffic light located in the vehicle's direction of travel, estimating whether the vehicle can decelerate with a given deceleration acceleration, and whether it can stop before the stop line, based on the location of the stop line. The method comprises selecting a traffic light corresponding to a stop line as a target traffic light in a case where it is judged that the vehicle cannot stop before the stop line, setting a detection area corresponding to the target traffic light in an image, and determining the display state of the target traffic light by performing image processing in the detection area. The traffic light recognition device comprises an imaging unit mounted on a vehicle and a controller.

EFFECT: increased accuracy of image recognition.

9 cl, 4 dwg

Description

FIELD OF TECHNOLOGY

[0001] The present invention relates to a traffic light recognition method and a traffic light recognition device.

BACKGROUND OF THE INVENTION

[0002] According to Patent Document 1, a method is provided in which a stop sign, a red light, or a yellow light from traffic light display states are detected from a camera image at each intersection, and in which an alarm is generated when an own vehicle is approaching to the stop line at some vehicle speed or higher.

[0003] In addition, according to Patent Document 2, a method is proposed in which, for two or more traffic lights located in front of a vehicle, in a case where the first traffic light closest to the vehicle indicates a state in which the vehicle cannot drive , and the second traffic light before the first traffic light indicates the state in which the vehicle can travel, an alarm is issued when the own vehicle's speed exceeds a certain upper limit.

LIST OF CITATIONS

PATENT DOCUMENT

[0004] Patent Document 1: Japanese Patent Publication Publication No. 2011-145892

Patent Document 2: Published Japanese Patent Publication No. 2018-092371

SUMMARY OF THE INVENTION

TECHNICAL PROBLEM

[0005] However, since the method described in Patent Document 1 is configured to sequentially detect the display state of a traffic light at each intersection, this method has a problem that generation of an alarm is delayed when intervals between intersections are relatively close. In addition, in the case where the display of a plurality of traffic lights located in front of the vehicle is simultaneously determined by using the technique described in Patent Document 2, since the second traffic light is always monitored in addition to the first traffic light, there is a problem that the computational load becomes large.

[0006] The present invention has been made in view of the above problems, and an object of the present invention is to provide a traffic light recognition method and a traffic light recognition apparatus capable of recognizing traffic lights to be detected while suppressing an increase in computational load even when the distance between intersections is relatively small.

THE SOLUTION OF THE PROBLEM

[0007] In order to solve the above problems, a traffic light recognition method and a traffic light recognition apparatus according to an aspect of the present invention judge whether a vehicle can decelerate at a given deceleration acceleration and whether it can stop before a stop line, based on the location of the stop line corresponding to the traffic light. located in the direction of travel of the vehicle, select a traffic light corresponding to the stop line as the target traffic light in the case where it is judged that the vehicle cannot stop before the stop line, set the detection area corresponding to the target traffic light in the image obtained by capturing the direction of travel of the vehicle, and determining the display state of the target traffic light by performing image processing in the detection area.

ADVANTAGES OF THE INVENTION

[0008] According to the present invention, it is possible to recognize traffic lights to be detected while suppressing an increase in computational load even when the distance between intersections is relatively short.

BRIEF DESCRIPTION OF THE DRAWINGS

[0009]

Fig. 1 is a block diagram illustrating the configuration of a traffic light recognition apparatus according to an embodiment of the present invention.

Fig. 2 is a flowchart illustrating a processing procedure of a traffic light recognition apparatus according to an embodiment of the present invention.

Fig. 3 is a schematic view showing an example of positional relationship between a vehicle and a plurality of traffic lights.

Fig. 4 is a schematic view showing an example of setting a detection area in a captured image.

DESCRIPTION OF EMBODIMENTS

[0010] Next, embodiments of the present invention will be described in detail with reference to the drawings. In the description, the same elements are identified by the same reference numerals, and the duplicate description will be omitted.

[0011] [Traffic light recognition device configuration]

Fig. 1 is a block diagram illustrating the configuration of a traffic light recognition device according to the present embodiment. As shown in FIG. 1, the traffic light recognition apparatus according to the present embodiment includes an imaging unit 71 and a controller 100, and the controller 100 is connected to the imaging unit 71, the vehicle sensor 73, the map information acquisition unit 75, and the vehicle control unit 400 via wired or wireless communication channel.

[0012] Here, the imaging unit 71, the in-vehicle sensor 73, and the vehicle control device 400 are installed on the vehicle (not shown), but the mapping information acquisition unit 75 and the controller 100 may be installed on the vehicle or may be installed outside the vehicle. funds.

[0013] The imaging unit 71 captures an image of the vehicle in the direction of travel. For example, the imaging unit 71 is a digital camera equipped with a solid state image sensor such as a CCD or CMOS and forms an image of the vehicle environment to capture a digital image of a peripheral area of the vehicle. The imaging unit 71 forms images in a predetermined range around the vehicle by setting the focal length, lens viewing angle, vertical and horizontal camera angles, and the like.

[0014] The image captured by the imaging unit 71 is output to the controller 100 and stored in a memory unit (not shown) for a predetermined period of time. For example, the imaging unit 71 captures images at predetermined time intervals, and images captured at predetermined time intervals are stored in the memory unit as past images. The past image can be deleted after a specified period has elapsed since the last image was captured.

[0015] The sensor 73 in the vehicle includes an object detection sensor mounted on the vehicle, such as a laser radar, a millimeter wave radar, and a camera that detects an object existing around the vehicle. The sensor 73 in the vehicle may include many different types of object detection sensors.

[0016] The sensor 73 in the vehicle detects the environment around the vehicle. For example, a sensor 73 in a vehicle can detect a moving object, including another vehicle, a motorcycle, a bicycle, a pedestrian, and a stationary object, including a stopped vehicle, as well as location, position, size, speed, acceleration, deceleration, yaw rate, etc. d. a moving object and a stationary object in relation to the vehicle. The sensor 73 in the vehicle may output, for example, the behavior of a two-dimensional object in a zenithal view (also referred to as a top view) viewed from the air above the vehicle as a detection result. In addition, the sensor 73 in the vehicle can detect a sign (a road sign or a sign displayed on a road surface), a guide rail, and the like existing around the vehicle. In addition, the sensor 73 in the vehicle can detect the slipperiness of the road surface in the lane on which the vehicle is traveling by detecting the rotation speed and the difference in the rotation speed of the wheels provided in the vehicle.

[0017] In addition, the sensor 73 in the vehicle detects the state of the vehicle in addition to the environment around the vehicle. For example, the sensor 73 in the vehicle can detect the vehicle's running speed (forward-backward running speed, left-right direction, turning speed), the steering angle of the wheels provided in the vehicle, and the change in steering angle speed.

[0018] In addition, the sensor 73 in the vehicle can measure the absolute position of the vehicle, that is, the position, attitude and speed of the vehicle relative to a given reference point, using a position sensor that measures the absolute position of the vehicle, such as GPS (global positioning system) and odometer.

[0019] The map information acquisition unit 75 obtains map information indicating the structure of the road on which the vehicle is traveling. The map information obtained by the map information acquisition unit 75 includes road structure information such as absolute lane positions, lane connection relationships, and relative positions. In addition, the map information obtained by the map information acquisition unit 75 may include object information such as a parking lot and a filling station. In addition, the mapping information may include information about the location of a traffic light, the type of traffic light, the location of a stop line corresponding to a traffic light, and the like. The map information acquisition unit 75 may have a map database in which map information is stored, or may acquire map information from an external map data server using cloud computing. In addition, the map information acquisition unit 75 can acquire map information using vehicle-vehicle communication and road-vehicle communication.

[0020] The vehicle control device 400 controls a vehicle (not shown) based on a traffic light recognition result obtained by the controller 100. For example, the vehicle control device 400 may control the vehicle by automatically driving according to a predetermined driving route, or may support a driving operation people in the vehicle. In addition, the vehicle control device 400 may be a notification device that notifies vehicle occupants of a traffic light detection result.

[0021] The controller 100 (an example of a control unit or a processing unit) is a general purpose microcomputer including a CPU (central processing unit), a memory, and an I/O unit. A computer program (traffic light recognition program) for operating as a traffic light recognition device is installed in the controller 100. By executing the computer program, the controller 100 functions as a plurality of information processing circuits (110, 120, 130, 140, 150, 160) included in the traffic light recognition device .

[0022] An example is shown here in which a plurality of information processing circuits (110, 120, 130, 140, 150, 160) included in the traffic light recognition apparatus are implemented by software. However, it is also possible to configure the information processing circuits (110, 120, 130, 140, 150, 160) by preparing specific hardware to perform each of the following information processing. In addition, a plurality of information processing circuits (110, 120, 130, 140, 150, 160) may be configured by separate hardware. In addition, the information processing circuit (110, 120, 130, 140, 150, 160) can also be used as an electronic control unit (ECU) used for other control related to the vehicle.

[0023] The controller 100, as a plurality of information processing circuits (110, 120, 130, 140, 150, 160), includes an own location acquisition unit 110, a stop location acquisition unit 120, a calculation unit 130, a detection area setting unit 140 , block 150 determine and block 160 output.

[0024] Own location acquisition unit 110 obtains the absolute position of the vehicle, that is, the current location of the vehicle relative to a given reference point, by means of a sensor 73 in the vehicle. In addition, the block 110 obtain own location can obtain information about the current speed, acceleration and position of the vehicle using the sensor 73 in the vehicle.

[0025] The stop location acquisition unit 120 searches for map information obtained by the map information acquisition unit 75, and retrieves a traffic light located in the direction of travel of the vehicle. Next, the stop location obtaining unit 120 obtains the location of the stop line corresponding to the retrieved traffic light from the map information. The stop location acquisition unit 120 may search for map information obtained by the map information acquisition unit 75 based on the current location and position of the vehicle obtained by the home location acquisition unit 110, and may retrieve a traffic light located in the direction of travel of the vehicle. In addition, the stop location acquisition unit 120 can extract a traffic light within a range that can be imaged by the imaging unit 71 . In addition, the stop location obtaining unit 120 can extract both a traffic light and a stop line corresponding to the extracted traffic light from the image captured by the imaging unit 71, and can obtain the location of the stop line corresponding to the extracted traffic light by obtaining the location of the traffic light and stop -line in relation to the vehicle. That is, the location of the stop line corresponding to the traffic light can be obtained without using the location of the own vehicle or map information, and the method for obtaining the location of the stop line corresponding to the traffic light can be changed accordingly.

[0026] In addition, the calculation unit 130 judges whether the vehicle can decelerate with a given deceleration acceleration and whether it can stop before the stop line based on the location of the stop line obtained by the stop location obtaining unit 120 . In particular, the calculation unit 130 calculates the distance D between the current location of the vehicle and the location of the stop line for each extracted traffic light. Then, the calculation unit 130 calculates the acceleration deceleration amount α when the vehicle stops at the stop line by dividing the square of the vehicle's current speed V by twice the distance D.

[0027] That is, the amount of acceleration α deceleration can be estimated from the following equation (1).

α=V^2/2D ... (1)

[0028] Next, the calculation unit 130 judges that the vehicle cannot stop before the stop line by decelerating at the predetermined deceleration acceleration β in the case where the calculated deceleration acceleration amount α is larger than the predetermined deceleration acceleration amount β. The predetermined deceleration acceleration β is a value obtained through experiments or the like. in advance so that the occupants of the vehicle do not experience discomfort due to sudden deceleration.

[0029] Instead of calculating the deceleration acceleration α and judging whether the vehicle can stop, the calculator 130 may calculate the estimated stop location when the vehicle decelerates at the predetermined deceleration acceleration β, and may judge that the vehicle cannot stop before the stop. - lines, if the stop line is located between the intended location of the stop and the vehicle. Specifically, the calculation unit 130 may calculate a predetermined distance DH that the vehicle travels until the vehicle stops while decelerating at a predetermined deceleration acceleration β by dividing the square of the current vehicle speed V by twice the predetermined deceleration acceleration β.

[0030] That is, the predetermined distance DH can be estimated from the following equation (2).

DH=V^2/2β ... (2)

[0031] Then, the calculation unit 130 may set a location ahead of the predetermined distance DH from the current location of the vehicle in the direction of travel of the vehicle as a suggested stop location, and may judge that the vehicle cannot be stopped in front of the stop line in the case when the stop line is located between the intended stop location and the vehicle. That is, in the case where the intended stop location is further than the stop line along the direction of travel of the vehicle as viewed from the vehicle, the calculation unit 130 may judge that the vehicle cannot stop before the stop line.

[0032] The calculation unit 130 selects, from the traffic lights retrieved by the stop location obtaining unit 120, a traffic light corresponding to the stop line at which it is judged that the vehicle cannot stop in front of the vehicle as a traffic light (target traffic light) to be defined.

[0033] The selection of the target traffic light will be described with reference to FIG. 3. FIG. 3 is a schematic view showing an example of positional relationship between a vehicle and a plurality of traffic lights. In FIG. 3 shows that a vehicle traveling at a speed V is at a location PS (coordinates x=0), in the direction of travel of the vehicle, there are traffic lights TS1, traffic lights TS2, and traffic lights TS3, and stop lines corresponding to each of the above traffic lights TS1, TS2, TS3 are at location P1 (coordinates x=D1), location P2 (coordinates x=D2), and location P3 (coordinates x=D3).

[0034] Here, it is assumed that the expected stop location is set at the location PT based on the predetermined distance DH calculated from Equation (2) above. In this case, even if the vehicle decelerates with the predetermined deceleration acceleration β, this means that the vehicle cannot stop before the stop line at the location P1 and the stop line at the location P2. This is because the stop line at location P1 and the stop line at location P2 exist between the intended stop location at location PT and the vehicle at location PS.

[0035] Therefore, the calculation unit 130 selects the traffic light TS1 corresponding to the stop line at location P1 and the traffic light TS2 corresponding to the stop line at location P2 as target traffic lights. On the other hand, the traffic light TS3 corresponding to the stop line at location P3 is not selected as the target traffic light.

[0036] According to equation (1), since the deceleration acceleration α and the distance D are inversely proportional, it can be seen that the amount of deceleration acceleration required to stop at the stop line at location P1 or the stop line at location P2 is larger than the value of the given acceleration β deceleration. This is because the distance D1 and the distance D2 are shorter than the specified distance DH. Thus, the target traffic light selected in the case where the calculation unit 130 calculates the deceleration acceleration α and judges whether the vehicle can be stopped corresponds to the target traffic light selected in the case where the calculation unit 130 judges whether the vehicle can be stopped based on the relative position between the stop line and the intended location of the stop.

[0037] The detection area setting unit 140 sets the detection area corresponding to the target traffic light in the image captured by the imaging unit 71 . Here, "detection area" means an area in which the presence of the target traffic light in the image is judged, and is the target range of the image processing by the determiner 150 described later. The location of the traffic light reflected in the imaging range of the image can be estimated based on the imaging direction of the imaging unit 71, the position and orientation of the vehicle at the time of imaging, and the location of the traffic light. The detection area setting unit 140 sets, for example, a portion of the captured image that includes the estimated location of the traffic light in the image as the detection area.

[0038] Setting the detection area will be described with reference to FIG. 4. FIG. 4 is a schematic view showing an example of setting a detection area in a captured image. In FIG. 4 shows how the detection area R1, the detection area R2, and the detection area R3 are set in accordance with the traffic light TS1, the traffic light TS2, and the traffic light TS3, respectively.

[0039] When traffic light TS1 and traffic light TS2 are selected as target traffic lights and traffic light TS3 is not selected as target traffic light, as shown in FIG. 3, the detection area R3 corresponding to the traffic light TS3 is not set.

[0040] Since the image for the traffic light corresponding to the stop line does not specify a detection area in which the vehicle can decelerate at a given deceleration acceleration and stop in front of it, it is possible to reduce the computational load in determining the target traffic light in the detection area and determining the display state of the target traffic light.

[0041] In addition, since the detection area is undoubtedly set in the image for a traffic light corresponding to a stop line where the vehicle cannot stop ahead even if the vehicle decelerates at a predetermined deceleration acceleration, it is possible to reliably determine the display state with a traffic light that is necessary detect as a detection target even if the distance between intersections is relatively close. Therefore, it is possible to recognize a traffic light to be detected while suppressing an increase in the computational load.

[0042] The determining unit 150 performs image processing in the detection area, detects the target traffic light in the detection area, and determines the display state of the target traffic light. The determiner 150 detects a traffic light, for example, by pattern matching. Pattern matching uses a standard traffic light image as a pattern, and scans the detection area while shifting the image one pixel at a time, and calculates the brightness distribution correlation, for example. Then, when the correlation reaches its highest value, it is found that the traffic light is located at the location in the image where the pattern is located.

[0043] The "color signal" indicated by the traffic light includes "green signal", "yellow signal", and "red signal". The meaning of the "color signal" is determined by the rules of the road that the vehicle must follow. For example, "green signal" means "you can pass" and "red signal" means "stop at the stop location". "Amber signal" means "stop at the stopping location, except when it is not possible to stop safely due to proximity to the stopping location."

[0044] Such a distinction between "green signal", "yellow signal" and "red signal" can be performed in such a way that it is judged that the "color signal" having the highest brightness level among the three "color signals" is lit.

[0045] In addition, the traffic light may indicate not only a "color signal" but also an "arrow signal" indicating the direction allowed for a vehicle to travel at an intersection where a traffic light is installed. For example, "arrow signal" is a "right turn signal", "straight ahead signal" and/or "left turn signal".

[0046] "Arrow signal" is not limited to "right turn signal", "direct traffic signal" and "left turn signal", and various options can be considered depending on the structure of the intersection where the traffic light is installed. The meaning of "arrow signal" is determined by the rules of the road that the vehicle must follow.

[0047] The determination unit 150 performs image processing in the detection area, and determines the lighting state of the "color signal" and "arrow signal" of the traffic light as the display state of the traffic light.

[0048] The processing of the traffic light detection image by the determiner 150 may use machine learning such as a support vector machine or a neural network. When a traffic light is detected, the recognition speed can be improved by preparing a training database that stores patterns of traffic lights of different sizes in advance, and using the training database to refer to it depending on the distance to the traffic light.

[0049] The output unit 160 outputs the target traffic light display state determined by the determine unit 150 . For example, the display state of the target traffic light is output from the output unit 160 to the vehicle control device 400 and is used to control the vehicle. In addition, the display state of the target traffic light can be output from the output unit 160 to a notification device (not shown) and communicated to a vehicle occupant via the notification device.

[0050] [Traffic Light Recognition Device Processing]

Next, with reference to the flowchart of FIG. 2, a traffic light recognition processing procedure of the traffic light recognition apparatus according to the present embodiment will be described. The traffic light recognition process shown in FIG. 2 may be executed each time the imaging unit 71 acquires an image, or may be executed every cycle when image processing is performed in the detection area after the imaging unit 71 acquires an image.

[0051] In step S101, the own location acquisition unit 110 obtains the absolute position of the vehicle using the sensor 73 in the vehicle.

[0052] In step S103, the stop location acquisition unit 120 searches for the map information obtained by the map information acquisition unit 75 and retrieves a traffic light located in the direction of travel of the vehicle.

[0053] In step S105, the stop location obtaining unit 120 obtains the location of the stop line corresponding to the retrieved traffic light from the map information.

[0054] In step S107, the calculation unit 130 calculates a selection condition for selecting a target traffic light from the retrieved traffic lights. Specifically, in the case of judging whether the vehicle can decelerate at a predetermined deceleration acceleration and stop before the stop line, the calculation unit 130 calculates the deceleration acceleration amount α in the case where the vehicle stops at the stop line. In addition, in the case of judging whether the vehicle can be stopped based on the positional relationship between the stop line and the proposed stop location, the calculation unit 130 calculates a predetermined distance DH at which the vehicle decelerates with a predetermined deceleration acceleration β and moves to stop the vehicle. funds.

[0055] In step S109, the calculation unit 130 selects the target traffic light from the extracted traffic lights using the calculated selection conditions. The following will be described assuming that N target traffic lights are selected.

[0056] For example, if the calculated deceleration acceleration amount α is larger than the deceleration predetermined acceleration amount β, the calculation unit 130 judges that the vehicle cannot stop before the stop line when the vehicle decelerates at the predetermined deceleration acceleration β, and selects a traffic light, corresponding stop line, as the target traffic light. In addition, the calculation unit 130 may select a traffic light corresponding to the stop line as the target traffic light, while the distance D between the vehicle and the stop line is less than the calculated predetermined distance DH.

[0057] In step S111, the detection area setting unit 140 sets the detection area corresponding to the target traffic light in the image captured by the imaging unit 71 .

[0058] In step S121, the determiner 150 sets the variable i to 1.

[0059] In step S123, the determining unit 150 determines the display state of the target traffic light (i-th target traffic light) closest to the vehicle of the N target traffic lights. Specifically, image processing is performed in a detection area corresponding to the i-th target traffic light closest to the vehicle, and the determination unit 150 determines the display state of the target traffic light.

[0060] In step S125, the determining unit 150 checks the variable i and determines whether "i=N" is satisfied, and if "i=N" is satisfied (YES in step S125), proceeds to step S131.

[0061] If "i=N" is not satisfied in step S125 (NO in step S125), in step S127, the determining unit 150 determines whether it is possible to move straight on the stop line (may go straight) corresponding to the i-th target traffic light , based on the display state of the i-th target traffic light.

[0062] The determining unit 150 may determine whether the vehicle can move straight ahead based on the vehicle's driving plan, and not based on the display state of the target traffic light. Specifically, based on the vehicle's driving plan, if the vehicle does not drive straight but turns left or right at the intersection located in front of the i-th target traffic light, the vehicle will not pass through the stop line corresponding to the i-th target traffic light. In this case, the determining unit 150 may determine that the vehicle cannot move straight at the i-th target traffic light.

[0063] If it is determined that "the vehicle can move straight" in step S127 (YES in step S127), the process proceeds to step S129, and the determination unit 150 adds 1 to the variable i.

[0064] If it is determined that "the vehicle cannot move straight" in step S127 (NO in step S127), the process proceeds to step S131.

[0065] In step S131, the output unit 160 outputs the display state of the target traffic light determined by the determine unit 150. The target traffic light display output state is used in the vehicle control device 400, for example.

[0066] In the flowchart of FIG. 2, it is explained that the determining unit 150 determines whether the "vehicle can go straight" at the i-th target traffic light based on the display state of the i-th target traffic light after the detection area setting unit 140 sets the detection area corresponding to the target traffic light to image. It will be explained that the determining unit 150 does not determine whether the "vehicle can not go straight" in the display state for the (i+1)th and subsequent target traffic lights in the case where it is determined that "the vehicle cannot go straight" on i- m target traffic light.

[0067] However, this embodiment is not limited to the example shown in the flowchart of FIG. 2. For example, after setting the detection area from the first to the i-th target traffic light out of N target traffic lights, it can be determined whether the "vehicle can move straight" based on the display state of the i-th target traffic light, and in the case where it is determined that "vehicle cannot move straight" at the i-th target traffic light, the detection area corresponding to the (i+1)-th and subsequent target traffic lights may not be set. Since the number of detection areas set in the image can be reduced, the computational burden on the detection area setting unit 140 and the determination unit 150 can be reduced.

[0068] In addition, the detection area setting unit 140 may set the detection area corresponding to the target traffic light in the image when the vehicle is accelerating or moving at a constant speed, and may not set the detection area corresponding to the target traffic light in the image when the vehicle slows down. When the vehicle slows down, the vehicle is assumed to stop at the stop line closest to the vehicle, and it is considered that there is no need to consider the traffic light corresponding to the stop line farthest from the vehicle. Therefore, when the vehicle slows down, the computational load can be reduced by not setting the detection area corresponding to the target traffic light.

[0069] [Effect of Embodiments]

According to the present invention, a traffic light recognition method and a traffic light recognition apparatus according to the present embodiment capture an image of a vehicle's driving direction with an imaging unit installed on the vehicle, obtain the position of a stop line corresponding to a traffic light located in the vehicle's travel direction, evaluate , whether the vehicle can decelerate with a given deceleration acceleration, and can stop in front of the stop bar, based on the position of the stop bar, select a traffic light corresponding to the stop bar as the target traffic light in the case where it is judged that the vehicle cannot stop in front of the stop bar. a stop line, setting a detection area corresponding to the target traffic light in the image, and determining a display state of the target traffic light by performing image processing in the detection area.

[0070] As a result, it is possible to determine the display state of the target traffic light corresponding to the stop line at which the vehicle cannot be stopped even if the vehicle decelerates at a predetermined deceleration acceleration, and it is possible to safely decelerate and/or stop the vehicle by outputting the display state .

[0071] In particular, the detection area is certainly set on the image for a traffic light corresponding to a stop line before which the vehicle cannot stop even if the vehicle decelerates with a predetermined deceleration acceleration, and thus it is possible to reliably determine the display state with the traffic light, to be detected as a detection target even if the distance between intersections is relatively close. As a result, the safety of the vehicle can be improved.

[0072] On the other hand, since the detection area is not set in the image for the traffic light corresponding to the stop line before which the vehicle can stop by decelerating at a given deceleration acceleration, it is possible to reduce the computational load in detecting the target traffic light in the detection area and determining the display state target traffic light.

[0073] Therefore, it is possible to recognize a traffic light to be detected while suppressing an increase in the computational load.

[0074] In addition, the traffic light recognition method and the traffic light recognition apparatus according to the present embodiment can calculate the deceleration acceleration for the vehicle to stop at the stop line, and can judge that the vehicle cannot stop before the stop line in when the amount of acceleration deceleration is greater than the amount of the specified acceleration deceleration. As a result, the target traffic light can be reliably selected with reference to the predetermined deceleration acceleration.

[0075] In addition, the traffic light recognition method and the traffic light recognition apparatus according to the present embodiment can calculate an estimated stop location when the vehicle decelerates at a predetermined deceleration acceleration, and can judge that the vehicle cannot stop in front of the stop line if the stop the line is between the proposed stop location and the vehicle. As a result, the target traffic light can be reliably selected with reference to the intended stop location determined based on the predetermined deceleration acceleration.

[0076] In addition, the traffic light recognition method and the traffic light recognition apparatus according to the present embodiment can obtain the location of the stop line corresponding to the traffic light based on the vehicle location and the map information, where the map information includes traffic light location information. As a result, the target traffic light can be reliably selected based on the position of the traffic light included in the map information.

[0077] In addition, the traffic light recognition method and the traffic light recognition apparatus according to the present embodiment can set the detection area corresponding to the target traffic light in the image if all the traffic lights between the stop line corresponding to the target traffic light and the vehicle indicate that the vehicle can move straight along the planned route of the vehicle. As a result, the number of detection areas set in an image can be reduced, so that the computational load can be reduced.

[0078] In addition, the traffic light recognition method and the traffic light recognition apparatus according to the present embodiment can acquire the acceleration of the vehicle and can set the detection area corresponding to the target traffic light in the image when the vehicle accelerates or moves at a constant speed. As a result, in a scene where it is necessary to determine not only the display state of a traffic light near the vehicle, but also the display state of a traffic light far away from the vehicle, it is possible to select a traffic light to be detected as the target traffic light, and it is possible to determine the display state of the traffic light, which needs to be discovered. As a result, the safety of the vehicle can be improved. On the other hand, when the vehicle slows down, the computational load can be reduced by not setting the detection area corresponding to the target traffic light.

[0079] In addition, the traffic light recognition method and the traffic light recognition apparatus according to the present embodiment can obtain a vehicle location, a vehicle speed, and a stop line location for each image processing execution cycle, and judge whether the vehicle can stop before the stop line. The computational burden associated with image processing is generally larger than the computational burden for obtaining the vehicle's location, vehicle speed, and stop line location, and the computational burden for estimating the stop of the vehicle in front of the stop bar. Thus, it is possible to prevent delaying other processing while waiting for image processing by performing other processing in each cycle in which image processing is performed.

[0080] In addition, the traffic light recognition method and the traffic light recognition apparatus according to the present embodiment can judge whether the vehicle can stop in front of the stop line by decelerating the vehicle with a predetermined deceleration acceleration in the case that all traffic lights between the stop line and the vehicle indicate that the vehicle can move directly along the planned route of the vehicle. As a result, the number of vehicle stop evaluations before the stop line can be reduced and the computational load can be reduced.

[0081] The respective functions described in the above embodiment may be implemented by one or multiple processing circuits. Processing circuits include programmed processing devices such as a processing device including an electrical circuit, and include devices such as an application specific integrated circuit (ASIC) and conventional circuit elements that are adapted to perform the functions described in the embodiment.

[0082] Although the contents of the present invention have been described above with reference to the embodiment, the present invention is not limited to these descriptions, and it will be apparent to those skilled in the art that various modifications and improvements can be made. It should not be construed that the present invention is limited to the descriptions and drawings, which form part of the present disclosure. Based on the present disclosure, various alternative embodiments, practical examples, and methods of operation will be apparent to those skilled in the art.

[0083] Needless to say, the present invention also includes various embodiments that are not described here. Therefore, the technical scope of the present invention should only be determined by the invention defining the subject matter of the patent according to the scope of the claims, as appropriately derived from the above descriptions.

LIST OF REFERENCES

[0084]

71 imaging units

73 sensor in the vehicle

75 block for obtaining cartographic information

100 controller

110 own location acquisition block

120 stop location acquisition block

130 calculation block

140 detection area setting unit

150 definition block

160 output block

400 vehicle control device.

Claims (33)

1. A method for recognizing a traffic light, comprising the steps of: capturing an image of the direction of travel of the vehicle using an imaging unit mounted on the vehicle, get the location of the stop line corresponding to the traffic light located in the direction of the vehicle, judging whether the vehicle can decelerate with a given deceleration acceleration and whether it can stop before the stop line, based on the location of the stop line, selecting a traffic light corresponding to the stop line as the target traffic light in the case where it is judged that the vehicle cannot stop before the stop line, set the detection area corresponding to the target traffic light in the image, and determining a display state of the target traffic light by performing image processing in the detection area. 2. The traffic light recognition method according to claim 1, further comprising the steps of: calculating the deceleration acceleration for the vehicle to stop at the stop line, and it is judged that the vehicle cannot stop before the stop line in the case where the deceleration acceleration amount is greater than the predetermined deceleration acceleration amount. 3. The traffic light recognition method according to claim 1 or 2, further comprising the steps of: calculating an estimated stop location when the vehicle decelerates at a given acceleration deceleration, and it is judged that the vehicle cannot stop in front of the stop line if the stop line is located between the intended stop location and the vehicle. 4. The method of recognizing a traffic light according to any one of paragraphs. 1-3, further comprising the step of: obtaining the location of the stop line corresponding to the traffic light based on the location of the vehicle and the map information, wherein the mapping information includes traffic light location information. 5. A method for recognizing a traffic light according to any one of paragraphs. 1-4, further comprising the step of: setting a detection area corresponding to the target traffic light in the image if all traffic lights between the stop line corresponding to the target traffic light and the vehicle indicate that the vehicle can move straight along the planned route of the vehicle. 6. The method of recognizing a traffic light according to any one of paragraphs. 1-5, further comprising the steps of: get the acceleration of the vehicle, and setting a detection area corresponding to the target traffic light in the image in a case where the vehicle is accelerating or moving at a constant speed. 7. The method of recognizing a traffic light according to any one of paragraphs. 1-6, further comprising the steps of: obtaining the vehicle location, vehicle speed, and stop line location for each image processing execution cycle, and assessing whether the vehicle can stop before the stop line. 8. A method for recognizing a traffic light according to any one of paragraphs. 1-7, further comprising the step of: assessing whether the vehicle can stop in front of the stop line by decelerating the vehicle with a given deceleration acceleration in case all traffic lights between the stop line and the vehicle indicate that the vehicle can move straight along the planned route of the vehicle. 9. A traffic light recognition device, comprising an imaging unit installed on a vehicle and a controller, wherein the controller captures an image of the direction of travel of the vehicle using the imaging unit, gets the location of the stop line corresponding to the traffic light located in the direction of the vehicle, estimates whether the vehicle can decelerate with a given deceleration acceleration and whether it can stop before the stop line, based on the location of the stop line, selects the traffic light corresponding to the stop line as the target traffic light in the case where it is judged that the vehicle cannot stop before the stop line, sets the detection area corresponding to the target traffic light in the image, and determines the display state of the target traffic light by performing image processing in the detection area.

Images