JP6228492B2 - Outside environment recognition device - Google Patents

Description

  The present invention relates to a vehicle environment recognition apparatus that identifies a lighting state or blinking state of a traffic light.

  2. Description of the Related Art Conventionally, a technique is known in which a front part of a host vehicle is imaged by an imaging unit, a traffic signal included in the captured image is detected, and driving assistance is performed according to the lighting state of the traffic signal. In recent years, switching from a conventional light bulb type traffic light to an LED traffic light that employs an LED as a light emitting part of the traffic light has been promoted.

  The LED traffic light is lit by a full-wave rectified wave obtained by full-wave rectifying the AC power of the commercial power supply. Therefore, while it is turned on, although it cannot be visually recognized by human eyes, it is turned off periodically and instantaneously according to the frequency of the AC power. Therefore, if the frame rate of the imaging unit and the LED signal extinction cycle are synchronized, LED signals that are momentarily extinguished in each frame, despite being actually controlled to be lit. May be captured. Therefore, a technique for avoiding synchronization between the frame rate of the imaging unit and the instantaneous turn-off period of the LED traffic light by shifting the frame rate and imaging timing of the imaging unit is disclosed (for example, Patent Documents 1 to 3).

JP 2008-293280 A JP 2008-134844 A JP 2012-34275 A

  By the way, the frequency of the AC power of the commercial power source is determined in advance, and the frame rate of the imaging unit is made different from the frequency of the AC power without performing a high processing load control as in Patent Documents 1 to 3. Problems due to synchronization can be avoided. However, even if the LED signal is not completely synchronized, the LED signal that is instantaneously extinguished may be captured as long as the LED signal is momentarily extinguished. In this case, there is a risk of erroneous determination that the LED traffic light is controlled to be turned off or blinked despite being controlled to be turned on.

  In view of such a problem, the present invention provides an outside environment recognition device capable of avoiding a situation in which a traffic signal controlled to be in a lighting state is erroneously determined to be controlled to be in a lighting state or a blinking state. The purpose is to do.

  In order to solve the above-described problems, an external environment recognition device according to the present invention includes an imaging unit that images a detection region at a frame rate different from the frequency of the AC power of the commercial power supply and a divisor of the frequency, The traffic light is turned off in a traffic light identifying unit that identifies the traffic signal included in the signal, a light-off determining unit that determines whether or not the traffic light for each frame is turned off, and a first predetermined number of consecutive frames set in advance. And a non-lighting determining unit that determines that the traffic light is turned off or blinking when the number of frames determined to exceed the first threshold value.

  The non-lighting determination unit has a second predetermined number equal to or greater than the first predetermined number of frames in which the traffic light is determined to be turned off in a predetermined first predetermined number of consecutive frames. If the ratio of the number of frames determined to be extinguished with respect to the second predetermined number of frames in consecutive frames exceeds the second threshold value, it may be determined that the signal is extinguished or blinking.

  In order to solve the above-described problem, another external environment recognition apparatus of the present invention has an imaging unit that images a detection region at a frame rate different from the frequency of the AC power of the commercial power supply and a divisor of the frequency, and the imaging unit images A second predetermined number in a traffic signal identifying unit that identifies a traffic signal included in the image, a turn-off determining unit that determines whether or not the traffic light for each frame is turned off, and a second predetermined number of consecutive frames set in advance. A non-lighting determining unit that determines that the traffic light is turned off or blinking when a ratio of the number of frames determined to be turned off with respect to the number of frames exceeds a second threshold. .

  ADVANTAGE OF THE INVENTION According to this invention, it becomes possible to avoid the situation which erroneously determines that the traffic signal currently controlled by the lighting state is controlled by the light extinction state or the blinking state.

It is the block diagram which showed the connection relation of the environment recognition system. It is the functional block diagram which showed the schematic function of the external environment recognition apparatus. It is explanatory drawing for demonstrating a color image and a distance image. It is explanatory drawing for demonstrating operation | movement of the target object specific | specification part. It is explanatory drawing for demonstrating the calculation which determines the frame rate of an imaging part. It is explanatory drawing for demonstrating the calculation result for every imaging cycle. It is explanatory drawing for demonstrating the lighting state and blinking state of a LED traffic light. It is a flowchart which shows the flow of a process of a non-lighting determination part.

  Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings. The dimensions, materials, and other specific numerical values shown in the embodiment are merely examples for facilitating understanding of the invention, and do not limit the present invention unless otherwise specified. In the present specification and drawings, elements having substantially the same function and configuration are denoted by the same reference numerals, and redundant description is omitted, and elements not directly related to the present invention are not illustrated. To do.

  In recent years, the on-board camera mounted on the vehicle images the road environment ahead of the host vehicle, the front of the host vehicle is captured by the imaging unit, the traffic light included in the captured image is detected, and the traffic light lighting state There is a known technology that provides driving assistance in response to the above. However, in the case of LED traffic lights that have been adopted in recent years, the LED is in a lighting state by a full-wave rectified wave obtained by full-wave rectification of AC power, and while it is in a lighting state, it cannot be visually recognized by human eyes. The light is turned off periodically and instantaneously according to the frequency of the AC power. For this reason, depending on the imaging timing, there is a risk of erroneous determination that the LED traffic light is controlled to be turned off or blinked despite being controlled to be lit. The present embodiment aims to avoid such erroneous determination. Hereinafter, an environment recognition system for achieving such an object will be described, and a vehicle exterior environment recognition apparatus as a specific component thereof will be described in detail.

  In the following description, continuous lighting, blinking, and extinguishing under the control of the LED traffic light are referred to as lighting state, blinking state, and extinguishing state, respectively, and lighting and extinguishing that occur periodically and instantaneously according to the frequency of AC power. In addition, lighting and extinguishing of the LED traffic light at a specific moment, such as when capturing an image of one frame, are simply referred to as lighting and extinguishing.

(Environment recognition system 100)
FIG. 1 is a block diagram showing a connection relationship of the environment recognition system 100. The environment recognition system 100 includes an outside environment recognition device 120 provided with an imaging unit 110 and a vehicle control device (ECU: Engine Control Unit) 130 provided in the host vehicle 1.

  The imaging unit 110 includes an imaging element such as a charge-coupled device (CCD) or a complementary metal-oxide semiconductor (CMOS), images an environment corresponding to the front of the host vehicle 1, and is represented by a color value. A color image can be generated. Here, the color value is a numerical group consisting of one luminance (Y) and two color differences (UV), or three hues (R (red), G (green), B (blue)). .

  In addition, the imaging units 110 are arranged in a substantially horizontal direction so that the optical axes of the two imaging units 110 are substantially parallel on the traveling direction side of the host vehicle 1. The imaging unit 110 continuously generates image data obtained by imaging an object existing in the detection area in front of the host vehicle 1 at a predetermined frame rate. The frame rate of the imaging unit 110 will be described in detail later.

  Here, the objects to be recognized are not only three-dimensional objects such as vehicles, pedestrians, traffic lights, roads (travel paths), guardrails, and buildings, but also three-dimensional objects such as taillights, blinkers, and lighting parts of traffic lights. The thing which can be specified as a part of is also included. Each functional unit in the following embodiment performs each process for each frame in response to such update of the image data.

  The vehicle environment recognition apparatus 120 acquires image data from each of the two imaging units 110, derives parallax using so-called pattern matching, and associates the derived parallax information (corresponding to a depth distance described later) with the image data. Generate a distance image. The color image and the distance image will be described in detail later. Further, the outside-vehicle environment recognition device 120 identifies an object in the detection area in front of the host vehicle 1 using the color value based on the color image and the depth distance from the host vehicle 1 based on the distance image.

  When the vehicle environment recognition apparatus 120 identifies the object, it performs a process of determining whether the traffic light is controlled to be in a lighting state, a blinking state, or a light-off state while tracking the target object, for example, a traffic light. Then, the outside environment recognition device 120 performs warning display (notification) to the driver through the display 122 installed in front of the driver according to the determination result, and makes a determination to the vehicle control device 130 as necessary. Outputs information indicating the result.

  The vehicle control device 130 receives a driver's operation input through the steering wheel 132, the accelerator pedal 134, and the brake pedal 136, and controls the host vehicle 1 by transmitting it to the steering mechanism 142, the drive mechanism 144, and the brake mechanism 146. In addition, the vehicle control device 130 controls the drive mechanism and the braking mechanism in accordance with instructions from the outside environment recognition device 120.

  Hereinafter, the configuration of the outside environment recognition device 120 will be described in detail. Here, the procedure for identifying the traffic signal controlled in the blinking state, which is characteristic of the present embodiment, will be described in detail, and description of the configuration unrelated to the features of the present embodiment will be omitted.

(Vehicle environment recognition device 120)
FIG. 2 is a functional block diagram showing a schematic function of the outside environment recognition device 120. As shown in FIG. 2, the vehicle exterior environment recognition apparatus 120 includes an imaging unit 110, an I / F unit 150, a data holding unit 152, and a central control unit 154. Since the imaging unit 110 has been described above, detailed description thereof is omitted here.

  The I / F unit 150 is an interface for performing bidirectional information exchange with the imaging unit 110 and the vehicle control device 130. The data holding unit 152 includes a RAM, a flash memory, an HDD, and the like. The data holding unit 152 holds various pieces of information necessary for the processing of each function unit described below, and temporarily holds image data received from the imaging unit 110. To do.

  The central control unit 154 is configured by a semiconductor integrated circuit including a central processing unit (CPU), a ROM storing a program, a RAM as a work area, and the like, and through the system bus 156, an I / F unit 150, a data holding unit 152 and the like are controlled. In the present embodiment, the central control unit 154 also functions as an image processing unit 160, a three-dimensional position deriving unit 162, an object specifying unit (traffic signal specifying unit) 164, a turn-off determining unit 166, and a non-lighting determining unit 168. . In the following, detailed operations of such functional units will be described in the order of image processing, object identification processing, traffic light state identification processing, based on the general purpose.

(Image processing)
The image processing unit 160 acquires image data from each of the two imaging units 110, and selects a block corresponding to a block arbitrarily extracted from one image data (for example, an array of 4 horizontal pixels × 4 vertical pixels) as the other image data. The parallax is derived using so-called pattern matching that is searched from the above. Here, “horizontal” indicates the horizontal direction of the screen, and “vertical” indicates the vertical direction of the screen.

  As this pattern matching, it is conceivable to compare the luminance (Y) between two pieces of image data in units of blocks indicating arbitrary image positions. For example, SAD (Sum of Absolute Difference) that takes a luminance difference, SSD (Sum of Squared Intensity Difference) that uses the difference squared, or NCC that takes the similarity of a variance value obtained by subtracting an average value from the luminance of each pixel There are methods such as (Normalized Cross Correlation). The image processing unit 160 performs such a block-unit parallax derivation process for all blocks displayed in the detection area (for example, horizontal 600 pixels × vertical 180 pixels). Here, the block is assumed to be horizontal 4 pixels × vertical 4 pixels, but the number of pixels in the block can be arbitrarily set.

  However, the image processing unit 160 can derive the parallax for each block, which is a unit of detection resolution, but cannot recognize what kind of target object the block is. Accordingly, the disparity information is derived independently not in units of objects but in units of detection resolution (for example, blocks) in the detection region. Here, an image in which the parallax information derived in this way is associated with image data is referred to as a distance image in distinction from the color image described above.

  FIG. 3 is an explanatory diagram for explaining the color image 210 and the distance image 212. For example, it is assumed that a color image (image data) 210 as illustrated in FIG. 3A is generated for the detection region 214 through the two imaging units 110. However, for ease of understanding, only one of the two color images 210 generated by each imaging unit 110 is schematically shown here. In the present embodiment, the image processing unit 160 obtains a parallax for each block from such a color image 210, and forms a distance image 212 as shown in FIG. Each block in the distance image 212 is associated with the parallax of the block. Here, for convenience of description, blocks from which parallax is derived are represented by black dots.

  Returning to FIG. 2, the three-dimensional position deriving unit 162 calculates disparity information for each block in the detection region 214 based on the distance image 212 generated by the image processing unit 160 using a so-called stereo method. The information is converted into three-dimensional position information in real space including the distance, height, and depth distance. Here, the stereo method is a method of deriving the depth distance of the target part from the imaging unit 110 from the parallax in the target part distance image 212 by using a triangulation method. At this time, the three-dimensional position deriving unit 162 determines the target part based on the depth distance of the target part and the detected distance on the distance image 212 between the point on the road surface and the target part at the same depth distance as the target part. The height from the road surface is derived. Then, the derived three-dimensional position information is associated with the distance image 212 again. Since various known techniques can be applied to the depth distance deriving process and the three-dimensional position specifying process, description thereof is omitted here.

(Object identification processing)
FIG. 4 is an explanatory diagram for explaining the operation of the object specifying unit 164. Here, the object specifying unit 164 will be described as an example of a process of specifying a red signal at the time of lighting among light emitting units of a traffic light as an object, but the object specifying unit 164 is a blue color at the time of lighting. It also performs processing for identifying signals and yellow signals when lit, and identifying other objects.

  The object specifying unit 164 determines that the color value of an arbitrary target part (pixel or block) in the color image 210 is a color value range of the red signal (for example, the reference value is the color value (R) and the color value (G) is It is determined whether or not the reference value (R) is 0.5 times or less and the color value (B) is 0.38 times or less of the reference value (R). And if it is contained in the color value range used as object, the identification number which shows the said object will be attached | subjected to the object part. Here, as shown in the enlarged view of FIG. 4, the identification number “1” is assigned to the target portion corresponding to the red signal.

  In addition, the object specifying unit 164 groups the target parts in the detection region 214 using the three-dimensional position information based on the distance image 212. Specifically, the object specifying unit 164 has an arbitrary target part as a base point, a difference between the target part, a horizontal distance, a height difference, and a depth distance within a predetermined range. Group other target sites. Here, the predetermined range is represented by a distance in the real space, and can be set to an arbitrary value (for example, 1.0 m). The object specifying unit 164 also sets the difference in the horizontal distance, the difference in the height, and the difference in the depth distance within the predetermined range with respect to the target part newly added by the grouping. Group some other target area. As a result, if the distance is within the predetermined range, all the target parts are grouped.

  Subsequently, the object specifying unit 164 is configured such that the grouped target part group 220 has a height range (for example, 4.5 to 7.0 m) and a width range (for example, 0.05 to 0.05) associated with the object. 0.2m), shape (for example, circular shape), etc., it is determined whether or not a predetermined condition is satisfied. Here, with respect to the shape, the shape is compared with reference to a template associated with the object in advance (pattern matching), and it is determined that the condition is satisfied when there is a correlation greater than or equal to a predetermined value. If the predetermined condition is satisfied, the grouped target part group 220 is determined as the target object.

  Moreover, if the target object specific | specification part 164 specifies the target object in any one of a red signal, a blue signal, and a yellow signal, it will search the target object of a traffic signal in the periphery. Similar to the red signal identification process, the traffic signal identification process determines whether or not the target part grouping and the grouped target part group 220 satisfy a predetermined condition associated with the traffic light. When the condition is satisfied, the traffic light is specified as the object.

  That is, the object specifying unit 164 functions as a traffic signal specifying unit that specifies a traffic signal included in an image captured by the imaging unit 110. Then, the object specifying unit 164 continues to specify the signal itself, the red signal, the blue signal, and the yellow signal by continuously tracking over the subsequent frames for the position of the specified signal device.

(Signal status identification processing)
Returning to FIG. 2, the extinction determination unit 166 determines whether or not a part corresponding to, for example, a red signal of the traffic light identified for each frame captured by the imaging unit 110 is extinguished at least when the frame is captured. judge.

  Specifically, the extinction determination unit 166 determines whether or not the color value of the light emitting unit of the red signal is included in the color value range corresponding to the red signal that is lit among the traffic lights included in the image, for example. To determine whether the red signal is extinguished. Similar processing is performed for the blue signal and the yellow signal.

  In the processing of the turn-off determination unit 166 and the non-lighting determination unit 168, the same processing is performed for any of the red signal, the blue signal, and the yellow signal. Therefore, in the following, it is not particularly mentioned whether the processing is for a red signal, a blue signal, or a yellow signal.

  As described above, while the LED traffic light is turned on by a full-wave rectified wave obtained by full-wave rectification of AC power, it is not visible to the human eye, but periodically and instantaneously turns off according to the frequency of the AC power. ing. When the commercial power source is used, the AC power is 50 Hz or 60 Hz. Therefore, even if the full-wave rectified LED traffic light is controlled to be in the lighting state, it is instantaneous at a frequency of 100 Hz or 120 Hz. Is unlit.

  Therefore, even in the case of an LED traffic light that is actually controlled to be turned on, the turn-off determination unit 166 may determine that the light is turned off. If the frame rate of the imaging unit 110 and the LED signal extinction cycle are synchronized, the LED signal that is controlled to be in the on state is continuously extinguished over many frames. It will be judged. As a result, there is a possibility that an LED traffic light that is controlled to be in a lighting state is erroneously determined as a blinking state.

  Therefore, in this embodiment, the frame rate of the imaging unit 110 is determined by the following calculation to avoid synchronization with the cycle of turning off the LED traffic light.

  FIG. 5 is an explanatory diagram for explaining calculation for determining the frame rate of the imaging unit 110. In the present embodiment, the image capturing unit 110, for each frame, generates an image for detecting a traffic light and an image for detecting a vehicle or the like that make it easy to detect the light emitting part of the traffic light by shortening the exposure time. Images are taken alternately. Therefore, the frame rate of the image for detection by the traffic light is halved with respect to the frame rate of all images in the imaging unit 110, and the imaging cycle is doubled.

  Here, an example that is not suitable as the imaging cycle of the imaging unit 110 will be described. Assuming that the imaging cycle is 62 msec, an image for detecting a traffic light is captured at a cycle of 124 msec. On the other hand, the instantaneous turn-off cycle of the LED traffic light controlled to be in the lighting state is 10 msec, which is the reciprocal of 100 Hz, in an area where the commercial power source is 50 Hz.

Moreover, in the LED traffic light controlled to be in the lighting state, the duty ratio of lighting: extinguishing is approximately 8: 2. Therefore, in one cycle of 10 msec, the lighting time is 8 msec and the extinguishing time t ₁ is 2 msec.

As described above, when the signal detection image is captured at a cycle of 124 msec, the LED traffic light blinks 12 times during the imaging period of the signal detection image, and in the next imaging cycle, the imaging cycle On the other hand, the blinking cycle of the LED traffic light is shifted (advanced) by the shift time t ₂ = 4 msec.

As shown in FIG. 5A, it is assumed that the LED traffic light starts to be turned off at the timing of starting the imaging of the imaging unit 110. Then, only the deviation time t ₂ each time the imaging cycle of image for the detection of traffic has passed one period shifted from each other in the period.

However, since the turn-off time t ₁ is shorter than the shift time t ₂ , when an image in which one frame is turned off is captured, in the next frame, the imaging timing is shifted to the turn-off time t ₁ or more and turned on. An image is captured. Therefore, the instantaneous turn-off of the LED traffic light is not captured continuously for two frames or more, but only once as shown by timing A in FIG.

On the other hand, in the region where the commercial power source is 60 Hz, the instantaneous extinguishing cycle of the LED traffic light controlled to be in the lighting state is 8.33 msec which is the reciprocal of 120 Hz. If the duty ratio of lighting: extinguishing is 8: 2, the lighting time is 6.67 msec and the extinguishing time t ₁ is 1.67 msec in one cycle of 8.33 msec.

During the imaging period of the image for detection of the traffic light, the LED traffic light blinks 14 times, and in the next imaging period, the blinking period of the LED traffic light is the deviation time t ₂ = 7.33 msec with respect to the imaging period. It will be shifted (delayed). In other words, it can be said that the blinking cycle of the LED traffic light is advanced with respect to the imaging cycle by the shift time t ₃ = 8.33−7.33 = 1 msec.

In this case, the turn-off time t ₁ is longer than the shift time t ₃ . Therefore, as shown in FIG. 5B, when an image in which one frame is turned off is captured, the imaging timing in the next frame is the turn-off time t with respect to the instantaneous turn-off cycle of the LED traffic light. Only progresses below ₁ . As a result, the instantaneous turn-off of the LED traffic light may be captured continuously for two frames as shown at timings B and C in FIG.

  It is desirable for the imaging unit 110 to have an imaging cycle in which the instantaneous turn-off of the LED traffic light is not captured continuously for two frames in both regions where the commercial power source is 50 Hz and 60 Hz. Therefore, when the imaging cycle of all the images of the imaging unit 110 is set to 0 to 150 msec, the above calculation is performed in each case, and an LED in which instantaneous turn-off of the LED traffic light is not captured continuously is specified.

  FIG. 6 is an explanatory diagram for explaining a calculation result for each imaging cycle. In FIG. 6, the horizontal axis indicates the imaging cycle of all images of the imaging unit 110, and the vertical axis indicates the flag. Here, the flag is calculated when the instantaneous turn-off of the LED traffic light is imaged for two or more frames continuously when the instantaneous turn-off of the LED traffic light is captured for one or more frames. Flag 0 is shown.

  In the present embodiment, the imaging cycle of all images of the imaging unit 110 is set to an imaging cycle in which the flag shown in FIG. That is, the imaging unit 110 images the detection area at a frame rate different from both the frequency of the AC power of the commercial power supply and the divisor thereof. In this way, in both the 50 Hz and 60 Hz commercial power supply regions, the situation where the instantaneous turn-off of the LED traffic light is imaged continuously for two frames is avoided.

  Returning to FIG. 2, the non-lighting determination unit 168 determines whether or not the traffic light is controlled to be in a blinking state. As described above, the frame rate of the imaging unit 110 is set so that the instantaneous turn-off of the LED traffic light is not captured continuously for two or more frames. That is, if it is determined that the traffic light tracked by the light turn-off determination unit 166 is continuously turned off for two frames or more, it is highly likely that the traffic light is controlled to be in a blinking state. . Therefore, the non-lighting determination unit 168 is controlled so as to be in a blinking state when it is determined that two or more frames out of the immediately preceding three frames are turned off in consideration of the error. It is determined that there is a high possibility.

  In this way, the non-lighting determination unit 168 determines that the number of frames determined to have the traffic light turned off in a predetermined first predetermined number (for example, three frames) of consecutive frames is the first threshold value (for example, 1 If it exceeds the (frame), it is determined that there is a high possibility that the traffic light is controlled to be in a blinking state.

  As described above, the imaging unit 110 divides an image for detecting a traffic light and an image for detection of a vehicle into “an image for detecting a traffic signal” → “an image for detection of a vehicle” → “an image of a traffic light”. Images such as “detection images” are alternately captured every frame. The non-lighting determination unit 168 selects continuous frames only for the detection image of the traffic light. That is, in the present embodiment, “continuous frames” means “frames of images for detection of continuous traffic lights”.

  Here, the non-lighting determination unit 168 does not compare the number of frames in which the traffic light is continuously turned off with a predetermined threshold value, but the number of frames in which the traffic light is determined to be turned off in three consecutive frames. Whether or not exceeds 1 is determined. For example, the non-lighting determination unit 168 determines whether or not the number of frames in which the traffic light is determined to be turned off out of three consecutive frames exceeds one frame. This is because even if the traffic light is actually turned off, due to error factors such as the positional relationship between the vehicle and the traffic light, imaging timing, ambient brightness, etc. This is because the frames to be determined may not be continuous. In the present embodiment, since it is determined whether or not the number of frames for which the traffic light has been turned off exceeds 1 in three consecutive frames, the frames that have been determined to be turned off are not continuous. However, the influence of errors can be suppressed.

  Further, the non-lighting determination unit 168 can determine whether or not the traffic light is controlled to be in a blinking state by other determination means. Other determination means of the non-lighting determination unit 168 will be described in detail with reference to FIG.

  FIG. 7 is an explanatory diagram for explaining the lighting state and blinking state of the LED traffic light. FIG. 7A shows a case where control is performed so as to be in the lighting state, and FIG. The case where it is controlled to be in a blinking state is shown.

  As shown in FIG. 7 (a), when the LED traffic light is controlled to be in the lighting state, the duty ratio of lighting: extinguishing is approximately 8: 2 as described above, so that the preset second 2 The ratio (ratio) of the number of frames determined by the extinction determination unit 166 to be extinguished with respect to the predetermined number (for example, 8 frames) is approximately 0.2.

  On the other hand, as shown in FIG. 7B, when the LED traffic light is controlled so as to blink, the ratio of the time of the lighting state to the time of the light-off state is 1: 1. That is, as shown in FIG. 7B, when the total time (cycle) in which lighting and extinguishing are performed once is set to 1, the lighting time and the lighting time are 0.5. In the lighting state, since the duty ratio of lighting: extinguishing is approximately 8: 2, the extinguishing time in the lighting state is 0.5 × 0.2 = 0.1. That is, the turn-off time in one cycle in the blinking state is 0.5 + 0.1 = 0.6. As a result, the ratio of the number of frames determined by the turn-off determining unit 166 to be turned off with respect to the second predetermined number of frames is approximately 0.6.

  Therefore, in consideration of the influence of errors, the second threshold value used for the determination process is set to 0.5 which is smaller than 0.6 and larger than 0.2. That is, if the ratio of the number of frames that the non-lighting determination unit 166 determines to be unlit with respect to the second predetermined number of frames exceeds the second threshold value of 0.5, It is determined that the traffic light is likely to be controlled to be in a blinking state.

  In the present embodiment, the non-lighting determination unit 168 improves the determination accuracy of the traffic light controlled to be in the blinking state by combining the above two determination means. Hereinafter, the process flow of the non-lighting determination unit 168 will be described with reference to a flowchart.

  FIG. 8 is a flowchart showing a process flow of the non-lighting determination unit 168. In the present embodiment, when the target object specifying unit 164 specifies a traffic signal included in the image, the target object tracking unit 164 tracks the traffic signal over subsequent successive frames. The processing shown in FIG. 8 is repeatedly executed every time the imaging unit 110 captures an image for detection of a traffic signal for each traffic signal to be tracked.

  Further, the processing shown in FIG. 8 is performed for a traffic signal that is a tracking target as a result of determining that the extinction determining unit 166 is lit, and the traffic signal that is controlled to be in the extinguished state is out of scope. That is, it is a process of determining whether the traffic light included in the image captured by the imaging unit 110 is controlled to be in a blinking state or controlled to be in a lighting state (non-flashing state). .

  The non-lighting determination unit 168 counts the number of frames that the light-off determination unit 166 tracks the traffic light (S300). Then, it is determined whether or not the number of consecutive frames tracking the traffic signal is greater than or equal to a second predetermined number (S302). That is, the non-lighting determination unit 168 determines whether or not the same traffic light is continuously recognized from the second predetermined number of previous frames. If the number of frames is not greater than or equal to the second predetermined number (NO in S302), the process ends.

  When the number of frames is equal to or greater than the second predetermined number (YES in S302), the non-lighting determination unit 168 determines the number of frames determined by the turn-off determination unit 166 to be turned off for the immediately preceding first predetermined number of frames. Confirm (S304).

  Then, the non-lighting determination unit 168 determines whether or not the number of frames that the light-off determination unit 166 has determined to be turned off exceeds the first threshold (S306). If the first threshold value is exceeded (YES in S306), the non-lighting determination unit 168 confirms the number of frames that the turn-off determination unit 166 has determined to be turned off for the immediately preceding second predetermined number of consecutive frames. (S308).

  Then, the non-lighting determination unit 168 determines whether or not the ratio of the number of frames determined by the turn-off determination unit 166 to be turned off with respect to the immediately preceding second predetermined number of frames exceeds the second threshold value. (S310). When the ratio of the number of frames exceeds the second threshold value (YES in S310), the non-lighting determination unit 168 adds 1 point to the tracking target traffic light (S312).

  On the other hand, when the number of frames determined to be turned off by the turn-off determining unit 166 is equal to or smaller than the first threshold (NO in S306), or the turn-off determining unit 166 is turned off for the immediately preceding second predetermined number of frames. If the ratio of the number of frames determined to be equal to or less than the second threshold value (NO in S310), one point is subtracted from the traffic signal point to be tracked (S314).

  Then, the non-lighting determination unit 168 determines whether or not the integrated value of the points of the traffic signal to be tracked is greater than or equal to the third threshold (S316). If it is greater than or equal to the third threshold (YES in S316), the non-lighting determination unit 168 determines that the traffic light is controlled to be in a blinking state (S318).

  If it is less than the third threshold (NO in S316), the non-lighting determination unit 168 determines that the traffic light is not controlled to be in a blinking state and is controlled to be in a non-flashing state. (S320). Here, as described above, the traffic signal that is controlled to be turned off is excluded, so that the traffic signal is controlled to be turned on.

  As described above, according to the outside environment recognition device 120, it is possible to avoid a situation in which a traffic light that is controlled to be in a lighting state is erroneously determined to be controlled in a blinking state.

  Further, since the non-lighting determination unit 168 reflects the determination results over a plurality of frames by the accumulation of points, it is possible to suppress the influence of errors and improve the determination accuracy of the traffic signal controlled to be in a blinking state. It becomes possible.

  Also provided are a program that causes a computer to function as the vehicle exterior environment recognition device 120 and a computer-readable storage medium that stores the program, such as a flexible disk, magneto-optical disk, ROM, CD, DVD, and BD. Here, the program refers to data processing means described in an arbitrary language or description method.

  As mentioned above, although preferred embodiment of this invention was described referring an accompanying drawing, it cannot be overemphasized that this invention is not limited to this embodiment. It will be apparent to those skilled in the art that various changes and modifications can be made within the scope of the claims, and these are naturally within the technical scope of the present invention. Is done.

  For example, in the above-described embodiment, the non-lighting determination unit 168 adds and subtracts points with respect to the traffic signal to be tracked, and when the integrated value of the points exceeds the third threshold, the traffic light is controlled to blink. It was determined that there was. However, the determination based on the result of point addition or subtraction need not be performed. That is, the non-lighting determination unit 168 has the second predetermined number of frames in which the number of frames determined to be off in the first predetermined number of consecutive frames exceeds the first threshold and the second predetermined number of consecutive frames. If the ratio of the number of frames determined that the traffic light is extinguished with respect to the predetermined number of frames exceeds the second threshold, it may be determined that the traffic light is blinking.

  In the above-described embodiment, the non-lighting determination unit 168 determines whether the number of frames for which the traffic light is determined to be turned off exceeds the first threshold in the first predetermined number of consecutive frames. In the continuous frames, the case where two determinations are made as to whether the ratio of the number of frames determined to have the traffic light turned off with respect to the second predetermined number of frames exceeds the second threshold has been described. However, the non-lighting determination unit 168 may perform only one of the two determinations and determine that the traffic light is blinking based on the determination result. However, the determination accuracy can be improved by using two determinations in combination.

  Further, in the above-described embodiment, a case has been described in which it is determined whether or not a traffic light that is controlled to be turned off is previously controlled and whether or not the traffic light is controlled to be blinked. However, the non-lighting determination unit 168 may determine whether or not the traffic light is controlled so as to be in the light-off state or the blinking state without previously excluding the traffic light that is controlled to be in the light-off state.

  Note that each step of the vehicle environment recognition method of the present specification does not necessarily have to be processed in time series in the order described in the flowchart, and may include processing in parallel or a subroutine.

  INDUSTRIAL APPLICABILITY The present invention can be used for an external environment recognition device that identifies a lighting state or blinking state of a traffic light.

DESCRIPTION OF SYMBOLS 1 Own vehicle 110 Image pick-up part 120 Out-of-vehicle environment recognition apparatus 130 Vehicle control apparatus 164 Target object specific | specification part (traffic signal specific part)
166 Light extinction determination unit 168 Non-lighting determination unit

Claims (3)

An imaging unit that images the detection region at a predetermined frame rate different from the frequency of the AC power of the commercial power supply and a divisor of the frequency;
A traffic light identifying unit for identifying a traffic signal included in an image captured by the imaging unit;
An extinction determining unit that determines whether or not the traffic light is extinguished for each frame;
In a predetermined first predetermined number of consecutive frames, a non-lighting determination unit that determines that the traffic light is turned off or blinks when the number of frames determined that the traffic light is turned off exceeds a first threshold. When,
A vehicle exterior environment recognition device comprising:   In the first predetermined number of consecutive frames set in advance, the non-lighting determination unit exceeds the first threshold when the number of frames for which it is determined that the traffic light is off, and is equal to or greater than the first predetermined number. In a second predetermined number of consecutive frames, when the ratio of the number of frames determined to be extinguished to the second predetermined number of frames exceeds a second threshold, the signal is extinguished or blinking. The outside environment recognition device according to claim 1, wherein: An imaging unit that images the detection region at a predetermined frame rate different from the frequency of the AC power of the commercial power supply and a divisor of the frequency;
A traffic light identifying unit for identifying a traffic signal included in an image captured by the imaging unit;
An extinction determining unit that determines whether or not the traffic light is extinguished for each frame;
In a predetermined second predetermined number of consecutive frames, when the ratio of the number of frames determined to be extinguished with respect to the second predetermined number of frames exceeds a second threshold, the signal is extinguished or A non-lighting determination unit that determines that it is blinking;
A vehicle exterior environment recognition device comprising: