JP2011216051A - Program and device for discriminating traffic light - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a traffic light discrimination program and a traffic light discrimination device which can accurately discriminate traffic light in a photographic image in real time regardless of weather, time zone, or the like without the necessity of preparing template data or the like for discrimination.SOLUTION: A traffic light discrimination device 3 is made to serve as: a still image acquisition part 61; a color component conversion part 64 which converts RGB components of respective pixels constituting a still image to HSV components; an extraction threshold storage part 53 wherein thresholds determining a hue range, a saturation range, and a brightness range for extracting pixels having hue, saturation, and brightness corresponding to a traffic light are stored; a pixel extracting part 65 which extracts pixels, the hue, the saturation, and the brightness of which are within the hue range, the saturation range, and the brightness range respectively, out of pixels converted by the color component conversion part 64; and an extracted image output part 72 which outputs an extracted image to an output device 4.

Description

  The present invention relates to a technique for identifying a signal lamp in a captured image, and more particularly to a signal lamp identification program and a signal lamp identification apparatus for identifying a signal lamp in real time regardless of the weather, time zone, or the like.

  In recent years, development of various driving support technologies using state-of-the-art information communication technology and in-vehicle sensors has been promoted. Given the future of an aging society, such driving assistance technologies are expected to evolve further. In particular, signal light automatic recognition technology is highly expected to prevent traffic accidents caused by oversight of signal lights. Moreover, since it has been pointed out that signal lights of LED light sources that have been introduced in recent years are difficult to see for people with low vision, it may be possible to support them with automatic recognition technology.

  As the above-described automatic signal lamp recognition technology, a method using electromagnetic waves and a method using a video camera are known. In ITS (Intelligent Transport System), emphasis is placed on the development of devices using electromagnetic waves, but the current situation is that the equipment costs are enormous and there are many unsolved problems.

  On the other hand, a method using a video camera can realize a system that is easy to spread at low cost, and an identification method based on template matching, an identification method based on color information, and the like have been proposed. In the former method, a template prepared in advance and a captured image are compared, and an area having the highest degree of correlation is identified as a signal lamp (Non-Patent Document 1). On the other hand, in the latter method, signal lamp color information is accumulated in advance, and a signal lamp is extracted from a captured image by threshold processing (Non-Patent Document 2).

Yuji Nakata, Daishi Koyasu, Hitoshi Maekawa "Traffic signal recognition from moving images using in-vehicle camera" (IEICE Technical Report, ITS2007-56, IE2007-239 (2008-02), pp.121- 125.) Fumika Kimura, Tomokazu Takahashi, Yoshito Mekada, Ichiro Ide, Hiroshi Murase “Signal Recognition in Various Shooting Environments to Support Safe Driving” (MIRU2006, pp.618-623 (2006-7))

  However, in the former method including the invention described in Non-Patent Document 1, the degree of correlation between the template and the captured image greatly affects the identification accuracy. For this reason, in order to avoid misrecognition of a signal lamp, pre-processing is required to bring the template close to the brightness of the shooting environment, the size of the signal lamp, and the like in advance. In addition, in the case of bad weather or at night, the shape of the signal light is difficult to identify, and there is a problem that the identification accuracy is lowered.

  On the other hand, the latter method including the invention described in Non-Patent Document 2 has a problem that it is difficult to adjust the threshold value and greatly affects the identification accuracy. In addition, since there is a lot of color information in the background that is confusing with the signal light, the processing time is slow and processing for reducing misrecognition is often used together. For this reason, the identification process cannot be performed in real time as the image range to be identified becomes wider.

  Furthermore, in any of the above methods, it is necessary to accumulate a large amount of template data and color information related to signal lights under various shooting conditions. For this reason, there are problems that it takes time to prepare data and the cost of the entire system also increases.

  The present invention has been made to solve such problems, and it is not necessary to prepare template data for identification in advance, and in high accuracy and real time regardless of the weather, time zone, etc. It is an object of the present invention to provide a signal lamp identification program and a signal lamp identification device that can identify a signal lamp in a captured image.

The signal lamp identification program according to the present invention uses the signal lamp identification apparatus to form each still image by using a still image acquisition unit that acquires a still image from a moving image and the following conversion equations (1) to (3). RGB component (red, green, blue) of the color component conversion unit that converts HSV components (hue, saturation, lightness), a threshold value that defines a hue range for extracting pixels of the hue corresponding to the signal light, and the signal light An extraction threshold value storage unit for storing a threshold value for determining a saturation range for extracting pixels of corresponding saturation and a threshold value for determining a lightness range for extracting pixels of brightness corresponding to a signal light; and the color component conversion The hue converted by the color component is within the hue range, the saturation converted by the color component conversion unit is within the saturation range, and the lightness converted by the color component conversion unit is within the lightness range. Extract It functions as an extracted image output unit that outputs an extracted image composed of pixels extracted by the pixel extracting unit and the pixel extracting unit to an output device, and the signal lamp identification device according to the present invention includes the above-described components. It is.
Hue = arctan (I _y / I _x ) (1)
Saturation = √ (I _x ² + I _y ² ) (2)
Intensity = I _z Expression (3)
However,
(R, g, b) is color information (luminance value) obtained from a still image.

  In the present invention, the extraction threshold storage unit has a hue range threshold for extracting a red signal or a yellow signal set to -10 degrees or more and 60 degrees or less, and for extracting a blue signal. The hue range threshold is set to 150 degrees or more and 180 degrees or less, the saturation range threshold for extracting light sources other than the white light source is set to 100 or more, and light sources other than the white light source are extracted. The brightness range threshold may be set to 50 or more and less than 225.

  Further, in the present invention, the extraction threshold storage unit further has a saturation range threshold for extracting a color-saturated blue signal set to less than 100, and brightness for extracting the color-saturated blue signal. The range threshold value may be set to 225 or more.

In the present invention, before the color component conversion unit performs conversion processing, a filtering threshold storage unit for storing a filtering threshold for filtering pixels having RGB components close to a signal lamp in advance, and the still image are configured. For each pixel to be processed, an RGB difference calculation unit that calculates a difference between RGB components, and a filtering in which the difference calculated by the RGB component calculation unit is obtained by filtering pixels within a range defined by the filtering threshold The signal light identification device functions as a unit, and the filtering threshold storage unit may be set with the following thresholds as filtering thresholds for a red signal, a yellow signal, and a blue signal.
Red signal: (R−G) + 128 ≧ 255 and (R−B) ≧ 100
Yellow signal: (R−G) +128 <255 and (R−B) ≧ 100
Green light: (R−G) +128 <28 and (B−G) ≦ 0
However, R: R component luminance value G: G component luminance value B: B component luminance value

  Furthermore, in the present invention, among the pixels extracted by the pixel extraction unit, a labeling unit that labels and groups all adjacent pixels, and a center point that calculates a center point of a labeling region that is grouped by the labeling unit A signal lamp identification device as a shape selection section for selecting a labeling area having a shape close to a signal lamp based on a positional relationship between a calculation section and a center point calculated by the center point calculation section and pixels constituting a peripheral portion of the labeling area The signal lamp identification device according to the present invention may have the above-described components.

  In the present invention, among the pixels extracted by the pixel extraction unit, a labeling unit that labels and groups all adjacent pixels, and a vertical distance and a horizontal distance of a labeling region grouped by the labeling unit Based on the difference, the signal lamp identification device may function as a shape selection unit that selects a labeling region having a shape different from that of the signal lamp, and the signal lamp identification device according to the present invention may include the above-described components. .

  Furthermore, in the present invention, among the pixels extracted by the pixel extraction unit, a labeling unit that labels and groups all adjacent pixels, and a center point that calculates a center point of a labeling region that is grouped by the labeling unit A calculation unit, an optical flow acquisition unit that acquires the optical flow of the center point calculated by the center point calculation unit, and a non-target that selects a labeling region other than the signal light based on the optical flow acquired by the optical flow acquisition unit The signal light identification device may function as an area selection unit, and the signal light identification device according to the present invention may include the above-described components.

  According to the present invention, it is not necessary to prepare identification template data or the like in advance, and it is possible to identify a signal lamp in a captured image with high accuracy and in real time regardless of weather, time zone, or the like.

It is a block diagram showing an embodiment of a signal light identification system including a signal light identification device according to the present invention. In the present embodiment, (a) a still image showing a color saturated signal light, and (b) an extracted image of FIG. 2 (a). In this embodiment, it is a figure which shows the labeling area | region corresponded to a signal lamp. In the present embodiment, (a) a still image obtained by photographing a signal lamp from a distance, and (b) an enlarged view of the extracted image of FIG. 4 (a). In this embodiment, it is a figure which shows the labeling area | region which classify | selects a shape. It is a flowchart figure which shows the process of the signal light identification device performed with the signal light identification program of this embodiment. In Example 1, (a) a still image obtained by capturing a blue signal, (b) a graph showing the relationship between saturation and lightness for pixels in the white frame region, (c) an enlarged image obtained by extracting pixels in the light source region 1, (D) An enlarged image obtained by extracting pixels in the light source region 2, and (e) an enlarged image obtained by extracting pixels in the light source region 3. In Example 1, (a) a still image obtained by photographing a red signal, (b) a graph showing the relationship between saturation and lightness for the pixels in FIG. 8 (a), and (c) pixels in the light source region 1 were extracted. It is an image. In Example 2, it is a table | surface which shows the imaging | photography date and time of the sample moving image used, and the weather at the time of imaging | photography. In Example 2, (a) The graph which shows the histogram distribution of a saturation component, (b) The graph which shows the histogram distribution of a brightness component. In Example 3, it is a figure which shows the hue value of all the pixels which comprise the blue signal of FIG.7 (c). In Example 4, it is (a) the still image containing a yellow signal, and (b) the enlarged image which shows a part of extracted image of Fig.12 (a). In Example 4, it is a figure which shows the hue value of all the pixels which comprise the yellow signal of FIG.12 (b). In Example 4, it is (a) the still image containing a red signal, and (b) the enlarged image which shows a part of extracted image of Fig.14 (a). In Example 4, it is a figure which shows the hue value of all the pixels which comprise the red signal of FIG.14 (b). In Example 5, it is a graph which shows the extraction rate of a light source about each sample moving image. In Example 6, it is three still images used for verification. In Example 6, it is an extraction image at the time of performing only a pixel extraction process. In Example 6, it is an extraction image at the time of performing a filtering process and a pixel extraction process. In Example 6, it is an extraction image at the time of performing a filtering process, a pixel extraction process, and a shape selection process. In Example 7, (a) the center point is plotted when passing in front of the green signal at a constant speed, and (b) the center point is plotted when the vehicle is gradually decelerated and stopped before the red signal. FIG.

  Hereinafter, embodiments of a signal lamp identification program and a signal lamp identification apparatus according to the present invention will be described with reference to the drawings.

  FIG. 1 is a block diagram showing a signal lamp identification system 1 of the present embodiment. Mainly, a photographing apparatus 2 for photographing an image including a signal lamp and a signal lamp identification for identifying a signal lamp in an image photographed by the photographing apparatus 2. The apparatus 3 includes an output device 4 that outputs various images and sounds based on the identification processing result of the signal light identification device 3. In the present invention, the signal lamp is a concept including the signal lamps of all traffic signals in addition to the signal lamps of automobile traffic signals composed of a blue signal, a yellow signal and a red signal.

  The photographing device 2 is configured by a digital video camera or the like that can be mounted on a car, a train, or the like, and photographs an image including a signal light. In the present embodiment, the image capturing apparatus 2 captures a moving image having RGB (RED: red, GREEN: green, BLUE: blue) components as digital information. The photographing device 2 is set so as to photograph the front of the automobile in the traveling direction.

  The output device 4 is for outputting video and audio for assisting driving to drivers such as cars and trains. In this embodiment, the output device 4 includes a liquid crystal display that displays an extracted image of the signal light output from the signal light identification device 3, a speaker that emits an alarm sound or the like when a red signal is detected, and the like. .

  The signal lamp identification device 3 is configured by a vehicle-mounted computer or the like. As shown in FIG. 1, the signal lamp identification program 3a of the present embodiment, various data, and the like are mainly stored, and the storage means. 5 and an arithmetic processing means 6 that acquires various data and performs arithmetic processing. In addition, although the signal light identification device 3 of this embodiment is comprised separately from the imaging device 2 and the output device 4, you may comprise as one or the device which united both. Hereinafter, each constituent means will be described in more detail.

  The storage unit 5 includes a ROM (Read Only Memory), a RAM (Random Access Memory), a hard disk, a flash memory, and the like. The storage unit 5 stores various data and a working area when the arithmetic processing unit 6 performs the calculation. It functions as.

  In the present embodiment, as shown in FIG. 1, the storage unit 5 mainly includes a program storage unit 51, a filtering threshold storage unit 52, and an extraction threshold storage unit 53. Hereinafter, each component will be described in more detail.

  The signal storage identification program 3a of this embodiment is installed in the program storage unit 51. The arithmetic processing means 6 executes the signal lamp identification program 3a, thereby causing the signal lamp identification apparatus 3 of this embodiment to function as each component described later. Note that the usage form of the signal lamp identification program 3a is not limited to the above configuration, but may be stored in a recording medium such as a CD-ROM, and directly started and executed from this recording medium.

  The filtering threshold storage unit 52 stores a filtering threshold for filtering in advance a pixel having an RGB component close to a signal lamp. As will be described later, the signal lamp identification process according to the present invention is mainly a process of identifying a signal lamp after converting an RGB component of a still image into an HSV component (hue, saturation, brightness). However, in the present embodiment, the number of pixels to be converted into the HSV component is reduced by executing the filtering process using the difference between the RGB components in advance, and the processing speed is improved by reducing the conversion processing amount. .

Specifically, in the present embodiment, the following threshold values are set in the filtering threshold value storage unit 52 as the filtering threshold values for the red signal, the yellow signal, and the blue signal.
Red signal: (R−G) + 128 ≧ 255 and (R−B) ≧ 100
Yellow signal: (R−G) +128 <255 and (R−B) ≧ 100
Green light: (R−G) +128 <28 and (B−G) ≦ 0
However, R: R component luminance value G: G component luminance value B: B component luminance value

  In the present embodiment, a value of 128 is added so that the difference value of (R−G) does not become negative. Further, the filtering threshold value is not limited to the above threshold value, and may be appropriately changed according to the performance of the signal light identification device 3 and the required identification accuracy. For example, when the signal light identification device 3 having a high processing speed is used, the filtering threshold value may be set in a wider range.

  The extraction threshold storage unit 53 stores an extraction threshold for extracting a pixel having an HSV component close to a signal lamp. In the present embodiment, the extraction threshold value storage unit 53 includes a threshold value that defines a hue range for extracting pixels having a hue corresponding to a signal light, and a saturation range for extracting pixels having a saturation corresponding to the signal light. A threshold value to be determined and a threshold value to determine a lightness range for extracting pixels having a lightness value corresponding to a signal light are stored.

  Specifically, according to the result of Example 4 described later, the threshold of the hue range for extracting the red signal or the yellow signal is set to −10 degrees or more and 60 degrees or less. Further, according to the result of Example 3 described later, the hue range threshold for extracting the blue signal is set to 150 degrees or more and 180 degrees or less. Further, according to the results of Examples 1 and 2 to be described later, the saturation range threshold for extracting light sources other than the white light source is set to 100 or more, and the brightness range for extracting light sources other than the white light source is set. Is set to 50 or more and less than 225.

  Of the three color signal lights, the blue signal often appears as white due to color saturation. For this reason, in the present embodiment, from the result of Example 1 described later, the extraction threshold storage unit 53 further sets the saturation range threshold for extracting the color-saturated blue light to less than 100, In addition, the threshold value of the brightness range for extracting a blue signal with color saturation is set to 225 or more. In the present embodiment, the color-saturated blue signal includes a white light source. In addition, in the case of shooting conditions where color saturation is unlikely to occur, this threshold need not be set.

  In the present embodiment, the hue, saturation, and lightness constituting the HSV component are called three attributes of color. Among them, the hue is a so-called hue and indicates a characteristic for distinguishing a chromatic color from other colors. The saturation indicates the degree of vividness of the color. Further, the intensity indicates the degree of color brightness.

  The arithmetic processing means 6 is composed of a CPU (Central Processing Unit) or the like, and by executing a signal lamp identification program 3a installed in the storage means 5, as shown in FIG. RGB difference calculation unit 62, filtering unit 63, color component conversion unit 64, pixel extraction unit 65, extracted image creation unit 66, labeling unit 67, center point calculation unit 68, shape selection unit 69, The signal light identification device 3 is caused to function as an optical flow acquisition unit 70, a non-target region selection unit 71, and an extracted image output unit 72. Hereinafter, each component will be described in more detail.

  The still image acquisition unit 61 is for acquiring a still image from a moving image captured by the imaging device 2. In the present embodiment, the still image acquisition unit 61 inputs moving image data from the photographing apparatus 2 and captures a still image from the moving image data at a predetermined time interval.

  The RGB difference calculation unit 62 calculates a difference between RGB components for each pixel constituting the still image. In the present embodiment, the RGB difference calculation unit 62 acquires color information related to RGB components for each pixel constituting the still image acquired by the still image acquisition unit 61. Then, the values of (R−G), (R−B) and (B−G) are calculated as differences between the RGB components.

  The filtering unit 63 obtains a pixel having an RGB component close to that of a signal lamp by filtering in advance. In the present embodiment, the filtering unit 63 acquires the values (R−G), (R−B), and (B−G) between the RGB components calculated by the RGB difference calculation unit 62, and these differences are obtained. The pixels within the range defined by the filtering threshold value in the filtering threshold value storage unit 52 are extracted.

  The reason why the filtering unit 63 uses the difference between the RGB components is that the color information of each pixel changes greatly due to weather or sunlight reflection. That is, by using the difference between the RGB components and excluding the color component that is greatly influenced by the weather from the comparison target, stable filtering processing is enabled. For example, the influence of the weather is reduced by excluding the G component if it is a red signal and excluding the B component if it is a blue signal.

  Further, by using the difference value of (R−G), it is possible to distinguish between a red signal and a yellow signal that are generally difficult to distinguish. That is, the RGB value of the red signal is in the vicinity of (255, 0, 0) and the RBG value of the yellow signal is in the vicinity of (255, 255, 0), but these change depending on the weather or the like. However, the G component is greatly different between the red signal and the yellow signal, and the value of (RG) of each color is found not to change so much depending on the weather, and the value of (RG) is used.

  Note that the filtering process is a process for improving the conversion processing speed from the RGB component to the HSV component as described above. For this reason, when using the signal light identification apparatus 3 which has the arithmetic processing means 6 with a high processing speed, it does not need to perform.

The color component conversion unit 64 is for converting the RGB component of each pixel into an HSV component. In the present embodiment, the color component conversion unit 64 acquires an RGB component for each pixel extracted by the filtering unit 63 and converts it into an HSV component using the following conversion formulas (1) to (3). Yes.
Hue = arctan (I _y / I _x ) (1)
Saturation = √ (I _x ² + I _y ² ) (2)
Intensity = I _z Expression (3)
However,
(R, g, b) is color information (luminance value) obtained from a still image.

  In the present embodiment, the reason for conversion to the HSV component is that the hue component has a value determined by the color. Specifically, among the blue signal, the yellow signal, and the red signal, the yellow signal is a mixed color of red and green, and the blue signal is a mixed color of green and blue. For this reason, in order to extract a yellow signal and a blue signal, two RGB components must be used as parameters, respectively, and it is difficult to specify a color. On the other hand, when converted into the HSV component, various colors can be extracted with high accuracy and easily simply by designating the hue component.

  In the present embodiment, since the filtering process described above is performed, the color component conversion unit 64 performs the conversion process only for the pixels extracted by the filtering unit 63. However, when the filtering process is not executed, the color component conversion unit 64 may acquire the still image directly from the still image acquisition unit 61 and execute the conversion process for all the pixels.

  The pixel extraction unit 65 is for extracting a pixel having an HSV component that matches the signal lamp. In the present embodiment, the pixel extraction unit 65 acquires the hue, saturation, and brightness for each pixel converted by the color component conversion unit 64, and the hue range, saturation, and color stored in the extraction threshold storage unit 53. Refer to the degree range and the brightness range. Then, pixels whose hue is within the hue range, whose saturation is within the saturation range, and whose brightness is within the brightness range are extracted.

  The extracted image creating unit 66 creates an extracted image in which only a pixel region corresponding to a signal lamp is clarified. In the present embodiment, the extracted image creating unit 66 leaves all the pixels extracted by the pixel extracting unit 65 as it is, and creates an extracted image by converting all other pixels to black. The pixels other than the signal light are not limited to black, and may be any color as long as the color can be distinguished from the signal light.

  The labeling unit 67 is for grouping pixels constituting an area corresponding to a signal lamp. In the present embodiment, the labeling unit 67 acquires information on all pixels extracted by the pixel extracting unit 65 in the extracted image created by the extracted image creating unit 66. Of the pixels, all adjacent pixels are labeled and grouped.

  In the present embodiment, labeling is a process of assigning the same label such as a number, a character, and a symbol to all the pixels constituting one signal lamp image. Further, each region grouped by this labeling process is defined as a labeling region. In general, the traffic light is prominently installed, and the periphery of the traffic light is the main body of the traffic light. For this reason, in the extracted image, it can be estimated that the region corresponding to the signal lamp is independent from the surroundings. Therefore, in the present embodiment, the labeling unit 67 can be distinguished from other regions simply by executing a grouping process.

  The center point calculation unit 68 is for calculating the center point of the labeling region. In general, when a signal lamp is photographed, as shown in FIG. 2 (a), the central portion of the signal lamp is often saturated with white color. For this reason, the labeling region in the extracted image has a donut shape with a hole in the center as shown in FIG. Therefore, in the present embodiment, the shape of the labeling region related to the signal light is determined regardless of the color saturated portion by using the center point as a reference.

Specifically, the center point calculation unit 68 first calculates coordinates (x _c , y _c ) serving as the center point for each labeling region grouped by the labeling unit 67 using the following calculation formula.
Here, N is the number of pixels constituting the edge (edge) of the labeling region, and (x _i , y _i ) is the xy coordinates of the i-th pixel among the N pixels constituting the edge of the labeling region. It is.

For example, when the center point is calculated for a labeling area corresponding to a circle as shown in FIG. 3, the pixels constituting the edge of the labeling area are hatched pixels, and N = 12. Then, the center point coordinates (x _c , y _c ) are calculated by dividing the sum of the x and y coordinates of each 12 pixel by 12.

Next, the center point calculation unit 68 acquires the xy coordinates of the four pixels that are the upper and lower ends and the left and right ends of the labeling region, and the coordinate X that is the center in the horizontal direction and the coordinate that is the center in the vertical direction by the following formula Y is calculated.
X = (x coordinate of right end + x coordinate of left end) / 2
Y = (y coordinate of upper end + y coordinate of lower end) / 2
When the error σ between the coordinates (X, Y) and the center point coordinates (x _c , y _c ) is within a predetermined range, the center point coordinates (x _c , y _c ) are determined to be correct. It has become.

  In the present embodiment, the error σ is set to about 2 pixels. However, the error σ is not limited to this, and may be appropriately changed according to required identification accuracy. In the present embodiment, as described above, the validity of the center point coordinates is calculated, but the processing can be advanced without executing the processing.

  The shape selecting unit 69 is for selecting a labeling region having a shape close to a signal lamp. Although the signal lamp is circular, as shown in FIG. 4A, when the signal lamp (arrow part in FIG. 4A) is photographed from a distance, the shape of the signal lamp in the extracted image is as shown in FIG. As shown, there are many cases where the shape is not a circle but a square. For this reason, in this embodiment, the shape selection part 69 selects the labeling area | region close | similar to circular or square as a signal lamp.

  Specifically, the shape selection unit 69 acquires, for each labeling region, the center point calculated by the center point calculation unit 68 and the position information of the four pixels farthest from the center point in the vertical and horizontal directions. Then, the labeling region in which the upper, lower, left, and right pixels are approximately equidistant from the center point is determined to be circular or square. On the other hand, the other labeling areas are determined not to be signal lights and are converted to black.

Note that the method of selecting the signal lamp by the shape selecting unit 69 is not limited to the above method, and it is also possible to select without using the center point. Specifically, when the labeling region D as shown in FIG. 5 is selected, the shape selection unit 69 determines whether or not the following formula is satisfied.
| (X _max −x _min ) − (y _max −y _min ) | ≦ C
However,
x _max : Maximum value of the x coordinate of the pixels constituting the labeling region x _min : Minimum value of the x coordinate of the pixels constituting the labeling region y _max : Maximum value of the y coordinate of the pixels constituting the labeling region y _min : Labeling region Minimum value of y-coordinates of the pixels constituting the C: constant

  If the above expression is not satisfied, that is, if the difference between the vertical distance and the horizontal distance of the labeling area is large, the labeling area is excluded because it cannot be considered as a circle or a square. As a result, objects with different length-to-width ratios, such as elongated signboards, are excluded, and the identification accuracy is improved. Further, according to this method, the processing speed is improved because the processing for calculating the center point is not required. The constant C may be set as appropriate according to the required sorting accuracy.

  In the present embodiment, the above-described processing by the labeling unit 67, the center point calculating unit 68, and the shape selecting unit 69 is an optional process for improving the identification accuracy by further selecting the labeling region in the extracted image by the shape. For this reason, the labeling unit 67, the center point calculation unit 68, and the shape selection unit 69 do not need to function when sufficient identification accuracy is obtained without executing the shape selection process.

  The optical flow acquisition unit 70 is for acquiring the optical flow of each labeling area. Here, the optical flow indicates a trajectory as to which direction an object moves in a temporally continuous still image. In the present embodiment, the optical flow acquisition unit 70 acquires the extracted images after the shape selection unit 69 performs the selection process in time series. Then, for each labeling region included in each extracted image, the center point calculated by the center point calculation unit 68 is sequentially acquired, and the optical flow that is the trajectory is acquired.

  In addition, the acquisition method of the optical flow by the optical flow acquisition part 70 is not specifically limited, For example, the gradient method, a block matching method, etc. are mentioned.

  The non-target area selection unit 71 is for selecting a labeling area other than the signal lamp based on the optical flow of the labeling area. In the present embodiment, the non-target region selection unit 71 acquires the optical flow acquired by the optical flow acquisition unit 70, and determines whether or not the optical flow can be approximated to a predetermined trajectory. As a result, when the approximation is possible, it is determined that the labeling area having the optical flow is a signal lamp. On the other hand, if it cannot be approximated, the labeling area is determined to be a tail lamp or a brake lamp other than the signal lamp, and is converted to black.

  In the present embodiment, based on the result of Example 7 to be described later, the non-target region selection unit 71 determines that it is a signal lamp when the optical flow can be approximated to a straight line or a curve.

  In the present embodiment, the above-described processing by the optical flow acquisition unit 70 and the non-target region selection unit 71 is an optional process that improves the identification accuracy by further selecting the labeling region in the extracted image by its movement locus. For this reason, the optical flow acquisition unit 70 and the non-target region selection unit 71 do not need to function when sufficient identification accuracy is obtained without executing the processing.

  The extracted image output unit 72 is for outputting an extracted image from which pixels matching the signal lamp are extracted to the output device 4. In the present embodiment, the extracted image output unit 72 acquires the extracted image subjected to the sorting process by the non-target region sorting unit 71 and outputs it to the output device 4. When the optical flow process is not executed, the extracted image output unit 72 acquires an extracted image from the shape selecting unit 69. When the shape selection process and the optical flow process are not executed, the extracted image output unit 72 acquires an extracted image directly from the extracted image creation unit 66.

  Next, the operation of the signal lamp identification apparatus 3 executed by the signal lamp identification program 3a of the present embodiment and the signal lamp identification system 1 including the signal lamp identification apparatus 3 will be described with reference to FIG.

  When the signal lamp in the moving image is identified in real time using the signal lamp identification system 1 of the present embodiment, first, the still image acquisition unit 61 acquires a still image from the moving image captured by the imaging device 2 (step) S1).

  Next, after the RGB difference calculation unit 62 calculates the difference between the RGB components for each pixel constituting the still image (step S2), the filtering unit 63 determines the difference between the RGB components as a filtering threshold value. Pixels within the specified range are extracted (step S3).

  Thereby, since the pixel which has RGB component close | similar to a signal lamp is extracted previously, the pixel number which carries out the conversion process by the following step S4 reduces, and processing speed improves. Further, since the difference between the RGB components is used so as to exclude the color component greatly influenced by the weather from the comparison target, the filtering process is stabilized. Furthermore, by using the difference between the RGB components, the discrimination accuracy between the red signal and the yellow signal is improved.

  Subsequently, the color component conversion unit 64 acquires an RGB component for each pixel filtered in step S3 and converts it into an HSV component (step S4). Thereby, since the color distribution in the hue becomes uniform, it becomes easy to accurately extract all colors only by designating the hue threshold. Further, since the histogram is distributed in a certain region regardless of the weather, time zone, etc., the light source is stably extracted. Furthermore, in this embodiment, since the filtering process is executed in step S3, the processing speed of this step S4 is improved.

  Next, the pixel extraction unit 65 acquires an HSV component for each pixel converted in step S4, and extracts a pixel in which each component is within the range defined by the extraction threshold (step S5). As a result, pixels corresponding to the hue, saturation, and brightness of the blue signal, yellow signal, and red signal are extracted. In the present embodiment, since the blue signal with color saturation is also extracted separately, the extraction rate of the blue signal is improved.

  The above steps S2 to S5 are processes executed on a pixel basis. For this reason, the processing load applied to the arithmetic processing means 6 is small, and even a moving image with a high frame rate can be processed in real time.

  Next, the extracted image creation unit 66 leaves the pixels extracted in step S5, while creating the extracted image by converting other pixels to black (step S6). As a result, the extracted image is clear because only the signal lamp stands out against the black background. For this reason, even an elderly person or a low vision person can notice the presence of a signal quickly and easily.

  Next, in the present embodiment, in order to further improve the accuracy of the extracted image created in step S6, the following steps S7 to S11 are executed. First, the labeling unit 67 acquires information on all pixels extracted in the extracted image, and labels all adjacent pixels (step S7). As a result, the extracted image is grouped for each pixel area corresponding to one object.

  Subsequently, the center point calculation unit 68 calculates center point coordinates for each labeling region grouped in step S7 (step S8). Thereby, when determining the shape of the signal lamp in the next step S9, the center point of the labeling area can be used as a reference. In step S10 described later, an optical flow is easily acquired. Furthermore, in the present embodiment, the center point is accurately calculated in order to check whether the center point coordinates calculated by the above calculation formula are correct.

  Next, the shape selection unit 69 acquires the center point calculated in step S8 and the position information of the peripheral pixel for each labeling region, and selects a labeling region having a shape close to a signal lamp (step S9). . As a result, light sources having shapes different from signal lights such as streets and neon signs are excluded, and the identification accuracy is improved. In addition, since the selection is performed based on the center point, the shape of the signal lamp having the color saturation can be accurately determined.

  Subsequently, the optical flow acquisition unit 70 sequentially acquires the extracted images subjected to the sorting process in step S9, and acquires an optical flow for the center point of each labeling region (step S10). Then, based on the optical flow acquired in step S10, the non-target area selection unit 71 selects a labeling area other than the signal lamp (step S11). This effectively eliminates the taillights, brake lights, etc. of automobiles that are difficult to distinguish from signal lights, and improves the identification accuracy.

  The extracted image obtained by the processes in steps S1 to S11 is output to the output device 4 by the extracted image output unit 72 (step S12). Thereby, the extracted image which extracted only the signal lamp is displayed on a vehicle-mounted liquid crystal display or the like. Further, based on the extracted image, a warning sound or warning announcement is appropriately issued from a speaker or the like. Thereby, even an elderly person or a low vision person easily recognizes the presence of a signal and its signal color. Moreover, since the extracted image is output in real time, it is suitable for assisting driving of a car or a train.

According to the present embodiment as described above, the following effects can be obtained.
1. Regardless of the weather, time zone, etc., it is possible to identify the signal lamp in the captured image with high accuracy and in real time.
2. It is not necessary to prepare data for identifying a signal lamp in advance, and a signal lamp identification system that can be easily spread at low cost can be realized.
3. An appropriate threshold value can be set to increase the extraction rate of the blue signal in which color saturation has occurred.
4). By pre-filtering pixels close to the signal light, the overall processing speed can be improved.
5. A labeling area having a shape close to that of a signal lamp can be selected to improve identification accuracy.
6). It is possible to improve the identification accuracy by removing tail lamps, brake lamps and the like that are difficult to identify.

  Next, a specific embodiment of the signal lamp identification system 1 including the signal lamp identification program 3a and the signal lamp identification apparatus 3 according to the present invention will be described. The technical scope of the present invention is not limited to the features shown by the following examples.

"Saturation range and lightness range selection experiment"
In Example 1, an experiment was performed to select an optimum saturation range and lightness range for extracting a signal lamp. Specifically, first, in the still image shown in FIG. 7A, a region including a blue signal (region surrounded by a white frame) is cut out, and the relationship between saturation and lightness is graphed for each pixel in this region. did. The result is shown in FIG.

  Subsequently, in FIG. 7B, the saturation and lightness threshold values were set to various values, and pixels within the range defined by the threshold values were extracted. As a result, when pixels included in the light source region 1 having a saturation of 100 or more and a lightness of 50 or more and less than 225 are extracted, as shown in FIG. Only the peripheral edge corresponding to the green light was extracted without being extracted.

  On the other hand, when only pixels included in the light source region 2 having a saturation of less than 100 and a lightness of 225 or more are extracted, as shown in FIG. . Further, among other pixels constituting the background, only pixels included in the light source region 3 (the elliptical region in FIG. 7B) having a saturation of 50 or more and less than 100 and a lightness of less than 225 are extracted. In this case, as shown in FIG. 7E, a ring region that is slightly larger than the outer peripheral edge of the signal lamp was extracted.

  On the other hand, an experiment similar to the above was performed for the red signal. Specifically, first, a region shown in FIG. 8A was cut out from a still image including a red signal, and the relationship between saturation and lightness was graphed for each pixel in this region. The result is shown in FIG.

  In FIG. 8B, saturation and lightness threshold values were set to various values, and pixels within a range defined by the threshold values were extracted. As a result, when pixels included in the light source region 1 with a saturation of 100 or more and a lightness of 50 or more and less than 225 are extracted, the background is not extracted and only the red signal is displayed as shown in FIG. Extracted.

  According to the first embodiment as described above, the optimum saturation range for extracting a light source other than a white light source such as a blue signal, a yellow signal, and a red signal is 100 or more, and a light source other than a white light source is extracted. It has been shown that the optimal brightness range for is greater than or equal to 50 and less than 225. Further, it was shown that the optimum saturation range for extracting the color-saturated blue signal is less than 100, and the optimum brightness range for extracting the color-saturated blue signal is 225 or more.

"Experiment to confirm the validity of saturation range and lightness range"
In the second embodiment, an experiment for confirming the validity of the saturation range and the lightness range defined in the first embodiment was performed. Specifically, as shown in FIG. 9, first, 10 sample moving images with different shooting seasons, weathers, and time zones are prepared, and 118 still images in which a red signal is reflected from each moving image are captured. did. Then, color conversion processing was executed for each still image, and the histogram distribution of HSV components was graphed.

  FIGS. 10A and 10B show histogram distributions of the saturation component and the brightness component, respectively. In addition, the horizontal axis of Fig.10 (a) has shown saturation (0-255), and the horizontal axis of FIG.10 (b) has shown lightness (0-255). In addition, the vertical axis in FIGS. 10A and 10B indicates the ratio (%) at which the histogram value is included in the pixels in the light source region.

  As shown in FIG. 10A, it can be confirmed that the saturation component of the red signal is distributed over 100. Further, as shown in FIG. 10B, it can be confirmed that the brightness component of the red signal is distributed in the range of 50 to less than 225. In the second embodiment, the red signal is verified. However, if the light source is other than the white light source, it is estimated that the same result can be obtained for the blue signal and the yellow signal.

  According to the second embodiment as described above, the saturation range and brightness range threshold values defined in the first embodiment are shown to be appropriate as threshold values for extracting light sources other than the white light source.

"Selection experiment of hue range to extract blue light"
In Example 3, an experiment for selecting an optimum hue range for extracting a blue signal was performed. Specifically, the still image shown in FIG. 7A is used as a still image in which a blue signal is lit, and pixels having a saturation of 100 or more and a brightness of 50 or more and less than 225 are extracted as in the first embodiment. did. And the hue value was investigated about all the pixels which comprise the said extraction image (FIG.7 (c)). The result is shown in FIG.

  As shown in FIG. 11, it can be confirmed that the hue value of the pixel corresponding to the blue signal does not exceed 180 degrees. Further, it can be seen that there are only 7 pixels having a hue value of less than 150. The hue value of each pixel is a value obtained by rounding off the decimal part. A pixel having a hue value of 0 is a pixel that has been converted to black as being other than a blue signal.

  According to the third embodiment as described above, it has been shown that the optimum hue range for extracting the blue signal is 150 degrees or more and 180 degrees or less.

"Selection experiment of hue range to extract yellow signal or red signal"
In Example 4, an experiment was conducted to select an optimum hue range for extracting a yellow signal or a red signal. Specifically, as shown in FIG. 12A, first, pixels having a saturation of 100 or more and a brightness of 50 or more and less than 225 were extracted from a still image in which a yellow signal is lit. Then, an area including a yellow signal as shown in FIG. 12B was cut out from the extracted image, and the hue value was examined for all the pixels constituting the cut-out area. The result is shown in FIG.

  As shown in FIG. 13, it can be confirmed that the hue value of the pixel corresponding to the yellow signal does not exceed 60 degrees. Further, no pixel having a hue value of less than 10 was present.

  Similarly, as shown in FIG. 14A, pixels having a saturation of 100 or more and a brightness of 50 or more and less than 225 are extracted from a still image in which a red signal is lit. Then, a region including a red signal as shown in FIG. 14B was cut out from the extracted image, and the hue value was examined for all the pixels constituting this cut-out region. The result is shown in FIG.

  As shown in FIG. 15, it can be confirmed that the hue values of the pixels corresponding to the red signal do not exceed 60 degrees. In addition, there were only two pixels having a hue value of less than −10. In FIG. 13 and FIG. 15, the hue value of each pixel is a value rounded off after the decimal point. A pixel having a hue value of 0 is a pixel that has been converted to black as being other than a signal light.

  According to Example 4 as described above, the optimum hue range for extracting the red signal or the yellow signal is shown to be −10 degrees or more and 60 degrees or less.

"Confirmation experiment of extraction rate"
In Example 5, an experiment was performed to confirm the extraction rate based on the extraction threshold values defined in Examples 1 to 4. Specifically, with respect to the 10 sample moving images used in Example 2, the light source extraction process (steps S1 to S6 above) using the signal lamp identification device 3 executed by the signal lamp identification program 3a of the present embodiment. Was executed. And about all the process frames, it confirmed visually and calculated the ratio from which the signal lamp was extracted in the extraction image. The result is shown in FIG.

  As shown in FIG. 16, the average value of the extraction rates for the blue, yellow, and red signals as a whole was 82.33%. In addition, when only the most important red signal was identified for identifying the signal lamp, the average value of the extraction rate was identified with a high accuracy of 95.32%. Of the sample moving images, signals 069 and signal 027 have frames that are extremely difficult to visually recognize the signal light.

  On the other hand, when verifying a frame that failed to be extracted, in most cases, the shooting distance was too far and the size of the signal lamp was not sufficient. For this reason, it is thought that said extraction rate fully satisfy | fills the extraction precision required practically.

  In addition, in the sample moving image at night (signal034, signal112), the blue signal is particularly saturated in color, and cannot be extracted as a light source, resulting in a decrease in extraction rate. However, as far as the red signal is concerned, a high extraction rate is obtained as described above, and it is very rare that the entire area of the red signal is saturated. For this reason, the above steps S7 to S11 are not executed, and this processing (step S1 to step S6) prioritizing the processing speed is at a level that can be sufficiently put into practical use. In addition, it is considered that the extraction rate is improved by using a threshold value for extracting the blue signal saturated in color as defined in the first embodiment.

  According to the fifth embodiment as described above, when a light source is extracted using the signal lamp identification device 3 executed by the signal lamp identification program 3a of the present embodiment, a high extraction rate can be obtained and at a level that can be sufficiently put into practical use. It was shown that there is.

"Verification experiment of filtering processing and shape selection processing"
In Example 6, the effect of the filtering process and the shape selection process on the identification result was verified. Specifically, with respect to the three still images shown in FIGS. 17A to 17C, when only pixel extraction processing based on the extraction threshold is performed, filtering processing based on the filtering threshold is performed together, and labeling An extracted image was output when the shape selection process was further executed. The extracted images in these cases are shown in FIGS.

  As shown in FIG. 18, when only pixel extraction processing is executed, signs and sky other than signal lights are also extracted, but signal lights are also extracted without omission. Further, as shown in FIG. 19, when the filtering process is executed before the pixel extraction process, the ratio of erroneously extracting signboards and sky other than the signal lights is suppressed as compared with FIG. Furthermore, as shown in FIG. 20, when the filtering process, the pixel extraction process, and the shape selection process are executed, there is no false extraction of a signboard or the sky, and only one tail lamp is erroneously extracted in FIG. It was.

  In FIG. 20, the area other than the signal lamp is white, and only the center point of the extracted labeling area is displayed. In FIG. 20, a small labeling region is intentionally removed from the viewpoint of easy viewing. For this reason, in FIG.20 (c), although the signal is not displayed, extraction itself is performed. Furthermore, it is considered that the tail lamp extracted in FIG. 20C is also removed by executing the optical flow process.

  According to the sixth embodiment as described above, it has been shown that even when only the pixel extraction processing according to the present invention is executed, an extracted image capable of identifying the signal lamp can be obtained. Moreover, it was shown that when the filtering process and the shape selection process are executed together, the identification accuracy is improved.

"Verification experiment of sorting process based on optical flow"
In Example 7, an experiment was conducted to verify the effectiveness of the sorting process based on the optical flow. Specifically, the trajectory was investigated by calculating and plotting the center point of the labeling area for each frame of a moving image in which a signal light was photographed sufficiently large. The result is shown in FIG.

  As shown in FIG. 21 (a), when passing in front of the green signal at a constant speed, it was confirmed that the center points appear at almost equal intervals, and the trajectory draws a straight line or a gentle curve. Further, as shown in FIG. 21B, when the vehicle is decelerated and stopped before the red signal, the distance between the center points gradually narrows, but the trajectory is substantially linear or gently curved. It was confirmed.

  As described above, since the traffic light is a fixed object, the central point of the signal light is considered to draw a locus that can be approximated to a straight line or a gentle curve. Further, the state of the vehicle (acceleration, deceleration, stop, etc.) can also be estimated based on the distance between the center points. Therefore, by examining the appearance tendency of the optical flow, it can be said that it is effective in removing tail lamps, brake lamps, and the like that are difficult to distinguish from signal lights only by color.

  According to the seventh embodiment as described above, when the optical flow can be approximated to a straight line or a curve, it is indicated that it is appropriate to determine that the light is a signal lamp.

"Real-time processing performance verification experiment"
In Example 8, an experiment was performed to verify the processing performance in real time for the signal lamp identification device 3 executed by the signal lamp identification program 3a of the present embodiment. Specifically, using a personal computer (CPU: Intel Pentium (registered trademark) 4 3 GHz, memory: 504 MB RAM), the above-described series of processing from step S1 to step S8 was performed every three frames. As a result, the average processing speed was 27 [fps].

  Considering that the analog television frame rate is currently about 30 [fps], it can be said that it is sufficiently possible to execute the identification processing according to the present invention in real time. Further, by reducing the number of frames to be processed, new processing can be added separately.

  According to the present Example 8 as described above, it has been shown that the signal lamp identification device 3 executed by the signal lamp identification program 3a of the present embodiment can identify a signal lamp in real time from a moving image.

  The signal lamp identification program 3a and the signal lamp identification apparatus 3 according to the present invention are not limited to the above-described embodiments and examples, and can be changed as appropriate.

  For example, in the above-described embodiment, a signal lamp in which color saturation has occurred is displayed in the extracted image in a donut shape as extracted. However, if real-time processing can be ensured, the pixel in the hollow portion that has become a donut shape may be separately converted into the same hue as the surroundings. As a result, the signal color of the extracted image is easily understood by the user, and the visibility is further improved.

DESCRIPTION OF SYMBOLS 1 Signal lamp identification system 2 Imaging device 3 Signal lamp identification apparatus 3a Signal lamp identification program 4 Output device 5 Memory | storage means 6 Arithmetic processing means 51 Program memory | storage part 52 Filtering threshold value memory | storage part 53 Extraction threshold value memory | storage part 61 Still image acquisition part 62 RGB difference calculation Unit 63 Filtering unit 64 Color component conversion unit 65 Pixel extraction unit 66 Extracted image creation unit 67 Labeling unit 68 Center point calculation unit 69 Shape selection unit 70 Optical flow acquisition unit 71 Non-target region selection unit 72 Extracted image output unit

Claims (14)

Signal light identification device
A still image acquisition unit for acquiring a still image from a moving image;
A color component conversion unit that converts RGB components (red, green, blue) of each pixel constituting the still image into HSV components (hue, saturation, brightness) using the following conversion equations (1) to (3) When,
To extract a threshold value for determining a hue range for extracting a pixel of hue corresponding to a signal light, a threshold value for determining a saturation range for extracting a pixel of saturation corresponding to the signal light, and a pixel of lightness corresponding to a signal light An extraction threshold value storage unit for storing a threshold value for determining the brightness range of
The hue converted by the color component conversion unit is within the hue range, the saturation converted by the color component conversion unit is within the saturation range, and the lightness converted by the color component conversion unit is within the lightness range. A pixel extractor for extracting pixels in
A signal lamp identification program that functions as an extracted image output unit that outputs an extracted image including pixels extracted by the pixel extracting unit to an output device.
Hue = arctan (I _y / I _x ) (1)
Saturation = √ (I _x ² + I _y ² ) (2)
Intensity = I _z Expression (3)
However,
(R, g, b) is color information (luminance value) obtained from a still image.   In the extraction threshold storage unit, a hue range threshold for extracting a red signal or a yellow signal is set to -10 degrees or more and 60 degrees or less, and a hue range threshold for extracting a blue signal is 150. The saturation range threshold for extracting light sources other than the white light source is set to 100 or more, and the brightness range threshold for extracting light sources other than the white light source is set. The signal lamp identification program according to claim 1, wherein the signal lamp identification program is set to 50 or more and less than 225.   In the extraction threshold storage unit, a saturation range threshold for extracting a color-saturated blue signal is set to less than 100, and a brightness range threshold for extracting a color-saturated blue signal is 225 or more. The signal lamp identification program according to claim 2, wherein A filtering threshold storage unit that stores a filtering threshold for pre-filtering pixels having RGB components close to a signal light before the color component conversion unit performs the conversion process;
For each pixel constituting the still image, an RGB difference calculation unit that calculates a difference between RGB components;
The difference calculated by the RGB component calculation unit functions as a filtering unit that acquires and filters pixels within the range defined by the filtering threshold,
4. The signal lamp identification program according to claim 1, wherein the filtering threshold storage unit is set with the following thresholds as filtering thresholds for a red signal, a yellow signal, and a blue signal. 5.
Red signal: (R−G) + 128 ≧ 255 and (R−B) ≧ 100
Yellow signal: (R−G) +128 <255 and (R−B) ≧ 100
Green light: (R−G) +128 <28 and (B−G) ≦ 0
However, R: R component luminance value G: G component luminance value B: B component luminance value Among the pixels extracted by the pixel extraction unit, a labeling unit that labels and groups all adjacent pixels;
A center point calculation unit for calculating a center point of a labeling region grouped by the labeling unit;
Based on the positional relationship between the center point calculated by the center point calculation unit and the pixels constituting the peripheral portion of the labeling region, the signal light identification device functions as a shape selection unit that selects a labeling region having a shape close to the signal light. The signal lamp identification program according to any one of claims 1 to 4. A labeling unit that labels and groups all adjacent pixels among the pixels extracted by the pixel extraction unit;
The signal light identification device is made to function as a shape selection unit for selecting a labeling region having a shape different from that of the signal light based on a difference between a vertical distance and a horizontal distance of the labeling region grouped by the labeling unit. Item 5. A signal lamp identification program according to any one of Items 4 to 6. A labeling unit that labels and groups all adjacent pixels among the pixels extracted by the pixel extraction unit;
A center point calculation unit for calculating a center point of a labeling region grouped by the labeling unit;
An optical flow acquisition unit that acquires the optical flow of the center point calculated by the center point calculation unit;
The signal lamp according to any one of claims 1 to 4, wherein the signal lamp identification device functions as a non-target area selection section that selects a labeling area other than the signal lamp based on the optical flow acquired by the optical flow acquisition section. Identification program. A still image acquisition unit for acquiring a still image from a moving image;
A color component conversion unit that converts RGB components (red, green, blue) of each pixel constituting the still image into HSV components (hue, saturation, brightness) using the following conversion equations (1) to (3) When,
To extract a threshold value for determining a hue range for extracting a pixel of hue corresponding to a signal light, a threshold value for determining a saturation range for extracting a pixel of saturation corresponding to the signal light, and a pixel of lightness corresponding to a signal light An extraction threshold value storage unit for storing a threshold value for determining the brightness range of
The hue converted by the color component conversion unit is within the hue range, the saturation converted by the color component conversion unit is within the saturation range, and the lightness converted by the color component conversion unit is within the lightness range. A pixel extractor for extracting pixels in
A signal lamp identification device comprising: an extracted image output unit that outputs an extracted image composed of pixels extracted by the pixel extraction unit to an output device.
Hue = arctan (I _y / I _x ) (1)
Saturation = √ (I _x ² + I _y ² ) (2)
Intensity = I _z Expression (3)
However,
(R, g, b) is color information (luminance value) obtained from a still image.   In the extraction threshold storage unit, a hue range threshold for extracting a red signal or a yellow signal is set to -10 degrees or more and 60 degrees or less, and a hue range threshold for extracting a blue signal is 150. The saturation range threshold for extracting light sources other than the white light source is set to 100 or more, and the brightness range threshold for extracting light sources other than the white light source is set. The signal light identification device according to claim 8, wherein the signal light identification device is set to 50 or more and less than 225.   In the extraction threshold storage unit, a saturation range threshold for extracting a color-saturated blue signal is set to less than 100, and a brightness range threshold for extracting a color-saturated blue signal is 225 or more. The signal light identification device according to claim 9, which is set as follows. A filtering threshold storage unit that stores a filtering threshold for pre-filtering pixels having RGB components close to a signal light before the color component conversion unit performs the conversion process;
For each pixel constituting the still image, an RGB difference calculation unit that calculates a difference between RGB components;
A filtering unit that obtains the difference calculated by the RGB component calculation unit by filtering pixels within a range defined by the filtering threshold;
The signal light identification device according to any one of claims 8 to 10, wherein the filtering threshold storage unit is set with the following thresholds as filtering thresholds for a red signal, a yellow signal, and a blue signal.
Red signal: (R−G) + 128 ≧ 255 and (R−B) ≧ 100
Yellow signal: (R−G) +128 <255 and (R−B) ≧ 100
Green light: (R−G) +128 <28 and (B−G) ≦ 0
However, R: R component luminance value G: G component luminance value B: B component luminance value A labeling unit that labels and groups all adjacent pixels among the pixels extracted by the pixel extraction unit;
A center point calculation unit for calculating a center point of a labeling region grouped by the labeling unit;
A shape selection unit that selects a labeling region having a shape close to a signal lamp based on a positional relationship between the center point calculated by the center point calculation unit and pixels that form a peripheral portion of the labeling region. The signal light identification device according to any one of claims 8 to 11. A labeling unit that labels and groups all adjacent pixels among the pixels extracted by the pixel extraction unit;
12. The shape selection unit for selecting a labeling region having a shape different from that of a signal light based on a difference between a vertical distance and a horizontal distance of the labeling region grouped by the labeling unit. The signal light identification device according to any one of the above. A labeling unit that labels and groups all adjacent pixels among the pixels extracted by the pixel extraction unit;
A center point calculation unit for calculating a center point of a labeling region grouped by the labeling unit;
An optical flow acquisition unit that acquires the optical flow of the center point calculated by the center point calculation unit;
The signal light identification device according to any one of claims 8 to 11, further comprising: a non-target region selection unit that selects a labeling region other than the signal light based on the optical flow acquired by the optical flow acquisition unit.