JP4423524B2 - Label detection apparatus, label detection method, and recording medium - Google Patents

Description

[0001]
BACKGROUND OF THE INVENTION
The present invention relates to a sign detection device and a sign detection method suitable for use in detecting road signs and using them for alerting a driver or for automatic vehicle control, for example.
[0002]
[Prior art]
A video camera mounted on the vehicle can be used to capture the road and its surroundings in the direction of travel of the vehicle, recognize road signs, etc. from the captured image, and use it to alert the driver or automatically control the vehicle (For example, refer to Japanese Patent Publication No. 6-52554, Japanese Patent Laid-Open No. 9-185703, Japanese Patent Laid-Open No. 11-73598, etc.).
[0003]
The road sign detection method proposed in the above-mentioned publications and the like basically uses pattern matching. For example, in the invention described in Japanese Patent Application Laid-Open No. 9-185703, a recognition method based on pattern matching in which a road sign has a specific combination of colors and a sign shape is used.
[0004]
[Problems to be solved by the invention]
However, in the case of a recognition method using pattern matching, it is necessary to prepare a template for the sign itself in advance, which is very troublesome. Moreover, every time a road sign is newly added, it is necessary to add the sign to the template, which is troublesome.
[0005]
In addition, measures against pattern matching defects caused by the movement of the vehicle are not taken into account in conventional sign detection methods such as the above-mentioned publications. In other words, as the scale of the road sign to be captured changes as the vehicle progresses, or the sign is tilted and captured, the recognition rate deteriorates if only one template is prepared for each road sign. To do. In order to improve this problem and recognize it well, it is necessary to prepare templates at every scale and every inclination for each road sign, but in practice it is not possible to prepare such a large number of templates. It is almost impossible, and even if it can be prepared, the recognition apparatus is increased in scale and cost, and the recognition speed is reduced.
[0006]
In view of the above points, the present invention provides a method and apparatus capable of successfully recognizing a sign even if the scale of the sign to be detected is changed or the sign is tilted as the vehicle moves. Is intended to provide.
[0007]
[Means for Solving the Problems]
In order to solve the above problems, a label detection apparatus according to the present invention provides:
Specific color area cutout means for cutting out an area having a specific color from the input color image;
An area dividing unit that divides the input color image into grid-like areas by dividing the input color image in the vertical direction and the horizontal direction so as to surround the area cut out by the specific color area cutting out unit,
For each divided area divided by the area dividing means, a sign area determination means for determining whether or not the area is a sign area that is a part of a sign,
In the label area determined by the label area determination means, label information detection means for detecting significant information inside the label area as label information,
It is characterized by providing.
[0008]
Further, the label detection method according to the present invention includes:
A specific color area cutout step of cutting out an area having a specific color from the input color image;
A region dividing step of dividing the input color image into a grid-like region by dividing the input color image in the vertical direction and the horizontal direction so as to surround the region cut out in the specific color region cutting step,
For each divided region divided in the region dividing step, a labeled region determination step for determining whether or not the region is a labeled region that is a portion of a marker,
In the label area determined in the label area determination step, a label information detection step for detecting significant information therein as label information,
It is characterized by providing.
[0009]
In the label detection apparatus and method according to the present invention having the above-described configuration, first, a region having a specific color used as a label is cut out from the input color image. In the extraction of the specific color region, it is not necessary to accurately extract only the label portion.
[0010]
Next, the input color image is divided in the vertical direction and the horizontal direction so as to surround the cut out region, and the input color image is divided into a lattice-like region. As described above, the cutout of the specific color area includes not only the label part but also other parts. Therefore, the divided grid area includes an area including the label part and an area not including the label part. Including.
[0011]
Therefore, next, for each grid-like divided region, it is determined whether the region is a marker region that is a marker part or another region. Then, as a result of this determination, in the label area determined to be a label portion, information that can be significant as a label, for example, is detected as label information.
[0012]
As described above, according to the present invention, the sign can be recognized regardless of the scale and inclination of the taken sign.
[0013]
DETAILED DESCRIPTION OF THE INVENTION
Hereinafter, embodiments of a sign detection device according to the present invention will be described with reference to the drawings in the case of a road sign detection device mounted on a vehicle.
[0014]
FIG. 2 shows an example of the entire system configuration of the sign detection apparatus of this embodiment, and this embodiment is an example when the system is mounted on a vehicle.
[0015]
In this embodiment, the video camera 1 is mounted on a vehicle so as to photograph a road in the forward direction of the vehicle and its surroundings. Color image information obtained by photographing with the video camera 1 is converted into a digital signal by the A / D converter 2 and supplied to the arithmetic processing unit 3.
[0016]
The arithmetic processing unit 3 includes a microcomputer. A CPU (Cetral Processing Unit) 31 and a ROM (Read Only Memory) 32 in which information such as programs and characters are stored are stored in the system bus 30. A work area RAM (Random Access Memory) 33, an image memory for image data 34M, an arithmetic image memory 34B, a display controller 35, a hard disk interface 36, and a CD-ROM (Compact Disc Read). An only memory) drive interface 38 is connected.
[0017]
A hard disk device 37 is connected to the hard disk interface 36, and a CD-ROM drive device 39 is connected to the CD-ROM drive interface 38. Further, the display 4 is connected to the display controller 35.
[0018]
A program for road sign detection processing to be described later can be stored in the hard disk device 37 in advance. Further, a CD-ROM in which the program is recorded can be loaded into a CD-ROM drive device 38, and the program can be read out from the CD-ROM and used. It is also possible to write on the hard disk of the device 37 for use.
[0019]
Further, although not shown, a communication interface connected to, for example, a mobile phone is connected to the system bus 30, the program is downloaded through the mobile phone, and the downloaded program data is written to the hard disk of the hard disk device for use. It can also be done.
[0020]
The color image data converted into a digital signal by the A / D converter 2 is taken into the image memory 34M of the arithmetic processing unit 3. Incorporation of color image data into the image memory 34M is controlled by the CPU 31. Color image data is taken into the image memory 34M as still image data at a time interval that takes into account the time required for road sign detection processing, which will be described later. Then, the arithmetic processing unit 3 performs a road sign detection process on the still image taken into the image memory 34M and displays the detection result on the display 4 to notify the driver.
[0021]
As shown in FIG. 1, the arithmetic processing unit 3 is functionally composed of a specific color region cutout unit 11, a region division unit 12, a marker region determination unit 13, and a marker information detection unit 14. The region dividing unit 12 includes a horizontal / vertical frequency distribution calculating unit 15, a change point detecting unit 16, and a grid-like region dividing unit 17. Each of these units corresponds to each step in the road sign detection method. Hereinafter, the configuration and processing operation of each unit will be described.
[0022]
[Specific Color Region Extraction Unit 11]
The color image data of the still image read from the image memory 34M is supplied to the specific color area cutout unit 11. The specific color area cutout unit 11 performs processing using the image memory 34B for calculation.
[0023]
Road signs use only certain specific colors, such as blue, red, yellow, black, and white, and there is no change in the level of these colors. The clipping unit 12 extracts these specific color areas from the color still image captured in the image memory 34M, and writes the extraction result in the image memory 34B.
[0024]
For example, when the color image signal is composed of three primary color signals of R (red), G (green), and B (blue), the specific color region is extracted using the R, G for detecting the specific color. , B are set in advance, and using these thresholds, it is determined whether each pixel unit or a plurality of pixel units of the color still image signal in the image memory 34 is in the specific color region. This is done by performing a determination process.
[0025]
In this case, the extracted pixel data of the specific color area is extracted as it is, but the pixel data of the areas other than the specific color area are all set to 0 for each of the R, G, and B values. To.
[0026]
As described above, since the colors used as road signs are limited and there is no fine level change in these colors, the number of thresholds prepared in advance can be limited.
[0027]
Further, when it is intended to accurately extract only the sign portion, the threshold values of the R, G, and B values for detecting a specific color region may be changed according to changes in ambient brightness. Although it is necessary, in this embodiment, as described later, since it is a two-stage process including a process of determining whether or not each area is a sign portion and removing an area other than the sign, the R , G, B threshold values are not strictly set, and there is no problem even if an area other than a sign is cut out by this specific color area cut-out process. That is, it is not necessary to finely set a threshold value for cutting out a specific color area according to a change in brightness or the like.
[0028]
For example, if the input color still image captured in the image memory 34M is as shown in FIG. 3, the processing by the specific color area cutout unit 11 causes the calculation image memory 34B to store The labeled region MS as shown in FIG. 4 is almost cut out and stored.
[0029]
[Area Divider 12]
The area dividing unit 12 divides the input color image into a plurality of grid areas using the data of the specific color area cut out by the specific color area cutout unit 11.
[0030]
The data of the specific color area cut out by the specific color area cutout unit 11 is supplied to the horizontal / vertical frequency distribution calculation unit 15 of the area division unit 12. The horizontal / vertical frequency distribution calculation unit 15 obtains horizontal and vertical histograms of the number of pixels having the specific color in the cut out specific color region.
[0031]
That is, as shown in FIG. 5, at each position X in the horizontal direction (x direction) of the area cut out by the specific color area cutout unit 11, the image data of that area is scanned in the vertical direction, and each horizontal position The number of pixels of a specific color in X is counted, and the counted value (frequency) Hh (x) is stored in, for example, the RAM 33 corresponding to each horizontal position coordinate X.
[0032]
Similarly, at each position Y in the vertical direction (y direction) of the area cut out by the specific color area cutout unit 11, the image data of that area is scanned in the horizontal direction, and the specific color at each vertical position Y is scanned. The number of pixels is counted, and the count value Hh (y) is stored in, for example, the RAM 33 corresponding to each position coordinate Y in the vertical direction. As described above, a histogram represented by the curves 21 and 22 in FIG. 5 is obtained.
[0033]
As described above, the horizontal and vertical histogram data calculated by the horizontal / vertical frequency distribution calculation unit 15 is supplied to the change point detection unit 16. In the change point detection unit 16, in the horizontal and vertical histograms, a significant change point (horizontal) that can distinguish a region including a sign (hereinafter, this region is referred to as a sign region) and other regions. Position coordinates in the direction and position coordinates in the vertical direction).
[0034]
In this embodiment, the change point detection unit 16 obtains position coordinates that serve as peak values of the histogram as significant change points. The peak point detection method in this embodiment is as follows.
[0035]
First, when detecting the peak point in the horizontal direction, assuming that the frequency at a certain horizontal coordinate X is Hh (x), the peak point is detected by threshold determination represented by the following equations (1) and (2). .
[0036]
Hh (x)> Hth (1)
{2 × Hh (x) −Hh (x−1) −Hh (x + 1)}> ΔHth (2)
Here, Hth and ΔHth are preset threshold values, and Hh (x−1) is a frequency in the horizontal coordinate (X−1) immediately before the horizontal coordinate X, and Hh (x + 1) is the frequency at the horizontal coordinate (X + 1) one after the horizontal coordinate X.
[0037]
Similarly, when detecting the peak point in the vertical direction, assuming that the frequency at a certain vertical coordinate Y is Hv (y), the peak point is detected by threshold judgment represented by the following equations (3) and (4). To do.
[0038]
Hv (y)> Vth (3)
{2 × Hv (y) −Hv (y−1) −Hv (y + 1)}> ΔVth (4)
Here, Vth and ΔVth are preset threshold values, and Hv (y−1) is a frequency in the vertical coordinate (Y−1) immediately before the vertical coordinate Y, and Hv (y + 1) is a frequency at a vertical coordinate (Y + 1) one after the vertical coordinate Y.
[0039]
The threshold values Hth, ΔHth, Vth, and ΔVth are determined in advance by preliminary experiments or the like.
[0040]
In the case of the example in FIG. 5, peak points Pxa and Pxb are detected from the horizontal histogram with the horizontal position coordinates Xa and Xb, and from the vertical histogram with the vertical position coordinates Ya and Yb. Peak points Pya and Pyb are detected. The position coordinate information of these detected peak points is supplied from the change point detection unit 16 to the grid area division unit 17.
[0041]
When there are a plurality of specific colors to be cut out as marker areas in the specific color area cutout section 11, the specific color area cutout section 11 adds information on the specific color of the area for each cut out area, Using the added specific color information, the horizontal / vertical frequency distribution calculation unit 15 calculates a histogram for each specific color, and uses it to detect a peak point from the histogram for each specific color. The detection is performed by the unit 16, and the detection result is supplied to the grid area dividing unit 17.
[0042]
In addition, the specific color region cutout unit 11, the horizontal / vertical frequency distribution calculation unit 15, and the change point detection unit 16 perform processing for each specific color to be cut out as a marker region, and check the change for each specific color. The output result may be supplied to the lattice area dividing unit 17.
[0043]
The grid area dividing unit 17 divides the input color still image stored in the image memory 34M into grid areas using the position coordinate information of all the change points supplied from the change point detecting unit 16 supplied thereto. To do. That is, in the case of FIG. 5 described above, the horizontal and vertical position coordinates Xa, Xb, Ya, Yb of the detected peak point are supplied to the lattice region dividing unit 17, so that the lattice region dividing unit In FIG. 17, as shown in FIG. 6, the input color still image is divided into three in the horizontal direction by dividing lines 23 and 24 passing through the horizontal position coordinates Xa and Xb. The input color still image is divided into three in the vertical direction by dividing lines 25 and 26 passing through the vertical position coordinates Ya and Yb. As a result, the input color image is divided into nine grid regions as shown in FIG. When the grid area is divided in this way, the contents stored in the calculation image memory 34M are cleared (all erased).
[0044]
[Signal area determination unit 13]
As described above, the image data for each grid area divided by the grid area dividing unit 17 is supplied to the marker area determining unit 13 as an output of the area dividing unit 12. The sign area determination unit 13 determines whether or not each area is a sign area from the feature of the image for each grid area from the area division unit 12.
[0045]
As described above, a road sign is provided with a specific color frame, and a specific symbol such as a sign character or an arrow is present inside the frame of the specific color. Therefore, the peak point of the histogram of the specific color pixel of the specific color region cut out by the specific color region cutout unit 11 corresponds to the frame portion, and the inside of the region surrounded by this peak point is inside the sign, It can be said that there are sign characters and symbols in them.
[0046]
However, as described above, in the extraction process of the specific color area, when the threshold is not strict, the color area other than the sign is also extracted, and thereby the peak of the histogram of the area other than the sign may be detected. . In addition, some input color images may include a plurality of signs.
[0047]
Therefore, in this embodiment, in each of the lattice areas divided by the area dividing unit 12 as described above, it is determined by this marked area determining unit 13 whether or not the area is inside the sign. .
[0048]
In the sign area determination unit 13 of this embodiment, it is determined whether or not it is a sign area by using the feature of the pixel value of the area inside the sign. That is, there are characters and symbols inside the sign area, but there is no fine change as a color pixel, and there are only two or three values as pixel values. For example, blue characters and symbols are described on a white background, and white characters and symbols are described on a blue background.
[0049]
Therefore, in the marker area determination unit 13 of this embodiment, paying attention to, for example, the G (green) signal of each pixel, whether or not the pixel value in each grid area is close to binary in this example, It is determined whether the grid area is inside the sign.
[0050]
FIG. 8 is a block diagram showing the function of the sign area determination unit 13 of this embodiment, and corresponds to the case where the sign area determination unit 13 has a hardware configuration. As shown in FIG. 8, the sign region determination unit 13 includes an in-region maximum value detection unit 131, a difference calculation unit 132, a maximum value vicinity pixel determination unit 133, a maximum value vicinity pixel counting unit 134, and an in-region minimum. The value detection unit 135, the difference calculation unit 136, the minimum value vicinity pixel determination unit 137, the minimum value vicinity pixel counting unit 138, the in-region total pixel number calculation unit 139, and the marker region determination extraction unit 130 are included.
[0051]
In the image data of each grid-like area unit from the area dividing unit 12, first, the in-area maximum value detecting unit 131 detects the level of the maximum G signal in the area. Then, the difference calculation unit 132 calculates the difference between the G value of each pixel in the region and the detected maximum G signal. The difference value is supplied to the maximum value neighboring pixel determination unit 133. The maximum value vicinity pixel determination unit 133 compares the difference value with a threshold value, and when the difference value is smaller than the threshold value, determines that the pixel is a pixel having a value in the vicinity of the maximum G value. The count value of the pixel counting unit 134 is incremented. Therefore, the maximum value vicinity pixel counting unit 134 counts the number Nb of pixels near the maximum G value among all the pixels in each grid area.
[0052]
Further, the image data of each grid area from the area dividing unit 12 is detected by the in-area minimum value detecting unit 135 to detect the level of the minimum G signal in the area. Then, the difference calculation unit 136 calculates the difference between the G value of each pixel in the region and the detected minimum G signal. The difference value is supplied to the minimum value neighboring pixel determination unit 133. The minimum value vicinity pixel determination unit 137 compares the difference value with the threshold value, and when the difference value is smaller than the threshold value, the minimum value vicinity pixel determination unit 137 determines that the pixel has a value in the vicinity of the minimum G value. The count value of the pixel counting unit 138 is incremented. Therefore, the minimum value vicinity pixel counting unit 138 counts the number Nc of pixels near the minimum G value among all the pixels in each grid area.
[0053]
In parallel with the above processing, the in-region total pixel number calculation unit 139 calculates the total number of pixels Na included in the grid region. The calculated total number of pixels Na is supplied to the sign area determination extraction unit 130. Further, the number Nb of pixels near the maximum G value counted by the maximum value vicinity pixel counting unit 134 and the number Nc of pixels near the minimum G value counted by the minimum value vicinity pixel counting unit 138 are also extracted from the sign region determination. Supplied to the unit 130.
[0054]
In this example, the sign area determination extraction unit 130 calculates (Nb + Nc) / Na, and when the calculation result is larger than a predetermined threshold, that is, the pixel value in the area is close to binary. The lattice area is determined as a sign area, and the data of the area is stored and held in, for example, the image memory 34B for calculation, and is used as a target of the sign information detection process in the next sign information detection unit 14.
[0055]
On the other hand, when the calculation result is equal to or smaller than the threshold value, the grid area is determined to be an area other than the marker area, and the data in the area is discarded without being stored in the calculation image memory 34B. The label information detection unit 14 does not perform the label information detection process.
[0056]
As described above, the marker area determination unit 13 according to this embodiment dynamically calculates the maximum value and the minimum value of predetermined information such as pixel brightness for each grid area, and calculates the maximum Since the pixel value in each grid area is determined to be close to binary based on the value and the minimum value, the sign area can be determined without depending on the surrounding brightness such as weather. Can do.
[0057]
FIG. 9 is a flowchart in the case where the sign area determination unit 13 of this embodiment is executed by the software (program) of the arithmetic processing unit 3.
[0058]
First, data of the first grid area to be determined is fetched (step S1), and the number of pixels Na in the area is detected (step S2). Next, the maximum value of the G signal is detected from the G signal of all the pixels in the grid area (step S3). Then, the difference between the G signal of each pixel in the grid area and the maximum value of the detected G signal is obtained, and the number Nb of pixels whose difference is smaller than a predetermined threshold value is calculated (step S4).
[0059]
Next, the minimum value of the G signal is detected from the G signals of all the pixels in the grid area (step S5). Then, the difference between the G signal of each pixel in the grid area and the minimum value of the detected G signal is obtained, and the number Nc of pixels whose difference is smaller than a predetermined threshold value is calculated (step S6).
[0060]
Next, (Nb + Nc) / Na is calculated using the pixel numbers Na, Nb, and Nc obtained in step S2, step S4, and step S6, and whether or not the calculation result is larger than a predetermined threshold value Nth. Determination is made (step S7). As a result of the determination, when the calculation result is larger than the threshold value Nth, the lattice area is determined to be a label area, the data of the area is held in the image memory 34B, and the next label information detection unit 14 It is set as a target for the detection process of the label information (step S8). When the calculation result is equal to or less than the threshold value Nth, the grid area is regarded as an area other than the sign area, and the data in the area is discarded and excluded from the target information detection process in the next sign information detection unit 14. (Step S9).
[0061]
Next, it is determined whether or not the determination for all grid areas has been completed (step S10). If not, the next grid area data is fetched (step S11), and the process returns to step S2. The above processing is repeated. Further, when the determination for all the lattice areas is completed, the marker area determination process is ended.
[0062]
In the case where the input color image is FIG. 6, the sign area determination unit 13 determines whether or not each of the nine grid areas is a sign area as described above. In the image memory 34B, a marker area as shown in FIG. 10 is stored as a determination result.
[0063]
In the above example, the marker area determination unit 13 focuses on the G signal and determines whether the pixel value in each grid area is close to binary, but other R (red ) Signal or B (blue) signal is not particularly limited to the G signal because there is no fine texture in the lattice area and the pixel value is close to binary or ternary. Further, instead of the G signal, a luminance signal Y is generated from each of the R, G, and B signals, and this luminance signal Y is processed in the same manner as described above, so that each grid area is inside the sign. It is also possible to determine whether or not there is.
[0064]
[Signal information detection unit 14]
The sign information detection unit 14 is determined as a sign area by the sign area determination unit 13 and is significant information as a sign such as a character or a symbol part such as an arrow from the image data of the lattice area held in the image memory 34B. Is detected. Three examples will be described as a method of detecting significant information in the sign information detection unit 14.
[0065]
[First example of sign information detection]
In this first example, since the color arrangement of information that is significant as a label is a plurality of predetermined color schemes as described above, the color information for each of the plurality of color arrangements is determined as a threshold value. . For this reason, in this first example, a threshold value is determined for each color signal component of R, G, B in each color arrangement, and the threshold value and each color signal of R, G, B of each pixel are determined. By comparing these, it is determined whether each pixel is a significant information part. The process for detecting the label information in the case of the first example will be described with reference to FIG. 11 and the subsequent flowchart of FIG.
[0066]
That is, first, threshold values are set for the R, G, and B color signals in the first color scheme to be detected among a plurality of color schemes (step S21). Then, the first pixel data in the marker area is read (step S22), the R signal of the pixel data is compared with the threshold value for the R signal set in step S21 (step S23), It is determined whether or not the significance condition is satisfied (step S24). If the significance condition for the R signal is not satisfied, for example, a flag indicating that is set as a pixel other than the significant information (step S30).
[0067]
When the significance condition for the R signal is satisfied, the G signal of the pixel data is compared with the threshold value for the G signal set in step S21 (step S25), and the significance for the G signal is compared. It is determined whether or not the sex condition is satisfied (step S26). If the significance condition for the G signal is not satisfied, for example, a flag indicating that is set as a pixel other than the significant information (step S30).
[0068]
When the significance condition for the G signal is satisfied, the B signal of the pixel data is compared with the threshold value for the B signal set in step S21 (step S27), and the significance for the B signal is compared. It is determined whether or not the sex condition is satisfied (step S28). In the case where the significance condition for the B signal is not satisfied, as a pixel other than the significant information, for example, a flag indicating that is set for the pixel (step S30).
[0069]
When the significance condition for the B signal is satisfied, that is, when the significance conditions for all of R, G, and B are satisfied, the pixel data is displayed as display image data on the display 4 as significant information. Is stored in the image memory 34B (step S29).
[0070]
Next, it is determined whether or not the detection of significance for all the pixels in the labeled area has been completed (step S31). If not completed, the next pixel data in the labeled area is read (step S32). ), The process returns to step S23, and the processes after step S23 described above are repeated.
[0071]
If it is determined in step S31 that the detection of the significance for all the pixels in the marker region is completed, the detection for all the color schemes prepared for detecting the significant information about the marker is completed. If it has not been completed, threshold values for the R, G, B color signals in the next color arrangement are set (step S34). Then, returning to step S22, the detection of significant information regarding a new color scheme is executed as described above for the pixels in the region.
[0072]
At this time, in the detection of the significant information about the previous color scheme, if it is determined that the pixel is other than the significant information and the flag to that effect is newly added as the significant information, the step In S29, the flag is erased and the pixel data that is regarded as significant information is retained.
[0073]
As described above, when the detection of the significant information for all the color schemes is completed, the process proceeds from step S33 to step S35, and it is determined whether or not the detection process of the significant information is completed for all the label areas. If there is another labeled area for which the detection process of significant information has not been completed, the process is shifted to that labeled area (step S36), and the processes from step S21 are repeated.
[0074]
If it is determined in step S35 that the detection processing of significant information has been completed for all the labeled areas, the pixel data flagged as pixels other than the significant information is deleted (step S37), and then this significant information detection is performed. The processing routine ends.
[0075]
As a result of the above processing, only the sign portion as shown in FIG. 10 remains in the image memory 34B, and this is read out and displayed on the display 4 so that an image similar to FIG. It is displayed on the screen and a road sign is notified to the user.
[0076]
[Second example of sign information detection]
This second example is based on the feature that the number of pixels that are significant information such as characters and symbols is smaller in the marker area than the number of other pixels. A process for detecting the label information in the case of the second example will be described with reference to the flowchart of FIG.
[0077]
First, the first labeled area where significant information is detected is designated (step S41), and the value of each pixel in the labeled area is quantized to binary or ternary in the R, G, B color space ( Step S42). Next, the number of pixels for each quantization level is counted (step S43), and the quantization level Qmin with the smallest number of pixels is detected (step S44).
[0078]
Then, for each pixel, it is determined whether or not the level after quantization corresponds to the detected quantization level Qmin (step S45), and the data of the corresponding pixel is stored in the image memory 34B as significant information (step S45). S46), the data of the non-corresponding pixels is deleted from the image memory 34B (step S47).
[0079]
Next, it is determined whether or not the significant information detection process has been completed for all the labeled areas (step S48). If there is another labeled area for which the significant information detection process has not been completed, the process is shifted to that labeled area. (Step S49), the processing from Step S42 is repeated.
[0080]
Also in the case of the second example, as in the first example described above, significant information such as characters and symbols in the sign area can be detected satisfactorily.
[0081]
[Third example of sign information detection]
This third example is a method that utilizes the fact that significant information inside the sign is gathered near the center of the sign, and there are a large number of pixels that are significant information in a minute area near the center. A process for detecting the label information in the case of the third example will be described with reference to the flowchart of FIG.
[0082]
First, the first marker area where significant information is detected is designated (step S51), the center coordinates in the area are calculated (step S52), and a minute grid area centered on the calculated center coordinates is defined. Set (step S53). Here, the minute area is sufficiently smaller than the label area.
[0083]
Next, binarization processing is performed on the pixel data within the minute grid area (step S54). Next, the number of pixels in the minute area is counted for each binarization level (step S55). Then, the level with the larger number of pixels is specified (step S56).
[0084]
Then, it is determined whether or not the binarization level of each pixel in the minute area corresponds to the level with the larger number of pixels (step S57), and the data of the corresponding pixel is stored in the image memory 34B as significant information. It retains (step S58), and the data of the non-corresponding pixel is deleted from the image memory 34B (step S59).
[0085]
Next, it is determined whether or not the significant information detection process has been completed for all the labeled areas (step S60). If there is another labeled area for which the significant information detection process has not been completed, the process is shifted to that labeled area. (Step S61), the processing from Step S52 is repeated.
[0086]
Also in the case of the third example, similar to the first example and the second example described above, significant information such as characters and symbols in the marker area can be detected well.
[0087]
[Other variations]
In the above-described embodiment, each of the plurality of types of color schemes assumed as road signs is processed in each unit and shifted to the next stage. However, for each color of the assumed color scheme, a specific color region is used. You may make it perform from a cutting-out to the process of a marker information detection.
[0088]
In the above-described embodiment, the change point detection unit 16 of the region division unit 12 uses the peak points of the horizontal and vertical histograms calculated by the horizontal / vertical frequency distribution calculation unit 15 as significant change points. Although detected, significant change points are not limited to peak points. For example, in a histogram, a point where the frequency gradually increases and then changes to a flat change or a point where the frequency changes so as to be gradually processed from the flat change may be detected as a significant change point. Good.
[0089]
In the above-described embodiment, the case where the detection target is a road sign has been described, but it goes without saying that the detection target sign of the present invention is not limited to a road sign.
[0090]
【The invention's effect】
As described above, according to the present invention, since the sign is detected based on the characteristic of the sign, not the simple pattern matching as in the prior art, it is possible to perform well without depending on the inclination of the sign or the scale of the sign. Can be detected.
[0091]
In addition, since the specific color area is extracted, the extraction result is used to divide into grid areas, and the marker portion is determined for each grid area. Even if there is a change in the brightness of the surrounding area, the sign can be reliably detected.
[Brief description of the drawings]
FIG. 1 is a functional block diagram of a main part of an embodiment of a label detection apparatus according to the present invention.
FIG. 2 is an overall system configuration diagram of an embodiment of a label detection apparatus according to the present invention.
FIG. 3 is a diagram for explaining a label detection processing operation according to the present invention;
FIG. 4 is a diagram for explaining a label detection processing operation according to the present invention.
FIG. 5 is a diagram for explaining a label detection processing operation according to the present invention;
FIG. 6 is a diagram for explaining a label detection processing operation according to the present invention.
FIG. 7 is a diagram for explaining a label detection processing operation according to the present invention.
FIG. 8 is a functional block diagram of an example of a sign area determination unit in the embodiment of the sign detection device according to the present invention.
FIG. 9 is a flowchart for explaining an example of a sign area determination unit in the embodiment of the sign detection device according to the present invention;
FIG. 10 is a diagram for explaining a label detection processing operation according to the present invention.
FIG. 11 is a part of a flowchart for explaining an example of a label information detection unit in the embodiment of the label detection apparatus according to the present invention;
FIG. 12 is a part of a flowchart for explaining an example of a label information detection unit in the embodiment of the label detection apparatus according to the present invention;
FIG. 13 is a flowchart for explaining another example of the label information detection unit in the embodiment of the label detection apparatus according to the present invention;
FIG. 14 is a flowchart for explaining still another example of the label information detection unit in the embodiment of the label detection apparatus according to the present invention;
[Explanation of symbols]
DESCRIPTION OF SYMBOLS 1 ... Video camera, 2 ... A / D converter, 3 ... Arithmetic processing part, 4 ... Display, 11 ... Specific color area cutout part, 12 ... Area division part, 13 ... Marking area determination part, 14 ... Sign information detection part , 15 ... Horizontal / vertical frequency distribution calculation unit, 16 ... Change point detection unit, 17 ... Lattice region division unit, 130 ... Marking region determination extraction unit, 131 ... In-region maximum value detection unit, 133 ... Maximum value neighborhood pixel determination 134: Maximum value neighboring pixel counting unit, 135 ... In-region minimum value detecting unit, 137 ... Minimum value neighboring pixel determining unit, 138 ... Minimum value neighboring pixel counting unit, 139 ... Total number of pixels in region calculating unit

Claims (19)

Specific color area cutout means for cutting out an area having a specific color from the input color image;
An area dividing unit that divides the input color image into grid-like areas by dividing the input color image in the vertical direction and the horizontal direction so as to surround the area cut out by the specific color area cutting out unit,
For each divided area divided by the area dividing means, a sign area determination means for determining whether or not the area is a sign area that is a part of a sign,
In the label area determined by the label area determination means, label information detection means for detecting significant information inside the label area as label information,
A sign detection apparatus comprising: The label detection apparatus according to claim 1,
The region dividing means includes
A frequency distribution calculating means for counting the pixels of the specific color in the horizontal direction and the vertical direction in the area cut out by the specific color area cutting out means to obtain a frequency distribution in each direction;
Change point detection means for detecting significant change points of the horizontal and vertical frequency distributions calculated by the frequency distribution calculation means;
A dividing unit that divides the input color image into a grid-like region by dividing the input color image in a vertical direction and a horizontal direction at the significant change point position detected by the change point detection unit;
A sign detection apparatus comprising: The label detection apparatus according to claim 1,
The specific color area cutout means extracts a specific color area by a process using a predetermined threshold for the color included in the sign, and sets all the pixel values to zero except for the specific color area. Characteristic label detection device. The label detection apparatus according to claim 2,
The said change point detection means detects the peak point of the frequency distribution of the said horizontal direction and a perpendicular direction as said significant change point. The label | marker detection apparatus characterized by the above-mentioned. The label detection apparatus according to claim 1,
The marker detection device, wherein the marker region determination means determines a marker region based on whether or not a value that can be taken by a pixel included in the region to be determined is close to a binary value or a ternary value. The label detection apparatus according to claim 1,
The sign area determination means includes
Detection means for detecting all the pixel numbers Na included in the region to be determined;
Maximum value detecting means for detecting a maximum value of predetermined information of pixels included in the region to be determined;
Minimum value detecting means for detecting a minimum value of the predetermined information of the pixels included in the region to be determined;
A maximum value neighboring pixel number counting unit for detecting a pixel number Nb whose difference from the maximum value is equal to or less than a predetermined value in the region to be determined;
In a region to be determined, a minimum value neighboring pixel number counting unit that detects a pixel number Nc whose difference from the minimum value is equal to or less than a predetermined value;
A determination means for determining the region to be determined as a region inside the sign or a region outside the sign based on a comparison result between the value of (Nb + Nc) / Na and a predetermined threshold;
A sign detection apparatus comprising: The label detection apparatus according to claim 1,
The sign information detection means compares the color of each pixel in the sign area determined by the sign area determination means with the color used for the sign, and if the difference between the two is smaller than a predetermined value, the pixel Is detected as a pixel of the significant information portion. The label detection apparatus according to claim 1,
The sign information detecting means includes
Means for quantizing the pixels in the label area determined by the label area determination means;
Means for determining the number of pixels for each quantization level for the quantized pixels in the label area;
Means for calculating the quantization level of the smallest number of pixels among the quantization levels;
Determining whether each pixel in the label area corresponds to the quantization level of the smallest number of pixels, and if so, means for detecting the pixel as a pixel of the significant information portion;
A sign detection apparatus comprising: The label detection apparatus according to claim 1,
The sign information detecting means includes
Means for determining the most frequent pixel level for pixels included in a minute region near the center in the marker region;
Determining whether each pixel in the marker region is a level close to the obtained pixel level, and if close, means for detecting the pixel as a pixel of the significant information portion;
A sign detection apparatus comprising: A specific color area cutout step of cutting out an area having a specific color from the input color image;
A region dividing step of dividing the input color image into a grid-like region by dividing the input color image in the vertical direction and the horizontal direction so as to surround the region cut out in the specific color region cutting step,
For each divided region divided in the region dividing step, a labeled region determination step for determining whether or not the region is a labeled region that is a portion of a marker,
In the label area determined in the label area determination step, a label information detection step for detecting significant information therein as label information,
A label detection method comprising: In the label | marker detection method of Claim 10,
The region dividing step includes
A frequency distribution calculating step in which the pixels of the specific color are counted in the horizontal direction and the vertical direction in the region cut out in the specific color region cutout step to obtain a frequency distribution in each direction;
A change point detection step of detecting a significant change point of the horizontal and vertical frequency distributions calculated in the frequency distribution calculation step;
A division step of dividing the input color image into grid-like regions by dividing the input color image in the vertical direction and the horizontal direction at the significant change point position detected in the change point detection step;
A label detection method comprising: In the label | marker detection method of Claim 10,
In the specific color region cut-out step, a specific color region is extracted by a process using a predetermined threshold for colors included in the sign, and pixel values other than the specific color region are all set to zero. Characteristic label detection method. The label detection method according to claim 12,
In the change point detection step, a peak point of the frequency distribution in the horizontal direction and the vertical direction is detected as the significant change point. In the label | marker detection method of Claim 10,
The sign detection method, wherein in the sign area determination step, a sign area is determined based on whether or not a value that can be taken by a pixel included in the area to be determined is close to binary or ternary. In the label | marker detection method of Claim 10,
The marker region determination step includes
A detection step of detecting all the pixel numbers Na included in the region to be determined;
A maximum value detecting step of detecting a maximum value of predetermined information of pixels included in the region to be determined;
A minimum value detecting step of detecting a minimum value of the predetermined information of the pixels included in the region to be determined;
In the region to be determined, a maximum value neighboring pixel number counting step of detecting a pixel number Nb whose difference from the maximum value is not more than a predetermined value;
In the region to be determined, a minimum value neighboring pixel number counting step of detecting a pixel number Nc whose difference from the minimum value is a predetermined value or less;
A determination step of determining the region to be determined as a region inside the sign or a region outside the sign based on a comparison result between the value of (Nb + Nc) / Na and a predetermined threshold;
A label detection method comprising: In the label | marker detection method of Claim 10,
In the marker information detection step, the color of each pixel in the marker region determined in the marker region determination step is compared with the color used for the marker, and when the difference between the two is smaller than a predetermined value, the pixel Is detected as a pixel of the significant information portion. In the label | marker detection method of Claim 10,
The label information detection step includes
Quantizing the pixels in the labeled area determined in the labeled area determining step;
Obtaining the number of pixels for each quantization level for the quantized pixels in the label area; and
Calculating the quantization level of the smallest number of pixels among the quantization levels;
Determining whether each pixel in the marker region corresponds to the quantization level of the smallest number of pixels, and if so, detecting the pixel as a pixel of the significant information portion;
A label detection method comprising: In the label | marker detection method of Claim 10,
The label information detection step includes
Obtaining the most frequent pixel level for the pixels contained in the minute region near the center in the labeled region; and
Determining whether each pixel in the labeled area is a level close to the obtained pixel level, and if so, detecting the pixel as a pixel of a significant information portion; and
A label detection method comprising: A specific color region cutout procedure for cutting out a region having a specific color from the input color image;
An area division procedure for dividing the input color image into a grid-like area by dividing the input color image in the vertical direction and the horizontal direction so as to surround the area cut out in the specific color area cut-out procedure;
For each divided area divided in the area dividing procedure, a sign area determination procedure for determining whether or not the area is a sign area that is a part of a sign;
In the label area determined by the label area determination procedure, a label information detection procedure for detecting significant information therein as label information,
A recording medium on which a program consisting of

Images