JP2006178694A - Method for detecting candidate of constitutional segment of road mark image, and program capable of detecting candidate of constitutional segment of road mark image - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To quickly detect a candidate of a segment constituting an overhead road mark image. <P>SOLUTION: A road area is detected from ground image data, and whether a vertical segment extended from each of a series of pixels on the center line of the road area up to an area in which a road end exists in a direction vertical to the center line of the road area is a candidate of a segment constituting a road mark image or not is discriminated in accordance with whether the number of light pixels whose lightness is lighter than a prescribed threshold is the prescribed number of pixels necessary for a condition that the vertical segment includes a segment constituting the overhead road mark image and more or not. In the discrimination, the number of light pixels in a partial pixel group in the vertical segment is counted, and under a supposition that all remaining pixels are light pixels, whether possibility that the number of light pixels in the vertical segment exceeds the prescribed number of pixels exists or not is judged. When the absence of the possibility is judged, the vertical segment is judged as no candidate. <P>COPYRIGHT: (C)2006,JPO&NCIPI

Description

  The present invention is based on image data (referred to as ground image data in the present specification) obtained by photographing the ground surface from a high place such as an artificial satellite or an aircraft (hereinafter sometimes referred to as an artificial satellite). Road sign image constituent line segment candidate detection method and road sign image for detecting candidate line segments constituting a road sign image representing an elevated road sign provided so as to pass over the area of at least one side The present invention relates to a program capable of detecting constituent line segment candidates.

  In recent years, the use of terrain image data has become widespread, and information that can be used for various purposes regarding a vast area on the terrain surface can be obtained at high speed from the terrain image data. For example, it is being used for the purpose of generating a map (for example, a road map), grasping a land utilization situation in an urban area, and monitoring a presence state or a growth state of trees in a forest or the like. On the other hand, car navigation apparatuses have become widespread, and map information including the current position of the vehicle as a center is displayed on the screen. The road map generated from the satellite image is used for such a car navigation device, for example. A GPS (Global Positioning System) device that measures the current position by receiving radio waves from a specific artificial satellite (GPS satellite) and measuring latitude and longitude information is used to detect the current position of the vehicle in the car navigation device. Has been.

  In many cases, the road map displayed on the screen of the car navigation device includes information indicating road signs (road sign information), and the road sign information included in the road map is a great assistance for the driver. ing. In addition to displaying road sign information on the road map and displaying it on the screen, the driver is informed of the road sign information in the vicinity of the driving point on the screen or by voice, and assists the driver in obtaining the road sign information while driving. Several ideas have been proposed. For example, see Patent Document 1.

The acquisition of road sign information indicating a road sign installed on the side of a road or a road sign drawn on a road surface is a vehicle equipped with a road sign information acquisition device (in this specification, simply a road sign information collection vehicle or information). A moving image obtained by running a road on the road and photographing the surrounding space with a video camera, and simultaneously recording the position of the information collecting vehicle while traveling using a GPS device The road sign information indicated by the road sign is obtained from the position information and the position information obtained from the GPS device is recorded in association with each other. Various improvements in technology for acquiring road sign information from such moving images have been proposed. For example, see Patent Document 2.
JP 2001-034887 A JP 2003-123197 A

  The acquisition of road sign information from the road sign can be acquired while the vehicle is running as described above. However, even if the information collection vehicle is run on the road where the road sign exists and the road sign information is acquired, In some cases, the road sign may not be visible at all from the position of the information collecting vehicle, or a part of it may not be visible. For example, when the inter-vehicle distance from the preceding vehicle is small, a part of the road sign drawn on the road surface may be blocked by the preceding vehicle and may not be visible from the information collecting vehicle behind. Similarly, when the distance between the preceding vehicle and the preceding vehicle is small, a part of the road sign placed on the side of the road may be blocked by the preceding vehicle and may not be seen from the information collecting vehicle.

  In such a case, even if a road sign is taken from a video camera mounted on the information collecting vehicle, a part of the road sign is not taken, and as a result, the road sign cannot be recognized correctly. As described above, in the method of driving the information collection / division collection vehicle and photographing the road sign without knowing in advance the location of the road sign, the acquisition of the road sign information may fail. When driving again on the same road and approaching the same place, increase the inter-vehicle distance from the preceding vehicle, check that the entire road sign is visible, take a picture of the road sign, The information collecting vehicle is caused to travel again such that road sign information is acquired from the obtained moving image.

  In view of this, the present inventor considered that it is desirable to obtain the location of the road sign in advance in order to prevent the failure of obtaining the road sign information due to the failure of photographing the road sign. That is, if the location of the road sign is known in advance, the information collection vehicle is driven while judging whether or not the information collection vehicle is approaching the road sign, and when the information collection vehicle approaches the road sign We thought that it would be possible to run the information collection vehicle so that the inter-vehicle distance with the preceding vehicle was made sufficiently long and the running speed was reduced.

  However, in order to know the position of the road sign in advance, driving an automobile that can detect the position of the road sign will reduce the amount of work compared to running the information collection vehicle suddenly and acquiring the road sign information. It will increase. Therefore, it is desirable that the location of the road sign can be detected in advance by another simple method. As one method for that purpose, the present inventor has examined to detect the installation position of the road sign from the ground image data obtained by photographing the ground surface from the sky. There are satellite images and aerial photographs as surface images currently available as surface image data. The resolution of the satellite image is low, and if the resolution is too high even in aerial photographs, the range that can be captured at one time becomes narrow, and the resolution cannot be made too high. Since the satellite image can be used for any region and can be used at a lower price than aerial photographs, the following description will be made on the case where the satellite image is used as the ground image. It can also be applied when using aerial photographs as the ground image.

  In ordinary urban roads, the road width is narrow, many road signs are installed on the side of the road, and buildings are close to the road edge. Therefore, in a situation where there are many objects in the vicinity of a road sign in an urban area, it is not always easy to detect the installation position of the road sign from a satellite image with low resolution. However, on roads with a large road width such as highways, elevated road signs provided so as to pass over the area of at least one side of the road are often used as road signs. Since such a road sign is provided across the road, the road sign includes a line segment extending perpendicularly to the road. However, not all vertical line segments that cross the road in a direction perpendicular to the road are line segments that constitute an elevated road sign. However, if the existence position of a line segment extending perpendicularly to such a road can be detected, the candidate for the existence position of the elevated road sign can be detected.

  On highways, a road sign for the highway may be installed on the side wall of the road that intersects the highway. In this case, the three-dimensional intersection road with such a road sign is used as an elevated road sign. Similarly, a road sign may be installed also on the side wall of a pedestrian bridge that crosses an expressway, and the pedestrian bridge on which the road sign is installed is also used as an elevated road sign. Therefore, in the following, these three-dimensional intersection roads and pedestrian bridges will be treated as candidates for elevated road signs that have the possibility of becoming elevated road signs.

  As described above, the ability to detect the position of the candidate for the position of the elevated road sign is useful in obtaining road sign information related to the elevated road sign. For example, it is possible to automatically analyze a pattern of an image around a line segment detected as a candidate by a computer to determine whether a road sign is installed there. Alternatively, an image of the vicinity of the detected line segment candidate is sequentially enlarged and displayed on the display device, and an elevated road sign is actually installed in the vicinity of the line segment candidate by a person who wants to generate road sign installation position information. You may make it determine visually whether it is carried out. In this way, after determining whether or not the vertical line segment detected as a candidate actually represents a road sign, the information collection vehicle is driven on the highway to photograph the road sign image to obtain the road sign information. When the information collection vehicle approaches the candidate position where it is determined that there is actually a road sign at the time of acquisition, the distance between the vehicle and the preceding vehicle is sufficiently maintained and the traveling speed is reduced. The photographing of the road sign at the candidate position can be ensured. Some vertical line segments detected as candidates may not be able to be determined whether or not they are the line segments that actually represent the road sign image. Assuming that it is expressed, it is sufficient to use a road sign information photographing apparatus that takes a picture if the road sign is installed by automatically judging whether the road sign is installed by driving a car locally.

Since the resolution of the satellite image is low, the vertical line segment constituting the elevated road sign image has a width of only about 1 to 3 pixels. Accordingly, it is necessary to detect a vertical line segment constituting an elevated road sign having a width of one pixel from the road existing area. Since a highway is very long, it takes a long time to detect such a narrow line segment from the entire area even if it is automatically detected by a computer. Therefore, it is desirable that the detection can be performed in as short a time as possible.
Accordingly, an object of the present invention is to detect a road sign image constituent line segment candidate and to detect a line segment candidate constituting an elevated road sign image provided so as to pass over the road at high speed, and To provide a program capable of detecting a road sign image constituent line segment candidate.

In order to solve the above-described object, the road sign image component line segment detection method according to the present invention is based on ground image data obtained by photographing the ground surface from above, and passes over the area of at least one side of the road. A road sign image constituent line segment candidate detection method for detecting a candidate line segment constituting a road sign image representing an elevated road sign provided to perform a road area detection step, a determination step, and a storage step Is included. The road area detecting step detects a road area where any road exists from the ground image data. The determining step includes, for each of a series of pixels extending along the extending direction of the road on the center line of the road area, reaching the area where the road edge exists from the center line, Among the vertical line segments corresponding to the pixel that extend in the direction perpendicular to the center line, the number of pixels whose brightness is higher than a predetermined threshold value, and the corresponding vertical line segment indicates the elevated road sign Whether or not the corresponding vertical line segment is a candidate for the line segment constituting the road sign image, depending on whether or not the predetermined number or more necessary for including the line segment constituting the image is included. Determine. The storing step stores position information indicating the position of the vertical line segment when it is determined that the corresponding vertical line segment is a line segment candidate constituting the road sign image.
Further, the determination step includes a possibility determination step and an early determination step, and the possibility determination step counts the number of bright pixels included in some pixel groups included in the corresponding vertical line segment. When it is assumed that the remaining pixels are all bright pixels, it is determined whether or not the total number of bright pixels included in the vertical line segment may reach the predetermined number, and depends on the determination result Then, it is determined whether or not there is a possibility that the corresponding vertical line segment becomes a line segment candidate constituting the road sign image. In the early determination step, as a result of the determination in the possibility determination step, when it is determined that the corresponding vertical line segment is not likely to be a line segment candidate constituting the road sign image, the partial pixel Without counting the number of bright pixels in the remaining partial pixel group other than the group, it is determined that the corresponding vertical line segment is not a line segment candidate constituting the road sign image.

  According to the present invention, a road sign representing an elevated road sign provided so as to pass over the area of at least one side of a road such as a highway from ground image data obtained by photographing the ground from the sky. It becomes possible to detect line segment candidates constituting the image at high speed. In particular, for each of a series of pixels extending along the road extending direction on the center line of the road, the pixels including the pixel and extending from the center line to the area where the road edge exists are perpendicular to the center line. A line segment extending in the direction is divided into a plurality of ordered partial pixel columns, and each of the partial pixel columns other than the last partial pixel column of the plurality of partial pixel columns up to the partial pixel column. Whether there is a possibility that the corresponding vertical line segment is a candidate for the line segment constituting the road sign image based on the total number of pixels whose brightness is higher than the threshold value included in all the partial pixel columns When the determination is made that there is no possibility, the determination of the possibility with respect to the subsequent partial pixel column is omitted, so that it is possible to quickly determine whether a line segment perpendicular to the center line is the candidate. To become It can be.

Hereinafter, a road sign image constituent line segment candidate detection method and a road sign constituent line segment candidate detection method for detecting candidate line segments constituting a road sign image representing an elevated road sign from ground image data according to the present invention. One embodiment of a possible program will be described in detail with reference to the drawings.
FIG. 1 is a schematic block diagram of an embodiment of an apparatus capable of recognizing an image road sign according to the present invention. Reference numeral 1 indicates the whole of one embodiment of the apparatus. The apparatus 1 includes, for example, a processing device 10 realized by a personal computer or a workstation, and a RAM (Random Access Memory) (not shown) used as a main memory and an auxiliary storage device (not shown) such as a magnetic disk storage device. And the input / output device 30. The input / output device 30 includes an input device 31 including a pointing device such as a keyboard and a mouse, and an output device such as a display device 32 such as a CRT display device or a printer 33. The input device 31 is used for inputting parameters and starting commands. The display device 32 or the printer 33 is used for displaying or printing an image to be a landmark detection target or an image showing a landmark detected from the image. When data is stored in the storage device 20, it is determined in advance for each piece of data whether the data is stored in a RAM (not shown) or an auxiliary storage device (not shown).

  The processing device 10 incorporates a program 40 capable of detecting road sign constituent line segment candidates. The program 40 includes modules of a road area detection unit 100, a binarization unit 200, and a line segment candidate detection unit 300. The line segment candidate detection unit 300 further includes sub-modules such as a left vertical line segment determination unit 350 and a right vertical line segment determination unit 370. The program 40 is stored in the storage device 20 and executed by the processing device 10. In FIG. 1, for ease of understanding, the plurality of modules constituting the program 40 are illustrated in a block showing the processing device 10. Yes. The storage device 20 stores road data 21 and ground image data 22 in advance. The road data 21 and the ground image data 22 are processed by the program 40 to obtain road area image data 23 and binarized road areas. Image data 24 and line segment candidate position data 25 are generated and stored in the storage device 20. The program 40 realizes one embodiment of a program capable of detecting road sign image constituent line candidates according to the present invention, and the apparatus 1 when the program 40 is executed by the apparatus 1 is a series of programs. A mode of executing the processing shows one embodiment of a method for detecting a road sign image constituent line segment candidate according to the present invention.

  Unless otherwise specified, it is assumed that the ground image data 22 is data of a monochrome multi-value image (grayscale image). The black and white shading of the ground image data 22 is called brightness. However, the present invention can also be applied to the case where the ground image data 22 is color image data. For example, when the ground image data 22 includes data relating to brightness, color, and hue, the brightness data may be used as the ground image data 22 in the following processing executed in the present embodiment. . When the ground image data 22 does not explicitly include brightness data, the brightness data obtained by converting the ground image data 22 into color data including brightness data is executed in the present embodiment. In this process, it may be used as image data.

  In the present embodiment, as an example of image data including a road sign, for example, ground image data obtained by photographing the ground surface from an altitude flying object such as an artificial satellite or an aircraft is used. Here, the resolution of the ground surface image data is the most accurate and is on the order of several tens of centimeters, while the elevated road sign is usually about several meters in size. Therefore, in many cases, the resolution of the ground image is a critical resolution for recognizing elevated road signs.

  In FIG. 1, when the program 40 is activated, it first executes the road area detection unit 100. The program 40 executes processing described below for each of a plurality of roads from the ground image data. The road data 21 is data indicating the location and range of roads existing on the ground surface represented by the ground surface image data 22. The road area image data 23 stores the image values of the plurality of pixels in the area where the road exists in the ground image represented by the ground image data 22 corresponding to the storage position of the image value in the ground image data 22. This is image data generated by storing in a position. First, the road area detection unit 100 reads the road data 21 and the ground surface image data 22 from the storage device 20, and based on the road data 21, the ground surface for each of a plurality of roads indicated by the road data 21 from the ground surface image data 22. The position of the road area that is the area where the road exists in the ground image represented by the image data 22 is calculated, the data portion in the ground image data 22 belonging to the road area is read out, and the road area image data 23 is obtained. Store in the storage device 20.

  FIG. 2 is a schematic diagram of a road. The road data 21 is composed of road vector data determined corresponding to one road. Each road is represented by a series of ordered polylines and road widths approximating the centerline of the road. A plurality of broken lines constituting the center line will be referred to as center line vectors. The road vector data of each road includes, for each of a plurality of centerline vectors constituting the road, centerline vector data representing the centerline and the road to which the centerline vector corresponds to each centerline vector. Road width data representing the road width w in the section. Each center line vector data includes the coordinates of the start point and end point of the corresponding center line vector. In the example of FIG. 2, a roadway exists between the centerline CL approximated by a plurality of centerline vectors and the road end lines E1 or E2 provided on both sides of the centerline CL. Divided into two lanes by lane boundary.

  FIG. 3A is a diagram showing an example of a road vector for the road shown in FIG. As shown in the figure, the center line CL is composed of three center line vectors CV1 to CV3, and each center line vector CV1 to CV3 is a set of a start point and an end point (S1 and G1, S2 and G2, S3 and G3), respectively. It is specified. The road widths in the road section where each centerline vector exists are w1 to w3, respectively. Here, it is assumed that the road width is uniform between both ends of each centerline vector for simplification. Furthermore, the width of the roadway on both sides of the road is assumed to be the same in the figure. Therefore, the road end lines E11 and E21, E12 and E22, E13 and E23 on both sides for each center line are all parallel to the center line vectors CV1, CV2, and CV3, and a set of road end lines for the same center line vector. (Eg E11 and E21) are equidistant from the centerline vector. However, the road width may change depending on the position on the center line vector, and the road widths of the left and right roads may be different.

  The end point (P) of the road end (E11) and the start point (Q) of the road end line E21 with respect to the connection point (eg G1 and S2) of two adjacent center line vectors (eg CV1, CV2) are the two centers. Because the direction of the line vector is changing, they do not match. Assume that these two points are connected by a suitable line segment (straight line or curve). In the present invention, it is assumed that the elevated road sign image to be detected is provided so as to extend perpendicularly to the centerline vector of the road, and therefore, the coupling point (G1 and S2) of the centerline vector and its Since there is no elevated road sign image to be detected in the combined area composed of two points (P and Q) on the road end line corresponding to the point, the above two points (P and Q) on the road end line do not exist As long as the present invention is concerned, the shape of the line segment which connects is not a problem. Therefore, in the following, no particular reference will be made to the road end line of this connecting portion.

  FIG. 3B is a diagram illustrating an example of the road data 21. The road data 21 includes road identification information 211 for identifying a road, center line vector identification information 212 for identifying a plurality of center line vectors in the road having the road identification information, and each center line vector. Of the coordinates of the start point and end point of 213 and the road width 214 in the section where each center line vector exists.

  Returning to FIG. 1, for each of a plurality of roads indicated by the road data 21, the road area detection unit 100, with respect to each of a plurality of centerline vectors constituting the road, on each side of each centerline vector. A partial road area including a road area that is located and a road edge line located further outside is determined, and an area connecting the partial road areas is determined as a road area for the road. The data portion in the road area is stored in the road area image data 23.

  The binarization unit 200 and the line segment candidate detection unit 300 described below process the road area detected by the road area detection unit 100. That is, only the road area image data 23 in the ground surface image data 22 is processed. As a result, it is possible to reduce the amount of processing required to detect the constituent line segment candidates of the elevated road sign image. When the road area is detected based on the road data 21, it is possible to avoid erroneously recognizing an area other than the road area as the road area. However, the present invention may detect the road region, further the center line vector, and the road edge line from the ground image data 22 without using the road data 21, and may be used for the following processing.

  First, the binarization unit 200 distinguishes bright pixels (bright pixels) from dark pixels by performing binarization processing using an appropriate threshold for the brightness of the road area image data 23. Elevated road signs are often white or a light color close to white. The road is made of concrete or asphalt, so the lightness is low and the color is dark. Therefore, it is expected that roads and elevated road signs can be distinguished by brightness. Various well-known techniques such as a mode method, a discriminant analysis method, a P tile method, and a differential histogram method can be applied to the lightness binarization processing. Which method is appropriate to perform the binarization process depends on the target image, and therefore, an appropriate method may be selected in accordance with the image. In general, when discriminant analysis is used, it is easy to distinguish a portion with high brightness and a portion with no brightness. The discriminant analysis method is also used in this embodiment. The binarization process is preferably performed by determining a threshold value for each partial road area for each centerline vector. Otherwise, a common threshold value is set for all road areas for all centerline vectors for one road. You may decide. The binarization unit 200 stores the binarized road area image data obtained by performing binarization processing on the road area image data 23 as the binarized road area image data 24 in the storage device 20. To do.

  FIG. 4 is a diagram schematically showing a partial road area including one centerline vector of a road to be processed among roads represented by the ground surface image data 22. In the partial road area shown in the figure, the road between the center line vector CV1 and the road edge lines E11 and E21 on both sides has three lanes each divided by two lane boundary lines, a rectangle T1, T2,... Indicate traveling vehicles, and the figure shows that the concentrations of these vehicles are different. R1 and R2 indicate two elevated road signs to be detected in the present embodiment. The elevated road sign R1 is provided so as to cross the upper roadway, and the elevated road sign R2 is provided so as to cross the lower roadway. Although not shown in the figure, an elevated road sign provided so as to cross the upper and lower roadways may be used. In the case of this last elevated road sign, the process described later causes an elevated road sign that crosses the upper roadway and the same position on the same centerline vector as the elevated road sign. An elevated road sign crossing the side roadway is detected.

  FIG. 5 is a diagram schematically showing a binarized partial road region image represented by the binarized road region image data 24 corresponding to the partial road region shown in FIG. If both the elevated road signs R1 and R2 are from the bright pixel group, they are displayed as images. The same applies to the center line vector CV, the lane boundary line, the bright car, and the slightly bright car. On the other hand, the road surface is represented by a black pixel group. Dark cars are also represented by black pixels, and are indistinguishable from the road surface.

  Returning to FIG. 1, next, the program 40 executes the line segment candidate detection unit 300 for the processing target road. Based on the road data 21, the line segment candidate detection unit 300 becomes a line segment constituting an elevated road sign image from the binarized road region image data 24 for each centerline vector of the road being processed. A possible candidate for a segment of an elevated road sign image is detected.

  FIG. 6 is a flowchart of processing of the line segment candidate detection unit 300. FIG. 7 is a diagram for explaining the left vertical line segment Vl and the right vertical line segment Vr processed by the line segment candidate detection unit 300 with respect to one center line vector of an arbitrary road. In FIG. 7A, the length of the center line vector CV is the road in the direction rotated counterclockwise by 90 ° with respect to the center line vector CV from the attention point M (Xm, Ym) on the center line vector CV. Half the width (w / 2) of the left vertical line segment Vl and the direction rotated 90 ° clockwise with respect to the center line vector CV from the point of interest M (Xm, Ym), and the length is half the road width The right vertical line segment Vr of (w / 2) is shown. The center line vector CV has a start point S (Xs, Ys) and an end point G (Xg, Yg). The line segment candidate detection unit 300 moves the attention point M from the start point S (Xs, Ys) to the end point G (Xg, Yg) by one pixel on the center line vector CV, and the left vertical line at each position of interest point. Whether the segment Vl and the right vertical line segment Vr are candidates for the line segment constituting the elevated road sign image is determined based on a predetermined criterion, and one of the left vertical line segment Vl and the right vertical line segment Vr is determined. Or, when both are determined to be candidates, the identification information given in advance to the center line vector CV and the attention point M for the left vertical line segment Vl or the right vertical line segment Vr determined to be a candidate. And the left vertical line segment Vl or the right vertical line segment Vr, that is, the vertical line segment determined to be a candidate is a line segment that crosses the left or right road area. Road area identification information for, Set of the determined by one or corresponds to that of the letter "l" or "r", "0" or "1" is stored in the storage device 20 as a line segment candidate location data 25 (FIG. 1) if e.

  6, the line segment candidate detection unit 300 first reads road data 21 and binarized road area image data 24 from an auxiliary storage device (not shown) in the storage device 20 into a RAM (not shown) in the processing device 10 (step S301) Unless one centerline vector process for all roads has been processed in step S302, one centerline vector CV of a plurality of centerline vectors constituting any road is selected (step S303). ). Unless it is determined in step S304 that all the pixels on the center line vector CV have been processed, one pixel on the selected center line vector CV is selected as the attention point M (Xm, Ym) (step S305). . For the attention point M, the pixel at the start point S of the center line vector CV is first selected. Each time step S305 is repeatedly executed, pixels on the center line vector CV adjacent to the start point S are sequentially selected.

  Next, the left vertical line segment discriminating unit 350 is activated. As will be described later, the left vertical line segment discriminating unit 350 determines whether or not the left vertical line segment Vl passing through the point of interest M and reaching the road end line can be a candidate for a line segment constituting the elevated road sign image. Determined. Next, the right vertical line segment discriminating unit 370 is executed for the right vertical line segment Vr, and the right vertical line segment Vr extending from the point of interest M to the road end line is an elevated road as in the left vertical line segment discriminating unit 350. It is determined whether or not the line segment constituting the sign image can be a candidate. The above processing is repeated until it is determined that all the pixels on the selected center line vector CV selected in step S304 have been processed. When it is determined in step S304 that all the pixels have been selected, the process proceeds to step S302, where a new center line vector is selected in step S303 until it is determined that all the center line vectors have been processed. The subsequent processing is repeated. Thereafter, when it is determined in step S302 that all center line vectors belonging to all roads have been processed, the line segment candidate detection unit 300 ends the process, and in FIG. 1, the program 40 ends the process.

  FIG. 8 is a schematic flowchart of processing of the left vertical line segment determination unit 350 executed by the line segment candidate detection unit 300. Before describing the details of the processing of the left vertical line segment determination unit 350, the principle of determination as to whether or not the vertical line segment is a road sign image constituent line segment candidate in the present embodiment will be described. In the present embodiment, the elevated road sign image extends linearly from the center of the road center line to the road edge line across the roadway, and all of the pixels are one or more pixels having bright brightness. It is assumed to consist of line segments. In order for the right vertical line segment or the left vertical line segment detected by the line segment candidate detection unit 300 to be such a line segment candidate, all of the pixels constituting the line segment are pixels with high brightness. However, when the upper part of the elevated road sign is dirty, for example, it is possible that the brightness of some of the pixels constituting the line segment is not high. Therefore, the brightness of a part of the pixels of the line segment after binarization by the binarization unit 200 may be “0”. However, from the viewpoint of the entire line segment, if the brightness of the pixels close to all the pixels is all “1”, it can be estimated that the line segment is composed of pixels having the brightness “1”. Therefore, in the present invention, the total number Nw of pixels having the brightness of “1” among all the pixels constituting the vertical line segment (hereinafter sometimes referred to as the number of bright pixels in the vertical line segment or the number of bright pixels in the line segment). ) Is equal to or greater than the product Nt × R of the total number of pixels Nt in the line segment and a fixed ratio R, that is, if the following equation (1) is satisfied, the line segment is a road sign image constituent line segment candidate. Judging that it is.

Nw ≧ Nt × R (1)
Here, the value of the ratio R may be determined by experiment for the binarized road area image data 24. On the road, there is a pattern whose brightness indicating a car or a lane boundary line is “1”, so even if the vertical line segment being processed passes over these patterns, the vertical line segment is It is necessary to determine the ratio R so that it is not erroneously detected as a road sign image configuration candidate. In general, even when a vertical line being processed passes over a pattern indicating a car or a lane boundary line with a lightness value of “1”, around the pattern indicating the car or the lane boundary line Since there are many dark pixels having a lightness of “0”, the vertical line segment is displayed when the vertical line segment passes over those patterns and when the road sign image passes. Since the total number of bright pixels having the value “1” included in the minute is greatly different, the ratio R can be determined so that these two cases can be distinguished. As the ratio R, for example, 0.7 or 0.8 can be used.

Hereinafter, the product is referred to as a candidate eligible minimum pixel number and may be described as Ntr. At this time, Formula 1 can be replaced by Formula 2.
Nw ≧ Ntr (2)

In order to determine whether or not the condition indicated by Expression 1 or Expression 2 is satisfied, it is necessary to count the number Nw of bright pixels in the line segment. The time required for this counting depends on the number of bright pixels Nw in the line segment. In the present embodiment, there is a possibility that the vertical line segment is a road sign image constituent line segment candidate even before the end of counting the number of bright pixels in the line segment Nw before the determination by Expression 1 or Expression 2. Note that there is a case where it is possible to determine that there is not, the presence or absence of the possibility is determined before the number Nw of bright pixels in the line segment is counted. That is, the number of pixels (sometimes referred to as the number of partial pixels) of a part of the vertical line segment to be determined is Np, and the number of bright pixels (sometimes referred to as the number of partial bright pixels) among those pixels. Is assumed to be Npw, and assuming that all the remaining pixels (Nt−Np) in the line segment are bright pixels, the line segment is a road sign image constituent line segment candidate. Equation 3 must be satisfied.
Npw ≧ Ntr− (Nt−Np) (3)

Conversely, when Expression 4 or Expression 5 is satisfied, the vertical line segment has no possibility of being a road sign image constituent line segment candidate. Here, the minimum value of Npl defined by Equation 5 may be referred to as the number of candidate partial minimum bright pixels, and may simply be expressed as Npl.
Npw <Ntr− (Nt−Np) (4)
Npw ≦ Npl = Ntr− (Nt−Np) −1 (5)

Here, as an example, the total number of pixels in the line segment Nt is 24 pixels. The road width is approximately 4 × 0.6 = 14.4 (m) when the image resolution is 0.6 (m). Assuming that the ratio R is 0.7, in this assumption, the number of bright pixels Nw in the line segment is Nw ≧ Ntr = in order for the vertical line segment to be a road sign image constituent line segment candidate according to Formula 1 or Formula 2. 24 × 0.7 = 16.8, that is, Equation 6 needs to be satisfied.
Nw ≧ 17 (6)
That is, the candidate eligible minimum pixel number Ntr is 17. Therefore, in order for a vertical line segment to be a road sign image constituent line segment candidate, a bright pixel having 17 or more pixels must be included in the vertical line segment.

Further, from Expression 5, the number Npl of candidate failed minimum partial bright pixels is represented by Expression 7.
Npl = Np-8 (7)
Therefore, there is no possibility that the vertical line segment becomes a road sign image constituent line segment candidate when the relationship between the number of partial bright pixels Npw and the number of partial pixels Np is represented by Expression 8.
Npw ≦ Npl = Np-8 (8)

  FIG. 9 is a diagram showing a comparison between the number Np of partial pixels and the number Npl of candidate partial minimum bright pixels that satisfy the relationship of Expression 8. In the figure, for example, when the number of partial pixels Np is 8, if the number of bright pixels in the partial pixel group is 0, a vertical line segment including the partial pixel group can be a candidate road sign image constituent line segment. There will be no sex. Similarly, for example, when the number of partial pixels Np is 16, if the number of bright pixels in the partial pixel group is 8 or less, the vertical line segment including the partial pixel group becomes a road sign image constituent line segment candidate. There will be no possibility. Furthermore, when the number of partial pixels Np is equal to the value 24 of the total number of pixels in the line segment, if the number of bright pixels in the line segment is 16 or less, the vertical line segment is not a road sign image constituent line segment candidate. It will be. Therefore, in order to determine whether the vertical line segment to be processed is a road sign image constituent line segment candidate, the vertical line segments are ordered before counting the total number of bright pixels in the vertical line segment. Divided into a plurality of partial pixel columns, the number Npw of partial bright pixels included in the first partial pixel column is counted, and the number of partial bright pixels Npw fails the candidate shown in FIG. 9 with respect to the partial pixel number Np. It is determined whether or not the number is the minimum partial bright pixel number Npl or not. Therefore, in this case, the number of bright pixels in the remaining (Nt−Np) pixels is counted, the number of bright pixels in the line segment is counted by reflecting the counted value, and the number of bright pixels in the line segment is counted. Even if it is not determined whether or not Nw is equal to or greater than the lower limit value 17 shown in Expression 6, it is possible to determine that the vertical line segment is not a road sign image constituent line segment candidate. The processing of unit 350 can be terminated at that time.

  If the number Npw of partial bright pixels in the first partial pixel column in the vertical line segment is larger than the number of candidate partial minimum bright pixels Npl shown in FIG. 9 with respect to the number of partial pixels in the partial pixel group Since it cannot be determined that there is no possibility of becoming a road sign image constituent line segment candidate only by the partial pixel column, the number of partial bright pixels in the next partial pixel column is counted, and the first partial bright pixel number and The new partial bright pixel number Npw is calculated based on the sum of the two, and based on the partial bright pixel number Npw, it is determined whether or not there is a possibility that the vertical line segment is a candidate road sign image component line segment according to FIG. What is necessary is just to judge. In this case, when it is determined that the vertical line segment is not likely to be a road sign image constituent line segment candidate, the processing of the left vertical line segment determination unit 350 for the vertical line segment is terminated at that time. In this way, the determination as to whether or not the vertical line segment to be processed is a road sign image constituent line segment candidate is made by dividing the vertical line segment into a plurality of partial pixel groups and sequentially increasing the partial pixel groups to be determined. By doing so, a vertical line segment that is not a road sign image constituent line segment candidate can be detected early.

  Hereinafter, based on FIG. 8, an outline of the processing of the left vertical line segment determination unit 350 will be described. Also in the following description, as assumed above, the total number of pixels in the line segment Nt is assumed to be 24 pixels, the ratio R is assumed to be 0.7, and the comparison table of FIG. 9 can be used with the above assumptions. In the following, since the vertical pixel column to be processed is composed of 24 pixels, a case where this vertical line segment is divided into three partial pixel columns will be described. Therefore, each partial pixel column consists of 8 pixels. Needless to say, the present invention can be applied to other division numbers.

  The left vertical line segment determination unit 350 is activated after the pixel of one attention point on the center line vector is selected in step S305 (FIG. 6) of the line segment candidate detection unit 300, and includes the attention point. For the left vertical line segment that is perpendicular to the center line vector and reaches the road edge line of the road area on the left side of the center line vector, it is determined whether or not the line segment is a road sign image constituent line segment candidate. For this purpose, the left vertical line segment determination unit 350 first sets the partial pixel group number n to 1 in step S351 in FIG. Next, in step S352, the number Npwn of partial pixel groups, which is the number of pixels having the brightness of “1”, in the nth partial pixel group of the left vertical line segment is counted. In step S353, it is determined whether or not the partial pixel group number n is 1. When the partial pixel group number n is 1, as in the present assumption, in step S354, the partial bright pixel number Npw is set to the partial pixel group bright pixel number. Equal to Npwn. In step S356, it is determined whether or not the partial pixel group number n is equal to the partial pixel group number np. If n = 1 as in the present assumption, this is not the case. In step S357, the left vertical line segment being processed is determined. Is likely to be a candidate road sign image constituent line segment. Specifically, the number Np of partial pixels is the number of pixels 8 in the first partial pixel group. From FIG. 9, the left vertical line segment being processed depends on whether or not the number Npw of partial bright pixels determined in step S354 is greater than the number of candidate partial minimum bright pixels Npl = 0 for the partial pixel number Np = 8. Is likely to be a candidate road sign image constituent line segment.

  When it is determined that there is no such possibility, the left vertical line segment determination unit 350 ends the process. When it is determined that there is such a possibility, the value of the partial pixel group number n is increased by 1 and changed to the value 2 in step S358, and the process returns to step S352. In step S352, the number Npwn of partial pixel groups, which is the number of bright pixels in the nth partial pixel group, is counted. Thereafter, in step S353, it is determined whether or not the partial pixel group number n is equal to 1, but in this case, this is not the case, so in step S355, the partial pixel group up to the (n−1) th partial pixel group. On the other hand, the partial bright pixel number Npw newly counted in step S352 is added to the partial bright pixel number Npw calculated at the time of the previous execution of step S355 to newly obtain the partial bright pixel number Npw. In step S356, it is determined whether or not the partial pixel group number n is equal to the partial pixel group number np. If n = 2 as in the present assumption, this is not the case. In step S357, the left vertical line segment being processed is determined. Is likely to be a candidate road sign image constituent line segment. Specifically, the number Np of partial pixels in the partial pixel group up to the second partial pixel group is equal to 16 in the present case. From FIG. 9, the left vertical line segment being processed depends on whether or not the number Npw of partial bright pixels determined in step S355 is larger than the number of candidate partial minimum bright pixels Npl = 8 for the partial pixel number Np = 16. Is likely to be a candidate road sign image constituent line segment.

  When it is determined that there is no such possibility, the left vertical line segment determination unit 350 ends the process. When it is determined that there is such a possibility, the value of the partial pixel group number n is increased by 1 and changed to the value 3 in step S358, and the process returns to step S352. In step S352, the number Npwn of partial pixel groups, which is the number of bright pixels in the nth partial pixel group, is counted. Thereafter, in step S353, it is determined whether or not the partial pixel group number n is equal to 1, but in this case, this is not the case, so in step S355, the partial pixels up to the (n−1) th partial pixel group are determined. The partial bright pixel count Npwn counted in step S352 is added to the partial bright pixel count Npw already calculated in step S355 for the group to newly obtain the partial bright pixel count Npw. In step S356, it is determined whether or not the partial pixel group number n is equal to the partial pixel group number np. If n = 3 as in the present assumption, this is the case. In step S359, the left vertical line being processed is determined. It is determined whether or not the minute is a road sign image constituent line segment candidate. Specifically, the number Np of partial pixels in the partial pixel group up to the third partial pixel group is equal to 24 in the present case. From FIG. 9, the left vertical line segment being processed depends on whether or not the number Npw of partial bright pixels determined in step S355 is larger than the number Npl = 16 of candidate partial minimum bright pixels with respect to the partial pixel number Np = 24. Is a road sign image constituent line segment candidate.

  If it is determined in step S359 that the candidate is not a candidate, the left vertical line segment determination unit 350 ends the process, and the process returns to the line segment candidate detection unit 300. When it is determined as a candidate, in step S360, the line segment candidate position data regarding the vertical line segment being processed is stored in the storage device 20 (FIG. 1) as a part of the line segment candidate position data 25. Thereafter, the left vertical line segment determination unit 350 ends the process, and the process returns to the line segment candidate detection unit 300.

  FIG. 10 shows an example of data stored in the line segment candidate position data 25. As described above, if it is determined in step S359 that the left vertical line segment being processed is a road sign image constituent line segment candidate, the identification information 251 of the road being processed and the identification of the centerline vector being processed The information 252, the coordinates 253 of the attention point M, and the character “l” indicating the left road area or the corresponding code “0” as the road area left / right discrimination information 254 are stored.

  Returning to FIG. 6, the line segment candidate detection unit 300 next executes the right vertical line segment determination unit 370. The right vertical line segment discriminating unit 370 is a left vertical line segment discriminating unit for a right vertical line segment that passes through the attention point M selected in step S305 and is perpendicular to the center line vector and extends to the road end line of the right side area of the road. In the same manner as 350, it is determined whether or not the right vertical line segment is a road sign image constituent line segment candidate. When it is determined that the right vertical line segment is a candidate, the line segment candidate position data regarding the right vertical line segment is determined. Is stored in the storage device 20 (FIG. 1) as a part of the line segment candidate position data 25. The line segment candidate position data stored by the right vertical line segment discriminating unit 370 includes the letter “r” indicating the right road area or the corresponding code “1” as the left / right discrimination information 254 of the road area in FIG. 10. The only difference is that Returning to FIG. 6, when the processing of the right vertical line segment discriminating unit 370 is completed, step S304 and subsequent steps are repeated, and thereafter step S302 and subsequent steps are repeated. The above processing is performed for all center line vectors on all roads. Repeat for the other pixels.

  FIG. 11 is a diagram showing a ground image represented by actual ground image data obtained by photographing the ground with an artificial satellite. The road shown in FIG. 11 is a road having three lanes on one side, as schematically shown in FIG. 4, and linearly extends from the lower left to the upper right. There are two elevated road signs surrounded by two circles near the center of FIG. FIG. 12 is a binarized road area image indicated by the binarized road area image data 24 obtained by performing binarization processing by the binarization unit 200 on the ground image of FIG. In FIG. 12, as indicated by two circles, most of the range from the center line vector to the road edge is a line segment that constitutes the road sign image and is perpendicular to the center line vector. There are line segments in which all the pixels are bright pixels, and those line segments are detected by the line segment candidate detection unit 300 as road sign image constituent line segment candidates.

  As described above, in the present embodiment, line segment candidates constituting a wide range of road sign images can be automatically detected from the ground image at high speed. In particular, elevated road signs are installed at very large intervals on the highway. Therefore, in many attention pixels on the center line, the line segment including the pixel is not a candidate for the line segment constituting the road sign image. In the present invention, since it can be determined at an early stage that the line segment is not a candidate for such a road sign image, the effect of speeding up the process is great. After the road sign image constituent line segment candidate is detected in this way, the pattern of the image around the line segment is analyzed to automatically determine whether or not a road notation is installed there. It is also possible. Alternatively, an image of the vicinity of the detected line segment candidate is sequentially enlarged and displayed on the display device, and an elevated road sign is actually installed in the vicinity of the line segment candidate by a person who wants to generate road sign installation position information. You may make it determine visually whether it is carried out. In this way, after determining whether or not the vertical line segment detected as a candidate actually represents a road sign, the information collection vehicle is driven on the highway to photograph the road sign image to obtain the road sign information. When acquiring, when the information collection vehicle approaches the candidate position where it is determined that there is actually a road sign, the distance between the vehicle and the preceding vehicle is sufficiently maintained and the traveling speed is reduced, etc. The shooting of the road sign at the candidate position can be ensured.

  In the above embodiment, the case where the road width for one centerline vector is 24 bits has been specifically described. The road width may change within one centerline vector. For example, the road width at the start point and end point of the center line vector may be different. Alternatively, the width of the road area may be different between the left side and the right side of the road. Thus, the present invention can also be applied when the width of the road presence area changes.

  For this purpose, the following processing may be performed. As the processing of the line segment candidate detection unit 300 in FIG. 6, the road width wl of the left road area is determined for each attention point between step S304 and the left vertical line segment 350, and the length of the road width wl is determined. The total number of pixels Nt included in the vertical line segment is determined, and an expression between the number of partial pixels Np and the number of candidate partial minimum bright pixels Npl is finally obtained using the ratio R used in the above embodiment. It is only necessary to determine a relational expression in place of 8 and then determine the division number n for dividing the vertical line segment into a plurality of appropriate partial pixel columns, and then execute the left vertical line segment 350. The road width wl (m) of the left road region at the point of interest is determined, for example, by interpolating the road widths at the start point and end point of the center line vector of the center line vector used in the above embodiment. Then, assuming that the resolution of the satellite image is P (m / pixel), the total number of pixels Nt included in the vertical line segment of the length of the road width from the point of interest on the center line vector to the road edge line is expressed as W / P Determined by the value of. Based on the number of pixels Nt and Expression 1, a condition that the number Nw of bright pixels necessary for the line segment perpendicular to the center line vector to be a candidate for the road sign image constituent line segment should be satisfied. Similarly, the candidate eligible minimum pixel number Ntr is determined from Equation 2. Thereafter, the relationship between the number of candidate partial minimum bright pixels Npl and the number of partial pixels Np is determined by Equation 5, and an equation in place of Equation 7 is determined. Using the result, as a condition that a line segment perpendicular to the center line vector cannot be a candidate for a road sign image constituent line segment, the number of partial pixels Np and the number of candidate partial minimum bright pixels Npl A relational expression instead of the expression 8 is determined. Using this relational expression, it is also possible to determine a comparison table between the number of partial pixels Np and the number of candidate rejected minimum partial bright pixels Npl similar to FIG.

  In the above embodiment, when the total number of pixels Nt of the vertical line segment is 24, this line segment is divided into three to determine three partial pixel columns each consisting of 8 pixels. Even when the total number of pixels Nt of the vertical line segment is different from 24, a method for dividing the vertical line segment into an appropriate number of partial pixel columns should be determined. For example, in the embodiment, one partial pixel row is composed of 8 pixels, and the length thereof is 4 (m) from the resolution of the satellite image. In the case of FIG. 4, there are three lanes in the road area on one side. Therefore, in the above embodiment, the number of pixels in one partial pixel column is 8 bits, which corresponds to the width of one lane. Accordingly, even when the total number of pixels Nt of the vertical line segment is different from 24, the number of pixels of one partial pixel column can be set to 8 bits or to correspond to the width of one lane.

  Thereafter, in FIG. 6, the left vertical line segment 350 is executed according to FIG. 8, and then the right vertical line segment is determined. Before the determination, when the road width of the right road area is different from the left road area, Before executing the right vertical line segment determination 370, as described above, the road width w of the right road region is determined, the total number of pixels Nt of the vertical line segment is determined, and as described above As a condition for determining the total number of pixels Nt and that there is no possibility that a line segment perpendicular to the center line vector is a candidate for a road sign image constituent line segment, the number of partial pixels Np and the number of candidate partial minimum bright pixels Npl A relational expression instead of the expression 8 is determined, and then the right vertical line segment determination 370 may be executed in the same manner.

1 is a schematic block diagram of an embodiment of an apparatus capable of recognizing a road sign of an image according to the present invention. It is a schematic diagram of a road. (A) is a figure which shows the example of the road vector with respect to the road shown by FIG. (B) is a figure showing an example of road data. It is a figure which shows typically the partial road area | region containing one centerline vector of a road. It is a figure which shows typically the binarization partial road area image corresponding to the partial road area shown by FIG. It is a flowchart of a process of a line segment candidate detection part. It is a figure for demonstrating the left vertical line segment and right vertical line segment which are processed by the line segment candidate detection part regarding one center line vector. It is a schematic flowchart of the process of the left vertical line segment discrimination | determination part performed by a line segment candidate detection part. It is a figure which compares and shows the number of partial pixels (Np) and the number of candidate partial minimum bright pixels (Npl). It is a figure which shows the example of line segment candidate position data. It is a figure which shows the ground-surface image represented by the actual ground-surface image data obtained by image | photographing the ground with an artificial satellite. 12 is a binarized road area image indicated by binarized road area data obtained by performing binarization processing on the ground surface image of FIG. 11 by a binarization unit.

Explanation of symbols

  CL: Center line, CV: Center line vector, E1, E11, E12, E13, E2, E21, E22, E23 ... Road end line, R1, R2 ... Elevated road sign, S1, S2, S3 ... Center line vector Start point, G1, G2, G3 ... end point of center line vector, w1, w2, w3 ... road width.

Claims (2)

Candidate line segments constituting a road sign image representing an elevated road sign provided so as to pass over the area of at least one side of the road are detected from ground image data obtained by photographing the ground surface from the sky. A method for detecting candidate road segment image line segments,
A road area detecting step for detecting a road area where any road exists from the surface image data;
For each of a series of pixels extending along the extending direction of the road on the center line of the road area, the pixel including the pixel and reaching the area where the road edge exists from the center line is perpendicular to the center line The number of bright pixels whose brightness is higher than a predetermined threshold among the vertical line segments corresponding to the pixel extending in a certain direction constitutes a road sign image in which the corresponding vertical line segment indicates the elevated road sign Discriminating whether or not the corresponding vertical line segment is a candidate for the line segment constituting the road sign image, depending on whether or not the predetermined number or more is necessary to include the line segment Steps,
A storage step of storing position information representing the position of the vertical line segment when the corresponding vertical line segment is determined to be a line segment candidate constituting the road sign image;
Including
The determination step includes
The number of bright pixels included in some of the pixel groups included in the corresponding vertical line segment is counted, and when it is assumed that all the remaining pixels are bright pixels, the bright pixels included in the vertical line segment are counted. It is determined whether or not the total number of pixels may reach the predetermined number, and depending on the determination result, the corresponding vertical line segment may be a candidate for the line segment constituting the road sign image. A possibility determination step of determining whether or not there is;
As a result of the determination in the possibility determination step, when it is determined that the corresponding vertical line segment is not likely to be a line segment candidate constituting the road sign image, the remaining pixels other than the partial pixel group are determined. An early determination step of determining that the corresponding vertical line segment is not a candidate for the line segment constituting the road sign image without counting the number of bright pixels in the partial pixel group,
A road sign image constituent line segment candidate detection method characterized by the above. A road sign image representing an elevated road sign that is located in the middle of the road and crosses over the area of at least one side of the road from the ground image data obtained by photographing the ground surface from the sky. A program capable of detecting a candidate for a line segment and detecting a line segment candidate for a road sign image,
Road area detecting means for detecting a road area where any road is present from the surface image data;
For each of a series of pixels extending along the extending direction of the road on the center line of the road area, the pixel including the pixel and reaching the area where the road edge exists from the center line is perpendicular to the center line The number of pixels whose brightness is higher than a predetermined threshold among the vertical line segments corresponding to the pixel extending in a certain direction constitutes a road sign image in which the corresponding vertical line segment indicates the elevated road sign. A discriminating means for discriminating whether or not the corresponding vertical line segment is a candidate for the line segment constituting the road sign image depending on whether or not the predetermined number or more is necessary to include the line segment. When,
Storage means for storing position information representing the position of the vertical line segment when the corresponding vertical line segment is determined to be a line segment candidate constituting the road sign image;
Function as a computer
The discrimination means includes
The number of bright pixels included in some of the pixel groups included in the corresponding vertical line segment is counted, and when it is assumed that all the remaining pixels are bright pixels, the bright pixels included in the vertical line segment are counted. It is determined whether or not the total number of pixels may reach the predetermined number, and depending on the determination result, the corresponding vertical line segment may be a candidate for the line segment constituting the road sign image. Possibility determination means for determining whether or not there is,
As a result of the determination by the possibility determination means, when it is determined that there is no possibility that the corresponding vertical line segment is a candidate for the line segment constituting the road sign image, the remaining pixels other than the partial pixel group are determined. Early determination means for determining that the corresponding vertical line segment is not a candidate for the line segment constituting the road sign image without counting the number of bright pixels in the partial pixel group;
As a computer to function,
A program capable of detecting a candidate road segment image line segment.