JP6526449B2 - Road attachment detection device and road attachment detection method - Google Patents

Description

  Embodiments of the present invention relate to a road appendage detection device and a road appendage detection method.

  Road accessories such as road signs, road mirrors, street lights, etc. are important to ensure safety for vehicles traveling on the road. For this reason, conventionally, a road appendage detection device is known which detects a road appendage based on a captured image around a road attachment.

Patent No. 4394476 Patent No. 3006471 JP, 2013-54522, A

  In such a road attachment detection device, it is desirable to detect as many types of road attachments as possible with high accuracy.

The road accessory detection device according to the embodiment includes a pillar area detection unit, a specific color detection unit, a specific shape detection unit, a determination unit, and an output unit. The pillar area detection unit detects a pillar area, which is an area of a pillar portion to be detected, from the captured image of the road on which the vehicle travels, based on the pixels of the captured image. The specific color detection unit detects a specific color area that is an area having a specific color to be detected from the captured image.
The specific shape detection unit detects a specific shape area, which is an area having a specific shape to be detected, from the captured image. The determination unit detects a pillar area, a specific color area, and a specific shape area from the captured image, and the distance between the areas on the captured image is less than a predetermined distance. It is determined that the area indicated by the area and the specific shape area is a road appendage disposed around the road. The output unit outputs information on road attachments.

FIG. 1 is a block diagram showing a functional configuration of the road accessory detection device of the first embodiment. FIG. 2 is a view showing an example of the configuration of the road attachment according to the first embodiment. FIG. 3 is a flowchart illustrating an example of a road accessory detection process according to the first embodiment. FIG. 4 is a flowchart illustrating an example of a procedure of pillar area detection processing according to the first embodiment. FIG. 5 is a diagram illustrating an example for explaining the edge enhancement processing of the first embodiment. FIG. 6 is a diagram showing an example of pixel extraction results of edge features in the first embodiment. FIG. 7 is a diagram for explaining the vertical direction accumulation process of the first embodiment. FIG. 8 is a flowchart illustrating an example of the procedure of the specific color detection process according to the first embodiment. FIG. 9 is a diagram for explaining the specific color detection process of the present embodiment. FIG. 10 is a diagram for explaining the specific color detection process of the present embodiment. FIG. 11 is a diagram for explaining the specific color detection process of the present embodiment. FIG. 12 is a flowchart illustrating an example of the procedure of the specific shape detection process according to the first embodiment. FIG. 13 is a diagram for explaining the specific shape detection process of the first embodiment. FIG. 14 is a flowchart illustrating an example of the procedure of the determination process of the first embodiment. FIG. 15 is a diagram illustrating an example of detection results of a pillar area detection process, a specific color detection process, and a specific shape detection process according to the first embodiment. FIG. 16 is a diagram illustrating an example of the determination result of the first embodiment. FIG. 17 is a diagram illustrating another example of the determination result of the first embodiment. FIG. 18 is a diagram illustrating another example of the determination result of the first embodiment. FIG. 19 is a diagram illustrating an example of the output result of the road appendage according to the first embodiment. FIG. 20 is a block diagram showing a functional configuration of the road accessory detection device of the second embodiment. FIG. 21 is a diagram showing an example of conversion of the shape pattern of the second embodiment. FIG. 22 is a flowchart illustrating an example of a road accessory detection process according to the second embodiment. FIG. 23 is a diagram illustrating an example of how a circular road mirror of the second embodiment is viewed from a vehicle. FIG. 24 is a block diagram showing a functional configuration of the road accessory detection device of the third embodiment. FIG. 25 is a diagram illustrating an example of imaging directions of two imaging cameras according to the third embodiment. FIG. 26 is a diagram for explaining an example of road accessory detection according to the third embodiment. FIG. 27 is a flowchart illustrating an example of a road accessory detection process according to the third embodiment.

  Hereinafter, embodiments will be described.

(Embodiment 1)
FIG. 1 is a block diagram showing a functional configuration of the road accessory detection device of the first embodiment. As shown in FIG. 1, the road accessory detection apparatus 100 according to the first embodiment includes an imaging camera 110, a pillar area detection unit 101, a specific color detection unit 102, a specific shape detection unit 103, and a determination unit 104. An output unit 105, a clock unit 106, a position detection unit 107, a color database 120 (hereinafter referred to as "color DB 120"), and a shape database 130 (hereinafter referred to as "shape DB 130") are mainly provided. ing. Here, the color DB 120 and the shape DB 130 are stored in a storage medium such as a hard disk drive (HDD), a solid state drive (SSD), or a memory.

  The road accessory detection device 100 according to this embodiment is a device that is mounted on a vehicle traveling on a road and detects a road attachment. Here, the road attachment is a facility or a work necessary for the maintenance of the structure of the road, securing of the traffic of the road safely and smoothly, and other management of the road (Road Act Article 2 Section 2). Examples of such road accessories include, but are not limited to, road signs, road mirrors, street lights and the like.

  FIG. 2 is a view showing an example of the configuration of the road attachment according to the first embodiment. A road sign or a road mirror, which is an example of a road attachment, generally includes an edge area 201, a central area 202, and a pillar area 203, as shown in FIG. The edge area 201 is the area of the edge of a sign or mirror. The central region 202 is a region of a portion inside the edge region 201 of the marker or mirror. The pillar region 203 is a region of the pillar portion that supports the portion consisting of the central region 202 and the edge region 201.

  Returning to FIG. 1, the imaging camera 110 is installed outside or inside the vehicle, captures an area around the road ahead of the traveling vehicle at a constant time interval, and outputs a captured image.

  The pillar area detection unit 101 inputs a captured image of a road around the image captured by the imaging camera 110, and adds a pair of lines in the vertical direction to the road based on the pixels constituting the captured image from the input captured image. Detect as a pillar area of an object. Specifically, the pillar region detection unit 101 performs edge enhancement on the captured image, extracts edge features from the edge-emphasized pixels, and accumulates the extracted pixels in the vertical direction of the captured image. A column portion consisting of a pair of lines is detected to form a column area, and the coordinate range of the column area is output.

  The specific color detection unit 102 inputs the captured image captured by the imaging camera 110, and uses the R value, the G value, and the B value of the captured image to have the color (specific color) of the road appendage to be detected. Detect a specific color area. Specifically, the specific color detection unit 102 compares the R value, the G value, and the B value of each pixel of the captured image with a predetermined range according to the specific color to be detected, and is included in the range A pixel is extracted, it is set as an area (specific color area) having a specific color, and the coordinate range of the specific color area is output. In the color DB 120, the maximum value and the minimum value of each of R, G, and B values that define the above range are registered.

  The specific shape detection unit 103 inputs a captured image captured by the imaging camera 110, and detects a specific shape region, which is a region having a shape (specific shape) of a road appendage to be detected, from the captured image. Specifically, the specific shape detection unit 103 performs edge enhancement on the captured image as in the pillar region detection unit 101, and extracts an edge feature from the edge-emphasized pixel. Then, the specific shape detection unit 103 performs pattern matching with a predetermined shape pattern of the road appendage on the extracted edge feature, extracts the specific shape of the road appendage, and sets it as a specific shape region, and identifies Output the coordinate range of the shape area. Here, the shape pattern of the road attachment is registered in the shape DB 130 for each road attachment.

  The determination unit 104 captures an image based on the pillar area detected by the pillar area detection unit 101, the specific color area detected by the specific color detection unit 102, and the specific shape area detected by the specific shape detection unit 103. In the image, it is determined whether there is a road attachment placed around the road. Specifically, when any of the pillar area, the specific color area, and the specific shape area is detected in the captured image, the determination unit 104 determines the distance on the captured image between the areas from the coordinate range of each area. Is determined to be less than a predetermined distance. Then, when the distance between the areas on the captured image is less than a predetermined distance, the determination unit 104 indicates that the area indicated by the pillar area, the specific color area, and the specific shape area in the captured image is a road attachment It is determined that On the other hand, when any of the distances on the captured image between the regions is equal to or greater than a predetermined distance, the determination unit 104 determines that there is no road attachment in the captured image.

  In addition, when any of the pillar area, the specific color area, and the specific shape area is not detected, the determination unit 104 determines that the road attachment is not present in the captured image.

  The clock unit 106 clocks the current time. The position detection unit 107 acquires the current position of the vehicle using a GPS (Global Positioning System) or the like.

  When the road attachment is detected by the determination unit 104, the output unit 105 outputs information on the road attachment to a monitor in the vehicle along with the current time and the current position of the vehicle.

  Next, the road accessory detection process of the present embodiment configured as described above will be described in detail. FIG. 3 is a flowchart illustrating an example of a road accessory detection process according to the first embodiment. Here, the processing of the flowchart shown in FIG. 3 is executed at predetermined time intervals such as every several seconds, for example.

  First, the imaging camera 110 captures an image around the front of the vehicle. The pillar area detection unit 101, the specific color detection unit 102, and the specific shape detection unit 103 input a captured image from the imaging camera 110 (S11). Then, the pillar area detection unit 101 performs a pillar area detection process on the captured image (S12). The specific color detection unit 102 performs a specific color detection process on the captured image (S13). The specific shape detection unit 103 performs a specific shape detection process on the captured image (S14). And the determination part 104 performs the determination process of a road appendage based on the detection result of S12, S13, S14 (S15).

  The details of the pillar area detection process of S12 will be described. FIG. 4 is a flowchart illustrating an example of a procedure of pillar area detection processing according to the first embodiment. The pole region detection unit 101 performs edge enhancement processing on the captured image (S31). Specifically, the pillar area detection unit 101 obtains the difference between each pixel of the captured image and the neighboring pixel. FIG. 5 is a diagram illustrating an example for explaining the edge enhancement processing of the first embodiment. For example, as shown in FIG. 5, the column region detection unit 101 obtains a sum of a pixel value obtained by multiplying eight peripheral pixels by a weight “−1” and a pixel value obtained by multiplying a central pixel by a weight “9”. Then, it is possible to emphasize pixels having a luminance peak in the central visual field.

  Next, the pillar area detection unit 101 performs an extraction process of pixels of the edge feature from the edge-emphasized captured image (S32). Specifically, the pillar area detection unit 101 extracts the pixel of the edge feature as a binary image with a certain threshold value to the edge-emphasized pixel. FIG. 6 is a diagram showing an example of pixel extraction results of edge features in the first embodiment. In FIG. 6, black lines indicate extracted pixels. As shown in the example of FIG. 6, not only the road attachment 501 but also pixels such as the end of the road are extracted as extraction pixels.

  The column region detection unit 101 performs the vertical direction accumulation processing on the image of the pixel extraction result of such edge features while setting the field of view while moving the small region 502 shown in FIG. 6 (S33). FIG. 7 is a diagram for explaining the vertical direction accumulation process of the first embodiment. The pillar region detection unit 101 sets the field of view 601 while moving the small region with respect to the captured image of the edge feature pixel extraction result as shown in FIG. 7 and accumulates the extracted pixels in the vertical direction with respect to the field of view 601 And one-dimensional accumulated data 602 is collected. Thereby, pillar field detection part 101 detects a pair of lines P1 and P2.

  Next, the pillar area detection unit 101 determines the pillar area with respect to the accumulated data 602 obtained in S33 (S34). Specifically, the pillar region detection unit 101 has a cumulative value equal to or greater than a predetermined threshold value Thr for the pair of lines P1 and P2 of the cumulative data 602, and the interval w between P1 and P2 is within a predetermined range. When it is inside, a pair of lines P1 and P2 are determined to be a pillar area, and the coordinate range is output as the range of the pillar area of the road attachment. Here, the coordinate range to be output is indicated by the upper left coordinates and the lower right coordinates of the rectangular column area.

  In addition, the method of a pillar area | region detection by the pillar area | region detection part 101 is not limited to this. For example, a pillar template may be prepared in advance, and the pillar region detection unit 101 may be configured to detect a pillar region using a matching method with a captured image of pixel extraction results of edge features.

  Next, the details of the specific color detection process of S13 will be described. FIG. 8 is a flowchart illustrating an example of the procedure of the specific color detection process according to the first embodiment. First, the specific color detection unit 102 reads, from the color DB 120, the minimum values RL, GL, and BL, and the maximum values RH, GH, and BH, which are ranges of colors (specific colors) to be detected, The R value, the G value, and the B value of the pixel are compared with the minimum values RL, GL, and BL, and the maximum values RH, GH, and BH, and pixels included in the corresponding range are extracted (S51). In the present embodiment, the R value, the G value, and the B value are used, but the present invention is not limited to the RGB color space. For example, L * a * b *, xy, and HSV values can be used as another color system.

  Next, the specific color detection unit 102 extracts connected pixel groups (connected components) in the extracted pixels (S52). Then, the specific color detection unit 102 obtains a rectangle of an outer shape including the extracted connected component (S53). The specific color detection unit 102 outputs the coordinate range of the outline rectangle as the determination result of the specific color, that is, the range of the specific color area of the road appendage (S54). Here, the output coordinate range is indicated by the upper left coordinates and the lower right coordinates of the outline rectangle.

  9 to 11 are diagrams for explaining the specific color detection process of this embodiment. FIG. 9 shows an example of the result in the case where only the edge area 201 is the color to be extracted with respect to the example of the road sign shown in FIG. FIG. 10 shows an example in which the edge area 201 and the column area 203 have the color to be extracted.

  For the example of FIG. 10, as indicated by a broken-line frame in FIG. 11, the specific color detection unit 102 obtains a circumscribed rectangle for connected pixel groups, and specifies the coordinate range of the outer rectangle as the specific color area 1101. Output as

  Next, the details of the specific shape detection process of S14 will be described. FIG. 12 is a flowchart illustrating an example of the procedure of the specific shape detection process according to the first embodiment. The specific shape detection unit 103 performs edge enhancement processing on the captured image as in the pillar area detection processing, and obtains a difference between each pixel of the image and its neighboring pixels (S71). The specific shape detection unit 103 performs the extraction process of the pixel of the edge feature from the captured image subjected to the edge enhancement as in the pillar area detection process (S72).

  Then, the specific shape detection unit 103 reads the shape pattern of the road annex to be detected from the shape DB 130, and performs pattern matching on the captured image of the extraction result of the edge feature while scanning the shape pattern (S73). Then, the specific shape detection unit 103 determines and outputs a rectangular coordinate range circumscribed with the region matching the shape pattern as the specific shape region of the road appendage (S74).

  FIG. 13 is a diagram for explaining the specific shape detection process of the first embodiment. As shown in FIG. 13, the specific shape detection unit 103 sets a local region as a field of view in the captured image, and performs matching processing with the shape pattern 1201 of the specific shape. For example, the shape pattern 1201 is registered in advance in the shape DB 130 as a binary image in which the portion of the circle is “1” and the others are “0”. The specific shape detection unit 103 moves in the captured image while changing the size of the shape pattern 1201, and detects a rectangular region 1202 circumscribing the region matching the shape pattern 1201 of the specific shape. Here, as a method of matching between the image from which the edge is extracted and the shape pattern 1201, a method of measuring the degree of coincidence between binary images or a method resistant to fluctuation such as normalized correlation can be used. As another method, the specific shape detection unit 103 may be configured to use detection by matching with a co-occurrence histogram of edge directions disclosed in Japanese Patent No. 5214367.

  Next, the details of the determination process of S15 will be described. FIG. 14 is a flowchart illustrating an example of the procedure of the determination process of the first embodiment. The column area detection unit 101, the specific color detection unit 102, and the specific shape detection unit 103 output the coordinate range of each area as the detection result as shown in FIG. First, the determination unit 104 determines whether or not all of the column area, the specific color area, and the specific shape area have been detected in the captured image (S1001).

  If any of the pillar area, the specific color area, and the specific shape area is not detected (S1001: No), the determination unit 104 determines that there is no road attachment in the captured image (S1005).

  On the other hand, when all of the pillar area, the specific color area, and the specific shape area are detected (S1001: Yes), the determination unit 104 determines the distance on the captured image between the areas from the coordinate range of each area. It calculates (S1002). Then, the determination unit 104 determines whether all the distances on the captured image between the regions are less than a predetermined distance (S1003). Then, if all the distances on the captured image between the respective regions are less than the predetermined distance (S1003: Yes), the determination unit 104 determines that the pole region in the captured image, the specific color region, and the specific shape region It is determined that the area indicated by is a road attachment (S1004). On the other hand, when any of the distances on the captured image between the respective regions is separated by a predetermined distance or more (S1003: No), the determination unit 104 determines that no road attachment is present in the captured image (S1003) S1005).

  FIG. 15 is a diagram illustrating an example of detection results of a pillar area, a specific color area, and a specific shape area according to the first embodiment. 16 to 18 are diagrams showing an example of the determination result of the first embodiment.

  In the target 1 of FIG. 15, although a pillar area | region and a specific color area | region are detected, the example without a detection result of a specific shape is shown. For example, the example of a detection result as shown in FIG. 16 corresponds, and the determination result of a road attachment becomes "not applicable." In the example of FIG. 16, a filled area in the image indicates a specific color, and the distance between the pillar area 1401 and the specific color area 1402 is extracted closer than a predetermined distance, but it has a specific shape. The specific shape area corresponding to the circle is not detected.

  In the object 2 of FIG. 15, all of the pillar area area, the specific color area, and the specific shape area are detected, and each area approaches within a predetermined distance as shown in FIG. For this reason, the determination unit 104 determines as “corresponding” as the detection result of the road attachment. In FIG. 17, a filled area in the image indicates a specific color area, and a pillar area 1501, a specific color area 1502, and a specific shape area 1503 are extracted close to each other. In this example, the area of the specific color (yellow) of the edge of the mirror is small, as in the road mirror, and the specific color area 1502 is partially detected. It becomes detectable well.

  Although a pillar area, a specific color, and a specific shape are detected in the target 3 in FIG. 15, as shown in FIG. 18, since the respective areas are separated by a predetermined distance or more, the determination unit 104 detects the road attachment As a result, it determines with "not applicable". In the example of FIG. 18, the filled area in the image indicates the specific color area 1602, and the column area 1601 and the specific shape area 1603 are close to each other, but the specific color area 1602 is separated and another target is separated. It is detected.

  Returning to FIG. 3, when the determination process by the determination unit 104 is completed, the clocking unit 106 acquires the current time (S16). Then, the position detection unit 107 acquires the current position of the vehicle (S17). For example, the position detection unit 107 can acquire GPS data as position information from GPS satellites. By acquiring such GPS data and using it as an output result, it becomes possible to use it in a navigation system etc. as reference data of road accessories.

  The output unit 105 stores the determination result by the determination unit 104, the current time, and the current position in a memory or the like. Then, the output unit 105 determines whether or not the detection is ended according to the user's instruction or the like (S18). If the output unit 105 determines that the detection is not completed (S18: No), the process returns to S11 and repeatedly executes S17.

  On the other hand, when the output unit 105 determines that the detection is completed (S18: Yes), the output result as information on the road attachment is generated using the current time and the current position as the determination result stored in the memory or the like. (S19). Here, the output unit 105 determines that the current position and the current time are within the predetermined range as the determination result of the same road attachment, and records only one determination result in the output result. It may be configured to The output unit 105 outputs the output result by displaying it on a monitor or the like (S20).

  FIG. 19 is a diagram illustrating an example of the output result of the road appendage according to the first embodiment. As shown in FIG. 19, the output result includes the collection date and time, the installation location of the road attachment, the type of the road attachment, the captured image, and the like. The output unit 105 uses the current time as the collection date and time, generates the contents of the installation location based on the position information, and determines the type of road attachment from the image of the detected road attachment. Then, the output unit 105 generates the collection date and time, the installation location of the road attachment, the type of the road attachment, and the captured image as an output result.

  The output result is not limited to the format shown in FIG. 19, and the output unit 105 can be configured to output in any format. For example, an image such as a road sign of a detected road attachment or a road mirror may be displayed at a coordinate position indicated by position information of an electronic map displayed by a navigation system or the like mounted on a vehicle.

  Thus, according to this embodiment, it is possible to detect many types of road accessories with high accuracy. That is, the conventional road attachment detection device can not cover all road attachments. In the method of detecting road attachments based on color information, for yellow and blue road attachments, there are cases where the color difference from the background such as the ground or sky is not large, making it difficult to detect road attachments. There is a case. Also, the yellow color used in road mirrors is used only in a limited portion of the edge of the mirror, making it difficult to capture features from the captured image.

  On the other hand, in the method of detecting the road attachment based on the shape, the shape at the time of imaging at the time of imaging may change depending on the attachment angle of the road attachment. For this reason, in the embodiment, the road accessory detection device 100 detects a pillar area, a specific color area, and a specific shape area, and comprehensively determines the detection result of each area to determine a road attachment. There is. For this reason, according to the present embodiment, it is possible to detect road attachments by combining the mounting pillar area, the specific color area, and the specific shape area, and to detect many types of road accessories with high accuracy. Is possible.

Second Embodiment
In the first embodiment, one shape pattern is used for one road attachment when detecting a specific shape, but in the second embodiment, one shape pattern is used for one road attachment. It is used while changing it.

  FIG. 20 is a block diagram showing a functional configuration of the road accessory detection device of the second embodiment. As shown in FIG. 20, the road accessory detection apparatus 3000 according to the second embodiment includes an imaging camera 110, a pillar area detection unit 101, a specific color detection unit 102, a specific shape detection unit 103, and a determination unit 104. An output unit 105, a clock unit 106, a position detection unit 107, a color DB 120, a shape DB 130, and a specific shape conversion unit 3010 are mainly provided. Here, functions of the imaging camera 110, column area detection unit 101, specific color detection unit 102, specific shape detection unit 103, determination unit 104, output unit 105, clock unit 106, position detection unit 107, color DB 120, shape DB 130, The configuration is the same as that of the first embodiment.

  The specific shape conversion unit 3010 reads the shape pattern of the specific shape to be detected from the shape DB 130 and converts the shape. FIG. 21 is a diagram showing an example of conversion of the shape pattern of the second embodiment. The specific shape conversion unit 3010 converts the shape pattern in accordance with the change in appearance as the appearance of the road attachment changes depending on the travel of the vehicle. The specific shape detection unit 103 detects a specific shape for each shape pattern that is changed by the specific shape conversion unit 3010.

  FIG. 22 is a flowchart illustrating an example of a road accessory detection process according to the second embodiment. Here, the processing of the flowchart shown in FIG. 22 is executed at predetermined time intervals such as every several seconds, for example. The input of the captured image in S11 to the specific color detection process in S13 are performed in the same manner as in the first embodiment. Next, the specific shape conversion unit 3010 reads the shape pattern of the specific shape to be detected from the shape DB 130, and converts only one pattern (S91). Then, the specific shape detection unit 103 performs a specific shape detection process using the shape pattern converted into one pattern (S14). The specific shape conversion unit 3010 converts the shape pattern one pattern at a time as shown in FIG. 21 each time the processing from S11 to S17 is repeated, and the specific shape detection unit 103 detects a specific shape. The processes after S15 are performed in the same manner as in the first embodiment.

  Although there is a method of detecting a road attachment based on the shape, depending on the attachment angle of the road attachment, the shape at the time of imaging at the time of imaging may change. For example, since a road mirror which is a circle when viewed from the front is attached obliquely to the traveling direction of the road, an elliptical image is obtained when observed from a traveling vehicle, and the road attachment detection processing based on the shape of the circle is applied as it is Can not.

  FIG. 23 is a diagram illustrating an example of how a circular road mirror of the second embodiment is viewed from a vehicle. The road mirror 2301 is not installed toward the front as viewed from the vehicle, and in a captured image captured from the vehicle, it is deformed into an ellipse as shown in FIG. The specific shape detection unit 103 detects road attachments installed at various angles using a plurality of types of shape patterns as described above.

  As described above, in the embodiment, the road attachment detection device 3000 detects a specific shape while changing the detection of the specific shape while converting the shape pattern of the road attachment. For this reason, according to the present embodiment, it is possible to detect the road attachment by combining the mounting pillar area, the specific color area, and the specific shape area, and correspond to the change in appearance depending on the travel of the vehicle. be able to. Therefore, according to the present embodiment, many types of road attachments can be detected with higher accuracy.

  In the present embodiment, one type of shape pattern is converted by the specific shape conversion unit 3010, but a plurality of shape patterns after conversion for one type of shape pattern are registered in the shape DB 130 in advance, The shape DB 130 and the specific shape detection unit 103 may be configured to use this in the specific shape detection.

(Embodiment 3)
In the third embodiment, a plurality of imaging cameras are mounted on a vehicle, and a road attachment is detected using two captured images obtained by imaging the front direction and the rear direction of the vehicle.

  FIG. 24 is a block diagram showing a functional configuration of the road accessory detection device of the third embodiment. As shown in FIG. 24, the road accessory detection device 5000 according to the second embodiment determines two imaging cameras 110a and 110b, a pillar area detection unit 101, a specific color detection unit 102, a specific shape detection unit 103, and A unit 2404, an output unit 105, a clock unit 106, a position detection unit 107, a color DB 120, a shape DB 130, and a specific shape conversion unit 3010 are mainly included.

  The imaging camera 110a is installed in the vehicle so as to be able to image around the road ahead of the vehicle. The imaging camera 110 b is installed in the vehicle so as to be able to image around the road behind the vehicle. FIG. 25 is a view showing an example of imaging directions of the two imaging cameras 110a and 110b of the third embodiment. In FIG. 25 (a), the imaging camera 110a uses the forward direction 2502 with the light axis aligned with the traveling direction of the vehicle 2501 as the imaging direction, and the imaging camera 110b captures the traveling direction of the vehicle 2501 and the direction 2503 behind 180 ° It shows an example installed as.

  In FIG. 25B, the imaging camera 110b takes a direction 2602 that is 45 ° oblique to the traveling direction of the vehicle 2501 as the imaging direction, and the imaging camera 110b does a direction 2603 that is 45 ° oblique from the rear of 180 ° of the traveling direction Shows an example in which the image pickup direction is set. The imaging direction of the imaging cameras 110a and 110b may be any of FIGS. 25 (a) and 25 (b) as long as it is a direction in which the road attachment installed around the road is included in the imaging range. It may be other than (b).

  An image captured by the imaging camera 110a is referred to as a forward captured image, and an image captured by the imaging camera 110b is referred to as a backward captured image. The front captured image is an example of a first captured image, and the rear captured image is an example of a second captured image.

  The function and configuration of the pillar area detection unit 101, the specific color detection unit 102, the specific shape detection unit 103, the output unit 105, the clock unit 106, the position detection unit 107, the color DB 120, the shape DB 130, and the specific shape conversion unit 3010 Similar to 2.

  However, the pillar area detection unit 101 detects a pillar area from each of the front captured image and the rear captured image. The specific color detection unit 102 detects a specific color area from each of the front captured image and the rear captured image. More specifically, the specific color detection unit 102 detects a specific color area having a specific color of the surface of the road attachment from the front captured image, and has a specific color of the back of the road attachment from the rear captured image Detect a specific color area. The specific color of the front surface and the specific color of the back surface may be the same or different depending on the road attachment. In addition, the specific color area on the front surface and the specific color area on the back surface may be the same or different depending on the road attachment. Here, the surface of the road attachment is an example of a first surface, and the back surface of the road attachment is an example of a second surface.

  The specific shape detection unit 103 detects a specific shape area from each of the front captured image and the rear captured image. More specifically, the specific shape detection unit 103 detects a specific shape area having a specific shape of the surface of the road attachment from the front captured image, and has a specific shape of the back surface of the road attachment from the rear captured image Detect a specific shape area. The specific shape of the front surface and the specific shape of the back surface may be the same or different depending on the road attachment. Also, the specific shape area on the front surface and the specific shape area on the back surface may be the same or different depending on the road attachment.

  The determination unit 2404 performs a first determination as to whether or not the surface of the road appendage is present in the forward captured image, based on the pillar region, the specific color region, and the specific shape region detected from the forward captured image. The determination unit 2404 performs a second determination as to whether or not the reverse side of the road appendage exists in the front captured image, based on the pillar region, the specific color region, and the specific shape region detected from the rear captured image. Here, each method of the first determination and the second determination is the same as that of the second embodiment.

  Further, the determination unit 2404 performs an overall determination process of determining whether a road attachment is present in the front captured image and the rear captured image from the result of the first determination and the result of the second determination. More specifically, when it is determined that the surface of the road attachment is present in the front captured image by the first determination and the back surface of the road attachment is present in the rear captured image by the second determination. It is determined that there is a road attachment. Therefore, when it is determined that the surface of the road attachment does not exist in the front captured image according to the first determination or the back side of the road attachment does not exist in the rear captured image according to the second determination. It is determined that there is no road attachment.

  FIG. 26 is a diagram for explaining an example of road accessory detection according to the third embodiment. FIG. 26 shows an overhead view of an example in which a road mirror 2601 is installed obliquely on the left side as viewed from the traveling direction when the vehicle 2501 travels from the top to the bottom of FIG.

  In this case, the appearance of the road mirror 2601 changes at the traveling position of the vehicle 2501. As shown in FIG. 26, when the vehicle 2501 is traveling at a position far from the road mirror 2601, the shape of the road mirror 2601 is imaged as an ellipse in the image taken by the imaging camera 110a for imaging the front (reference numeral 2602). ). Then, as the vehicle 2501 approaches the road mirror 2601, the shape of the road mirror 2601 in the image captured by the imaging camera 110a changes into a circular shape (reference numeral 2603). At this time, the road mirror 2601 is not imaged (not photographed) by the imaging camera 110 b imaging the rear of the vehicle 2501.

  In addition, after the vehicle 2501 passes the position of the road mirror 2601, the back 2604 of the road mirror 2601 is photographed by the imaging camera 110b that images the rear.

  In the road mirror 2601, the areas painted in yellow are the edge and the back of the mirror. For this reason, after the vehicle 2501 passes the position of the road mirror 2601, the detection camera 110b for picking up the rear side picks up the back of the road mirror 2601 with the imaging camera 110b. can do. At this time, the road mirror 2601 is not imaged (not photographed) by the imaging camera 110a that images the front of the vehicle 2501.

  Next, the road appendage detection process of Embodiment 3 configured as described above will be described. FIG. 27 is a flowchart illustrating an example of a road accessory detection process according to the third embodiment. Here, the processing of the flowchart shown in FIG. 27 is executed at predetermined time intervals such as every several seconds, for example.

  First, the imaging camera 110a that images the front of the vehicle images the surroundings of the front of the vehicle. The pillar area detection unit 101, the specific color detection unit 102, and the specific shape detection unit 103 input a front captured image from the imaging camera 110a (S1301). Then, the pillar area detection unit 101 performs a pillar area detection process on the front captured image (S1302). The specific color detection unit 102 performs a specific color detection process on the front captured image (S1303). The specific shape conversion unit 3010 converts the shape pattern stored in the shape DB 130 (S1304). The specific shape detection unit 103 performs a specific shape detection process on the front captured image using a shape pattern (S1305). Then, based on the detection results of S1302, S1303, and S1305, the determination unit 2404 performs determination processing (first determination) of the surface of the road attachment (S1306). The processes of S1302, S1303, S1305, and S1306 are performed as in the first embodiment.

  Then, the determination unit 2404 determines whether the detection target is out of the range of the front captured image (S1307). That is, the determination unit 2404 determines whether the vehicle has passed the position of the detection target (road attachment). If the detection target is within the range of the front captured image (S1307: No), the vehicle has not yet passed the detection target, the process returns to S1301, and the processing up to S1306 is repeatedly executed.

  On the other hand, when the detection target is out of the range of the front captured image (S1307: Yes), the vehicle passes through the detection target, and the imaging camera 110b for imaging the rear of the vehicle is located behind the vehicle. Take pictures of the surroundings. The pillar area detection unit 101, the specific color detection unit 102, and the specific shape detection unit 103 input a rear captured image from the imaging camera 110b (S1308). Then, the pillar area detection unit 101 performs a pillar area detection process on the rearward captured image (S1309). The specific color detection unit 102 performs a specific color detection process on the rearward captured image (S1310). The specific shape conversion unit 3010 converts the shape pattern stored in the shape DB 130 (S1311). The specific shape detection unit 103 performs a specific shape detection process on the rearward captured image using a shape pattern (S1312). Then, based on the detection results of S1309, S1310, and S1312, the determination unit 2404 performs determination processing (second determination) of the back surface of the road attachment (S1313). The processes of S1309, S1310, S1312, and S1313 are performed in the same manner as in the first embodiment.

  Then, the determination unit 2404 determines whether the detection target is out of the range of the rear captured image (S1314). That is, determination unit 2404 determines whether the vehicle is far away from the position of the detection target (road attachment) and becomes undetectable. If the detection target is within the range of the backward captured image (S1314: No), the process returns to S1308, and the process to S1313 is repeatedly executed.

  On the other hand, when the detection target is out of the range of the front captured image (S 1314: Yes), that is, when the vehicle is far away from the position of the detection target (road attachment) and becomes undetectable. The determination unit 2404 performs an overall determination process (S1315) that determines whether a road attachment has been detected from the result of the first determination (S1306) and the result of the second determination (S1313).

  Next, the clock unit 106 acquires the current time (S1316). Then, the position detection unit 107 acquires the current position of the vehicle using a GPS or the like mounted on the vehicle (S1317).

  The output unit 105 stores the determination result by the determination unit 104, the current time, and the current position in a memory or the like. Then, the output unit 105 determines whether the detection is ended or not according to the user's instruction or the like (S1318). If the output unit 105 determines that the detection is not completed (S1318: No), the process returns to S1301 and repeatedly executes up to S1317.

  On the other hand, when the output unit 105 determines that the detection is completed (S1318: Yes), an output result is generated using the current time and the current position as the determination result stored in the memory or the like (S1319). The output unit 105 outputs the output result by displaying it on a monitor or the like (S1320).

  As described above, in the present embodiment, the two imaging cameras 110a and 110b are mounted on a vehicle, and the road accessory detection device 5000 is an image obtained by imaging the front and the rear of the vehicle Road attachments are detected using images). That is, before the vehicle passes the position of the road attachment, the road attachment detection device 5000 detects the pillar detection area, the specific color area, and the specific shape area from the front captured image, and detects the surface of the road attachment. Determine In addition, after the vehicle passes the position of the road attachment, the road attachment detection device 5000 detects the pillar detection area, the specific color area, and the specific shape area from the rear captured image, and detects the back surface of the road attachment. Determine Then, the road attachment detection device 5000 determines that the road attachment is present when both of the front and back sides of the road attachment are detected.

  For this reason, according to the present embodiment, it is possible to detect the road attachment by combining the mounting pillar area, the specific color area, and the specific shape area, and correspond to the change in appearance depending on the travel of the vehicle. be able to. For this reason, according to this embodiment, many types of road attachments can be detected with higher accuracy.

  In the above embodiment, the road accessory detection devices 100, 3000, and 5000 are mounted on a vehicle, and the road accessory detection devices 100, 3000, and 5000 are used to capture the road surroundings with the imaging camera 110 while the vehicle is traveling. Although the captured image was sequentially acquired and the presence of the road attachment was determined in real time, it is not limited to this. For example, the road attachment detection devices 100, 3000, 5000 and the imaging camera 110 are separately provided, and the road attachment detection devices 100, 3000, 5000 are provided separately from the vehicle. Then, while the vehicle is traveling, the area around the road is imaged by the imaging camera 110 and stored as a moving image. Then, the road attachment detection apparatus 100, 3000, 5000 may be configured to obtain moving images later and detect the pillar area, the specific color area, and the specific shape area to detect the presence of the road attachment. .

  Moreover, in the said embodiment, although a road sign, a road mirror, and a street light were mentioned as an example and demonstrated as a road attachment, it is not limited to these. For example, the road attachment detection apparatus 100, 3000, 5000 can be configured to detect a traffic light, a guide plate or the like as a road attachment. This makes it possible to detect more types of road accessories with high accuracy.

  Moreover, in the said embodiment, although the road appendage detection apparatus 100, 3000, 5000 was detecting the road appendage arrange | positioned at the side of the road, it is not limited to this. For example, road attachment detection devices 100, 3000, 5000 can be configured to detect road attachments located above or below a road. This makes it possible to detect more types of road accessories with high accuracy.

  Furthermore, in the above embodiment, the pillar area detection unit 101 detects a pair of black portions in the vertical direction in the captured image as a pillar area, but the present invention is not limited to this. For example, the pillar area detection unit 101 can be configured to detect a pair of black portions in the lateral direction in the captured image as a pillar area. In this case, a guide plate or the like supported by laterally extending columns can also be detected with high accuracy as a road attachment.

  The road accessory detection devices 100, 3000, and 5000 of the above embodiments include a control device such as a CPU, a storage device such as a ROM (Read Only Memory) or a RAM, an external storage device such as an HDD or an SSDCD drive device, and a monitor. Etc., and an input device such as a touch panel or an operation button.

  The road appendage detection program executed by the road appendage detection apparatus 100, 3000, 5000 of the above embodiment is incorporated in advance in a ROM or the like and provided as a computer program product.

  The road attachment detection program executed by the road attachment detection apparatus 100, 3000, 5000 according to the above embodiment is a file in an installable format or an executable format, and the file is CD-ROM, flexible disk (FD), CD-R. , And may be recorded on a computer-readable recording medium such as a DVD (Digital Versatile Disk) and provided as a computer program product.

  Furthermore, a computer program by storing the road attachment detection program executed by the road attachment detection apparatus 100, 3000, 5000 of the above embodiment on a computer connected to a network such as the Internet and downloading via the network It may be configured to be provided as a product. Further, the road attachment detection program executed by the road attachment detection apparatus 100, 3000, 5000 of the above embodiment may be provided or distributed as a computer program product via a network such as the Internet.

  The road appendage detection program executed by the road appendage detection apparatus 100, 3000, 5000 according to the above-described embodiment includes the above-described units (the pillar area detection unit 101, the specific color detection unit 102, the specific shape conversion unit 3010, the specific shape detection The module configuration includes the unit 103, the determination unit 104, 2404, the output unit 105, the clock unit 106, and the position detection unit 107), and the CPU reads the road attachment detection program from the ROM as actual hardware. The above-described units are loaded on the main storage device by executing the column area detection unit 101, the specific color detection unit 102, the specific shape conversion unit 3010, the specific shape detection unit 103, the determination units 104 and 2404, the output unit 105 The unit 106 and the position detection unit 107 are generated on the RAM.

  While certain embodiments of the present invention have been described, these embodiments have been presented by way of example only, and are not intended to limit the scope of the invention. These novel embodiments can be implemented in various other forms, and various omissions, substitutions, and modifications can be made without departing from the scope of the invention. These embodiments and modifications thereof are included in the scope and the gist of the invention, and are included in the invention described in the claims and the equivalent scope thereof.

100, 3000, 5000 Road attachment detection device 110, 110a, 110b Imaging camera 101 Column area detection unit 102 Specific color detection unit 103 Specific shape detection unit 104, 2404 Determination unit 105 Output unit 106 Timekeeping unit 107 Position detection unit 120 Color DB
130 Shape DB
3010 Specific shape converter

Claims (9)

A pole area detection unit that detects a pole area that is an area of a pole portion to be detected based on pixels of the captured image from a captured image of a road on which the vehicle travels;
A specific color detection unit which detects a specific color area which is an area having a specific color of the detection target from the captured image;
A specific shape detection unit that detects a specific shape region that is a region having a specific shape of the detection target from the captured image;
If the pillar area, the specific color area, and the specific shape area are detected from the captured image, and the distance between the areas on the captured image is less than a predetermined distance, A determination unit that determines that the area indicated by the specific color area and the specific shape area is a road appendage disposed around a road;
An output unit that outputs information on the road attachment;
Road attachment detection device with. The specific shape detection unit matches a shape pattern corresponding to the shape of the road appendage with the captured image while scanning the captured image, and matches the shape image with the captured image while changing the shape pattern. Detecting the specific shape area by repeating;
The road accessory detection device according to claim 1 . A specific shape conversion unit that changes the shape pattern,
The road accessory detection device according to claim 2 , further comprising: The captured image includes a first captured image in front of the vehicle and a second captured image in back of the vehicle,
The pillar region detection unit detects the pillar region from each of the first captured image and the second captured image,
The specific color detection unit detects the specific color area from each of the first captured image and the second captured image,
The specific shape detection unit detects the specific shape area from each of the first captured image and the second captured image.
The determination unit determines whether the road appendage is present in the first captured image based on the pillar area, the specific color area, and the specific shape area detected from the first captured image. Whether the road attachment is present in the second captured image based on the pillar area, the specific color area, and the specific shape area detected from the second captured image. A second determination as to whether the road attachment exists in the first captured image and the second captured image based on the result of the first determination and the result of the second determination judge,
The road accessory detection device according to any one of claims 1 to 3 . The specific color detection unit detects the specific color area having the specific color of the first surface of the road appendage from the first captured image, and detects the road appendage from the second captured image. Detecting the specific color area having the specific color of the second surface;
The specific shape detection unit detects the specific shape region having the specific shape of the first surface of the road appendage from the first captured image, and detects the road appendage from the second captured image. Detecting the specific shape area having a specific shape of the second surface;
The determination unit determines, as the first determination, whether or not a first surface of the road appendage is present in the first captured image, and, as the second determination, determines the second captured image. It is determined whether or not the second surface of the road attachment is present, and according to the first determination, the first surface of the road attachment is present in the first captured image, and the second determination is made. Determining that the road attachment is present when it is determined that the second surface of the road attachment is present in the second captured image,
The road accessory detection device according to claim 4 . The determination unit detects any of the pillar area, the specific color area, and the specific shape area, and any one of the distances between the areas on the captured image is equal to or more than the predetermined distance. Determining that the road attachment is not present in the captured image;
A road accessory detection device according to claim 5 . The determination unit determines that the road attachment does not exist in the captured image, when any one of the pillar area, the specific color area, and the specific shape area is not detected.
The road accessory detection device according to any one of claims 1 to 6 . An imaging unit configured to capture an area around a road on which the vehicle travels to obtain the captured image;
The road attachment detection device according to any one of claims 1 to 7 , further comprising: A pillar area which is an area of a pillar portion to be detected is detected from a pickup image of a periphery of a road on which a vehicle travels, based on pixels of the pickup image,
Detecting a specific color area which is an area having a specific color of the detection target from the captured image;
Detecting a specific shape area, which is an area having a specific shape of the detection target, from the captured image;
If the pillar area, the specific color area, and the specific shape area are detected from the captured image, and the distance between the areas on the captured image is less than a predetermined distance, It is determined that the area indicated by the specific color area and the specific shape area is a road appendage disposed around a road,
Output information about the road attachment,
Road attachment detection method including.

Images