JP2016162436A - Road appanage detection device and road appanage detection method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To accurately detect many kinds of road appanages, for one example.SOLUTION: A road appanage detection device of an embodiment comprises: a pole area detection part; a specific color detection part; a specific shape detection part; a determination part; and an output part. The pole area detection part detects a pole area which is an area of a pole part of a detection target, based on pixels of a photographed image from the photographed image of a road where a vehicle travels. The specific color detection part detects a specific color area which is an area having specific color of the detection target from the photographed image. The specific shape detection part detects a specific shape area which is an area having a specific shape of the detection target. The determination part determines whether or not there is a road appanage(s) arranged in the vicinity of the road, based on the pole area, specific color area, and specific shape area. The output part outputs information related to the road appanage(s).SELECTED DRAWING: Figure 1

Description

  Embodiments described herein relate generally to a road accessory detection apparatus and a road accessory detection method.

  Road accessories such as road signs, road mirrors and street lights are important for ensuring safety for vehicles traveling on the road. For this reason, conventionally, a road accessory detection device that detects a road accessory based on a captured image around the road accessory is known.

Japanese Patent No. 4394476 Japanese Patent No. 3006471 JP 2013-54522 A

  In such a road accessory detection device, it is desired to detect as many types of road accessories as possible with high accuracy.

  The road accessory detection device according to the embodiment includes a pillar region detection unit, a specific color detection unit, a specific shape detection unit, a determination unit, and an output unit. The column area detection unit detects a column area, which is an area of a column part to be detected, from a captured image of a road on which the vehicle travels based on pixels of the captured image. The specific color detection unit detects a specific color region that is a region having a specific color to be detected from the captured image. The specific shape detection unit detects a specific shape region that is a region having a specific shape to be detected from the captured image. The determination unit determines whether or not a road accessory arranged around the road exists in the captured image based on the pillar region, the specific color region, and the specific shape region. The output unit outputs information on road accessories.

FIG. 1 is a block diagram illustrating a functional configuration of the road accessory detection apparatus according to the first embodiment. FIG. 2 is a diagram illustrating an example of a configuration of a road accessory according to the first embodiment. FIG. 3 is a flowchart illustrating an example of a procedure of the road accessory detection process according to the first embodiment. FIG. 4 is a flowchart illustrating an example of the procedure of the column area detection process according to the first embodiment. FIG. 5 is a diagram illustrating an example for explaining the edge enhancement processing according to the first embodiment. FIG. 6 is a diagram illustrating an example of a pixel extraction result of edge features according to the first embodiment. FIG. 7 is a diagram for explaining the vertical accumulation processing of the first embodiment. FIG. 8 is a flowchart illustrating an example of the procedure of the specific color detection process according to the first embodiment. FIG. 9 is a diagram for explaining the specific color detection process of the present embodiment. FIG. 10 is a diagram for explaining the specific color detection process of the present embodiment. FIG. 11 is a diagram for explaining the specific color detection processing of the present embodiment. FIG. 12 is a flowchart illustrating an example of a procedure of the specific shape detection process according to the first embodiment. FIG. 13 is a diagram for explaining the specific shape detection process of the first embodiment. FIG. 14 is a flowchart illustrating an example of a determination processing procedure according to the first embodiment. FIG. 15 is a diagram illustrating an example of detection results of the column area detection process, the specific color detection process, and the specific shape detection process according to the first embodiment. FIG. 16 is a diagram illustrating an example of a determination result according to the first embodiment. FIG. 17 is a diagram illustrating another example of the determination result of the first embodiment. FIG. 18 is a diagram illustrating another example of the determination result of the first embodiment. FIG. 19 is a diagram illustrating an example of an output result of the road appendage according to the first embodiment. FIG. 20 is a block diagram illustrating a functional configuration of the road accessory detection apparatus according to the second embodiment. FIG. 21 is a diagram illustrating an example of shape pattern conversion according to the second embodiment. FIG. 22 is a flowchart illustrating an example of a procedure of the road accessory detection process according to the second embodiment. FIG. 23 is a diagram illustrating an example of how a circular road mirror according to the second embodiment is seen from a vehicle. FIG. 24 is a block diagram illustrating a functional configuration of the road accessory detection device according to the third embodiment. FIG. 25 is a diagram illustrating an example of the imaging directions of the two imaging cameras according to the third embodiment. FIG. 26 is a diagram for explaining an example of road accessory detection according to the third embodiment. FIG. 27 is a flowchart illustrating an example of a procedure of road accessory detection processing according to the third embodiment.

  Hereinafter, embodiments will be described.

(Embodiment 1)
FIG. 1 is a block diagram illustrating a functional configuration of the road accessory detection apparatus according to the first embodiment. As shown in FIG. 1, the road accessory detection apparatus 100 according to the first embodiment includes an imaging camera 110, a column region detection unit 101, a specific color detection unit 102, a specific shape detection unit 103, a determination unit 104, An output unit 105, a time measuring unit 106, a position detection unit 107, a color database 120 (hereinafter referred to as “color DB 120”), and a shape database 130 (hereinafter referred to as “shape DB 130”) are mainly provided. ing. Here, the color DB 120 and the shape DB 130 are stored in a storage medium such as an HDD (HARD Disk Drive), an SSD (Solid State Drive), or a memory.

  The road accessory detection apparatus 100 of this embodiment is an apparatus that is mounted on a vehicle traveling on a road and detects a road accessory. Here, a road accessory is a facility or work required for road structure maintenance, ensuring safe and smooth road traffic, and other road management (Article 2, Paragraph 2 of the Road Law). Examples of such road accessories include, but are not limited to, road signs, road mirrors, street lamps, and the like.

  FIG. 2 is a diagram illustrating an example of a configuration of a road accessory according to the first embodiment. As shown in FIG. 2, a road sign or road mirror, which is an example of a road accessory, generally includes an edge area 201, a center area 202, and a pillar area 203. The edge area 201 is an area of the edge of a sign or mirror. The central area 202 is an area inside the sign or mirror edge area 201. The column region 203 is a column portion region that supports a portion including the central region 202 and the edge region 201.

  Returning to FIG. 1, the imaging camera 110 is installed outside or inside the vehicle, images the road surrounding the road ahead of the traveling vehicle at regular time intervals, and outputs a captured image.

  The column area detection unit 101 inputs a captured image of the periphery of the road imaged by the imaging camera 110, and adds a pair of vertical lines to the road from the input captured image based on the pixels constituting the captured image. Detect as a pillar area of an object. Specifically, the pillar region detection unit 101 performs edge enhancement on the captured image, extracts edge features from the edge-enhanced pixels, and accumulates the extracted pixels in the vertical direction of the captured image. A column part composed of a pair of lines is detected as a column region, and the coordinate range of the column region is output.

  The specific color detection unit 102 inputs a captured image captured by the imaging camera 110, and has a color (specific color) of a road accessory that is a detection target using the R value, the G value, and the B value of the captured image. Detect a specific color area. Specifically, the specific color detection unit 102 compares the R value, the G value, and the B value of each pixel of the captured image with a predetermined range according to the specific color to be detected, and is included in the range. Pixels are extracted and set as a region having a specific color (specific color region), and the coordinate range of the specific color region is output. In the color DB 120, the maximum value and the minimum value of R, G, and B values that define the above range are registered.

  The specific shape detection unit 103 receives a captured image captured by the imaging camera 110, and detects a specific shape region that is a region having a shape (specific shape) of a road appendage that is a detection target from the captured image. Specifically, like the pillar region detection unit 101, the specific shape detection unit 103 performs edge enhancement on the captured image, and extracts edge features from the edge-enhanced pixels. Then, the specific shape detection unit 103 performs pattern matching with a predetermined shape pattern of the road appendage on the extracted edge feature, extracts the specific shape of the road appendage, and sets it as the specific shape region. Output the coordinate range of the shape area. Here, in the shape DB 130, the shape pattern of the road accessory is registered for each road accessory.

  The determination unit 104 captures an image based on the column region detected by the column region detection unit 101, the specific color region detected by the specific color detection unit 102, and the specific shape region detected by the specific shape detection unit 103. It is determined whether or not a road accessory arranged around the road exists in the image. Specifically, when any of the column area, the specific color area, and the specific shape area is detected in the captured image, the determination unit 104 determines the distance on the captured image between the areas from the coordinate range of each area. Is less than a predetermined distance. When the distance on the captured image between the regions is less than the predetermined distance, the determination unit 104 determines that the region indicated by the column region, the specific color region, and the specific shape region in the captured image is a road accessory. It is determined that (exists). On the other hand, the determination unit 104 determines that no road appendage exists in the captured image when any of the distances between the regions on the captured image is greater than or equal to a predetermined distance.

  The determination unit 104 determines that no road appendage exists in the captured image when any of the pillar region, the specific color region, and the specific shape region is not detected.

  The timer unit 106 measures the current time. The position detection unit 107 acquires the current position of the vehicle using GPS (Global Positioning System) or the like.

  When the determination unit 104 detects a road accessory, the output unit 105 outputs information on the road accessory together with the current time and the current position of the vehicle to a monitor in the vehicle.

  Next, the road accessory detection process of the present embodiment configured as described above will be described in detail. FIG. 3 is a flowchart illustrating an example of a procedure of the road accessory detection process according to the first embodiment. Here, the process of the flowchart shown in FIG. 3 is executed at predetermined time intervals such as every several seconds.

  First, the imaging camera 110 images the surroundings in front of the vehicle. The column area detection unit 101, the specific color detection unit 102, and the specific shape detection unit 103 input a captured image from the imaging camera 110 (S11). Then, the column area detection unit 101 performs a column area detection process on the captured image (S12). The specific color detection unit 102 performs specific color detection processing on the captured image (S13). The specific shape detection unit 103 performs specific shape detection processing on the captured image (S14). And the determination part 104 performs the determination process of a road attachment based on the detection result of S12, S13, S14 (S15).

  Details of the column area detection processing in S12 will be described. FIG. 4 is a flowchart illustrating an example of the procedure of the column area detection process according to the first embodiment. The column area detection unit 101 performs edge enhancement processing on the captured image (S31). Specifically, the pillar area detection unit 101 obtains a difference between neighboring pixels for each pixel of the captured image. FIG. 5 is a diagram illustrating an example for explaining the edge enhancement processing according to the first embodiment. For example, as illustrated in FIG. 5, the pillar region detection unit 101 obtains the sum of a pixel value obtained by multiplying the surrounding 8 pixels by a weight “−1” and a pixel value obtained by multiplying the center pixel by a weight “9”. Thus, a pixel having a luminance peak in the central visual field can be emphasized.

  Next, the pillar region detection unit 101 performs edge feature pixel extraction processing from the edge-enhanced captured image (S32). Specifically, the pillar region detection unit 101 extracts pixels having edge features as binary images with a certain threshold value for the edge-enhanced pixels. FIG. 6 is a diagram illustrating an example of a pixel extraction result of edge features according to the first embodiment. In FIG. 6, black lines indicate extracted pixels. As shown in the example of FIG. 6, not only the road appendage 501 but also pixels such as road edges are extracted as extracted pixels.

  The column area detection unit 101 sets the field of view while moving the small area 502 shown in FIG. 6 on the image of the pixel extraction result of the edge feature, and performs vertical accumulation processing (S33). FIG. 7 is a diagram for explaining the vertical accumulation processing of the first embodiment. As shown in FIG. 7, the column area detection unit 101 sets a field of view 601 while moving a small area with respect to the captured image of the pixel extraction result of the edge feature, and accumulates extracted pixels in the vertical direction with respect to the field of view 601. One-dimensional accumulated data 602 is collected. Thereby, the column area detection unit 101 detects the pair of lines P1 and P2.

  Next, the column area detection unit 101 determines a column area for the accumulated data 602 obtained in S33 (S34). Specifically, the column area detection unit 101 has an accumulated value equal to or greater than a certain threshold Thr for the pair of lines P1 and P2 of the accumulated data 602, and the interval w between P1 and P2 is a predetermined range. If it is within the range, the pair of lines P1 and P2 are determined as the column area, and the coordinate range is output as the range of the column area of the road appendage. Here, the output coordinate range is indicated by the upper left coordinate and the lower right coordinate of the rectangular pillar region.

  Note that the method of detecting the column region by the column region detection unit 101 is not limited to this. For example, a pillar template may be prepared in advance, and the pillar region detection unit 101 may be configured to detect a pillar region using a matching method with a captured image of a pixel extraction result of an edge feature.

  Next, details of the specific color detection process in S13 will be described. FIG. 8 is a flowchart illustrating an example of the procedure of the specific color detection process according to the first embodiment. First, the specific color detection unit 102 reads out the minimum values RL, GL, and BL and the maximum values RH, GH, and BH that are the range of the color (specific color) to be detected from the color DB 120, and each of the captured images. The R value, G value, and B value of the pixel are compared with the minimum values RL, GL, and BL and the maximum values RH, GH, and BH, and pixels included in the corresponding range are extracted (S51). In the present embodiment, the R value, the G value, and the B value are used, but the present invention is not limited to the RGB color space. For example, L * a * b *, xy, and HSV values can be used as other color systems.

  Next, the specific color detection unit 102 extracts connected pixel groups (connected components) in the extracted pixels (S52). Then, the specific color detection unit 102 obtains a rectangular shape that includes the extracted connected components (S53). The specific color detection unit 102 outputs the coordinate range of the outer rectangle as the specific color determination result, that is, the range of the specific color region of the road accessory (S54). Here, the output coordinate range is indicated by the upper left coordinates and the lower right coordinates of the outline rectangle.

  9 to 11 are diagrams for explaining the specific color detection processing of the present embodiment. FIG. 9 shows an example of the result when only the edge region 201 is the color to be extracted, with respect to the example of the road sign shown in FIG. FIG. 10 shows an example in which the edge area 201 and the column area 203 are colors to be extracted.

  In the example of FIG. 10, as indicated by the dashed frame in FIG. 11, the specific color detection unit 102 obtains a circumscribed rectangle for the connected pixel group, and sets the coordinate range of the outer rectangle as the specific color region 1101. Output as.

  Next, details of the specific shape detection process of S14 will be described. FIG. 12 is a flowchart illustrating an example of a procedure of the specific shape detection process according to the first embodiment. The specific shape detection unit 103 performs edge enhancement processing on the captured image in the same manner as the pillar region detection processing, and obtains a difference from a neighboring pixel for each pixel of the image (S71). The specific shape detection unit 103 performs an edge feature pixel extraction process from the edge-enhanced captured image in the same manner as the pillar region detection process (S72).

  Then, the specific shape detection unit 103 reads the shape pattern of the road appendage to be detected from the shape DB 130, and performs pattern matching while scanning the shape pattern with respect to the captured image of the edge feature extraction result (S73). Then, the specific shape detection unit 103 determines and outputs a rectangular coordinate range that circumscribes the region that matches the shape pattern as the specific shape region of the road appendage (S74).

  FIG. 13 is a diagram for explaining the specific shape detection process of the first embodiment. As illustrated in FIG. 13, the specific shape detection unit 103 sets a local region as a field of view in a captured image, and performs a matching process with a shape pattern 1201 having a specific shape. The shape pattern 1201 is registered in advance in the shape DB 130 as a binary image in which, for example, the circle portion is “1” and the others are “0”. The specific shape detection unit 103 moves within the captured image while changing the size of the shape pattern 1201, and detects a rectangular region 1202 circumscribing a region that matches the shape pattern 1201 of the specific shape. Here, as a method for matching the image from which the edge is extracted and the shape pattern 1201, a method for measuring the degree of coincidence between binary images, or a method resistant to fluctuations such as a normalized correlation can be used. As another method, the specific shape detection unit 103 may be configured to use the detection based on the co-occurrence histogram of the edge direction disclosed in Japanese Patent No. 5214367.

  Next, details of the determination process of S15 will be described. FIG. 14 is a flowchart illustrating an example of a determination processing procedure according to the first embodiment. From the column area detection unit 101, the specific color detection unit 102, and the specific shape detection unit 103, the coordinate range of each area as a detection result is output as shown in FIG. First, the determination unit 104 determines whether all of the column area, the specific color area, and the specific shape area are detected in the captured image (S1001).

  When any of the column area, the specific color area, and the specific shape area is not detected (S1001: No), the determination unit 104 determines that no road appendage exists in the captured image (S1005).

  On the other hand, when all of the column area, the specific color area, and the specific shape area are detected (S1001: Yes), the determination unit 104 calculates the distance on the captured image between each area from the coordinate range of each area. Calculate (S1002). Then, the determination unit 104 determines whether or not all the distances on the captured image between the regions are less than a predetermined distance (S1003). And when all the distances on the captured image between each area | region are less than predetermined distance (S1003: Yes), the determination part 104 is the column area | region in a captured image, a specific color area | region, a specific shape area | region, It is determined that the area indicated by is a road accessory (S1004). On the other hand, if any of the distances between the regions on the captured image is a predetermined distance or more (S1003: No), the determination unit 104 determines that there is no road appendage in the captured image ( S1005).

  FIG. 15 is a diagram illustrating an example of detection results of the column area, the specific color area, and the specific shape area according to the first embodiment. 16 to 18 are diagrams illustrating examples of determination results according to the first embodiment.

  FIG. 15 shows an example in which a pillar region and a specific color region are detected, but there is no detection result of a specific shape. For example, the example of the detection result as shown in FIG. 16 corresponds, and the determination result of the road attachment is “not applicable”. In the example of FIG. 16, the filled area in the image indicates a specific color, and the distance between the column area area 1401 and the specific color area 1402 is closer than a predetermined distance and is extracted, but has a specific shape. The specific shape area corresponding to the circle is not detected.

  In the object 2 of FIG. 15, all of the pillar region, the specific color region, and the specific shape region are detected, and each region is approaching less than a predetermined distance as shown in FIG. 17. For this reason, the determination part 104 determines with "applicable" as a detection result of a road appendage. In FIG. 17, a filled area in the image indicates a specific color area, and a column area area 1501, a specific color area 1502, and a specific shape area 1503 are extracted close to each other. In this example, the area of the specific color (yellow) at the edge of the mirror is small and the specific color region 1502 is partially detected like a road mirror, but accuracy can be improved by combining with the other two features. It can be detected well.

  In the object 3 of FIG. 15, the pillar region, the specific color, and the specific shape are detected. However, as shown in FIG. 18, the determination unit 104 detects the road appendage because each region is separated by a predetermined distance or more. As a result, “not applicable” is determined. In the example of FIG. 18, the filled area in the image indicates the specific color area 1602, and the column area area 1601 and the specific shape area 1603 are close to each other, but the specific color area 1602 is separated and another object is selected. Detected.

  Returning to FIG. 3, when the determination process by the determination unit 104 is completed, the time measuring unit 106 acquires the current time (S16). And the position detection part 107 acquires the present position of a vehicle (S17). For example, the position detection unit 107 can acquire GPS data as position information from a GPS satellite. By acquiring such GPS data and using it as an output result, it can be used as reference data for road accessories in a navigation system or the like.

  The output unit 105 stores the determination result by the determination unit 104, the current time, and the current position in a memory or the like. Then, the output unit 105 determines whether or not to end the detection according to a user instruction or the like (S18). If the output unit 105 determines that the detection is not finished (S18: No), the process returns to S11 and repeats up to S17.

  On the other hand, when the output unit 105 determines that the detection is finished (S18: Yes), an output result as information on the road accessory is generated using the determination result stored in the memory or the like, the current time, and the current position. (S19). Here, the output unit 105 determines that the determination result that the current position and the current time are within a predetermined range is the determination result of the same road accessory, and records only one determination result as the output result. You may comprise. The output unit 105 outputs the output result by displaying it on a monitor or the like (S20).

  FIG. 19 is a diagram illustrating an example of an output result of the road appendage according to the first embodiment. As shown in FIG. 19, the output result includes the collection date and time, the location of the road accessory, the type of the road accessory, the captured image, and the like. The output unit 105 uses the current time as the collection date and time, generates the contents of the installation location based on the position information, and determines the type of the road appendage from the detected road appendage image. And the output part 105 produces | generates a collection date and time, the installation location of a road attachment, the kind of road attachment, and a captured image as an output result.

  The output result is not limited to the format shown in FIG. 19, and the output unit 105 can be configured to output in an arbitrary format. For example, an image such as a road sign or a road mirror of a detected road accessory may be displayed at a coordinate position indicated by position information of an electronic map displayed by a navigation system or the like mounted on a vehicle.

  As described above, according to this embodiment, it is possible to detect many types of road accessories with high accuracy. That is, the conventional road accessory detection device cannot target all road accessories. In the method of detecting road accessories based on color information, there are cases where the color difference from the background such as the ground or sky is not large for yellow or blue road accessories, making it difficult to detect road accessories. There is a case. In addition, yellow used in road mirrors is used only in a limited portion of the edge portion of the mirror, and it is difficult to capture features from the captured image.

  On the other hand, in the method of detecting the road accessory based on the shape, the shape at the time of photographing may change depending on the attachment angle of the road accessory. For this reason, in the embodiment, the road accessory detection device 100 detects a pillar accessory, a specific color region, and a specific shape region, and comprehensively determines the detection result of each region to determine the road accessory. Yes. For this reason, according to the present embodiment, it is possible to detect a road appendage by combining a mounting column region, a specific color region, and a specific shape region, and to detect many types of road appendages with high accuracy. Is possible.

(Embodiment 2)
In the first embodiment, one shape pattern is used for one road appendage when detecting a specific shape. In the second embodiment, one shape pattern is used for one road appendage. Used while changing.

  FIG. 20 is a block diagram illustrating a functional configuration of the road accessory detection apparatus according to the second embodiment. As shown in FIG. 20, the road accessory detection device 3000 according to the second embodiment includes an imaging camera 110, a column area detection unit 101, a specific color detection unit 102, a specific shape detection unit 103, a determination unit 104, It mainly includes an output unit 105, a time measuring unit 106, a position detection unit 107, a color DB 120, a shape DB 130, and a specific shape conversion unit 3010. Here, the functions of the imaging camera 110, the column area detection unit 101, the specific color detection unit 102, the specific shape detection unit 103, the determination unit 104, the output unit 105, the timing unit 106, the position detection unit 107, the color DB 120, and the shape DB 130, The configuration is the same as in the first embodiment.

  The specific shape conversion unit 3010 reads a shape pattern of a specific shape to be detected from the shape DB 130 and converts the shape. FIG. 21 is a diagram illustrating an example of shape pattern conversion according to the second embodiment. The specific shape conversion unit 3010 converts the shape pattern in accordance with the change in the appearance because the appearance of the road appendage varies depending on the traveling of the vehicle. The specific shape detection unit 103 detects a specific shape for each shape pattern that is changed by the specific shape conversion unit 3010.

  FIG. 22 is a flowchart illustrating an example of a procedure of the road accessory detection process according to the second embodiment. Here, the process of the flowchart shown in FIG. 22 is executed at predetermined time intervals such as every several seconds. The process from the input of the captured image in S11 to the specific color detection process in S13 is performed in the same manner as in the first embodiment. Next, the specific shape conversion unit 3010 reads the shape pattern of the specific shape to be detected from the shape DB 130 and converts only one pattern (S91). Then, the specific shape detection unit 103 performs a specific shape detection process with the shape pattern converted by one pattern (S14). The specific shape conversion unit 3010 converts the shape pattern one pattern at a time as shown in FIG. 21 every time the processes of S11 to S17 are repeated, and the specific shape detection unit 103 detects the specific shape. The processes after S15 are performed in the same manner as in the first embodiment.

  Although there is a method of detecting a road accessory based on the shape, the shape at the time of shooting may change depending on the attachment angle of the road accessory. For example, a road mirror that is a circle when viewed from the front is attached obliquely with respect to the direction of travel of the road, so an ellipse image is obtained when observed from a traveling vehicle, and the road accessory detection process based on the shape of the circle is applied as it is. Can not.

  FIG. 23 is a diagram illustrating an example of how a circular road mirror according to the second embodiment is seen from a vehicle. The road mirror 2301 is not installed in front of the vehicle and is deformed into an ellipse as shown in FIG. 23 in the captured image taken from the vehicle. The specific shape detection unit 103 detects road appendages installed at various angles using a plurality of types of shape patterns.

  Thus, in the embodiment, the road accessory detection device 3000 detects a specific shape while changing the detection of the specific shape while converting the shape pattern with respect to the road accessory. For this reason, according to the present embodiment, it is possible to detect the road appendage by combining the attached column region, the specific color region, and the specific shape region, and cope with a change in appearance according to the traveling of the vehicle. be able to. Therefore, according to the present embodiment, many types of road accessories can be detected with higher accuracy.

  In the present embodiment, one type of shape pattern is converted by the specific shape conversion unit 3010. However, a plurality of converted shape patterns are registered in advance in the shape DB 130 for one type of shape pattern, The shape DB 130 and the specific shape detection unit 103 may be configured to use this for specific shape detection.

(Embodiment 3)
In the third embodiment, a plurality of imaging cameras are mounted on a vehicle, and road accessories are detected using two captured images obtained by imaging the front direction and the rear direction of the vehicle.

  FIG. 24 is a block diagram illustrating a functional configuration of the road accessory detection device according to the third embodiment. As shown in FIG. 24, the road accessory detection device 5000 according to the second embodiment includes two imaging cameras 110a and 110b, a column area detection unit 101, a specific color detection unit 102, a specific shape detection unit 103, and a determination. A unit 2404, an output unit 105, a timing unit 106, a position detection unit 107, a color DB 120, a shape DB 130, and a specific shape conversion unit 3010 are mainly provided.

  The imaging camera 110a is installed in the vehicle so as to be able to capture an image around the road ahead of the vehicle. The imaging camera 110b is installed in the vehicle so as to be able to capture an image around the road behind the vehicle. FIG. 25 is a diagram illustrating an example of the imaging directions of the two imaging cameras 110a and 110b according to the third embodiment. FIG. 25A shows that the imaging camera 110a has a forward direction 2502 in which the optical axis is aligned with the traveling direction of the vehicle 2501 as the imaging direction, and the imaging camera 110b has the traveling direction of the vehicle 2501 and the direction 2503 behind 180 °. The example installed as is shown.

  In FIG. 25B, the imaging camera 110 b has a direction 2602 inclined 45 ° with respect to the traveling direction of the vehicle 2501 as the imaging direction, and the imaging camera 110 b has a direction 2603 inclined 45 ° from 180 ° behind the traveling direction of the vehicle 2501. Is shown as an imaging direction. The imaging directions of the imaging cameras 110a and 110b are any of FIGS. 25A and 25B, or FIG. 25A as long as the imaging range includes a road accessory installed around the road. ), Other than (b).

  An image captured by the imaging camera 110a is referred to as a front captured image, and an image captured by the imaging camera 110b is referred to as a rear captured image. The front captured image is an example of a first captured image, and the rear captured image is an example of a second captured image.

  Embodiments of functions and configurations of the column area detection unit 101, specific color detection unit 102, specific shape detection unit 103, output unit 105, timing unit 106, position detection unit 107, color DB 120, shape DB 130, and specific shape conversion unit 3010 Same as 2.

  However, the column area detection unit 101 detects a column area from each of the front captured image and the rear captured image. The specific color detection unit 102 detects a specific color region from each of the front captured image and the rear captured image. More specifically, the specific color detection unit 102 detects a specific color region having a specific color on the surface of the road appendage from the front captured image, and has a specific color on the back surface of the road appendage from the rear captured image. Detect a specific color area. The specific color on the front and the specific color on the back may be the same or different depending on the road accessories. The specific color area on the front surface and the specific color area on the back surface may be the same or different depending on the road accessories. Here, the surface of the road appendage is an example of the first surface, and the back surface of the road appendage is an example of the second surface.

  The specific shape detection unit 103 detects a specific shape region from each of the front captured image and the rear captured image. More specifically, the specific shape detection unit 103 detects a specific shape region having a specific shape on the front surface of the road appendage from the front captured image, and has a specific shape on the back surface of the road appendage from the rear captured image. A specific shape area is detected. The specific shape on the front surface and the specific shape on the back surface may be the same or different depending on the road accessories. Further, the specific shape region on the front surface and the specific shape region on the back surface may be the same or different depending on the road accessories.

  The determination unit 2404 performs a first determination as to whether or not the surface of the road appendage exists in the front captured image based on the column region, the specific color region, and the specific shape region detected from the front captured image. The determination unit 2404 performs a second determination as to whether or not the back surface of the road appendage exists in the front captured image based on the column region, the specific color region, and the specific shape region detected from the rear captured image. Here, each method of the first determination and the second determination is the same as in the second embodiment.

  In addition, the determination unit 2404 performs comprehensive determination processing for determining whether or not a road accessory exists in the front captured image and the rear captured image from the first determination result and the second determination result. More specifically, when the determination unit 2404 determines that the front captured image includes the front surface of the road appendage in the first determination and the second determination determines that the rear captured image includes the back surface of the road appendage. It is determined that there is a road accessory. Therefore, the determination unit 2404, when it is determined by the first determination that the surface of the road appendage does not exist in the front captured image, or when the second determination determines that the rear surface of the road appendage does not exist in the rear captured image, It is determined that there are no road accessories.

  FIG. 26 is a diagram for explaining an example of road accessory detection according to the third embodiment. 26 shows an overhead view of an example in which a road mirror 2601 is obliquely installed on the left side when viewed from the traveling direction when the vehicle 2501 travels on the road from the upper part to the lower part of FIG.

  In this case, the appearance of the road mirror 2601 changes depending on the travel position of the vehicle 2501. As shown in FIG. 26, when the vehicle 2501 is traveling at a position far from the road mirror 2601, the shape of the road mirror 2601 is captured as an ellipse in the captured image by the imaging camera 110a that captures the front (reference numeral 2602). ). Then, as the vehicle 2501 approaches the road mirror 2601, the shape of the road mirror 2601 in the image captured by the imaging camera 110a changes to a circle (reference numeral 2603). At this time, in the imaging camera 110b that captures the rear of the vehicle 2501, the road mirror 2601 is not captured (not captured).

  In addition, after the vehicle 2501 passes the position of the road mirror 2601, the back 2604 of the road mirror 2601 is photographed by the imaging camera 110b that captures the rear.

  In the road mirror 2601, the areas painted in yellow are the edge and back surface of the mirror. Therefore, after the vehicle 2501 passes the position of the road mirror 2601, the backside of the road mirror 2601 is imaged by the imaging camera 110 b that captures the rear, thereby increasing the detection accuracy of the yellow specific color region by the specific color detection unit 102. can do. At this time, the imaging mirror 1101 that images the front of the vehicle 2501 does not capture (does not capture) the road mirror 2601.

  Next, the road accessory detection process of the third embodiment configured as described above will be described. FIG. 27 is a flowchart illustrating an example of a procedure of road accessory detection processing according to the third embodiment. Here, the processing of the flowchart shown in FIG. 27 is executed at predetermined time intervals such as every several seconds, for example.

  First, the imaging camera 110a that captures the front of the vehicle images the surroundings in front of the vehicle. The pillar region detection unit 101, the specific color detection unit 102, and the specific shape detection unit 103 input a front captured image from the imaging camera 110a (S1301). Then, the column area detection unit 101 performs a column area detection process on the front captured image (S1302). The specific color detection unit 102 performs a specific color detection process on the front captured image (S1303). The specific shape conversion unit 3010 converts the shape pattern stored in the shape DB 130 (S1304). The specific shape detection unit 103 performs a specific shape detection process on the front captured image using the shape pattern (S1305). Then, the determination unit 2404 performs a determination process (first determination) on the surface of the road appendage based on the detection results of S1302, S1303, and S1305 (S1306). Each process of S1302, S1303, S1305, and S1306 is performed in the same manner as in the first embodiment.

  Then, the determination unit 2404 determines whether or not the detection target object is outside the range of the forward captured image (S1307). That is, the determination unit 2404 determines whether or not the vehicle has passed the position of the detection object (road accessory). If the detection target is within the range of the forward captured image (S1307: No), the vehicle has not yet passed the detection target, the process returns to S1301, and the processes up to S1306 are repeatedly executed.

  On the other hand, when the detection object is out of the range of the front captured image (S1307: Yes), the vehicle has passed the detection object, and the imaging camera 110b that captures the rear of the vehicle Image the surroundings. The pillar region detection unit 101, the specific color detection unit 102, and the specific shape detection unit 103 input a rear captured image from the imaging camera 110b (S1308). Then, the column area detection unit 101 performs a column area detection process on the rear captured image (S1309). The specific color detection unit 102 performs specific color detection processing on the rear captured image (S1310). The specific shape conversion unit 3010 converts the shape pattern stored in the shape DB 130 (S1311). The specific shape detection unit 103 performs a specific shape detection process on the rear captured image using the shape pattern (S1312). Then, the determination unit 2404 performs determination processing (second determination) on the back side of the road appendage based on the detection results of S1309, S1310, and S1312 (S1313). Each process of S1309, S1310, S1312, and S1313 is performed similarly to the first embodiment.

  Then, the determination unit 2404 determines whether or not the detection target object is outside the range of the rear captured image (S1314). That is, the determination unit 2404 determines whether or not the vehicle is far away from the position of the detection target (road accessory) and cannot be detected. If the detection target is within the range of the rear captured image (S1314: No), the process returns to S1308, and the processes up to S1313 are repeatedly executed.

  On the other hand, when the detection object is out of the range of the forward captured image (S1314: Yes), that is, when the vehicle is far away from the position of the detection object (road accessory) and cannot be detected. The determination unit 2404 performs comprehensive determination processing for determining whether or not a road accessory is detected from the result of the first determination (S1306) and the result of the second determination (S1313) (S1315).

  Next, the time measuring unit 106 acquires the current time (S1316). And the position detection part 107 acquires the present position of a vehicle using GPS etc. which were mounted in the vehicle (S1317).

  The output unit 105 stores the determination result by the determination unit 104, the current time, and the current position in a memory or the like. Then, the output unit 105 determines whether or not to end the detection based on a user instruction or the like (S1318). When the output unit 105 determines that the detection is not finished (S1318: No), the process returns to S1301 and repeatedly executes up to S1317.

  On the other hand, when the output unit 105 determines that the detection is finished (S1318: Yes), an output result is generated using the determination result saved in the memory or the like, the current time, and the current position (S1319). The output unit 105 outputs the output result by displaying it on a monitor or the like (S1320).

  As described above, in this embodiment, the two imaging cameras 110a and 110b are mounted on the vehicle, and the road accessory detection device 5000 captures two captured images (a front captured image and a rear captured image) that capture the front direction and the rear direction of the vehicle. Image) is used to detect road accessories. That is, before the vehicle passes through the position of the road accessory, the road accessory detection device 5000 detects the column detection area, the specific color area, and the specific shape area from the front captured image, thereby detecting the surface of the road accessory. Determine. In addition, after the vehicle passes through the position of the road accessory, the road accessory detection device 5000 detects the column detection area, the specific color area, and the specific shape area from the rear captured image, and detects the back surface of the road accessory. Determine. Then, the road accessory detection device 5000 determines that a road accessory exists when both the front and back surfaces of the road accessory are detected.

  For this reason, according to the present embodiment, it is possible to detect the road appendage by combining the attached column region, the specific color region, and the specific shape region, and cope with a change in appearance according to the traveling of the vehicle. be able to. For this reason, according to this embodiment, many types of road accessories can be detected with higher accuracy.

  In the above-described embodiment, the road accessory detection devices 100, 3000, and 5000 are mounted on a vehicle, and the road accessory detection devices 100, 3000, and 5000 capture an image of the periphery of the road with the imaging camera 110 while the vehicle is running. Although captured images were sequentially acquired and the presence of road accessories was determined in real time, the present invention is not limited to this. For example, the road accessory detection devices 100, 3000, and 5000 and the imaging camera 110 are separately provided, and the road accessory detection devices 100, 3000, and 5000 are provided separately from the vehicle. Then, while the vehicle is running, the surroundings of the road are imaged by the imaging camera 110 and stored as a moving image. Then, the road accessory detection device 100, 3000, or 5000 may be configured to acquire a moving image at a later date and detect the pillar region, the specific color region, and the specific shape region to detect the presence of the road accessory. .

  Moreover, although the said embodiment demonstrated and demonstrated the road sign, the road mirror, and the street light as an example of a road accessory, it is not limited to these. For example, the road accessory detection devices 100, 3000, and 5000 can be configured to detect traffic lights, guide plates, and the like as road accessories. This makes it possible to detect a greater number of types of road accessories with high accuracy.

  Moreover, in the said embodiment, although the road appendage detection apparatus 100, 3000, 5000 detected the road appendage arrange | positioned at the side of a road, it is not limited to this. For example, the road accessory detection devices 100, 3000, and 5000 can be configured to detect a road accessory located above or below the road. This makes it possible to detect a greater number of types of road accessories with high accuracy.

  Moreover, in the said embodiment, although the pillar area | region detection part 101 has detected a pair of black part of the vertical direction in a captured image as a pillar area | region, it is not limited to this. For example, the column region detection unit 101 can be configured to detect a pair of horizontal black portions in the captured image as a column region. In this case, a guide plate or the like supported by a column extending in the lateral direction can be detected with high accuracy as a road accessory.

  The road accessory detection device 100, 3000, 5000 of the above embodiment includes a control device such as a CPU, a storage device such as a ROM (Read Only Memory) or a RAM, an external storage device such as an HDD or an SSDCD drive device, and a monitor. And a hardware configuration including an input device such as a touch panel and operation buttons.

  The road accessory detection program executed by the road accessory detection device 100, 3000, 5000 of the above embodiment is preinstalled in a ROM or the like and provided as a computer program product.

  The road adjunct detection program executed by the road adjunct detection apparatus 100, 3000, 5000 according to the above embodiment is a file in an installable format or an executable format, and is a CD-ROM, a flexible disk (FD), a CD-R. The program may be recorded on a computer-readable recording medium such as a DVD (Digital Versatile Disk) and provided as a computer program product.

  Further, a computer program is stored by storing a road accessory detection program executed by the road accessory detection apparatus 100, 3000, 5000 of the above-described embodiment on a computer connected to a network such as the Internet and downloading the program via the network. You may comprise so that it may provide as a product. Moreover, you may comprise so that the road accessory detection program performed with the road accessory detection apparatus 100,3000,5000 of the said embodiment may be provided or distributed via networks, such as the internet, as a computer program product.

  The road accessory detection program executed by the road accessory detection devices 100, 3000, and 5000 of the above embodiment includes the above-described units (the column region detection unit 101, the specific color detection unit 102, the specific shape conversion unit 3010, the specific shape detection). Unit 103, determination units 104 and 2404, output unit 105, timing unit 106, and position detection unit 107). As actual hardware, the CPU reads the road accessory detection program from the ROM. By executing this, the above-described units are loaded on the main storage device, and the column area detection unit 101, specific color detection unit 102, specific shape conversion unit 3010, specific shape detection unit 103, determination units 104 and 2404, output unit 105, timekeeping The unit 106 and the position detection unit 107 are generated on the RAM.

  Although several embodiments of the present invention have been described, these embodiments are presented by way of example and are not intended to limit the scope of the invention. These novel embodiments can be implemented in various other forms, and various omissions, replacements, and changes can be made without departing from the scope of the invention. These embodiments and modifications thereof are included in the scope and gist of the invention, and are included in the invention described in the claims and the equivalents thereof.

100, 3000, 5000 Road accessory detection device 110, 110a, 110b Imaging camera 101 Column region detection unit 102 Specific color detection unit 103 Specific shape detection unit 104, 2404 Determination unit 105 Output unit 106 Timing unit 107 Position detection unit 120 Color DB
130 Shape DB
3010 Specific shape converter

Claims (10)

From a captured image of a road on which the vehicle travels, based on the pixels of the captured image, a pillar region detection unit that detects a pillar region that is a region of a pillar part to be detected;
A specific color detection unit that detects a specific color region that is a region having the specific color of the detection target from the captured image;
A specific shape detection unit that detects a specific shape region that is a region having the specific shape of the detection target from the captured image;
Based on the pillar region, the specific color region, and the specific shape region, a determination unit that determines whether the captured image includes a road accessory arranged around a road;
An output unit for outputting information on the road accessories;
Road attachment detection device with The specific shape detection unit matches a shape pattern corresponding to the shape of the road appendage with the captured image while scanning the captured image, and matches the captured image with changing the shape pattern. By repeating, detecting the specific shape region,
The road accessory detection device according to claim 1. A specific shape conversion unit for changing the shape pattern;
The road accessory detection device according to claim 2, further comprising: The captured image includes a first captured image in front of the vehicle and a second captured image behind the vehicle,
The pillar area detection unit detects the pillar area from each of the first captured image and the second captured image,
The specific color detection unit detects the specific color region from each of the first captured image and the second captured image,
The specific shape detection unit detects the specific shape region from each of the first captured image and the second captured image,
The determination unit determines whether the road accessory exists in the first captured image based on the pillar region, the specific color region, and the specific shape region detected from the first captured image. Whether the road appendage exists in the second captured image based on the pillar region, the specific color region, and the specific shape region detected from the second captured image. Whether or not the road appendage exists in the first captured image and the second captured image based on the result of the first determination and the result of the second determination. judge,
The road accessory detection apparatus according to any one of claims 1 to 3. The specific color detection unit detects the specific color region having the specific color of the first surface of the road appendage from the first captured image, and detects the road appendage from the second captured image. Detecting the specific color region having the specific color of the second surface;
The specific shape detection unit detects the specific shape region having a specific shape of the first surface of the road appendage from the first captured image, and detects the road appendage from the second captured image. Detecting the specific shape region having the specific shape of the second surface;
The determination unit determines, as the first determination, whether or not a first surface of the road appendage exists in the first captured image, and as the second determination, determines the second captured image. It is determined whether or not the second surface of the road appendage exists, the first determination causes the first surface of the road appendage to exist in the first captured image, and the second determination. When it is determined that the second surface of the road appendage exists in the second captured image, it is determined that the road appendage exists.
The road accessory detection device according to claim 4. The determination unit detects the pillar region, the specific color region, and the specific shape region, and when the distance between the regions on the captured image is less than a predetermined distance, And determining that the region indicated by the specific color region and the specific shape region is the road appendage,
The road accessory detection device according to any one of claims 1 to 5. The determination unit detects all of the pillar region, the specific color region, and the specific shape region, and when any of the distances between the regions on the captured image is equal to or greater than the predetermined distance. Determining that the road accessory does not exist in the captured image;
The road accessory detection apparatus according to claim 6. The determination unit determines that the road accessory does not exist in the captured image when any of the pillar region, the specific color region, and the specific shape region is not detected.
The road accessory detection apparatus according to claim 6 or 7. An imaging unit that captures an image of a periphery of a road on which the vehicle travels to obtain the captured image;
The road accessory detection device according to claim 1, further comprising: From the captured image around the road on which the vehicle travels, based on the pixels of the captured image, detect a column region that is a region of the column portion to be detected,
Detecting a specific color region that is a region having the specific color of the detection target from the captured image;
From the captured image, a specific shape region that is a region having the specific shape of the detection target is detected,
Based on the pillar region, the specific color region, and the specific shape region, it is determined whether or not a road accessory arranged around a road exists in the captured image,
Outputting information about the road accessories,
Road appendage detection method.

Images