WO2015001747A1

WO2015001747A1 - Travel road surface indication detection device and travel road surface indication detection method

Info

Publication number: WO2015001747A1
Application number: PCT/JP2014/003301
Authority: WO
Inventors: 丙辰王; 展彦井上; 宗昭松本
Original assignee: 株式会社デンソー
Priority date: 2013-07-02
Filing date: 2014-06-19
Publication date: 2015-01-08
Also published as: JP2015011665A; JP6032141B2

Abstract

A travel road surface indication detection device includes a generation means (13, S2) and an indicator detection means (16, S7). The generation means generates a bird's-eye image as seen from a prescribed virtual viewpoint, on the basis of multiple photographic images which have been photographed by multiple cameras (3) that photograph the periphery of a vehicle, and at least a portion of the photographic areas of which do not overlap. The indicator detection means detects road surface indicators from the bird's-eye view image generated by the generation means. For each overlapping portion (41a, 42a), which is a portion where the photographic areas in two or more bird's-eye images overlap each other, the indicator detection means detects indicators in an assigned area which has been assigned to the respective overlapping portion.

Description

Travel road marking detection device and travel road marking detection method

Cross-reference of related applications

This disclosure is based on Japanese Application No. 2013-138970 filed on July 2, 2013, the contents of which are incorporated herein by reference.

The present disclosure relates to a traveling road marking detection apparatus that detects a road marking object by performing image processing on a captured image captured by a camera that captures the periphery of a vehicle.

Conventionally, a system for detecting markings on road markings such as lane markings such as white lines and roadsides using a camera mounted on a vehicle has been proposed. In order to calculate the positional relationship between the vehicle and the sign object, images taken by a plurality of cameras may be converted into a bird's-eye view image (see Patent Document 1).

In order to suppress the detection omission of signs around the vehicle, it is preferable to photograph the periphery of the vehicle with a plurality of cameras without any gaps. Therefore, it is conceivable that a plurality of cameras are arranged so that a part of the shooting range overlaps so that no gap occurs in the shooting range.

When the camera is arranged as described above, an overlapping portion is generated in the captured image and the bird's-eye image converted from the captured image. Although the same road surface is photographed in the overlapping portion, if the image processing for detecting the sign object is performed on all the overlapping portions, the load on the processing apparatus may be increased.

JP 2010-245802 A

The present disclosure has been made in view of the above points, and an object of the present disclosure is to provide a traveling road marking detection device and a traveling road marking detection method that can reduce the detection load of road marking objects.

The traveling road surface marking detection device according to the first aspect of the present disclosure includes a generation unit and a marking object detection unit. The generation unit generates a bird's-eye image viewed from a predetermined virtual viewpoint, based on a plurality of captured images captured by a plurality of cameras that capture the periphery of the vehicle and at least a part of the imaging range does not overlap. The sign object detection means detects a sign object on the road surface from the bird's-eye view image generated by the generation means.

The sign object detection means detects a sign object in each of the overlapping portions where the shooting ranges in two or more bird's-eye images overlap each other in the assigned range assigned to each of the overlapping portions.

According to the above road surface sign detection device, since the detected object is detected in the allocated range in the overlapping portion of the plurality of bird's-eye images, it is not necessary to perform the detection process of the labeled object in the portion other than the allocated range, and the detection load by the image processing Can be reduced.

The traveling road surface marking detection device according to the second aspect of the present disclosure includes a generation unit, a range setting unit, and a marking object detection unit.

The generating means includes a first photographed image taken by a first camera having a first photographing range and a second photograph taken by a second camera having a second photographing range that partially overlaps the first photographing range. Based on the image, a first bird's-eye image and a second bird's-eye image viewed from a predetermined virtual viewpoint are generated. The first bird's-eye image has a first overlapping portion in which an overlapping shooting range partially overlapping in the first shooting range and the second shooting range is shot, and the second bird's-eye image is the overlapping shooting range. Has a second overlapping portion taken.

The range setting means is configured to detect a detection target range and a remaining portion of the detection target range in each of the first overlapping portion and the second overlapping portion based on the resolution of the first captured image and the resolution of the second captured image. A detection unnecessary range is set.

In the first captured image, the sign detection means detects a road surface sign from the remaining portion excluding the first overlapping portion and the detection target range of the first overlapping portion, and in the second captured image, The sign on the road surface is detected from the remaining part excluding the second overlapping part and the detection target range of the second overlapping part.

The range setting means sets the detection target range of the first overlapping portion to be the same range as the detection unnecessary range of the second overlapping portion, and the detection unnecessary range of the first overlapping portion is the second overlapping portion. It is set to indicate the same range as the detection target range.

The resolution of the portion corresponding to the detection target range of the first overlapping portion in the first captured image is larger than the resolution of the portion corresponding to the detection unnecessary range of the second overlapping portion in the second captured image.

The resolution of the portion corresponding to the detection target range of the second overlapping portion in the second captured image is larger than the resolution of the portion corresponding to the detection unnecessary range of the first overlapping portion in the first captured image.

According to the traveling road marking detection device, the same functions and effects as the traveling road marking detection device according to the first aspect of the present disclosure can be achieved.

A traveling road marking detection device that detects a road marking object by performing image processing on at least a plurality of captured images captured by a plurality of cameras that capture the periphery of the vehicle according to the third aspect of the present disclosure. The road marking detection method used in the method generates a plurality of bird's-eye images viewed from a predetermined virtual viewpoint based on each of a plurality of photographed images, detects a sign from the generated bird's-eye images, and detects two or more For each of the overlapping portions, which are portions where the shooting ranges in the bird's-eye view image overlap each other, a sign object is detected from the assigned range assigned to each of the overlapping portions.

By using the traveling road marking detection method described above, the same operations and effects as the traveling road marking detection device according to the first aspect of the present disclosure can be achieved.

The above and other objects, features and advantages of the present disclosure will become more apparent from the following detailed description with reference to the accompanying drawings. The drawing
FIG. 1 is a block diagram illustrating a configuration of a sign recognition system according to an embodiment of the present disclosure. FIG. 2 is a flowchart for explaining the processing procedure of the lane departure warning process. FIG. 3A is a diagram illustrating an example of an image obtained by converting a captured image around the vehicle into a bird's-eye view image, and FIG. 3B is a diagram of the captured image around the vehicle illustrated in FIG. Yes, FIG. 4A is an enlarged view in which a part of an overlapping part in one bird's-eye image is enlarged, and FIG. 4B is an enlarged view in which a part of the overlapping part in another bird's-eye image is enlarged. 4 (c) is a diagram of the bird's-eye image shown in FIG. 4 (a), FIG. 4 (d) is a diagram of the bird's-eye image shown in FIG. 4 (b), FIG. 5A and FIG. 5B are diagrams for explaining an example of an allocation range setting method. FIG. 6A and FIG. 6B are diagrams for explaining the handling of inappropriate parts in setting the allocation range. FIG. 7A and FIG. 7B are diagrams for explaining an example of an allocation range setting method.

Hereinafter, embodiments of the present disclosure will be described with reference to the drawings.

[Example]
(1) Configuration of Sign Object Recognition System 1 The sign object recognition system 1 is a system used by being mounted on a vehicle such as an automobile. As shown in FIG. 1, a plurality of in-vehicle cameras 3, display means 5, And a road surface marking detection device 7.

The sign recognition system 1 detects a white line such as a roadway center line or a roadway outer line, and displays a warning when a traveling vehicle is predicted to deviate from the lane. In the present embodiment, a configuration that targets a white line as an example of a road surface marking object is illustrated, but it can be configured to detect a lane marking other than the white line, a roadway, or the like.

The in-vehicle camera 3 is a photographing device for photographing the periphery of the vehicle, and for example, a known Charge-Coupled Device (CCD) image sensor, Complementary Metal-Oxide-Semiconductor (CMOS) image sensor, or the like can be used. The in-vehicle camera 3 is disposed on the front, rear, left, and right sides of the vehicle, and photographs the front diagonally lower, the rear diagonally downward, the left diagonally downward, and the right diagonally downward of the vehicle, respectively. Each in-vehicle camera 3 captures the periphery of the vehicle at a predetermined time interval (1/15 s as an example), and outputs the captured image to the traveling road marking detection device 7.

The imaging range of the in-vehicle camera 3 is arranged so that a part thereof overlaps with the imaging range of the other in-vehicle camera 3 and the other parts except for the part do not overlap with the imaging range of the other in-vehicle camera 3.

The display means 5 is a liquid crystal display that displays an image, and displays the image according to a signal input from the traveling road marking detection device 7. In this embodiment, a warning screen is displayed when it is predicted that the traveling vehicle will deviate from the traveling lane.

The traveling road surface marking detection device 7 acquires a photographed image photographed by the in-vehicle camera 3, performs image processing to detect a white line, and predicts whether or not the vehicle deviates from the traveling lane.

This road surface sign detection device 7 includes a CPU (not shown), a ROM that stores a program executed by the CPU, a RAM that is used as a work area when the CPU executes a program, and a flash memory that can electrically rewrite data. And a computer system including an NVRAM as a nonvolatile memory such as an EEPROM, and executes a predetermined process by executing a program.

This traveling road surface marking detection device 7 inputs traveling information 9 from an ECU and various sensors mounted on the vehicle. The travel information 9 includes information such as a shift range, vehicle speed, steering, or yaw rate.

In addition, the road surface sign detection device 7 includes a vehicle motion calculation unit 11, a camera-mounted parameter correction unit 12, a viewpoint conversion unit 13, a resolution distribution calculation unit 14, an image range selection processing unit 15, an image detection processing unit 16, and a sign recognition result. The integrated determination unit 17 functions.

The camera-mounted parameter correction unit 12 stores information that can specify the shooting range of each vehicle-mounted camera 3 as the camera-mounted parameter 12a indicating the mounting position of the vehicle-mounted camera 3. Specifically, the information on the position information (three-dimensional coordinates) and the photographing direction (pitch, roll, yaw angle) of each in-vehicle camera 3 with respect to the vehicle is stored.

(2) Processing by the road marking detection device 7 The lane departure warning processing executed by the road marking detection device 7 will be described based on the flowchart shown in FIG. As an example, this process is started when the vehicle starts running (when the vehicle speed exceeds a predetermined threshold) and is repeatedly executed at predetermined time intervals.

In S1, a photographed image photographed by each vehicle-mounted camera 3 is acquired.

In S2, the acquired photographed image is converted into a bird's-eye view image. This process is realized by the viewpoint conversion unit 13 described above.

The viewpoint conversion unit 13 acquires captured images from each of the in-vehicle cameras 3, and generates a plurality of bird's-eye images viewed from a predetermined virtual viewpoint based on the acquired captured images. In this embodiment, the virtual viewpoint is directly above the vehicle, and a bird's-eye view image in which the road surface is viewed vertically is generated using a predetermined conversion formula corresponding to the camera mounting parameter 12a indicating the camera mounting position. Although a method for generating a bird's-eye view image is known, for example, a technique described in JP-A-10-211849 can be used. The viewpoint conversion unit 13 is an example of a generation unit in the present disclosure.

The bird's-

eye images

21, 22, 23, and 24 shown in FIG. 3A are bird's-eye images showing the front, rear, left, and right sides of the vehicle, respectively, and are taken by each vehicle-mounted camera 3 at the same timing. It is a bird's-eye view image based on an image. The same road surface around the vehicle is photographed in the overlapping portion where the shooting ranges in each bird's-eye view image overlap each other.

For confirmation, the bird's-eye view images in FIG. 3 (a) are merely arranged according to the shooting direction centered on the vehicle, and are not synthesized according to the actual position.

In subsequent S3, fixed noise such as dirt and edge suitability values are detected. This process is realized by the image detection processing unit 16 described above.

The image detection processing unit 16 executes three processes: (a) fixed noise detection, (b) edge suitability value calculation, and (c) white line detection. Since the above (a) and (b) are executed in S3, these processes will be described here.

(A) Fixed noise detection The image detection processing unit 16 detects fixed noise generated continuously in the same part such as lens dirt of the in-vehicle camera 3 based on the photographed image. A method for detecting fixed noise such as dirt is known, but as an example, the method described in Japanese Patent Application Laid-Open No. 2003-259358 can be used. This fixed noise detection may be performed based on a bird's-eye view image. The image detection processing unit 16 is an example of a noise detection unit in the present disclosure.

(B) Edge Suitability Calculation The image detection processing unit 16 detects an edge from each bird's-eye view image, and a parameter indicating the degree of ease of detecting a white line for the detected edge (hereinafter, also simply referred to as an edge suitability value). ) Is calculated. This parameter is obtained from the edge strength, the number of edges, and the edge angle.

The edge strength is the contrast (brightness difference) between the white line and the road surface. The greater the contrast, the greater the edge suitability value. The number of edges is the number of detected edges, and the edge suitability value increases as this number increases. The edge angle is the direction in which the edges are arranged, and the edge suitability value increases as the edges are arranged at an appropriate angle in consideration of the traveling direction of the vehicle and the direction of the white line detected in the past.

In other words, a part with a high edge suitability value is a part suitable for white line detection with few problems that are inappropriate for white line detection, for example, overexposure, excessive darkness, and many inappropriate edges are detected. It is.

In subsequent S4, the accuracy of the camera mounting parameter 12a is confirmed, and if there is an error, correction is performed. This process is realized by the above-described camera mounting parameter correction unit 12.

The camera-mounted parameter correction unit 12 estimates current position information and current shooting direction information of the in-vehicle camera 3 based on the shot image. The estimated camera mounting parameter is compared with the stored camera mounting parameter 12a. If the estimated camera mounting parameter is different by a predetermined threshold or more, the newly estimated parameter is updated as the camera mounting parameter 12a.

The camera-mounted parameter correction unit 12 is an example of a position detection unit in the present disclosure.

In subsequent S5, the resolution distribution is calculated based on the camera mounting position. This process is realized by the resolution distribution calculation unit 14 described above.

The resolution distribution calculation unit 14 calculates the resolution distribution using the camera mounting parameters 12a. The resolution distribution is information indicating the original resolution of each pixel of the bird's-eye view image. The original resolution here refers to a portion per pixel of a portion corresponding to one pixel of a bird's-eye image after conversion in a captured image before conversion by the viewpoint conversion unit 13 (hereinafter also simply referred to as an original captured image). This parameter indicates the actual distance on the road surface.

The above parameters are merely examples of parameters that can be used as the original resolution. For example, the number of pixels of a captured image per unit length (for example, 1 cm) of the road surface can be used as the original resolution. Also, the original resolution may be calculated for each set of pixels, not for each pixel. The resolution distribution may be calculated only in the overlapping portion described later.

In the captured image, the road surface relatively close to the vehicle-mounted camera 3 has a shorter road surface distance per pixel than the road surface relatively far from the vehicle-mounted camera 3. When converted into a bird's-eye view image, the road surface relatively closer to the in-vehicle camera 3 has a smaller relative degree of enlargement and the road surface farther has a larger degree of relative expansion. The smaller the magnification, the finer the screen.

4 (a) and 4 (b) show enlarged images obtained by enlarging

portions

22a and 23a where the same photographing range is shown in the bird's-

eye images

22 and 23 of FIG. 3 (a). The portion 23a of the bird's-eye image 23 has a relatively small original resolution, so the degree of enlargement is large, and the boundary between the white line 31 and the surrounding road surface portion 33 is ambiguous. On the other hand, the portion 22a of the bird's-eye image 22 has a relatively high original resolution, so the degree of enlargement is small, and the boundary between the white line 31 and the road surface portion 33 is relatively clear.

When performing white line detection, edges are detected by image processing, but when the boundary between the white line and the road surface is clear, for example, when there is a large difference in brightness between the white line and the road surface, the edge should be detected with high accuracy. Can do. Therefore, even when images are taken of the same road surface, edges can be detected well by using an image with a higher original resolution.

The resolution distribution calculation unit 14 described above is an example of calculation means in the present disclosure.

In subsequent S6, an image processing range for each bird's-eye view image corresponding to each vehicle-mounted camera 3 is determined. The image processing range in the bird's-eye image includes a portion where the shooting range does not overlap with another bird's-eye image, and a portion set as an allocation range by the image range selection processing unit 15 among overlapping portions where the shooting range overlaps with another bird's-eye image. , And the combined range.

The image range selection processing unit 15 sets an allocation range for each overlapping portion of the two bird's-eye images.

A specific allocation range setting method will be described with reference to FIGS. 5 (a) and 5 (b). The bird's-

eye images

41 and 42 are obtained by converting captured images captured by different on-vehicle cameras 3 into bird's-eye images. Portions overlapping each other in the bird's-

eye images

41 and 42 are referred to as overlapping

portions

41a and 42a, respectively. Since these are the same size, the pixel arrangement is the same, and the pixels at the same position correspond to the same shooting range.

The original resolution of each pixel of the overlapping

portions

41 a and 42 a is obtained by the resolution distribution calculation unit 14. Of the overlapping portion 41a, a portion where the original resolution of the corresponding pixel is larger than that of the overlapping portion 42a is set as the allocation range 41b. On the other hand, a portion having an original resolution larger than the overlapping portion 41a in the overlapping portion 42a is set as the allocation range 42b.

The bird's-eye image 41 is an example of the first bird's-eye image in the present disclosure, and the bird's-eye image 42 is an example of the second bird's-eye image in the present disclosure. The overlapping part 41a of the bird's-eye image 41 is an example of a first overlapping part in the present disclosure, and the overlapping part 42a of the bird's-eye image 42 is an example of a second overlapping part in the present disclosure. In the first overlapping portion 41a, the allocation range 41b is an example of a detection target range in the present disclosure, and the remaining portion is an example of a detection unnecessary range in the present disclosure. In the second overlapping portion 42a, the allocation range 42b is an example of a detection target range in the present disclosure, and the remaining portion is an example of a detection unnecessary range in the present disclosure.

As shown in FIGS. 5 (a) and 5 (b), the detection target range 41b of the first overlapping portion 41a is set to indicate the same range as the unnecessary detection range of the second overlapping portion 42a, and the first overlapping The detection unnecessary range of the portion 41a is set to indicate the same range as the detection target range 42b of the second overlapping portion 42a.

The actual distance of the road surface per pixel of the portion corresponding to the detection target range 41b of the first overlapping portion 41a in the first captured image is the portion of the portion corresponding to the detection unnecessary range of the second overlapping portion 42a in the second captured image. It is shorter than the actual distance of the road surface per pixel. The actual distance of the road surface per pixel of the portion corresponding to the detection target range 42b of the second overlapping portion 42a in the second captured image is the portion of the portion corresponding to the detection unnecessary range of the first overlapping portion 41a in the first captured image. It is shorter than the actual distance of the road surface per pixel.

However, the portion where the fixed noise is detected in S3 and the portion whose edge suitability value calculated in S3 is smaller than the predetermined threshold (hereinafter, any one of these portions will be simply referred to as an inappropriate portion) is the target of the allocation range. Excluded from. That is, the allocation range is set from a portion where no fixed noise is detected and the edge suitability value is equal to or greater than a predetermined threshold.

As shown in FIGS. 6A and 6B, when there is an inappropriate portion 41c in the overlapping portion 41a, the inappropriate portion 41c is not set as the allocation range 41b, and that portion is the original resolution in the overlapping portion 42a. Regardless, it is set as the allocation range 42b.

When the original resolution is calculated for each set of a plurality of pixels, an allocation range may be set based on the original resolution.

In addition, the allocation range is not set strictly based on the comparison results of the original resolutions of a plurality of overlapping portions, and is determined based on the comparison results of the original resolutions as shown in FIGS. 7 (a) and 7 (b). The allocated range may be set with the boundary line 43 as a boundary. The boundary line 43 is set so that a portion (pixel) having a higher original resolution than the other overlapping portion in one overlapping portion is included in the allocation range of the one overlapping portion.

When setting the overlapping portion as shown in FIG. 7, it is preferable that the portion having the original resolution higher than the other overlapping portion in one overlapping portion occupies the main portion of the allocation range in one overlapping portion. . In addition, it may be set such that a portion having an original resolution higher than that of the other overlapping portion in one overlapping portion is included in the allocation range of the one overlapping portion by a predetermined ratio or more.

In addition, when the part of the same original resolution arises in the overlapping

parts

41a and 42a, it can be included in the allocation range of either of the overlapping

parts

41a and 42a about the part.

The image range selection processing unit 15 described above is an example of a range setting unit in the present disclosure. In addition, a pixel or a set of pixels having a higher original resolution than the overlapping portion 42a in the overlapping portion 41a is an example of the first region in the present disclosure when the overlapping portion 41a is used as a reference, and is more than the overlapping portion 41a in the overlapping portion 42a. Also, a pixel or a set of pixels with a small original resolution is an example of the second region in the present disclosure.

In S7, white line detection processing is performed in the image processing range determined in S6 for each bird's-eye view image. This process is the process (c) realized by the image detection processing unit 16 described above.

(C) White Line Detection The image detection processing unit 16 extracts an edge from each bird's-eye view image and performs white line detection. Since a specific method is publicly known, description thereof is omitted. In each overlapping portion where the shooting ranges in each bird's-eye view image overlap each other, white line detection is performed in the portion set as the allocation range.

When white line detection is performed, an edge detected based on a previously captured image is also used. The vehicle motion calculation unit 11 calculates vehicle movement parameters (traveling direction and speed) based on the travel information 9. Using this calculated movement parameter, an edge movement destination based on the vehicle over time is estimated.

The image detection processing unit 16 is an example of a sign detection unit in the present disclosure.

In subsequent S8, the results of the white line detection processing in S7 are integrated to determine the position of the white line with reference to the vehicle. This process is realized by the above-described sign recognition result integration determination unit 17.

The sign recognition result integration determination unit 17 determines the position of the white line based on the vehicle based on the white line information detected in each bird's-eye view image by the image detection processing unit 16. Specifically, the likelihood of the white line detected from each bird's-eye view image, the degree of coincidence and difference with the white line detected from other bird's-eye images, and the information of the white line calculated in the past in the same manner as in S7 are considered. Then, the current position of the white line is determined. As the likelihood of the white line, for example, an edge suitability value can be used.

In subsequent S9, based on the position of the white line determined in S8 and the vehicle movement parameter calculated by the vehicle motion calculation unit 11, the vehicle departs from the lane within a predetermined time (crossing the white line). A lane departure determination is performed to determine whether or not the vehicle is passing, and if it is determined that the vehicle will deviate from the lane, a warning screen for indicating a danger to the driver is displayed on the display means 5. This process is also realized by the above-described sign recognition result integration determination unit 17 as in S8.

(3) Effect In the sign recognition system 1 of the present embodiment, the white line is detected in the allocation range in the overlapping portion where a plurality of bird's-eye images overlap. Since the overlapping ranges of the overlapping portions do not overlap each other, it is possible to suppress the detection of white lines overlapping the same imaging range, and to reduce the detection load due to image processing.

In addition, since the allocation range is set based on the original resolution that is the resolution of the original captured image of the overlapping portion, based on the image of the plurality of overlapping portions that has a small degree of enlargement and the detection target is not blurred. A white line can be detected, and a decrease in white line detection accuracy can be suppressed.

Further, since the mounting position of the in-vehicle camera 3 is detected, the original resolution of the overlapped portion is calculated based on the detected mounting position, and the allocation range is set, the fixing device for fixing the in-vehicle camera 3 to the vehicle is aged. The assigned range can be appropriately changed based on the deterioration or the change in the road surface angle accompanying the traveling of the vehicle, and the decrease in the detection accuracy of the white line can be suppressed.

Further, since the white line is detected by excluding the part having fixed noise such as dirt in the overlapping part or the part having a low edge suitability value from the allocation range, it is possible to suppress the decrease in the white line detection accuracy.

The setting method of the above allocation range is not particularly limited. For example, the first area that occupies a part or all of the overlapping part, and the original resolution that is the resolution of the part corresponding to the first area in the captured image is the same as the first area and the capturing range in the other bird's-eye images. The allocation range may be set so as to include the first area larger than the original resolution of the overlapping second area.

The road surface shot by the camera attached to the vehicle is larger as the distance from the camera is closer, that is, the road surface is shot with a larger resolution. Therefore, when a captured image is converted into a bird's-eye view image, the distance from the camera on the road surface is farther, and the degree of magnification increases as the image has a smaller resolution. Since the image is blurred at the greatly enlarged portion, there is a possibility that the accuracy of detection of the sign object is lowered.

However, as described above, by setting the allocation range based on the original resolution that is the resolution of the original captured image of the overlapped portion, the sign object is detected based on the bird's-eye image with a smaller degree of enlargement and a detection target that is not blurred. It is possible to suppress the decrease in the detection accuracy of the sign object.

[Modification]
Although the embodiments of the present disclosure have been described above, the present disclosure is not limited to the above embodiments, and it is needless to say that various forms can be employed as long as they belong to the technical scope of the present disclosure.

For example, in the above embodiment, in setting the allocation range based on the original resolution of the overlapping portion, the configuration in which the portion where the fixed noise is detected and the portion having a low edge suitability value are excluded from the allocation range is exemplified. Only one of the configurations may be considered, or the allocation range may be set based on only the original resolution without considering either.

Further, the image range selection processing unit 15 may set the allocation range based on the edge suitability value without considering the original resolution. For example, it is conceivable to set, as the allocation range, a range including a portion having a larger edge suitability value compared to other overlapping portions showing the same imaging range among the overlapping portions.

In the above embodiment, the camera mounting parameter 12a is updated as needed, and an appropriate allocation range is calculated based on the camera mounting parameter 12a. However, the allocation range does not change and is constant. Also good.

Moreover, in the said Example, although the structure which detects a white line for every bird's-eye view image was illustrated, before detecting a white line, each bird's-eye view image is synthesize | combined to one image, and a white line is detected from the synthesized bird's-eye image. You may comprise. When combining the images, it is preferable to use an allocation range for combining the overlapping portions.

In the above-described embodiment, the configuration in which the allocation range is set so as not to overlap in a plurality of overlapping portions is exemplified, but a configuration in which some overlap may be employed.

Further, the present disclosure includes, in addition to the above-described traveling road marking detection device and traveling road marking detection method, a system including the traveling road marking detection device as a component, a program for causing a computer to execute the traveling road marking detection method, It can be realized in various forms such as a recording medium on which the program is recorded.

The flowchart or the process of the flowchart described in the present disclosure includes a plurality of means (or referred to as steps), and each means is expressed as, for example, S1. Furthermore, each means can be divided into a plurality of sub means, while a plurality of means can be combined into one means. Further, each means configured in this way can be referred to as a module, means.

Although the present disclosure has been described based on the embodiments, it is understood that the present disclosure is not limited to the embodiments and structures. The present disclosure includes various modifications and modifications within the equivalent range. In addition, various combinations and forms, as well as other combinations and forms including only one element, more or less, are within the scope and spirit of the present disclosure.

Claims

Generating means (13, S2) for generating a bird's-eye image viewed from a predetermined virtual viewpoint, based on a plurality of captured images captured by a plurality of cameras (3) that capture the periphery of the vehicle and at least a part of the imaging range does not overlap. )When,
Marking object detection means (16, S7) for detecting a road surface marking object from the bird's-eye view image generated by the generation means,
The sign object detection means, for each of the overlapping portions (41a, 42a) in which the shooting ranges in two or more bird's-eye images overlap each other, in the assigned range assigned to each of the overlapping portions, A road marking detector that detects objects.
The first region occupying part or all of the overlapping portion, and the original resolution that is the resolution of the portion corresponding to the first region in the captured image is the first region and the capturing range in the other bird's-eye image The road marking detection device according to claim 1, wherein the assigned range is set so as to include the first region that is larger than the original resolution of the second region in which the two overlap.
The road marking detection device according to claim 2, further comprising range setting means (15, S6) for setting the allocation range based on the original resolution of each of the overlapping portions.
Position detection means (12, S4) for detecting the mounting position of the camera;
Calculation means (14, S5) for calculating the original resolution of the overlapping portion based on the mounting position detected by the position detection means;
The travel road marking detection device according to claim 3, wherein the range setting means sets the allocation range of the overlapping portion based on the original resolution calculated by the calculation means.
The sign detection means detects an edge from the bird's-eye image and detects the sign based on the edge,
The range setting means is configured such that, from among the overlapped portions, a parameter indicating a degree of ease of detection of the sign detected at the edge detected by the sign detection means is equal to or greater than a predetermined threshold value. The travel road marking detection device according to claim 3 or 4, wherein the allocation range is set.
The sign detection means detects an edge from the bird's-eye image and detects the sign based on the edge,
A range including a portion in which the parameter indicating the degree of ease of detection of the sign object of the edge detected by the sign object detection unit is larger than the overlap part in the other bird's-eye view image among the overlap parts. The travel road marking detection device according to claim 1, further comprising range setting means (15) that sets the allocation range as the allocation range.
Noise detection means (16, S3) for detecting fixed noise, which is noise generated continuously in the same part of the captured image,
The travel according to any one of claims 3 to 6, wherein the range setting unit sets the allocation range from a portion of the overlapping portion where the fixed noise is not detected by the noise detection unit. Road marking detection device.
The first photographed image taken by the first camera (3) having the first photographing range and the second photographed by the second camera (3) having the second photographing range partially overlapping with the first photographing range. Two generation means (13, S2) for generating a first bird's-eye image and a second bird's-eye image viewed from a predetermined virtual viewpoint based on two photographed images, and the first bird's-eye image includes the first photographing range and the A first overlapping portion in which an overlapping imaging range partially overlapping in the second imaging range is captured, and the second bird's-eye image has a second overlapping portion in which the overlapping imaging range is captured;
Based on the resolution of the first captured image and the resolution of the second captured image, a detection unnecessary range that is a detection target range and a remaining portion of the detection target range in each of the first overlap portion and the second overlap portion. Range setting means (15, S6) for setting
In the first photographed image, a road surface marking object is detected from the remaining part excluding the first overlapping part and the detection target range of the first overlapping part, and the second overlapping part is excluded from the second photographed image. A road surface sign detection device comprising: sign object detection means (16, S7) for detecting an object on the road surface from the detection target range of the remaining part and the second overlapping part.
The range setting means sets the detection target range of the first overlapping portion to be the same range as the detection unnecessary range of the second overlapping portion, and the detection unnecessary range of the first overlapping portion is the second overlapping portion. Set to indicate the same range as the detection target range of
The resolution of the portion corresponding to the detection target range of the first overlapping portion in the first captured image is larger than the resolution of the portion corresponding to the detection unnecessary range of the second overlapping portion in the second captured image,
The resolution of the portion corresponding to the detection target range of the second overlapping portion in the second captured image is larger than the resolution of the portion corresponding to the detection unnecessary range of the first overlapping portion in the first captured image. The described road surface marking detection device.
Running road marking detection used in a running road marking detection apparatus that detects a road marking object by image processing a plurality of captured images taken by a plurality of cameras that capture the periphery of the vehicle and at least a part of the shooting range does not overlap. A method,
Based on each of the plurality of captured images, a plurality of bird's-eye images viewed from a predetermined virtual viewpoint are generated (S2),
For each of the overlapping portions, which are portions where the shooting ranges of the two or more generated bird's-eye images overlap each other, the sign is detected from the assigned range assigned to each of the overlapping portions (S7). Including road surface marking detection method.